DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
jin	74bd33301f	feat:optimizing code (#2245 ) * feat:optimizing code * add licence and format code	5 years ago
qiaozhanwei	055bb28de4	worker fault tolerance modify (#2212 ) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * Encapsulate the parameters required by sqltask * 1，Encapsulate the parameters required by sqltask 2，SQLTask optimization * AbstractTask modify * ProcedureTask optimization * MasterSchedulerService modify * TaskUpdateQueueConsumer modify * test * DataxTask process run debug * DataxTask process run debug * add protobuf dependency，MR、Spark task etc need this * TaskUpdateQueueConsumer modify * TaskExecutionContextBuilder set TaskInstance workgroup * WorkerGroupService queryAllGroup modify query available work group * 1，get workergroup from zk modify 2，SpringConnectionFactory repeat load modify * master and worker register ip use OSUtils.getHost() * ProcessInstance host set ip:port format * worker fault tolerance modify * Constants and .env modify Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
qiaozhanwei	e31b48fdb2	1，get workergroup from zk modify 2，SpringConnectionFactory repeat load modify (#2201 ) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * Encapsulate the parameters required by sqltask * 1，Encapsulate the parameters required by sqltask 2，SQLTask optimization * AbstractTask modify * ProcedureTask optimization * MasterSchedulerService modify * TaskUpdateQueueConsumer modify * test * DataxTask process run debug * DataxTask process run debug * add protobuf dependency，MR、Spark task etc need this * TaskUpdateQueueConsumer modify * TaskExecutionContextBuilder set TaskInstance workgroup * WorkerGroupService queryAllGroup modify query available work group * 1，get workergroup from zk modify 2，SpringConnectionFactory repeat load modify Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
Rubik-W	69633a29c6	fix: Api server startup fail (#2198 )	5 years ago
t1mon	f6f29f4abb	Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. (#2193 )	5 years ago
gabry.wu	61ebbf2946	Adapting partial code(file name start with L) to the sonar cloud rule (#2171 )	5 years ago
gabry.wu	daad5efbab	use static method in static class JSON (#2156 )	5 years ago
tswstarplanet	e3065dbfcc	fix field access modifier (#2150 )	5 years ago
lgcareer	bbe2cd4864	refactor worker group (#2132 ) * replace worker group id with worker group * add worker group field in ddl * remove worker group id	5 years ago
dailidong	f551913249	split logback config into seperate child module logback (#2120 ) * update logback * update log	5 years ago
gabry.wu	450a1f56fc	Adapting partial code(file name start with E) to the sonar cloud rule (#1999 ) * Adapting partial code(file name start with E) to the sonar cloud rule * remove isEmpty invoke * resolve conflicts	5 years ago
qiaozhanwei	eb45ff9e83	TaskPriority refactor (#2097 ) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
qiaozhanwei	d5c795bd45	fix ugly logger usage (#2063 ) * fix ugly logger usage * remove space	5 years ago
qiaozhanwei	3e5a939913	delete unused import (#2066 )	5 years ago
qiaozhanwei	72e8f8d195	host add host:port format (#2070 ) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format	5 years ago
Rubik	1ea48375f4	Fixbug i18n #2008 (#2035 ) * fix：add chinese message * fix：add chinese message * fix：'getLocale()' can return not null * fix：optimize message content	5 years ago
gabry.wu	5ca4862ea9	remove String.format for org.slf4j.Logger (#2001 )	5 years ago
Yelli	619fb30c50	#1310 add executor user to the process instance & task instance pages (#1973 ) * add executor name in processDefinitionInstance & taskInstance * modify processInstanceTest * modify processInstanceTest * #1310 add executor user to the process instance & task instance pages * add processInstanceUT & taskInstanceUT * add processInstanceUT & taskInstanceUT * modify processInstanceTest & taskInstanceTest * Remove duplicate code * add userServiceUT	5 years ago
qiaozhanwei	9b75bebf1b	1，add sendAsync method 2，refactor LoggerClient (#1965 ) * 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module (#1925) * 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module * ProcessUtils modify * Refactor architecture (#1926) * move version to parent pom * move version properties to parent pom for easy management * remove freemarker dependency * delete CombinedApplicationServer * #1871 correct spelling * #1873 some updates for TaskQueueZkImpl * #1875 remove unused properties in pom * #1878 1. remove tomcat dependency 2. remove combined_logback.xml in api module 3. format pom.xml for not aligning * #1885 fix api server startup failure 1. add jsp-2.1 dependency 2. remove jasper-runtime dependency * add stringutils ut (#1921) * add stringutils ut * Newfeature for #1675. (#1908) Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell. * Add modify user name for process definition (#1919) * class overrides equals() and should therefore also override hashCode() * #1862 add modify user in process difinition list * #1862 add pg-1.2.2 ddl.sql * modify ScriptRunnerTest * add updateProessDifinition UT * modify updateProcessDifinition UT * modify updateProcessDifinition UT * modify mysql 1.2.2 ddl.sql&dml.sql * add scope test to mysql in pom * modify pg-1.2.2 ddl.sql * refactor module * updates Co-authored-by: khadgarmage <khadgar.mage@outlook.com> Co-authored-by: zhukai <boness@qq.com> Co-authored-by: Yelli <amarantine@my.com> * dolphinscheduler-common remove spring (#1931) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * SpringApplicationContext class title add license (#1932) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * add license (#1934) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * move datasource classes to dao module * fix send4LetterWord bug * Refactor architecture (#1936) * move datasource classes to dao module * fix send4LetterWord bug * exclude jasper-compiler in case of runtime conflict * LoggerServiceTest remove ProcessDao (#1944) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * LoggerServiceTest remove ProcessDao * exclude jasper-compiler in case of runtime conflict (#1938) * move datasource classes to dao module * fix send4LetterWord bug * exclude jasper-compiler in case of runtime conflict * DataAnaylysisServiceTest and ProcessDefinitionService modify * remote module add comment * OSUtilsTest modify * add finally block to close channel * add finally block to close channel (#1951) * move datasource classes to dao module * fix send4LetterWord bug * exclude jasper-compiler in case of runtime conflict * add finally block to close channel * refactor log client service * add sendSync method * 1，quartz.properties add conf category 2，dolphinscheduler-daemon.sh modify * dolphinscheduler-binary.xml modify * add sendAsync method (#1962) * add sendAsync method * add sendAsync method * 1，add sendAsync method 2，refactor LoggerClient * LogClientService modify Co-authored-by: Tboy <technoboy@yeah.net> Co-authored-by: khadgarmage <khadgar.mage@outlook.com> Co-authored-by: zhukai <boness@qq.com> Co-authored-by: Yelli <amarantine@my.com>	5 years ago
gabry.wu	99ac739851	rename vaild to valid (#1961 )	5 years ago
qiaozhanwei	c1ee133392	Refactor Architecture Basic modification #1658 (#1946 ) * 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module (#1925) * 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module * ProcessUtils modify * Refactor architecture (#1926) * move version to parent pom * move version properties to parent pom for easy management * remove freemarker dependency * delete CombinedApplicationServer * #1871 correct spelling * #1873 some updates for TaskQueueZkImpl * #1875 remove unused properties in pom * #1878 1. remove tomcat dependency 2. remove combined_logback.xml in api module 3. format pom.xml for not aligning * #1885 fix api server startup failure 1. add jsp-2.1 dependency 2. remove jasper-runtime dependency * add stringutils ut (#1921) * add stringutils ut * Newfeature for #1675. (#1908) Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell. * Add modify user name for process definition (#1919) * class overrides equals() and should therefore also override hashCode() * #1862 add modify user in process difinition list * #1862 add pg-1.2.2 ddl.sql * modify ScriptRunnerTest * add updateProessDifinition UT * modify updateProcessDifinition UT * modify updateProcessDifinition UT * modify mysql 1.2.2 ddl.sql&dml.sql * add scope test to mysql in pom * modify pg-1.2.2 ddl.sql * refactor module * updates Co-authored-by: khadgarmage <khadgar.mage@outlook.com> Co-authored-by: zhukai <boness@qq.com> Co-authored-by: Yelli <amarantine@my.com> * dolphinscheduler-common remove spring (#1931) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * SpringApplicationContext class title add license (#1932) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * add license (#1934) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * Refactor architecture (#1936) * move datasource classes to dao module * fix send4LetterWord bug * LoggerServiceTest remove ProcessDao (#1944) * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * dolphinscheduler-common remove spring * LoggerServiceTest remove ProcessDao * exclude jasper-compiler in case of runtime conflict (#1938) * move datasource classes to dao module * fix send4LetterWord bug * exclude jasper-compiler in case of runtime conflict * DataAnaylysisServiceTest and ProcessDefinitionService modify * remote module add comment * OSUtilsTest modify * add finally block to close channel (#1951) * move datasource classes to dao module * fix send4LetterWord bug * exclude jasper-compiler in case of runtime conflict * add finally block to close channel Co-authored-by: Tboy <technoboy@yeah.net> Co-authored-by: khadgarmage <khadgar.mage@outlook.com> Co-authored-by: zhukai <boness@qq.com> Co-authored-by: Yelli <amarantine@my.com>	5 years ago
Jave-Chen	cdd53a7673	Fix socket bug: read before sock connect. Add UT for FourLetterWordMain. (#1954 ) * Fix socket bug: read before sock connect. Add UT for FourLetterWordMain. * Fix code smell Remove @Ignore and @Test from testCmd(). * correct comment	5 years ago
Technoboy-	8e154ecbe4	refactor log client service	5 years ago
Tboy	1e812ca078	add finally block to close channel (#1951 ) * move datasource classes to dao module * fix send4LetterWord bug * exclude jasper-compiler in case of runtime conflict * add finally block to close channel	5 years ago
Technoboy-	e39d6ba973	add finally block to close channel	5 years ago
Yeleights	6c90575a39	add process null check Remove duplicate code in processDefinitionService add processDefinitionService UT	5 years ago
gabry.wu	44b76468a8	Adapting partial code(file name start with C) to the sonar cloud rules (#1918 ) * Adapting partial code(file name start with C) to the sonar cloud rules * add some UT * add some UT(rollback JSONUtils.checkJsonVaild) * add some UT(fix OSUtils.exeCmd UT)	5 years ago
Yelli	768b5d9dfd	add state null check (#1939 ) add process definition UT	5 years ago
lilin	735034c72f	modify DataAnalysisServiceTest by mock	5 years ago
Tboy	d3f8bb0a2b	Refactor architecture (#1936 ) * move datasource classes to dao module * fix send4LetterWord bug	5 years ago
Technoboy-	52c87a6fda	fix send4LetterWord bug	5 years ago
Technoboy-	b1a68c645d	move datasource classes to dao module	5 years ago
Tboy	29f2b41ed0	Refactor architecture (#1926 ) * move version to parent pom * move version properties to parent pom for easy management * remove freemarker dependency * delete CombinedApplicationServer * #1871 correct spelling * #1873 some updates for TaskQueueZkImpl * #1875 remove unused properties in pom * #1878 1. remove tomcat dependency 2. remove combined_logback.xml in api module 3. format pom.xml for not aligning * #1885 fix api server startup failure 1. add jsp-2.1 dependency 2. remove jasper-runtime dependency * add stringutils ut (#1921) * add stringutils ut * Newfeature for #1675. (#1908) Continue to finish the rest works, add the cache feature for dependence,mr,python,sub_process,procedure and shell. * Add modify user name for process definition (#1919) * class overrides equals() and should therefore also override hashCode() * #1862 add modify user in process difinition list * #1862 add pg-1.2.2 ddl.sql * modify ScriptRunnerTest * add updateProessDifinition UT * modify updateProcessDifinition UT * modify updateProcessDifinition UT * modify mysql 1.2.2 ddl.sql&dml.sql * add scope test to mysql in pom * modify pg-1.2.2 ddl.sql * refactor module * updates Co-authored-by: khadgarmage <khadgar.mage@outlook.com> Co-authored-by: zhukai <boness@qq.com> Co-authored-by: Yelli <amarantine@my.com>	5 years ago
qiaozhanwei	9a70e77955	1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module (#1925 ) * 1，remove dolphinscheduler-rpc module 2，add dolphinscheduler-remote module 3，add dolphinscheduler-service module 4，refactor LoggerServer module * ProcessUtils modify	5 years ago
Yelli	d937a6ae94	Add modify user name for process definition (#1919 ) * class overrides equals() and should therefore also override hashCode() * #1862 add modify user in process difinition list * #1862 add pg-1.2.2 ddl.sql * modify ScriptRunnerTest * add updateProessDifinition UT * modify updateProcessDifinition UT * modify updateProcessDifinition UT * modify mysql 1.2.2 ddl.sql&dml.sql * add scope test to mysql in pom * modify pg-1.2.2 ddl.sql	5 years ago
khadgarmage	c447bb489f	add stringutils ut (#1921 ) * add stringutils ut	5 years ago
gabry.wu	b77dddfeef	Adapting partial code(file name start with B) to the sonar cloud rules (#1915 ) * Adapting partial code(file name start with B) to the sonar cloud rules * add @Ignore to BaseTaskQueueTest class	5 years ago
lilin	e00ef4915d	modify loggerService UT by Mock	5 years ago
zhangzy	9d944d3f76	Optimize duplicate code Use public methods to simplify duplicate code	5 years ago
zhangzy	662233e0dd	Remove ScheduleUtil，use the existing CronUtils CronUtils is a public quartz tools, then the project dolphinscheduler-api can remove dolphinscheduler-server	5 years ago
dailidong	c63cd3e237	combine logback config of master/worker/alert/api server to one (#1898 ) * combine logback config of master/worker/alert/api server to one logback.xml	5 years ago
dailidong	835383105c	combine logback config of master/worker/alert/api server to one logback.xml	5 years ago
Yelli	5169689c35	class overrides equals() and should therefore also override hashCode() (#1882 )	5 years ago
Tboy	6b0debec79	some updates for pom (#1879 ) * move version to parent pom * move version properties to parent pom for easy management * remove freemarker dependency * delete CombinedApplicationServer * #1871 correct spelling * #1873 some updates for TaskQueueZkImpl * #1875 remove unused properties in pom * #1878 1. remove tomcat dependency 2. remove combined_logback.xml in api module 3. format pom.xml for not aligning	5 years ago
Jave-Chen	ff9009a5ae	#1864 fix dependency and createCommand date check (#1865 )	5 years ago
gabry.wu	a1b918c364	[Feature] createSchedule should return schedule's Id (#1861 ) issue: https://github.com/apache/incubator-dolphinscheduler/issues/1860	5 years ago
liwenhe1993	30db5c1467	Api module relp server module/1756 (#1766 ) * Remove server module on api module 1. Remove server module and add dao module. 2. Move quartz in server module to dao module. 3. Update config and class. to resolve conflicts，I deleted CombinedApplicationServer.java Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
Jave-Chen	16913bacd7	fix return, obj error (#1855 )	5 years ago
Jave-Chen	d371745ddf	Fix bug: Use try-with-resources or close this "Socket" in a "finally" clause. (#1716 ) * #1714	5 years ago
Jave-Chen	c522ea7ebb	Fix bug: NullPointerException etc. (#1721 ) * #1720 Fix bug: NullPointerException etc. * fix code smell * Update DateInterval.java refactor equals method Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago

... 14 15 16 17 18 ...

967 Commits (769a20e98698ef6769c82c88ee31a93f5717743c)