DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
tlhhup	396b1716e4	fix bug:#2735 (#2770 ) 1. store process id in zk. 2. resolve host and port through zk path. 3. modify ui key.	5 years ago
zixi0825	6d0375e46d	fix sql task bug #2739 (#2741 ) * Solve the failure to execute non-query sql * fix sql task bug Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com> Co-authored-by: lgcareer <18610854716@163.com>	5 years ago
xuchangqun	cfd83235a2	Hourly dependency configuration， 'currentHour' and 'last24Hours' (#2696 ) * [feature] Add hourly dependency scenarios * [Feature] #2680 add DependentUtilsTest UT Co-authored-by: changqun.xu <changqun.xu@17zuoye.com> Co-authored-by: xuchangqun <changqun.x@gmail.com>	5 years ago
bao liang	184d64e852	[BUG FIX] fix bug: dependent task failed when conditions task exists (#2768 ) * fix bug 2464: change dependent task for process. * remove unused code * add ut * add ut * update comments Co-authored-by: baoliang <baoliang@analysys.com.cn>	5 years ago
qiaozhanwei	2330cc8872	master select worker filter high load worker #2704 (#2733 ) * add LoggerServerTest UT * add LoggerServerTest UT * add LoggerServerTest UT add RemoveTaskLogRequestCommandTest UT add RemoveTaskLogResponseCommandTest * master select worker filter high load worker #2704 * master select worker filter high load worker #2704 * master select worker filter high load worker #2704 * master select worker filter high load worker #2704 * master select worker filter high load worker #2704 * master select worker filter high load worker #2704 Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
gabry.wu	772a8e2b37	add retryCallSilent to RetryerUtils making it easy to use (#2729 ) * add retryCallSilent to RetryerUtils making it easy to use * add more detail to java doc of retryCallSilent	5 years ago
bao liang	a9da86abb4	[bug fix] fix bug:2634:Process shows success,when the task of the process is killed in the terminal (#2731 ) * fix bug: some bugs about worker group from 1.2.0 to 1.3.0 * fix bug: scheduler from 1.2.0 to 1.3.0 do not work * fix bug: scheduler from 1.2.0 to 1.3.0 do not work * fix ut bug * fix bug: Process shows success,when the task of the process is killed in the termina * fix bug #2634: Process shows success,when the task of the process is killed in the terminal Co-authored-by: baoliang <baoliang@analysys.com.cn>	5 years ago
break60	d588d62620	Modify ip configuration	5 years ago
xingchun-chen	7010bb2ef2	Modify HDFS/S3 path (#2736 ) * Modify HDFS/S3 path * Modify HDFS/S3 path * Update common.properties Co-authored-by: chenxingchun <438044805@qq.com> Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
gabry.wu	1771aa50d5	change variable name ,make it easy to understand (#2647 ) Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
gabry.wu	c28a697ee8	add RetryerUtils to retry async & sync function gracefully (#2697 ) * add RetryerUtils tool class for async & sync retry invoking * add RetryerUtils tool class for async & sync retry invoking * add guava-retrying-2.0.0.jar to known-dependencies.txt * fix javadoc * fix code smell Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>	5 years ago
qiaozhanwei	83ede1037e	blank in eamil and left font align #2648 (#2673 ) * script variable has "processDefinitionId" is error #2664 * blank in eamil and left font align #2648 Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
JinyLeeChina	daea87a515	'ExecutionStatus' (#2635 ) Co-authored-by: dailidong <dailidong66@gmail.com> Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>	5 years ago
s751167520	cb5285384e	Fix some code specification issues (#2482 ) * Fix some code specification issues * Pre-compile regular expressions to improve efficiency and solve concurrency problems * Solve the nullpoint problem that Object.equals may bring * Modify if else structure, add {} to improve code readability Co-Authored-By: daili <daili@users.noreply.github.com> * Update ResourceTreeVisitor.java re-run test process Co-authored-by: daili <daili@users.noreply.github.com> Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
dailidong	8493aa8209	simplify common config (#2613 ) * update README about DolphinScheduler * Update issue templates * update * regularize api pom xml update rpc maven compile to 1.8 * regularize api pom xml * change commons.lang3.StringUtils to common.utils.StringUtils * update pom.xml * update * correct equals method * jasper-runtime is needed when api server start * jasper-runtime jar is needed when api server start * combine logback config of master/worker/alert/api server to one logback.xml * remove tomcat runtime jar * add UT * add license * remove jasper-runtime jar, not need anymore * sovle jar conflict， remove servlet-api 2.5 jar * remove servlet 2.5 * add embedded database h2 and update embedded zookeeper * delete logs * add license * change log level to INFO * simplify and optimize config * add development state * simplify common config Co-authored-by: lidongdai <escheduler@outlook.com> Co-authored-by: dailidong <dolphinschedule@gmail.com>	5 years ago
dailidong	175475e0a4	simplify common config	5 years ago
dailidong	6a92bef5e3	add development state config (#2609 ) * update README about DolphinScheduler * Update issue templates * update * regularize api pom xml update rpc maven compile to 1.8 * regularize api pom xml * change commons.lang3.StringUtils to common.utils.StringUtils * update pom.xml * update * correct equals method * jasper-runtime is needed when api server start * jasper-runtime jar is needed when api server start * combine logback config of master/worker/alert/api server to one logback.xml * remove tomcat runtime jar * add UT * add license * remove jasper-runtime jar, not need anymore * sovle jar conflict， remove servlet-api 2.5 jar * remove servlet 2.5 * add embedded database h2 and update embedded zookeeper * delete logs * add license * change log level to INFO * simplify and optimize config * add development state Co-authored-by: lidongdai <escheduler@outlook.com> Co-authored-by: dailidong <dolphinschedule@gmail.com>	5 years ago
dailidong	0d0d844efe	add development state	5 years ago
hgaol	906ffa09a9	resolve code checks	5 years ago
hgaol	3c6d813617	add license to plugin api pom file	5 years ago
hgaol	85f3ac10a0	feat: add plugin management for alert service	5 years ago
lgcareer	fd79be39a9	verify whether the filename start with '/' (#2544 ) * fix #2442 when the resource name contains '$',need translate it to '\$' * fix #2442 when the resource name contains '$',need translate it to '/$' * verify whether the filename start with '/' Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com> Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
dailidong	eec92ed1fc	simplify and optimize config (#2469 ) * [Refactor worker] simplify and optimize config (#2386) * simplify config * simplify config * simplify and optimize config * Update HadoopUtils.java optimize HadoopUtils * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java * Update HttpUtils.java * Update pom.xml * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java * Update HadoopUtilsTest.java	5 years ago
dailidong	a989e60558	Update HadoopUtilsTest.java	5 years ago
dailidong	ee34a95f3d	Update HadoopUtilsTest.java	5 years ago
dailidong	51bf150b94	Update HadoopUtilsTest.java	5 years ago
dailidong	037ab78aaf	Update HadoopUtilsTest.java	5 years ago
dailidong	176c637757	Update HadoopUtilsTest.java	5 years ago
dailidong	437605e665	Update HttpUtils.java	5 years ago
dailidong	a8437f0f02	Update HadoopUtilsTest.java	5 years ago
bao liang	4946e88872	move dependent and conditions to master server (#2423 ) * move condition and dependent to master * move conditions to master * move conditions and dependent task to master * move conditions to master * update test * add log * add test for dependent task * add test for dependent task * update * update * refactor complexity code * refactor complexity code * add conditions task test * add conditions task test * update * update host to host:port * update logback.xml Co-authored-by: baoliang <baoliang@analysys.com.cn>	5 years ago
dailidong	a563c82c6d	Update HadoopUtilsTest.java	5 years ago
dailidong	8e7dcc02b2	Update HadoopUtilsTest.java	5 years ago
dailidong	b85b59375e	Update HadoopUtils.java optimize HadoopUtils	5 years ago
dailidong	5e043c31bc	simplify and optimize config	5 years ago
qiaozhanwei	d6798c6db7	1，task status statistics and process status statistics bug fix (#2357 ) 2，worker group bug fix (#2430 ) * dispatch task fail will set task status failed * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，no worker condition , master will while ture wait for worker startup 2，worker response task status sync wait for result * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix * 1，task status statistics and process status statistics bug fix (#2357) 2，worker group bug fix Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
lgcareer	733acdfd76	It is necessary to check whether the resource is valid (#2399 ) * init full_name in dml of version 1.2.2 * redefine request parameter resourceIds * redefine request parameter resourceIds * Actually grant resource file if choose the directory * To cancel authorized resource need check whether it is used by the process definition which is online * If release the process definition online,It is necessary to check whether resource is valid * update ResourceServiceTest and ResourceMapperTest * add batchUpdateResourceTest * add getHdfsFileNameTest and getHdfsResourceFileNameTest * update ResourceServiceTest and ResourceMapperTest * extract getResourceProcessMap to ResourceProcessDefinitonUtils Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
dailidong	b854d6b77a	[Refactor worker] simplify and optimize config (#2386 ) * simplify config * simplify config * simplify and optimize config	5 years ago
qiaozhanwei	43b990656a	modify UT	5 years ago
qiaozhanwei	49944a4ff3	javadoc compile error modify	5 years ago
qiaozhanwei	223bdec85a	refactor-worker merge to dev bug fix	5 years ago
qiaozhanwei	433b41dd7f	refactor-worker merge to dev bug fix	5 years ago
qiaozhanwei	70d6360149	startup conflict reslove	5 years ago
dailidong	c9620ab557	remove zookeeper as task queue config (#2344 ) * update logback * update log * refactor worker registry (#2107) * Refactor worker (#2115) * refactor worker registry * refactor master server * Modify workgroupid parameter name (#2105) * Delete worker group management page * Modify workgroupid parameter name * Refactor worker (#2121) * refactor worker registry * refactor master server * refactor MasterSchedulerService * cancelTaskInstance set TaskExecutionContext host,logPath,executePath (#2126) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn> * not exist in openjdk,just delete * add master and worker properties * add master and worker properties * add master and worker properties * fix cpu 100% bug * simplify master、 worker、alert、dao properties * add master and worker properties * add master and worker properties * add master and worker properties * Optimize code , reduce cost time from 60ms to 0.5ms * merge code form remote * remove zookeeper as task queue relevant config * correct config error and remove unuse properties that may affect performance * correct config error and remove unuse properties that may affect performance * remove zookeeper as task queue config * remove zookeeper as task queue config Co-authored-by: Tboy <guo.jiwei@immomo.com> Co-authored-by: break60 <790061044@qq.com> Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com> Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
dailidong	fce6af0c6b	[Refactor worker] correct config error and remove unuse properties that may affect performance (#2343 ) * update logback * update log * refactor worker registry (#2107) * Refactor worker (#2115) * refactor worker registry * refactor master server * Modify workgroupid parameter name (#2105) * Delete worker group management page * Modify workgroupid parameter name * Refactor worker (#2121) * refactor worker registry * refactor master server * refactor MasterSchedulerService * cancelTaskInstance set TaskExecutionContext host,logPath,executePath (#2126) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn> * not exist in openjdk,just delete * add master and worker properties * add master and worker properties * add master and worker properties * fix cpu 100% bug * simplify master、 worker、alert、dao properties * add master and worker properties * add master and worker properties * add master and worker properties * Optimize code , reduce cost time from 60ms to 0.5ms * merge code form remote * remove zookeeper as task queue relevant config * correct config error and remove unuse properties that may affect performance * correct config error and remove unuse properties that may affect performance * remove zookeeper as task queue config Co-authored-by: Tboy <guo.jiwei@immomo.com> Co-authored-by: break60 <790061044@qq.com> Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com> Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
gabry.wu	718e4b15d2	Adapting partial code(file name start with O) to the sonar cloud rule (#2259 ) * Adapting partial code(file name start with O) to the sonar cloud rule * resolve conflict with dev branch Co-authored-by: dailidong <dailidong66@gmail.com>	5 years ago
dailidong	07cc10c34b	[Refactor worker] correct config error and remove unuse properties that may affect performance (#2337 ) * update logback * update log * refactor worker registry (#2107) * Refactor worker (#2115) * refactor worker registry * refactor master server * Modify workgroupid parameter name (#2105) * Delete worker group management page * Modify workgroupid parameter name * Refactor worker (#2121) * refactor worker registry * refactor master server * refactor MasterSchedulerService * cancelTaskInstance set TaskExecutionContext host,logPath,executePath (#2126) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn> * not exist in openjdk,just delete * add master and worker properties * add master and worker properties * add master and worker properties * fix cpu 100% bug * simplify master、 worker、alert、dao properties * add master and worker properties * add master and worker properties * add master and worker properties * Optimize code , reduce cost time from 60ms to 0.5ms * merge code form remote * remove zookeeper as task queue relevant config * correct config error and remove unuse properties that may affect performance * correct config error and remove unuse properties that may affect performance Co-authored-by: Tboy <guo.jiwei@immomo.com> Co-authored-by: break60 <790061044@qq.com> Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com> Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
songgg	f7e89e4561	some annotation optimizations (#2333 ) Co-authored-by: songqh <songquanhe@foxmail.com>	5 years ago
dailidong	195dfb508c	remove zookeeper as task queue relevant config (#2330 ) * update logback * update log * refactor worker registry (#2107) * Refactor worker (#2115) * refactor worker registry * refactor master server * Modify workgroupid parameter name (#2105) * Delete worker group management page * Modify workgroupid parameter name * Refactor worker (#2121) * refactor worker registry * refactor master server * refactor MasterSchedulerService * cancelTaskInstance set TaskExecutionContext host,logPath,executePath (#2126) * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * 1, master persistent task 2. extract master and worker communication model * add license * modify javadoc error * TaskExecutionContext create modify * buildAckCommand taskInstanceId not set modify * java doc error modify * add comment * ExecutorManager interface add generic type * add TaskInstanceCacheManager receive Worker report result * TaskInstance setExecutePath * add TaskInstanceCacheManager to receive Worker Task result report * TaskInstanceCacheManager add remove method * add license * add dispatcht task method * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * AbstractCommandExecutor remove db access * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * taskInstanceCache is null ,need load from db * 1，worker TaskPros use TaskExecutionContext replase 2，Master kill Task , KillTaskProcessor modify * worker remove db * ShellTask modify * master persistence processId and appIds * master persistence processId and appIds * master add kill task logic * master add kill task logic * master add kill task logic * javadoc error modify * remove chinese log * executeDirectly method add Override * remote module modify * TaskKillResponseProcessor command type modify * create buildKillCommand * host add host:port format * host add host:port format * TaskAckProcessor modify * TaskAckProcessor modify * task prioriry refator * remove ITaskQueue * task prioriry refator * remove ITaskQueue * TaskPriority refactor * remove logs * WorkerServer refactor * MasterSchedulerService modify * WorkerConfig listen port modify * modify master and worker listen port * cancelTaskInstance set TaskExecutionContext host,logPath,executePath * cancelTaskInstance set TaskExecutionContext host,logPath,executePath Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn> * not exist in openjdk,just delete * add master and worker properties * add master and worker properties * add master and worker properties * fix cpu 100% bug * simplify master、 worker、alert、dao properties * add master and worker properties * add master and worker properties * add master and worker properties * Optimize code , reduce cost time from 60ms to 0.5ms * merge code form remote * remove zookeeper as task queue relevant config Co-authored-by: Tboy <guo.jiwei@immomo.com> Co-authored-by: break60 <790061044@qq.com> Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com> Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>	5 years ago
liwenhe1993	526e5c91f1	Connection mode of adding Sid to Oracle (#2254 ) * Connection mode of adding Sid to Oracle * Remove code * Add asf * Add unit test * Add unit test * Add unit test * solve the conflict	5 years ago

... 8 9 10 11 12 ...

710 Commits (23d8aaf735496aa10f1984433f19eccaa8ca788f)