DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
caishunfeng	0cc0ee77fa	[Bug][Master] fix master task failover (#10065 ) * fix master task failover * ui	3 years ago
WangJPLeo	dbdbfeaeee	[Fix-9975] The selected task instance was recreated when the Master service fail… (#9976 ) * The selected task instance was recreated when the Master service failed over. * Returns the expression result directly. * Use Recovery to determine whether to use the old task instance.	3 years ago
Tq	d4aeee16e5	[Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (#9952 ) * fix cmd param to overwrite global param when executing complement * fix cmd param to overwrite global param when executing complement	3 years ago
Paul Zhang	8562f6a878	[Feature][Log]Add timezone information in log output (#9913 )	3 years ago
WangJPLeo	a1b6b033ad	[Fix-9906] After the serial wait execution strategy stops the running workflow instance, the instance will be woken up and executed if there is a wait instance. (#9907 ) * After the serial wait execution strategy stops the running workflow instance, the instance will be woken up and executed if there is a wait instance. * clear logic * Resource overloading	3 years ago
WangJPLeo	fb0f96ed94	[Fix-9868] A task flow definition isolates the runs of different execution strategies by version numbers. (#9869 ) * The thread cache task flow definition should get the latest version. * Coverage on New Code * Coverage on New Code * Coverage on New Code * use an existing method. * Increase unit test coverage. * Task flow definitions enforce policy isolation.	3 years ago
WangJPLeo	31cd1b5e61	Serial wait for subsequent execution (#9847 )	3 years ago
WangJPLeo	3cea039239	Task queue status update. (#9832 )	3 years ago
WangJPLeo	5c0be8a3d7	A task instance that normally queries the serial wait state. (#9777 ) Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>	3 years ago
WangJPLeo	897d7cb555	Add the host address of the execution server to the sub task task instance. (#9758 ) Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>	3 years ago
Jiajie Zhong	de50f43de6	[common] Make dolphinscheduler_env.sh work when start server (#9726 ) * [common] Make dolphinscheduler_env.sh work * Change dist tarball `dolphinscheduler_env.sh` location from `bin/` to `conf/`, which users could finish their change configuration operation in one single directory. and we only need to add `$DOLPHINSCHEDULER_HOME/conf` when we start our sever instead of adding both `$DOLPHINSCHEDULER_HOME/conf` and `$DOLPHINSCHEDULER_HOME/bin` * Change the `start.sh`'s path of `dolphinscheduler_env.sh` * Change the setting order of `dolphinscheduler_env.sh` * `bin/env/dolphinscheduler_env.sh` will overwrite the `<server>/conf/dolphinscheduler_env.sh` when start the server using `bin/dolphinsceduler_daemon.sh` or `bin/install.sh` * Change the related docs	3 years ago
WangJPLeo	7bcec7115a	[Fix-9717] The failure policy of the task flow takes effect (#9718 ) * Failure policy takes effect. * Coverage on New Code * correct description logic * Compatible with all scenarios * clearer logic Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>	3 years ago
caishunfeng	5657cb9aec	[Bug-9719][Master] fix failover fail because task plugins has not been loaded (#9720 )	3 years ago
gaojun2048	ebc4253d50	[fix][Service] BusinessTime should format with schedule timezone (#9714 ) * BusinessTime should format with schedule timezone * fix test error * fix test error * fix test error	3 years ago
caishunfeng	88d2803fe1	fix task dispatch error overload resource pool of task group (#9667 )	3 years ago
caishunfeng	63638601b0	fix process pause and rerun (#9568 )	3 years ago
sparklezzz	508ed9769a	[Fix][Master Server] handle warn+failed timeout strategy in workflow execute thread of master server (#8077 ) (#9485 ) Co-authored-by: xudong.zhang <xudong.zhang@nio.com>	3 years ago
Paul Zhang	3815a86a3b	[Improvement][Master] Fix typo for MasterTaskExecThreadTest (#9513 )	3 years ago
caishunfeng	66d148872d	[Bug-9501][Worker] fix kill task error before running (#9509 )	3 years ago
czeming	706cdb6a8e	[Fix-9174] [Alert] Fix deduplication of alarm information (#9371 ) * feat(issue #9174): Fix-9174	3 years ago
worry	69923546a1	[DS-9263][Improvement][master]optimize failover (#9281 ) - add FailoverService.java - move failover method from MasterRegistryClient to FailoverService - move failover code from FailoverExecuteThread to FailoverService This closes #9263	3 years ago
caishunfeng	b285ccf930	[Future-9396]Support output parameters transfer from parent workflow to child work flow (#9410 ) * [Future-9396]Support output parameters transfer from parent workflow to child work flow * fix note	3 years ago
worry	dce3c132ca	[DS-9387][refactor]Remove the lock in the start method of the MasterRegistryClient class (#9389 )	3 years ago
Eric Gao	fd6b43bc81	[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223 ) (#9299 )	3 years ago
caishunfeng	40e44f6bc4	fix repeated submit task (#9323 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
guoshupei	3e5781b6e1	[Fix-9065] [master] when task submit failed, remove from queue and add state event to thread (#9186 ) * [Fix-9065] [master] when task submit failed, remove from queue and add state event to thread This closes #9065 * Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java yes, your idea is better Co-authored-by: caishunfeng <caishunfeng2021@gmail.com> * [Bug-9065] [master] update logger info Co-authored-by: guoshupei <guoshupei@lixiang.com> Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	3 years ago
caishunfeng	1073fcae44	[Bug-9295][Master] fix repeated submit task (#9304 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
caishunfeng	52ba2c6475	[Improvement-9288][Master] add task event thread pool (#9293 ) * add task event thread * license heander * ci test * delete unuse file * fix CI test Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
caishunfeng	ddf1ff98fa	[Improvement-7697][Master/Worker] Change the task ack to runnning callback (#8719 ) * rebase dev * change task state to dispatch if dispatch success * update task host when dispatch * add dispatch task event * test * check tenant after enable auto create * handle dispatch state Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
worry	b4b5241797	[DS-9266][feat][server]Implement show dolphinscheduler banner at launch (#9273 ) - add banner.txt - modify application.yaml This closes #9266	3 years ago
wangxj3	fd76436d87	[Feature-#8318] [master-server] Sub process transfer parameters to father process (#8522 ) * load param from subprocess * unit test * modify code style Co-authored-by: wangxj <wangxj31>	3 years ago
caishunfeng	df519f8044	remove common task log (#9274 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
guoshupei	e2c1cc0579	[Fix-9222] [master] Support for deploying multiple masters on one node (#9240 ) This closes #9222 Co-authored-by: guoshupei <guoshupei@lixiang.com>	3 years ago
JinYong Li	85e56b5e2a	fix 9243 (#9244 )	3 years ago
worry	d3251c9bcc	[Improvement-9227][master]implement use the slot to scan the database (#9228 ) when the master assigns tasks by slot,implement use the slot to scan the database. This closes #9227	3 years ago
caishunfeng	263791a63e	[Bug-9137][Server] fix timezone init (#9219 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
guoshupei	258285e6bb	[Fix-8980] [master] fixed repeat processing command (#9220 ) This closes #8980 Co-authored-by: guoshupei <guoshupei@lixiang.com>	3 years ago
Kerwin	d7d756e7b0	Fix DataX and Sqoop Task execute NullPointerException (#9192 )	3 years ago
nobolity	0e3cafec1d	[Feature-8612][RESOURCE] extend s3 to the storage of ds (#8637 ) * feat(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.extend the type of storage 3.add the s3utils to manager resource 4.automatic inject the storage in addition to your config * fix(resource manager): update the dependency * fix(resource manager): extend s3 to the storage of ds fix the constant of hadooputils * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * fix(resource manager): merge the unitTest: 1.TenantServiceImpl 2.ResourceServiceImpl 3.UserServiceImpl * fix(resource manager): extend s3 to the storage of ds merge the resourceServiceTest * fix(resource manager): test cancel the test method createTenant verifyTenant * fix(resource manager): merge the code follow the check-result of sonar * fix(resource manager): extend s3 to the storage of ds fit the spell question * fix(resource manager): extend s3 to the storage of ds revert the common.properties * fix(resource manager): extend s3 to the storage of ds update the storageConfig with None * fix(resource manager): extend s3 to the storage of ds fix the judge of resourceType * fix(resource manager): extend s3 to the storage of ds undo the compile-mysql * fix(resource manager): extend s3 to the storage of ds delete hadoop aws * fix(resource manager): extend s3 to the storage of ds update the know-dependencies to delete aws 1.7.4 update the e2e file-manager common.properties * fix(resource manager): extend s3 to the storage of ds update the aws-region * fix(resource manager): extend s3 to the storage of ds fix the storageconfig init * fix(resource manager): update e2e docker-compose update e2e docker-compose * fix(resource manager): extend s3 to the storage of ds revent the e2e common.proprites print the resource type in propertyUtil * fix(resource manager): extend s3 to the storage of ds 1.println the properties * fix(resource manager): println the s3 info * fix(resource manager): extend s3 to the storage of ds delete the info and upgrade the s3 info to e2e * fix(resource manager): extend s3 to the storage of ds add the bucket init * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * fix(resource manager): extend s3 to the storage of ds upgrade the s3 endpoint * fix(resource manager): withPathStyleAccessEnabled(true) * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * fix(resource manager): upgrade the s3client builder * fix(resource manager): correct the s3 point to s3client * fix(resource manager): update the constant BUCKET_NAME * fix(resource manager): e2e s3 endpoint -> s3:9000 * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * style(resource manager): add info to createBucket * style(resource manager): debug the log * ci(resource manager): test test s3 * ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql * fix(resource manager): update the h2 sql * fix(resource manager): solve to delete the tenant * style(resource manager): merge the style end delete the unuse s3 config * fix(resource manager): extend s3 to the storage of ds UPDATE the rename resources when s3 * fix(resource manager): extend s3 to the storage of ds 1.fix the code style of QuartzImpl * fix(resource manager): extend s3 to the storage of ds 1.impoort restore_type to CommonUtils * fix(resource manager): update the work thread * fix(resource manager): update the baseTaskProcessor * fix(resource manager): upgrade dolphinscheduler-standalone-server.xml * fix(resource manager): add user Info to dolphinscheduler_h2.sql * fix(resource manager): merge the resourceType to NONE * style(upgrade the log level to info): * fix(resource manager): sysnc the h2.sql * fix(resource manager): update the merge the user tenant * fix(resource manager): merge the resourcesServiceImpl * fix(resource manager): when the storage is s3 ,that the directory can't be renamed * fix(resource manager): in s3 ,the directory cannot be renamed * fix(resource manager): delete the deleteRenameDirectory in E2E * fix(resource manager): check the style and recoverd the test * fix(resource manager): delete the log.print(LoginUser)	3 years ago
caishunfeng	852597c9bf	[Bug-9070][Server]fix taskExecutionContext json convert error (#9072 ) * [Bug-9070][Server]fix taskExecutionContext json convert error * test * add license header Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
guoshupei	3114c91748	[Bug-8810] [master] add try catch to onSuccess method in the WorkflowExecuteThreadPool (#8896 ) This closes #8810 Co-authored-by: guoshupei <guoshupei@lixiang.com>	3 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	3 years ago
Kerwin	c4fc59c9bb	[Fix][Master] Fix master-server execution logic thread unsafe problem (#8707 ) * Fix thread safety issue in master service * optimize code.	3 years ago
Martin Huang	b0fc6e7a69	[Feature-5752][server]Add blocking task (#6272 ) * fix python task execution error * modify code style * delete AbstractTask#setCommand * migrate blocking task alert feature and common attribute * blocking task migrate(complete base features) * migrate blocking task test * resolve TaskNode.java conflict * fix package problem in BlockingParameters * resolve duplicate code * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * optimise code based on pr#6283 in ds master repo * add log * modify log * resolve conflicts * change hardcode to variable * Add blocking task UI * refactor old code to compatable with the newest branch * add override annotation * remove old code * remove never used packages * add missing packages declaration * Refine the i18n description * add UT * resolve some code smell problems * add fetch-depth * enable unit test in sonar analysis * disable unit test * remove unused packages * resolve duplicate codes * add logs when task status changed and rename some variables * add BLOCK & READY_BLOCK status and corresponding processing code * revoke UI * revert unit-test Co-authored-by: Jave-Chen <baicai.chen@gmail.com>	3 years ago
Yao WANG	67a6813e9c	remove no useful code (#8552 ) Co-authored-by: ywang46 <ywang46@paypal.com>	3 years ago
xiangzihao	74afdcf8f3	[Fix-8506][Master] process instance stop result wrong (#8529 ) * fix bug_8506 * recover success state	3 years ago
yimaixinchen	a72742b9de	remove unuse import jsonFormat (#8548 )	3 years ago
caishunfeng	84f2cfec39	[Feature][Server] Custom timezone - Add timezone handler unified (#8477 ) * date convert of timezone * remove @JsonFormat * add unit test * fix time preview in scheduler * optimization & add env config Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
kezhenxu94	fa19006a18	Add a property to enable/disable metrics module (#8484 )	3 years ago
xiangzihao	234f399488	remove worker group lowercase (#8448 )	3 years ago

1 2 3 4

179 Commits (251756f4108d8f89f8cac6523f8c22af33c9bfce)