DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
caishunfeng	852597c9bf	[Bug-9070][Server]fix taskExecutionContext json convert error (#9072 ) * [Bug-9070][Server]fix taskExecutionContext json convert error * test * add license header Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
guoshupei	3114c91748	[Bug-8810] [master] add try catch to onSuccess method in the WorkflowExecuteThreadPool (#8896 ) This closes #8810 Co-authored-by: guoshupei <guoshupei@lixiang.com>	3 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	3 years ago
Kerwin	c4fc59c9bb	[Fix][Master] Fix master-server execution logic thread unsafe problem (#8707 ) * Fix thread safety issue in master service * optimize code.	3 years ago
Martin Huang	b0fc6e7a69	[Feature-5752][server]Add blocking task (#6272 ) * fix python task execution error * modify code style * delete AbstractTask#setCommand * migrate blocking task alert feature and common attribute * blocking task migrate(complete base features) * migrate blocking task test * resolve TaskNode.java conflict * fix package problem in BlockingParameters * resolve duplicate code * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * optimise code based on pr#6283 in ds master repo * add log * modify log * resolve conflicts * change hardcode to variable * Add blocking task UI * refactor old code to compatable with the newest branch * add override annotation * remove old code * remove never used packages * add missing packages declaration * Refine the i18n description * add UT * resolve some code smell problems * add fetch-depth * enable unit test in sonar analysis * disable unit test * remove unused packages * resolve duplicate codes * add logs when task status changed and rename some variables * add BLOCK & READY_BLOCK status and corresponding processing code * revoke UI * revert unit-test Co-authored-by: Jave-Chen <baicai.chen@gmail.com>	3 years ago
Yao WANG	67a6813e9c	remove no useful code (#8552 ) Co-authored-by: ywang46 <ywang46@paypal.com>	3 years ago
xiangzihao	74afdcf8f3	[Fix-8506][Master] process instance stop result wrong (#8529 ) * fix bug_8506 * recover success state	3 years ago
yimaixinchen	a72742b9de	remove unuse import jsonFormat (#8548 )	3 years ago
caishunfeng	84f2cfec39	[Feature][Server] Custom timezone - Add timezone handler unified (#8477 ) * date convert of timezone * remove @JsonFormat * add unit test * fix time preview in scheduler * optimization & add env config Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
kezhenxu94	fa19006a18	Add a property to enable/disable metrics module (#8484 )	3 years ago
xiangzihao	234f399488	remove worker group lowercase (#8448 )	3 years ago
xiangzihao	8200a3f15a	add try catch (#8433 )	3 years ago
xiangzihao	864ceb164f	[Fix-Typo] Fix typo in dev branch. (#8395 ) * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * test * fix dev_bug_8337 * fix dev_bug_8337 * fix dev_bug_8337 * fix bug_8337 * fix typo	3 years ago
xiangzihao	08f642c756	[Fix-8337][Master] Process instance can not be kill when task is failure and can be retry (#8387 ) * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * fix bug_8337 * test * fix dev_bug_8337 * fix dev_bug_8337 * fix dev_bug_8337 * fix bug_8337	3 years ago
Wenjun Ruan	66dcf2376a	Optimize some code (#8324 )	3 years ago
zixi0825	4ba6d667fc	[Feature][DataQuality] Add Data quality Module #4283 (#6718 ) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA * fix Constants bug * remove the unused class * add unit test * fix code style error * add unit test * refactor data quality common entity * fix code style error * add unit test * close e2e test * fix code smell bug * modify dataquality enum value to 14 in TaskType * add data qualtiy task * update * add getDatasourceOptions interface * fix checkstyle * close e2e test * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * add unit test and alter dataquality task result * fix checkstyle * fix process service test error * add unit test and fix code smells * fix checkstyle error * fix unit test error * fix checkstyle error * change execute sql type name * revert ui pom.xml * fix data quality task error * fix checkstyle error * fix dq task src_connector_type ui select bug * fix spark rw postgresql bug * change mysql driver scope * fix form-create json bug * fix code smell * fix DolphinException Bug * fix ui validate rule and Alert title * fix target connection param bug * fix threshold validate change * add rule input entry index * change statistic_comparison_check logic * remove check type change * add DateExpressionReplaceUtil * fix null point expetion * fix null point expetion * fix test error * add more sql driver * fix test error and remove DateExprReplaceUtil * add get datasource tables and columns * add get datasource tables and columns * remove hive-jdbc in pom.xml * fix code smells * update sql * change the pom.xml * optimize multi_table_accuracy ui * fix v-show error * fix code smells * update sql * [Feature][DataQuality] Add data quality task ui (#5054) * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * fix dq task src_connector_type ui select bug * fix threshold validate change * remove check type change * add get datasource tables and columns * optimize multi_table_accuracy ui * fix v-show error * fix code smells Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality module (#4830) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * update * close e2e test * fix spark rw postgresql bug * change mysql driver scope * add more sql driver * remove hive-jdbc in pom.xml * change the pom.xml Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality task backend (#4883) * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * refactor data_quality_module * add header license * data quality module refactor * fix unit test error * fix checkstyle error * fix unit test error * fix checkstyle error * fix unit test error * fix code smell * fix check style * fix unit test error * task statistics value add unique code * fix unit test error * fix checkstyle error * fix checkstyle * fix security hotspot * fix unit test error * fix security hotspot * fix check * add data quality task error handling * fix unit test error * add unit test * add unit test * optimize data quality result alert * fix unit test * fix sql script error * fix bug * update sql script * fix checkstyle * add license * fix checkstyle * fix checkstyle * fix unit test * add jacoco dependencies * fix unit test * fix unit test * add jacoco dependencies * add unit test * add unit test * add license * fix checkstyle * fix pom * fix checkstyle * fix checkstyle * merge dev * fix ui error * fix pom error * fix pom error * fix test error * fix test error * mssql-jdbc exclude azure-keyvault * fix test error * merge dev and add unit test * add notes * rollback the CollectionUtils * fix * update sql * fix * fix * fix query rule page error * change dq.jar path * fix sql error * fix ui error * fix(dq): jar path&task enum description * add notes on DataQualityApplication * fix dq result jump error * fix(ui): page condition * feat(ui): add show error output path * change version * remove all chinese word in sql * merge Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>	3 years ago
Wenjun Ruan	f88def8ef7	[IMPROVEMENT-8178] Add Netty processor in Spring container (#8179 ) * Add Netty processor in Spring container	3 years ago
Kerwin	103ccb01da	change version to 2.0.4-SNAPSHOT (#8188 )	3 years ago
wind	b5fa54b6be	[Bug-7865][MasterServer] retry logic optimization (#8156 ) * submit task optimization * cloneAndReset * update * tolerant task restart when init * fix snoar check * fix test * delete unuse file * taskInstance key * code style * skip complete when retry Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
天仇	90ea6ebb6f	[Improvement-7907][refactor] Optimization for query task instances list when build dag flow (#7915 ) Optimization for query task instances list when build dag flow This closes #7907	3 years ago
Assert	12d7e6261e	[Improvement][MasterServer] process complement data optimization #7925 (#7976 ) * [Improvement][MasterServer] process complement data optimization #7925 * [Improvement][MasterServer] process complement data optimization #7925 Co-authored-by: shangeyao <sgy960921>	3 years ago
wind	a8c592bd93	[Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry (#7809 ) * [Bug-7788] fix submit duplicate tasks sometimes when retry * [Bug-7788][MasterServer] fix submit duplicate tasks sometimes when retry Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
JinYong Li	24a97fda75	[Bug] [dolphinscheduler-api]support workflow instance to definition (#7930 ) * add task save and binds workflow * add task update with upstream * support workflow instance to definition * fix ut	3 years ago
wind	e4cdd82b7c	[Improvement][TaskLog] Unified task log (#7831 ) * task logger * update * log format * remove master config task-logger * udpate log level Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wind	4b22ad6cf6	[Implement][MasterServer]TaskProcessor code optimization (#7754 ) * task processor optimization * fix test Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
aaronlinv	707e3a6ec1	[Improvement][Logger] Logger server integrate into master and worker (#6894 )	3 years ago
wind	d3bd7309fb	[Bug-7686][Server]fix restart server after kill force (#7688 ) * [DS-7686][Server]fix restart server after kill force * update registry logic Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
aaronlinv	9a1cea402e	[Fix-7062][MasterServer] NPE when run switch conditions sub-process (#7687 )	3 years ago
wind	8808c0a700	[Bug-7474][MasterServer] fix failover when node host is null (#7475 ) * fix failover when node host is null * add failover execute thread * worker handle dead server * fix task instance failover time check * fix upgrade sql * failover logic update Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
lhjzmn	3af4d765c2	[Fix-7538] [server] Fix when there is a forbidden node in dag, the execution flow is abnormal (#7613 ) * when there is a forbidden node in dag, the execution flow is abnormal Co-authored-by: hongjie.li <hongjie.li@dmall.com>	3 years ago
Assert	e172015cd3	fix conflict pr 6997 (#7571 ) Co-authored-by: shangeyao <sgy960921>	3 years ago
wangxj3	9e763ad0d1	[Feature-#6422] [api-server] task group queue (#7491 ) * add task group * modify task group * pull dev * add license header * modify code style * fix code style * fix sql error * fix error * fix test * fix test * fix test * fix test * fix code style * fix ut * code style * fix unit test * test ut * ut * add unittest * test ut * modify back ut * majorization code * fix conflict * fix ut * add task group api * reset file * fix ut * fix lost column * fix ut * fix ut * fix ut * fix ut * delete duplicate code * fix code style 、name * fix ut * fix mapper Co-authored-by: wangxj <wangxj31>	3 years ago
xiangzihao	1efa85ca27	[Fix-7534][Master] zookeeper failover error (#7562 ) * fix bug_7534 * fix bug_7534 * fix bug_7534 * fix bug_7534 * fix bug_7534 * fix bug_7534 Co-authored-by: SbloodyS <sbloodys@qq.com>	3 years ago
SbloodyS	9f56123a26	fix bug_7537 (#7551 ) Co-authored-by: SbloodyS <sbloodys@qq.com>	3 years ago
Kerwin	b23f756c22	[Fix-7146][server]Fix the task processor thread is not safe. (#7488 ) * Fix the task processor thread is not safe. * fix code style	3 years ago
wind	41bf1a955e	[Improvement-7213][MasterServer] execute thread pool code optimization (#7258 ) * threadpool optimization * threadpool params * rebase dev * ut check fix * add return * rebase dev * event loop Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wind	1cbc86b9e6	add task-logger config (#7423 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wind	99b8ec6492	[Bug-7319][MasterServer] fix taskNode NPE when switch else branch is empty (#7320 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Kerwin	939fa0c3ba	fix TaskResponseProcessor process NullPointerException (#7318 )	3 years ago
kezhenxu94	67cc260d52	Split the components into individual package, refactor configurations to use Spring properties (#7160 ) * Split the components into individual package A follow-up PR will be made to build dedicated Docker images for each component, so that every component Docker image has minimal jars, which is easy to maintain and good for security fixes. * Split the components into individual package A follow-up PR will be made to build dedicated Docker images for each component, so that every component Docker image has minimal jars, which is easy to maintain and good for security fixes. * Split the components into individual package A follow-up PR will be made to build dedicated Docker images for each component, so that every component Docker image has minimal jars, which is easy to maintain and good for security fixes.	3 years ago

40 Commits (23b24508abb14a96687dba856efb5915a33a331c)