DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
Frank-Liao	e97f6a41ed	Check whether the key exists when "start param to overwrite global param" (#15676 ) (#15677 ) Co-authored-by: Rick Cheng <rickchengx@gmail.com>	3 months ago
Wenjun Ruan	89742332a8	Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time (#15528 )	5 months ago
Wenjun Ruan	5b6b0ceb31	Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355 )	6 months ago
Wenjun Ruan	9c7ad6df0a	[Improvement] Clean up Scheduler logic (#15198 ) * Clean up Scheduler logic * Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java Co-authored-by: 旺阳 <wang@lqwang.net> --------- Co-authored-by: 旺阳 <wang@lqwang.net>	7 months ago
旺阳	5afd94136c	[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue (#14992 ) * change data quality field name * fix-dao * fix-dao * fix-dao * fix-dao * update * test * update * update	7 months ago
Wenjun Ruan	e6d94632b9	Add WorkflowExecuteContext (#14544 )	11 months ago
Jay Chung	221df4a278	revert: Datasource need test binding to create prod (#14381 ) We should keep our step as less as possible, for now, we have to test datasource and binding it to the prod datasource, I think it will make our users do not thing to use datasource, so I do like to revert it and keep step as less as possible related to https://github.com/apache/dolphinscheduler/pull/11670	12 months ago
Wenjun Ruan	aa2d9a7f05	Add baseDao (#14316 )	1 year ago
Drake Youngkun Min	801216bd35	RECOVER_TOLERANCE_FAULT_PROCESS CommandType needs the start parameters (#13958 )	1 year ago
Aaron Wang	45586f71d5	[BUG][Resource Center] Task can not select main JAR (#13536 )	1 year ago
caishunfeng	9d9ae9ad54	[improvement] support self-dependent (#13818 )	1 year ago
Aaron Wang	8895921d87	[Improvement-13651] Remove useless resources_task (#13721 )	1 year ago
qianli2022	8be32d4145	[Feature][Api] When use api to run a process we want get processInstanceId (#13184 ) * add sql * add mapper * add dao * add excutor Co-authored-by: qianl4 <qianl4@cicso.com>	1 year ago
Wenjun Ruan	8a479927f3	Add projectCode in t_ds_process_instance and t_ds_task_instance to remove join (#13284 )	1 year ago
Wenjun Ruan	14ec4a2398	Remove dao module in worker (#13242 )	2 years ago
JieguangZhou	66e20271ad	[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks (#13194 ) * Supports task instance cache operation * add task plugin cache * use SHA-256 to generate key * Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql Co-authored-by: Jay Chung <zhongjiajie955@gmail.com> * Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql Co-authored-by: Jay Chung <zhongjiajie955@gmail.com> * Optimizing database Scripts * Optimize clear cache operation Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>	2 years ago
HanayoZz	489e7fe4e2	[Feature-10495][Resource Center] Resource Center Refactor (#12076 ) * resource center refactor - S3 services connection Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
Yann Ann	04aa125ba2	[Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. (#12505 ) * migrate some task DAO to dao modules. * Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceMapDaoImpl.java Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
caishunfeng	dedff70f90	[Improvement] Merge spi.utils into common.utils (#12502 ) * improve JSONUtil * merge spi.utils into common.utils * split constants && remove some copy utils method * remove StringUtils	2 years ago
Yann Ann	b936b882bb	[Refactor] Migrate all command-related interface functions from ProcessServiceImpl (#12474 ) * migrate all command-related interface functions to CommonService	2 years ago
Eric Gao	d46e4659d9	[Improvement][Style] Fix remaining style errors and wildcard imports (#12412 ) * Fix remaining style errors and wildcard imports * Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link * Fix spotless matching scope of markdown files	2 years ago
Yann Ann	b357781df2	[Migrate][Test] Migrate junit4 -> junit5 test cases in common, service, and spi modules (#12363 )	2 years ago
kezhenxu94	db1d319159	Reorganize some classes in common module, remove duplicate classes (#12321 )	2 years ago
Eric Gao	d1e409e9a2	[Improvement][Test] Remove powermock in service and server modules (#12164 ) * Remove powermock and refactor some related code in dolphinscheduler-service and dolphinscheduler-server modules * Remove redundant comments * Add null check	2 years ago
insist777	5b384f3fab	[Feature-11473][Task]Support test task (#11670 ) * [improve] support test tasks * [improve] support test tasks * [improve] support test tasks * [improve] support test tasks * [improve] support test tasks * Update TaskExecuteThread.java * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * Update DataSource.java * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * Update messages_zh_CN.properties * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * Update messages.properties * Update messages_en_US.properties * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks	2 years ago
kezhenxu94	efedc9a53c	Bump up dependencies to fix cves (#11765 )	2 years ago
caishunfeng	f8d46a26c1	[Feature-11530] add state history for process instance (#11757 ) * [Feature] add state history for process instance (#97) * add state history for process instance * upsertProcessInstance * remove unuse method * fix UT Co-authored-by: caishunfeng <534328519@qq.com>	2 years ago
JiPeng Wang	13d400ceb6	[Fix-11465] Serial first, the last executing instance will be stopped (#11466 ) * Serial first, the last executing instance will be stopped * ut fix	2 years ago
longtb	be66035dee	[Improvement] Avoid using search in for and start using testSaveTaskDefine (#11383 ) * [Improvement] Avoid using search in for and start using testSaveTaskDefine() * [Improvement] not import * Co-authored-by: zhangshunmin <zhangshunmin@kezaihui.com>	2 years ago
Wenjun Ruan	2a67866718	Catch exception when check state in StateWheelExecuteThread (#10908 ) * Catch exception when check state	2 years ago
Wenjun Ruan	3f69ec8f28	[Fix-10842] Fix master/worker failover will cause status incorrect (#10839 ) * Fix master failover will not update task instance status * Add some failover log * Fix worker failover will rerun task more than once * Fix workflowInstance failover may rerun already success taskInstance	2 years ago
Wenjun Ruan	426567348e	Remove quartz in service (#10748 ) * Remove quartz in service	2 years ago
WangJPLeo	8f621ff98b	[Optimization] Calculate global parameter and local parameter at master. (#10704 ) * Global parameter and local parameter calculation external expansion. * k8s task ut fix. * TimePlaceholderUtils import DateUtils fix * follow the review comments to fix. * follow the review comments to fix. * e2e rerun	2 years ago
Wenjun Ruan	66624c5c86	[Bug] [Master] Worker failover will cause task cannot be failover (#10631 ) * fix worker failover may lose event	2 years ago
WangJPLeo	b5184138fa	[Feature] Time function analysis extension. (#10624 ) * Time function analysis extension. * param add. * clear useless logs and update method notes * permission omission fix. * extending time functions to optimize static methods * e2e rerun.	2 years ago
Wenjun Ruan	2d3be6b36c	Add dolphinscheduler-scheduler module (#10360 ) * Add dolphinscheduler-scheduler module	2 years ago
cadl	13af2adfc1	[fix] Fix the SERIAL_DISCARD execution_type not working (#10258 ) * fix always stop process_instance when execution_type is SERIAL_DISCARD * test: no running_process, submit it	2 years ago
WangJPLeo	fb0f96ed94	[Fix-9868] A task flow definition isolates the runs of different execution strategies by version numbers. (#9869 ) * The thread cache task flow definition should get the latest version. * Coverage on New Code * Coverage on New Code * Coverage on New Code * use an existing method. * Increase unit test coverage. * Task flow definitions enforce policy isolation.	2 years ago
gaojun2048	ebc4253d50	[fix][Service] BusinessTime should format with schedule timezone (#9714 ) * BusinessTime should format with schedule timezone * fix test error * fix test error * fix test error	2 years ago
gaojun2048	5ef3f9d668	[optimization] [Service] Optimization ProcessService and add ProcessService interface (#9370 )	2 years ago
worry	d3251c9bcc	[Improvement-9227][master]implement use the slot to scan the database (#9228 ) when the master assigns tasks by slot,implement use the slot to scan the database. This closes #9227	2 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	2 years ago
zixi0825	4ba6d667fc	[Feature][DataQuality] Add Data quality Module #4283 (#6718 ) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA * fix Constants bug * remove the unused class * add unit test * fix code style error * add unit test * refactor data quality common entity * fix code style error * add unit test * close e2e test * fix code smell bug * modify dataquality enum value to 14 in TaskType * add data qualtiy task * update * add getDatasourceOptions interface * fix checkstyle * close e2e test * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * add unit test and alter dataquality task result * fix checkstyle * fix process service test error * add unit test and fix code smells * fix checkstyle error * fix unit test error * fix checkstyle error * change execute sql type name * revert ui pom.xml * fix data quality task error * fix checkstyle error * fix dq task src_connector_type ui select bug * fix spark rw postgresql bug * change mysql driver scope * fix form-create json bug * fix code smell * fix DolphinException Bug * fix ui validate rule and Alert title * fix target connection param bug * fix threshold validate change * add rule input entry index * change statistic_comparison_check logic * remove check type change * add DateExpressionReplaceUtil * fix null point expetion * fix null point expetion * fix test error * add more sql driver * fix test error and remove DateExprReplaceUtil * add get datasource tables and columns * add get datasource tables and columns * remove hive-jdbc in pom.xml * fix code smells * update sql * change the pom.xml * optimize multi_table_accuracy ui * fix v-show error * fix code smells * update sql * [Feature][DataQuality] Add data quality task ui (#5054) * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * fix dq task src_connector_type ui select bug * fix threshold validate change * remove check type change * add get datasource tables and columns * optimize multi_table_accuracy ui * fix v-show error * fix code smells Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality module (#4830) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * update * close e2e test * fix spark rw postgresql bug * change mysql driver scope * add more sql driver * remove hive-jdbc in pom.xml * change the pom.xml Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality task backend (#4883) * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * refactor data_quality_module * add header license * data quality module refactor * fix unit test error * fix checkstyle error * fix unit test error * fix checkstyle error * fix unit test error * fix code smell * fix check style * fix unit test error * task statistics value add unique code * fix unit test error * fix checkstyle error * fix checkstyle * fix security hotspot * fix unit test error * fix security hotspot * fix check * add data quality task error handling * fix unit test error * add unit test * add unit test * optimize data quality result alert * fix unit test * fix sql script error * fix bug * update sql script * fix checkstyle * add license * fix checkstyle * fix checkstyle * fix unit test * add jacoco dependencies * fix unit test * fix unit test * add jacoco dependencies * add unit test * add unit test * add license * fix checkstyle * fix pom * fix checkstyle * fix checkstyle * merge dev * fix ui error * fix pom error * fix pom error * fix test error * fix test error * mssql-jdbc exclude azure-keyvault * fix test error * merge dev and add unit test * add notes * rollback the CollectionUtils * fix * update sql * fix * fix * fix query rule page error * change dq.jar path * fix sql error * fix ui error * fix(dq): jar path&task enum description * add notes on DataQualityApplication * fix dq result jump error * fix(ui): page condition * feat(ui): add show error output path * change version * remove all chinese word in sql * merge Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>	2 years ago
天仇	90ea6ebb6f	[Improvement-7907][refactor] Optimization for query task instances list when build dag flow (#7915 ) Optimization for query task instances list when build dag flow This closes #7907	2 years ago
JinYong Li	24a97fda75	[Bug] [dolphinscheduler-api]support workflow instance to definition (#7930 ) * add task save and binds workflow * add task update with upstream * support workflow instance to definition * fix ut	2 years ago
wind	ba2b2a67c2	[Feature-6988][MasterServer] add cache manager for workflow (#7090 ) * add cache manager for workflow * [DS-6988][MasterServer] add cache manager for workflow * cache evict code optimization * test Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wangxj3	2b519f31e9	[Feature-#6422] task group queue (#6722 ) * add task group * modify task group * pull dev * add license header * modify code style * fix code style * fix sql error * fix error * fix test * fix test * fix test * fix test * fix code style * fix ut * code style * fix unit test * test ut * ut * add unittest * test ut * modify back ut * majorization code * fix conflict * fix ut Co-authored-by: wangxj <wangxj31>	3 years ago
wind	8e93939121	[DS-6963][MasterServer]add cache manager for tenant and user (#6974 ) * cache manager for tenant and user * add test * fix checkstyle * add dependencies * fix sonar check * use spring boot cache configuration * add license * check style * check style * add license * change proxy name to processor * add license header * check style * add license file * add license file * add license Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wind	595e4843d0	[DS-6849][MasterServer] fetch more commands and handle in parallel (#6850 ) * [DS-6849][MasterServer] fetch more commands and handle in parallel * add return * handle command with check * add test * delete master prefix Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
OS	880c1d28ff	[Feature][master] change sub process id to code. (#6846 ) * 6597 change sub process id to code. * 6597 change sub process id to code. * 6597 change sub process id to code. * 6597 change sub process id to code. * 6597 change sub process id to code. * 6597 change sub process id to code. * code style * fix ut process * fix ut process	3 years ago

1 2

84 Commits (e97f6a41edd361d719f22d4952c392873eee4b15)