DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
wind	85b3b2d29d	[DS-7686][Server]fix restart server after kill force (#7689 ) * [DS-7686][Server]fix restart server after kill force * update registry logic Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
zwZjut	32fb3f84df	[Bug] [dolphinscheduler-server] memory leak of logger #7661 (#7665 ) * [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238) * to #7237 * rerun test Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `05aef27` and handle conflicts * to #7065: fix ExecutorService and schedulerService (#7072) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082) * to #7081 * fix #7081 * to #7081 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `8ebe060` and handle conflicts * cherry-pick `1f18444` and handle conflicts * fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com> * add default constructor (#6780) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * to #7108 (#7109) * add conf && pick #7562 to worker * to #7661 * to #7661 * to #7661:fix ut Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com>	3 years ago
zwZjut	df40cdc116	[Bug] [deploy on k8s] add default config of quartz and zookeeper in common configmap && pick #7562 fix to worker #7639 (#7641 ) * [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238) * to #7237 * rerun test Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `05aef27` and handle conflicts * to #7065: fix ExecutorService and schedulerService (#7072) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082) * to #7081 * fix #7081 * to #7081 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `8ebe060` and handle conflicts * cherry-pick `1f18444` and handle conflicts * fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com> * add default constructor (#6780) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * to #7108 (#7109) * add conf && pick #7562 to worker Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com>	3 years ago
wind	744577f252	remove taskInstancnCacheManager (#7636 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Jiajie Zhong	9e068d23e2	[cherry-pick][python] Make dolphinscheduler python API works to 2.0.2 (#7608 ) * [cherry-pick][python] Make it work to 2.0.2 * Remove unused ProcessExecutionTypeEnum * Add queryByName to project * Add checkTenantExists to tenant * Add queryByTenantCode to tenant * Add queryQueueName to queue * Add all content from dev branch * Add gitignore * Add pydolphinscheduler content * Add ds-py to bin test * Py merge to 202 * Fix version * Fix missing variable * Add py4j as known deps * Fix core database bug	3 years ago
wind	8c5f364796	[cherry-pick-2.0.2][BUG-7554] Fix database lose data in standalone (#7621 ) * [cherry-pick][BUG-7554] Fix database lose data in standalone * Fix thread starts too early Co-authored-by: caishunfeng <534328519@qq.com> Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>	3 years ago
zwZjut	6c819ee16d	[Bug] [dolphinscheduler-server] workflow is always running when worker sendResult success but master not received (#7610 ) * [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238) * to #7237 * rerun test Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `05aef27` and handle conflicts * to #7065: fix ExecutorService and schedulerService (#7072) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082) * to #7081 * fix #7081 * to #7081 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `8ebe060` and handle conflicts * cherry-pick `1f18444` and handle conflicts * fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com> * add default constructor (#6780) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * to #7108 (#7109) * to #7609 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com>	3 years ago
zwZjut	f8942bf798	[Improvement][MasterServer] event response handle parallel (#7560 ) * [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238) * to #7237 * rerun test Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `05aef27` and handle conflicts * to #7065: fix ExecutorService and schedulerService (#7072) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082) * to #7081 * fix #7081 * to #7081 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `8ebe060` and handle conflicts * cherry-pick `1f18444` and handle conflicts * fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com> * add default constructor (#6780) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * to #7108 (#7109) * to #7450 * to #7450: fix parallel bug * add index * expose config to user * fix bug * fix bug * add delay delete * fix bug * add License * fix ut * fix ut * fix name Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com>	3 years ago
BaoLiang	8fa61cf050	[FIX-7287] change description of 'master.exec.threads' (#7588 ) * [2.0.2] #7287 change description of 'master.exec.threads' * [2.0.2] #7287 change description of 'master.exec.threads'	3 years ago
xiangzihao	5c297a594c	fix 7534 (#7567 ) Co-authored-by: SbloodyS <sbloodys@qq.com>	3 years ago
SbloodyS	ab1467e622	cherry-pick bug_7537 (#7555 )	3 years ago
zwZjut	c956185e00	[Feature] 2.0.2-prepare bug fix of Pressure tests #7511 (#7540 ) * [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238) * to #7237 * rerun test Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `05aef27` and handle conflicts * to #7065: fix ExecutorService and schedulerService (#7072) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082) * to #7081 * fix #7081 * to #7081 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `8ebe060` and handle conflicts * cherry-pick `1f18444` and handle conflicts * fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com> * add default constructor (#6780) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * to #7108 (#7109) * to #7511 * to #7511 * to #7511 * to #7511 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com>	3 years ago
SbloodyS	0b6f3fb0c3	[Cherry-pick-2.0.2[Feature-7460][]Feature-7451][Fix-7392] (#7483 ) * remove date+1 when the data is supplemented (#7452) * resolve conflict * [Fix-7392][dolphinscheduler-datasource] Add hive datasource failed (#7393) * fix bug_7392 * fix bug_7392 * fix bug_7392 Co-authored-by: SbloodyS <sbloodys@qq.com> * cherry-pick-2.0.2 Co-authored-by: SbloodyS <sbloodys@qq.com>	3 years ago
zwZjut	255670fda9	[Feature][dev-2.0.2-prepare] cherry-pick pull requests from dev to 2.0.2-prepare #7357 (#7358 ) * [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238) * to #7237 * rerun test Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `05aef27` and handle conflicts * to #7065: fix ExecutorService and schedulerService (#7072) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082) * to #7081 * fix #7081 * to #7081 Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * chery-pick `8ebe060` and handle conflicts * cherry-pick `1f18444` and handle conflicts * fix #6807: dolphinscheduler.zookeeper.env_vars - > dolphinscheduler.registry.env_vars (#6808) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com> * add default constructor (#6780) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * to #7108 (#7109) Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: Kirs <acm_master@163.com>	3 years ago
lenboo	54a89e9e24	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	ec0e57c53d	[maven-release-plugin] prepare release 2.0.1	3 years ago
BaoLiang	921f388077	[FIX-7338] set h2 for default DATABASE_TYPE for standalone-server (#7339 ) * [Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139) 1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。 2、modify index start_time_index in init script and upgrade scripts. * [BUG][SQL] fix the process list page load slowly. (#7166) * fix #7601 move ddl to 2.0.1 * fix #7601 move ddl to 2.0.1 * [2.0.1] remove unused foreign_key * [2.0.1] add version in sql * [2.0.1] change serial for postgre * [2.0.1] set h2 for default DATABASE_TYPE for standalone-server Co-authored-by: GaoTianDuo <gaotianduo_yewu@cmss.chinamobile.com>	3 years ago
lenboo	14888518db	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	bef4fa8321	[maven-release-plugin] prepare release 2.0.1	3 years ago
wind	79ab27299e	[cherry-pick-2.0.1][Bug-7319][MasterServer] fix NPE when switch else branch is empty (#7314 ) * fix taskNode NPE * fix switch node NPE when no else branch Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wind	5dad635ff0	[cherry-pick-2.0.1][Bug-7307][MasterServer] timeout event was filtered (#7303 ) * remove task filter when task timeout * get lastest task instance info Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
lenboo	87e091076a	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	8b239a0728	[maven-release-plugin] prepare release 2.0.1	3 years ago
wind	41c05b53c7	[cherry-pick-2.0.1][Bug][MasterServer] fix timeout event (#7283 ) * fix timeout alert * fix timeout Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
lenboo	9a49077fdf	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	29a4323540	[maven-release-plugin] prepare release 2.0.1	3 years ago
wind	9a307dc71a	fix failover logic (#7276 )	3 years ago
lenboo	a97bc2c756	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	c011f6487a	[maven-release-plugin] prepare release 2.0.1	3 years ago
kezhenxu94	a8edeac1e3	Fix env vars are not passed in from docker-compose (#7240 )	3 years ago
wind	30848478bf	fix timeout (#7224 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wind	c2d0acd713	fix timeout (#7222 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Kerwin	b4c422e977	fix waterdrop task plugin (#7216 )	3 years ago
kezhenxu94	e9d27015c3	Add missing property for API and worker (#7198 )	3 years ago
Jiajie Zhong	1e0757bd8e	Change default database from mysql to pg (#7190 )	3 years ago
wind	0b376f7f90	[2.0.1-cherry-pick] fix bug : Tez state, complement data kill, dependent node (#7188 ) * [Fix-7037][dolphincheduler-server] process instance can not be killed when complement data (#7140) * fix bug 7037 * update processComplementData Co-authored-by: SbloodyS <sbloodys@qq.com> * [Fix-5551][common] Fix ExecutionStatus for hive-on-Tez application (#6289) * add ENDED state * Update HadoopUtils.java * [Fix-7093][dolphincheduler-server] fix dependent node stuck in running (#7116) * fix bug 7093 * remove redundant else Co-authored-by: SbloodyS <sbloodys@qq.com> Co-authored-by: SbloodyS <460888207@qq.com> Co-authored-by: SbloodyS <sbloodys@qq.com> Co-authored-by: Manhua <kevinjmh@qq.com>	3 years ago
kezhenxu94	449c098aef	Fix quartz not work, worker / alert server should not listen or 8080 port (#7159 )	3 years ago
Jiajie Zhong	423d7ea3a1	[deploy] Add export database type for before daemon file (#7151 ) * [deploy] Use install_config.conf export datasource properties Directly user install_config.conf to set spring datasource properties, delete datasource.properties file for more clearly * Remove unused properties	3 years ago
JinYong Li	e96a39f57c	[2.0.1-cherrypick][Feature][API] API operation optimization (#7136 ) * pick-6863 * [Improvement][API][num-7] add task delete check (#6887) * upgrade add sub_process * add interface method * add interface method * fix ut * add task delete check * add task delete check * [Improvement][API][num-5] The workflow of the main table version is not allowed to be deleted (#6893) * feature:deleteProcessDefinitionVersion * pref:delete duplicate enum definition Co-authored-by: 北笙 <“zhanqian@cai-inc.com”> * [DS-6931][fix] swagger failed to start due to improper use of request param annotation (#6933) Co-authored-by: eye <eye.gu@aloudata.com> * [fix #6878] implement interface queryUpstreamRelation queryDownstreamRelation and add UT (#6888) * fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT * fix: #6878, implement interface queryUpstreamRelation queryDownstreamRelation and add UT * fix: #6878, add license * fix: #6878, code style * fix: #6878, code style * fix: #6878, fix Duplication * fix: #6878 * fix: #6878 , fix style * to #6878: fix style * to #6878: fix style * to #6878: add buildTaskDefinition to overwrite equals and hashCode Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> * [Improvement][API][num-6] TaskDefinition releaseTaskDefinition (#6965) * finished releaseTaskDefinition * add releaseTaskDefinition unit test * change import * add import * bug fix * change import Co-authored-by: 时光 <caizhedong@cai-inc.com> * pick-6966 * update task_definition_log field typ (#6967) * update task_definition_log field typ update task_definition_log field task_params type,modify text to longtext to be consistent with the main table * update t_ds_task_definition_log field task_params update t_ds_task_definition_log field task_params * pick-6970 * pick-6977 * pick-6984 * add processDefinition releaseWorkflowAndSchedule (#7044) * [Improvement][API][num-1] save schedule when workflow is offline in interface 'createEmptyProcessDefinition' (#7047) * [Improvement] save schedule when workflow is offline * [Improvement] rollback transactional when creating/updating schedule failed. * [Improvement] resolve merge conflict * [Improvement] do not set projectName and processDefinitionName when saving schedule * [Improvement] do not set projectName and processDefinitionName when saving schedule Co-authored-by: edward-yang <yangjianh210@gmail.com> * pick-7053 * pick-7098 * pick add 6772 relation code * fix ut Co-authored-by: Jeff Zhan <360400325@qq.com> Co-authored-by: 北笙 <“zhanqian@cai-inc.com”> Co-authored-by: ououtt <734164350@qq.com> Co-authored-by: eye <eye.gu@aloudata.com> Co-authored-by: zwZjut <zw15068835817@163.com> Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com> Co-authored-by: 时光 <41109695+yikeshiguang@users.noreply.github.com> Co-authored-by: 时光 <caizhedong@cai-inc.com> Co-authored-by: zhang <60912722+boy-xiaozhang@users.noreply.github.com> Co-authored-by: EdwardYang <yangjianh210@126.com> Co-authored-by: edward-yang <yangjianh210@gmail.com>	3 years ago
wind	36fd0c83f5	[Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (#7046 ) (#7059 ) Co-authored-by: Hua Jiang <jianghuachinacom@163.com>	3 years ago
Kirs	2bf3b22d64	[2.0.1-cherrypick] SPI(Task ,Alert ,DataSource) (#7041 ) * data source spi * Fix datasource not work in tests and standalone server, and run all tests by default (#6743) * Refactor alert plugins and simplify its usage (#6723) * [Fix-6792] resolve API and alert port conflicts (#6792) (#6815) * [Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792) * [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490) (#6828) * [Feature][datasource] Change DataSource Connection Pool from Druid to HiKariCP (#6490) * fix pom dep scope * filter alert * filter alert * filter alert * fix checkstyle * remove hikaricp version * update pom * add properties * add properties * rename properties * fix e2e fail * rename properties * [Fix-6792] resolve API and alert port conflicts (#6792) (#6815) [Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792) * fix ut fail * fix license * fix license * update package * update package * remove unused license Co-authored-by: mask <39329477+narcasserun@users.noreply.github.com> Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>	3 years ago
wind	19cb1237b3	[cherry-pick][DS-7004][MasterServer]fix master still work when it lose zk connection (#7058 )	3 years ago
Kirs	6821c7a9ac	[2.0.1-prepare#6712]Refactor registry plugin and simplify its usage (#6999 ) * [2.0.1-prepare#6712]Refactor registry plugin and simplify its usage (#6712) * upgrade version and fix dep err * Clean up the utility codes (#6732) * remove code style check * upgrade docker version Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>	3 years ago
lenboo	8a27ae4593	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	b31338b950	[maven-release-plugin] prepare release 2.0.0	3 years ago
lenboo	b7d2b11d56	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	549824a036	[maven-release-plugin] prepare release 2.0.0	3 years ago
lenboo	61e86b242b	[maven-release-plugin] prepare for next development iteration	3 years ago
lenboo	80c4aaaea5	[maven-release-plugin] prepare release 2.0.0	3 years ago
lenboo	8585b48402	[maven-release-plugin] prepare for next development iteration	3 years ago

1 2 3 4 5 ...

763 Commits (720e0ad36059fef4581a4ba228c3a0f56f09e437)