DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
JieguangZhou	66e20271ad	[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks (#13194 ) * Supports task instance cache operation * add task plugin cache * use SHA-256 to generate key * Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql Co-authored-by: Jay Chung <zhongjiajie955@gmail.com> * Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql Co-authored-by: Jay Chung <zhongjiajie955@gmail.com> * Optimizing database Scripts * Optimize clear cache operation Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>	2 years ago
sssqhai	7a0a2c2a46	Solve the deadlock problem caused by queuing (#13191 ) * Solve the deadlock problem caused by queuing * Solve the deadlock problem caused by queuing * Solve the deadlock problem caused by queuing * Solve the deadlock problem caused by queuing,move the event to the tail by throwing a exception Co-authored-by: wfs <wangfushun@cdqcp.cpm>	2 years ago
Wenjun Ruan	be81b222d4	Optimize event loop (#13193 )	2 years ago
JieguangZhou	e4b9b67255	Allow execute task in workflow instance (#13103 )	2 years ago
Wenjun Ruan	70ccffeee2	Format task parameter as pretty json (#13173 )	2 years ago
Yann Ann	6ef74073cc	[Improve-13001]migrate commons-collections -> commons-collections4 (#13002 )	2 years ago
JieguangZhou	a7ecc5a8b3	fix retry task failure (#13077 )	2 years ago
Wenjun Ruan	169168ef34	Add plugin-all module (#13079 )	2 years ago
Kevin.Shin	12a6138d33	fix issue 13035 (#13065 ) Co-authored-by: shenk-b <shenk-b@glodon.com>	2 years ago
Wenjun Ruan	ffc9fb280a	Add gc timestampt (#13059 )	2 years ago
Wenjun Ruan	1a8811cb41	Set max loop times when consume StateEvent to avoid dead loop influence the thread. (#13007 )	2 years ago
Yann Ann	3106054ea7	[Improvement-12907] Change heartbeat log level to debug (#12980 )	2 years ago
Kerwin	50779ea1e6	[Bug-12963] [Master] Fix dependent task node null pointer exception (#12965 ) * Fix that there are both manual and scheduled workflow instances in dependent nodes, and one of them will report a null pointer exception during execution.	2 years ago
rickchengx	38b876733c	[Feature-10498] Mask the password in the log of sqoop task (#11589 )	2 years ago
fuchanghai	3747029cc0	[fix-#12932] when subprocess's processInstance is fail,not notify parent processInstance (#12933 )	2 years ago
John Bampton	27c37b8828	Fix grammar and spelling (#12937 )	2 years ago
Kerwin	c916c60853	fix NPE while retry task (#12903 )	2 years ago
Wenjun Ruan	d99ba29b66	Fix master cluster may loop command unbalanced (#12891 ) (cherry picked from commit 3b2b86661be76b7c1404a910c865d78b7936313d)	2 years ago
JieguangZhou	229c554912	[feature][task] Add Kubeflow task plugin for MLOps scenario (#12843 )	2 years ago
ZhenjiLiu	7cdb926a5f	[Improvement][Batch Query] Batch query ProcessDefinitions belongs to need failover ProcessInstance. (#12506 )	2 years ago
Wenjun Ruan	9e0c9af1a5	Fix the waiting strategy cannot recovery if the serverstate is already in running (#12651 )	2 years ago
Aaron Wang	08335b1032	[Improvement][Task] Improved way to collect yarn job's appIds (#12197 ) * Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code. * Add user property `appId.collect` for user to decide how to collect applicationId. * Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh` * Update docs to declare how to use aop way. * Update `LogUtils` to support fetch applicationId in different ways based on the user property. Co-authored-by: gabrywu <gabrywu@apache.com>	2 years ago
Wenjun Ruan	e6da1ccf81	Add worker-group-refresh-interval in master config (#12601 ) * Add worker-group-refresh-interval in master config * Set interval cannot smaller than 10s * Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/config/MasterConfig.java Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>	2 years ago
kezhenxu94	065d5caccc	Only expose necessary actuator endpoints (#12571 )	2 years ago
HanayoZz	489e7fe4e2	[Feature-10495][Resource Center] Resource Center Refactor (#12076 ) * resource center refactor - S3 services connection Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
Yann Ann	04aa125ba2	[Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. (#12505 ) * migrate some task DAO to dao modules. * Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceMapDaoImpl.java Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
liqiquan	4aa9d60683	[Bug][Master] Add the aws-java-sdk-s3 jar package to the master module (#12259 ) (#12512 ) Co-authored-by: qiquan.li <qiquan.li@linkflowtech.com>	2 years ago
caishunfeng	dedff70f90	[Improvement] Merge spi.utils into common.utils (#12502 ) * improve JSONUtil * merge spi.utils into common.utils * split constants && remove some copy utils method * remove StringUtils	2 years ago
Yann Ann	b936b882bb	[Refactor] Migrate all command-related interface functions from ProcessServiceImpl (#12474 ) * migrate all command-related interface functions to CommonService	2 years ago
Eric Gao	d46e4659d9	[Improvement][Style] Fix remaining style errors and wildcard imports (#12412 ) * Fix remaining style errors and wildcard imports * Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link * Fix spotless matching scope of markdown files	2 years ago
Yann Ann	38b643f69b	make sure all failed task will save in errorTaskMap (#12424 )	2 years ago
kezhenxu94	4ea7eec750	Use temurin Java docker images instead of deprecated ones (#12334 )	2 years ago
Yann Ann	bc80f34056	[Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in master, worker, and remote modules (#12362 ) * migrate junit4 -> junit5 in master, worker, and remote modules	2 years ago
Eric Gao	2f37da0dbc	[Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom (#12311 ) * move mockito dependencies from sub-modules to root pom * Add check in CI to block the usage of powermock	2 years ago
kezhenxu94	db1d319159	Reorganize some classes in common module, remove duplicate classes (#12321 )	2 years ago
rickchengx	05099bd8f7	[Improvement-12296][Test] Close static mock (#12300 )	2 years ago
sgw	165b9a58de	[DS-12131][master] Optimize the log printing of the master module acc… (#12152 ) * [DS-12131][master] Optimize the log printing of the master module according to the log specification.	2 years ago
Yann Ann	c87f6d89a4	[fix#12195] Change date to timestamp to support cross diff time zones (#12239 ) * fix DS support cross time zone, use timestamp replace date	2 years ago
xuhhui	55388be21f	[Improvement][Master] LowerWeightRoundRobin doSelect may be null (#12159 )	2 years ago
caishunfeng	40090d459e	[Improvement] remove log-server and server module (#12206 ) * remove log-server module * remove dolphinscheduler-server module	2 years ago
Wenjun Ruan	aacabc6ec7	Remove workerGroup in registry (#12217 )	2 years ago
fuchanghai	e27c79974d	[fix-#11753] send alert error alert data id (#11774 ) * [fix-#11753] send alert error alert data id Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>	2 years ago
Wenjun Ruan	d526dd89f6	[hotfix][UI] Remove SURVEIL and NEXT_LOOP in ui (#12160 ) * Remove SURVEIL and NEXT_LOOP in ui * Fix UT	2 years ago
Eric Gao	cc82206c33	Remove powermock in dolphinscheduler-master module (#12143 ) * Remove powermock related code in dolphinscheduler-master module * Remove redundant comments, useless code and unnecessary stubbings	2 years ago
Tq	cf5a8894e2	[BUG][BE-12113]fix htrace dependency conflicts (#12126 ) * fix htrace dependency conflicts * replace htrace-core with hbase-noop-htrace	2 years ago
Yann Ann	ada7cf71d5	[fix#12000]Cannot remove the WorkerGroup from the master service (#12050 ) * [Bug] [Master] Cannot remove the WorkerGroup from the master service. #12000 * remove unnecessary locks * Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java Co-authored-by: caishunfeng <caishunfeng2021@gmail.com> Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
insist777	5b384f3fab	[Feature-11473][Task]Support test task (#11670 ) * [improve] support test tasks * [improve] support test tasks * [improve] support test tasks * [improve] support test tasks * [improve] support test tasks * Update TaskExecuteThread.java * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * try solve e2e q * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * Update DataSource.java * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * Update messages_zh_CN.properties * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * Update messages.properties * Update messages_en_US.properties * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks * [Feature] support test tasks	2 years ago
Yann Ann	e938fdbe96	[Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 (#11864 ) * [Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838	2 years ago
caishunfeng	6868876a29	fix workflow keep running when task fail (#11930 )	2 years ago
Wenjun Ruan	d3a77c68e6	Make LogServiceClient Singleton (#11777 )	2 years ago

1 2 3 4 5

211 Commits (b2336b0ce9e3ebbe57a92969c57a613964e5ccf2)