DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
huangxinan1990	d772f2492b	fix: when sudo enable is false, execute command will failed (#13860 )	2 years ago
Aaron Wang	d6fcf60534	[Improvement-13751][Worker] Support real-time pod log collection (#13752 ) * Support real-time pod log collection * datax task test have to mock ShellCommandExecutor with whole dependencies * replace \n with EMPTY_STRING * modify error log msg * Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java Co-authored-by: caishunfeng <caishunfeng2021@gmail.com> * throw exception in pod log collection thread --------- Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
Wenjun Ruan	9df89cf0cf	Use bootstrap user as the default tenant (#13825 )	2 years ago
Rick Cheng	85605bbef9	[Improvement-13653][Remote Logging] Support api-server to get task log from remote target (#13654 )	2 years ago
Wenjun Ruan	d1b6e6f02c	Take over task instance in master failover (#13798 )	2 years ago
Aaron Wang	b98566e070	[Bug-13782] match ')' in setValue regular expression (#13787 )	2 years ago
Rick Cheng	2dceb6b8ca	[Feature-13428][Resource Center] Add support for Google Cloud Storage in the resource center (#13562 )	2 years ago
Wenjun Ruan	aef5524ee7	Clearer task runnable. (#13689 ) * Clean unused method in AbstractTask * Kill task by process.destroy * wait 5/s after destroy process	2 years ago
Wenjun Ruan	c9066e8de9	Use MDC to filter task instance log (#13673 ) * Use MDC to collect task instance log * Use MDCAutoClosableContext to remove the MDC key	2 years ago
Aaron Wang	fd720880c7	[Improvement] Add missing placeholder (#13663 ) * Add missing placeholder * Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/KubernetesApplicationManager.java Co-authored-by: kezhenxu94 <kezhenxu94@apache.org> --------- Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>	2 years ago
Wenjun Ruan	ca9df868f1	Fix task instance log order might be wrong due to async print (#13660 )	2 years ago
QuantumXiecao	7586710d49	[fix-13625] Modify systemd-run from MemoryMax to MemoryLimit for more generality (#13627 ) Co-authored-by: xiecao <haolong.zhl@alibaba-inc.com>	2 years ago
Wenjun Ruan	4351a25f2a	Add execute function to handle the workflow instance operation (#13610 )	2 years ago
fuchanghai	7bf3e3cdd6	[improve-#13201] update pid during running (#13206 )	2 years ago
Aaron Wang	047fa2f65e	[Feature-13511] Submit Spark task directly on Kubernetes (#13550 )	2 years ago
Rick Cheng	2bd65fb2df	[Feature][Remote Logging] Add support for writing task logs to OSS (#13332 )	2 years ago
JieguangZhou	16b193454b	[Improvement][Task Plugin] Support Task Plugin output response as a parameter (#13529 ) * support http output response as param * add UT * fix sqoop UT	2 years ago
seedscoder	8d12dc0702	[Improvement-13491] Use lombok @Slf4j annotation to generate logger (#13509 )	2 years ago
Wenjun Ruan	5a9cb7ee64	Fix task log might delay until task finished due to batch flush (#13502 )	2 years ago
Tq	78ba71bdbd	[Feature][task]add azure datafactory task type (#13417 ) * init data factory * add task-type-config.yaml * add data factory ui * submit doc * submit license --------- Co-authored-by: devosend <devosend@gmail.com>	2 years ago
caishunfeng	ba0a253f09	fix k8s task run fail (#13348 )	2 years ago
Wenjun Ruan	6634ff98af	Add source env list in common.properties (#13317 )	2 years ago
Wenjun Ruan	14ec4a2398	Remove dao module in worker (#13242 )	2 years ago
youzipi	042ec74a24	[refactor] cache compiled regex pattern (#13208 ) * [refactor] cache compiled regex pattern * [refactor] cache compiled regex pattern * [refactor] cache compiled regex pattern	2 years ago
Wenjun Ruan	70ccffeee2	Format task parameter as pretty json (#13173 )	2 years ago
Aaron Wang	560d232cbd	[Bug][Worker] Fix duplicated key bug when have out parameters of same name (#12930 )	2 years ago
Wenjun Ruan	32a8241ef2	Don't source env file in task, this will override tenant's environment (#13028 )	2 years ago
Kerwin	d480c271ec	[Bug-12994] [Worker] Fix kill process does not take effect (#12995 )	2 years ago
Wenjun Ruan	92a9b7e243	Use System#lineSeparator to replace /n (#12889 )	2 years ago
JieguangZhou	229c554912	[feature][task] Add Kubeflow task plugin for MLOps scenario (#12843 )	2 years ago
simsicon	43d69ae157	Upgrade clickhouse jdbc driver (#12639 )	2 years ago
Kerwin	c4fb555883	Modify shell env to bash (#12742 )	2 years ago
Wenjun Ruan	eaef68856b	Change command file permission to 755 (#12678 ) (cherry picked from commit 95484d930762712f4ab68ece27ca3b65a2073ab2)	2 years ago
JieguangZhou	a47b4d0672	[Feature][Task] Transfer files between tasks (#12552 ) * add data transfer between tasks * add delete DATA_TRANSFER API * convert Result to DeleteDataTransferResponse * add api UT * fix final * fix doc	2 years ago
amao	b71d4983dd	[Feature-6586][Server]add some ds process definition demo when init (#11759 ) * [Feature-6586][Server]add some ds process definition demo when init 1.add some ds process definition demo when init, to display what task type can run and make user easy to use ds. 2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service 3.need modify the tenant information in it	2 years ago
rickchengx	3c31ddfd9f	[Improvement-12536][k8s] Support the command for the container in k8s task plugin (#12538 )	2 years ago
Aaron Wang	08335b1032	[Improvement][Task] Improved way to collect yarn job's appIds (#12197 ) * Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code. * Add user property `appId.collect` for user to decide how to collect applicationId. * Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh` * Update docs to declare how to use aop way. * Update `LogUtils` to support fetch applicationId in different ways based on the user property. Co-authored-by: gabrywu <gabrywu@apache.com>	2 years ago
caishunfeng	dedff70f90	[Improvement] Merge spi.utils into common.utils (#12502 ) * improve JSONUtil * merge spi.utils into common.utils * split constants && remove some copy utils method * remove StringUtils	2 years ago
fuchanghai	f0a34cddec	[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE (#12449 )	2 years ago
Eric Gao	d46e4659d9	[Improvement][Style] Fix remaining style errors and wildcard imports (#12412 ) * Fix remaining style errors and wildcard imports * Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link * Fix spotless matching scope of markdown files	2 years ago
insist777	1e11034049	[Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly (#12253 ) * [Fix][Task] dependent task can not predicate the status of the corresponding task correctly	2 years ago
simsicon	ba538067f2	[fix] Change the default unix shell executor from sh to bash (#12180 ) Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>	2 years ago
Kerwin	55004bebe0	[Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function (#12392 ) * Workflow definitions that contain logical task nodes support the copy function.	2 years ago
Yann Ann	7aa8a77fb2	[Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @Ignore to @Disable	2 years ago
rickchengx	7b44612f28	[Improvement-12372][k8s] Update the deprecated k8s api (#12373 )	2 years ago
kezhenxu94	db1d319159	Reorganize some classes in common module, remove duplicate classes (#12321 )	2 years ago
Eric Gao	875682d267	[Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example (#12299 ) * Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example	2 years ago
Yann Ann	c87f6d89a4	[fix#12195] Change date to timestamp to support cross diff time zones (#12239 ) * fix DS support cross time zone, use timestamp replace date	2 years ago
冯剑	779a0b9c75	[Improvement-#11608][task-plugin] New time variables are added to facilitate business development (#11667 ) Add new time expressions for users' convenience Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com> Co-authored-by: David <dailidong66@gmail.com>	2 years ago
zhuxt2015	e6832220c3	[Improvement][SQL Task]use default sql segment separator (#10869 )	2 years ago

1 2 3

137 Commits (4439c1ec9096bd4ae3a7e4ad7edb939992a4f7ce)