fuchanghai
67f50e3904
[fix-#13940] add status of ``` stop``` in class of TaskExecutionStatus ( #14016 )
1 year ago
Aaron Wang
c3809fbecf
[Improvement-13868][K8S Task] Support the custom labels for k8s task ( #14067 )
...
* [Improvement-13868] Support the custom labels for k8s task
* fix front-end type mismatched
1 year ago
JieguangZhou
a3b2cc0275
[CI] Try fix api test error ( #14094 )
...
* try fix api test error
* add workergroupapitest to ci and fix log output
* fix log output
* skip remote shell UT
---------
Co-authored-by: SbloodyS <460888207@qq.com>
1 year ago
Rick Cheng
3eb4c0ed1d
[Improvement-14054][Worker] Kill multiple yarn apps at the same time ( #14055 )
...
* [Improvement-14054][Worker] Kill multiple yarn apps at the same time
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/YarnApplicationManager.java
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2 years ago
Rick Cheng
84edea7593
[Improvement-14052][Log] Remove the useless filter in getAppIdsFromLogFile ( #14053 )
2 years ago
Aaron Wang
3bad68a942
[Bug-13929][K8S Task] Set command contains single quote is invalid ( #13930 )
...
* [Fix-13929][K8S Task] Set command contains single quote is invalid
* fix UT
2 years ago
Wenjun Ruan
cf91544b99
Refactor logic task ( #13948 )
2 years ago
Jeremy
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin ( #13866 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2 years ago
fuchanghai
ec29220977
[improve-#13966] mask datax log ( #13969 )
...
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2 years ago
Zzih
34f5b009e1
[Fix 13941]Fix replace time variable ( #13975 )
...
* fix replace time variable
Co-authored-by: <ziheng.zhan@longbridge.sg>
2 years ago
Aaron Wang
45586f71d5
[BUG][Resource Center] Task can not select main JAR ( #13536 )
2 years ago
zzm
35bb533a50
[Improvement][Task Plugin] Integrate zeppelin password authentication with DS zeppelin task plugin ( #13861 )
...
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2 years ago
Aaron Wang
119f08d2ba
[Bug-13882][Worker] Upgrade kubernetes-client version to 6.0.0 ( #13891 )
2 years ago
Assert
4439c1ec90
[Feature-13052][Task Plugin] Support Apache Seatunnel Connector-V2 ( #13086 )
2 years ago
huangxinan1990
d772f2492b
fix: when sudo enable is false, execute command will failed ( #13860 )
2 years ago
Aaron Wang
d6fcf60534
[Improvement-13751][Worker] Support real-time pod log collection ( #13752 )
...
* Support real-time pod log collection
* datax task test have to mock ShellCommandExecutor with whole dependencies
* replace \n with EMPTY_STRING
* modify error log msg
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* throw exception in pod log collection thread
---------
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2 years ago
Wenjun Ruan
a1861bc2df
Use DATAX_PYTHON to specify a datax python version ( #13849 )
...
(cherry picked from commit 752452ecbd2f8094b819eae219f3eec0f6a6427b)
2 years ago
JieguangZhou
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin ( #13801 )
2 years ago
Wenjun Ruan
9df89cf0cf
Use bootstrap user as the default tenant ( #13825 )
2 years ago
Rick Cheng
85605bbef9
[Improvement-13653][Remote Logging] Support api-server to get task log from remote target ( #13654 )
2 years ago
Wenjun Ruan
d1b6e6f02c
Take over task instance in master failover ( #13798 )
2 years ago
Aaron Wang
b98566e070
[Bug-13782] match ')' in setValue regular expression ( #13787 )
2 years ago
Rick Cheng
2dceb6b8ca
[Feature-13428][Resource Center] Add support for Google Cloud Storage in the resource center ( #13562 )
2 years ago
Wenjun Ruan
aef5524ee7
Clearer task runnable. ( #13689 )
...
* Clean unused method in AbstractTask
* Kill task by process.destroy
* wait 5/s after destroy process
2 years ago
fuchanghai
6218f818c0
[improve-#13665] improve datax parser if sql contain * ( #13680 )
...
* [improve-#13665] improve datax parser if sql contain *
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxTask.java
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2 years ago
Wenjun Ruan
c9066e8de9
Use MDC to filter task instance log ( #13673 )
...
* Use MDC to collect task instance log
* Use MDCAutoClosableContext to remove the MDC key
2 years ago
Aaron Wang
fd720880c7
[Improvement] Add missing placeholder ( #13663 )
...
* Add missing placeholder
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/KubernetesApplicationManager.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
---------
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2 years ago
Wenjun Ruan
ca9df868f1
Fix task instance log order might be wrong due to async print ( #13660 )
2 years ago
旺阳
43d79e45b4
Fix data quality pwd error ( #13643 )
2 years ago
QuantumXiecao
7586710d49
[fix-13625] Modify systemd-run from MemoryMax to MemoryLimit for more generality ( #13627 )
...
Co-authored-by: xiecao <haolong.zhl@alibaba-inc.com>
2 years ago
Wenjun Ruan
4351a25f2a
Add execute function to handle the workflow instance operation ( #13610 )
2 years ago
fuchanghai
79d3ed0744
Update DataxUtils.java ( #13604 )
2 years ago
fuchanghai
7bf3e3cdd6
[improve-#13201] update pid during running ( #13206 )
2 years ago
Aaron Wang
047fa2f65e
[Feature-13511] Submit Spark task directly on Kubernetes ( #13550 )
2 years ago
Rick Cheng
2bd65fb2df
[Feature][Remote Logging] Add support for writing task logs to OSS ( #13332 )
2 years ago
fuchanghai
37939add0c
[fix-13492]add single quotes ( #13534 )
...
* [fix-13492]add single quotes
* [improve-#13201] fix test error
---------
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2 years ago
JieguangZhou
16b193454b
[Improvement][Task Plugin] Support Task Plugin output response as a parameter ( #13529 )
...
* support http output response as param
* add UT
* fix sqoop UT
2 years ago
seedscoder
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2 years ago
shenyun
8fa90c2d8f
feature flinksql support standalone deploy mode ( #13495 )
2 years ago
Wenjun Ruan
5a9cb7ee64
Fix task log might delay until task finished due to batch flush ( #13502 )
2 years ago
Tq
78ba71bdbd
[Feature][task]add azure datafactory task type ( #13417 )
...
* init data factory
* add task-type-config.yaml
* add data factory ui
* submit doc
* submit license
---------
Co-authored-by: devosend <devosend@gmail.com>
2 years ago
LucasClt
a1f5675012
[Bug][TaskPlugin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string ( #13162 )
...
* [Bug] [TaskPlugin-Zeppelin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string. #13161
2 years ago
caishunfeng
ba0a253f09
fix k8s task run fail ( #13348 )
2 years ago
Wenjun Ruan
6634ff98af
Add source env list in common.properties ( #13317 )
2 years ago
JieguangZhou
df491f78cc
fix sagemaker sync status ( #13264 )
2 years ago
Wenjun Ruan
14ec4a2398
Remove dao module in worker ( #13242 )
2 years ago
Wenjun Ruan
b2336b0ce9
Remove datasource in procedure task ( #13198 )
2 years ago
youzipi
042ec74a24
[refactor] cache compiled regex pattern ( #13208 )
...
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
2 years ago
HuangWei
43e6ac32dd
[Fix-13187] refactor OpenmldbParameters ( #13190 )
2 years ago
Wenjun Ruan
838a0df783
Remove logic task plugin module ( #13171 )
2 years ago