9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
1 year ago
d6a4e99e19
fix process utils comment typo ( #14613 )
1 year ago
04800a48d1
[Feature-13763][K8S Task] collect real-time log ( #14379 )
...
* [Improvement-13763][K8S Task] collect real-time log
* fix codesmell
* get pod watcher until pod is ready
* fix codesmell
* specify container name & loop waiting pod creation
* sleep when pod is not initialized
---------
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
1 year ago
c30cca9d9a
Support execute shell in different interceptor ( #14582 )
1 year ago
e6d94632b9
Add WorkflowExecuteContext ( #14544 )
1 year ago
09afa75a3d
[Improvement][Worker] No need to kill remote app when appid is empty ( #14534 )
1 year ago
c1cffba349
[Improvement] Remove the unused newDaemonSingleThreadExecutor ( #14520 )
1 year ago
175d976cd2
[Feature][Data Quality] Data Quality Support Choose Database ( #14406 )
...
* add dataquality database api
* change ui
* api change
* update
* fix spotless
* fix h2
* fix pg
* fix-dead-line
* update
* fix-spotless
* update pg sql
* add ut
* fix ut
1 year ago
0b69236b04
[Feature-14251][Task] Support yarn queue definition in yarn task ( #14310 )
1 year ago
742a2dc780
[Fix-14447][K8S Task] Fix IndexOutofBoundsException for pod name without dash ( #14448 )
1 year ago
cff53feaeb
[Improvement-14464][Task] Remove the useless taskResultString ( #14465 )
1 year ago
65b6a4b097
[Feature-14421][K8S Task] Configurable image pull policy ( #14426 )
1 year ago
b49b91c45d
[Improvement-14331][common] Remove the duplicate ParameterUtils ( #14332 )
1 year ago
a55612e4f7
Fix switch task use boolean type error ( #14326 )
1 year ago
1cd5070b5d
[Improvement-14329][common] Remove the duplicate BusinessTimeUtils ( #14330 )
1 year ago
4fa0ab4d7e
[feature][Resource] Add support for Azure Blob Storage in the resource center ( #14082 )
1 year ago
66e09dc66e
[Improvement-14295][common] Remove the duplicate FileUtils ( #14301 )
1 year ago
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters ( #14127 )
...
* add dynamic task plugin
* fix ui
* update code
* add DOC
* ADD UT
* fix doc
* fix sanity check
* add upgrade ddl
* fix sql
1 year ago
2c057d6a8f
[Improvement-14292][Common] Remove the duplicate OSUtils ( #14293 )
1 year ago
4d798e774e
[Improvement-14280][Task] update the task execution status check ( #14281 )
1 year ago
d587b5f03d
[Bug][Master][Fixes 14216]workflow state keep READY_STOP when sub_process state … ( #14221 )
...
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes #14216
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes mvn spotless:apply#14216
* Update testFile.txt.crc
* Update testFile.txt.crc
* Delete testFile.txt.crc
1 year ago
9e0991145e
[Improvement-14146][K8S Task] print complete error logs in task log file ( #14147 )
1 year ago
4e78248238
[Feature][K8S Task] support node selector ( #14126 )
...
* feat: support K8S node selector
* fix CI
1 year ago
af7da0630e
[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables ( #14099 )
...
* [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables
1 year ago
606b56403e
[Fix-13381][plugin] fix error while parsing hive load sql ( #13378 )
...
* [fix][task plugin][task sql] fix parsing error while using variables sometimes parsing exception happened if task sql contains a variable followed by \" or \' .
for example:
input sql: load inpath '/tmp/test_table/dt=${dt}'
into table test_table partition(dt=${dt})
after replace: preparing : load inpath '/tmp/test_table/dt=?
into table test_table partition(dt=?)
1 year ago
8315279620
[Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes ( #14150 )
...
* Added a dependent task failure policy in the dependent task node
1 year ago
67f50e3904
[fix-#13940] add status of ``` stop``` in class of TaskExecutionStatus ( #14016 )
1 year ago
c3809fbecf
[Improvement-13868][K8S Task] Support the custom labels for k8s task ( #14067 )
...
* [Improvement-13868] Support the custom labels for k8s task
* fix front-end type mismatched
1 year ago
3eb4c0ed1d
[Improvement-14054][Worker] Kill multiple yarn apps at the same time ( #14055 )
...
* [Improvement-14054][Worker] Kill multiple yarn apps at the same time
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/YarnApplicationManager.java
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2 years ago
84edea7593
[Improvement-14052][Log] Remove the useless filter in getAppIdsFromLogFile ( #14053 )
2 years ago
3bad68a942
[Bug-13929][K8S Task] Set command contains single quote is invalid ( #13930 )
...
* [Fix-13929][K8S Task] Set command contains single quote is invalid
* fix UT
2 years ago
cf91544b99
Refactor logic task ( #13948 )
2 years ago
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin ( #13866 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2 years ago
45586f71d5
[BUG][Resource Center] Task can not select main JAR ( #13536 )
2 years ago
119f08d2ba
[Bug-13882][Worker] Upgrade kubernetes-client version to 6.0.0 ( #13891 )
2 years ago
d772f2492b
fix: when sudo enable is false, execute command will failed ( #13860 )
2 years ago
d6fcf60534
[Improvement-13751][Worker] Support real-time pod log collection ( #13752 )
...
* Support real-time pod log collection
* datax task test have to mock ShellCommandExecutor with whole dependencies
* replace \n with EMPTY_STRING
* modify error log msg
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* throw exception in pod log collection thread
---------
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2 years ago
9df89cf0cf
Use bootstrap user as the default tenant ( #13825 )
2 years ago
85605bbef9
[Improvement-13653][Remote Logging] Support api-server to get task log from remote target ( #13654 )
2 years ago
d1b6e6f02c
Take over task instance in master failover ( #13798 )
2 years ago
b98566e070
[Bug-13782] match ')' in setValue regular expression ( #13787 )
2 years ago
2dceb6b8ca
[Feature-13428][Resource Center] Add support for Google Cloud Storage in the resource center ( #13562 )
2 years ago
aef5524ee7
Clearer task runnable. ( #13689 )
...
* Clean unused method in AbstractTask
* Kill task by process.destroy
* wait 5/s after destroy process
2 years ago
c9066e8de9
Use MDC to filter task instance log ( #13673 )
...
* Use MDC to collect task instance log
* Use MDCAutoClosableContext to remove the MDC key
2 years ago
fd720880c7
[Improvement] Add missing placeholder ( #13663 )
...
* Add missing placeholder
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/KubernetesApplicationManager.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
---------
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2 years ago
ca9df868f1
Fix task instance log order might be wrong due to async print ( #13660 )
2 years ago
7586710d49
[fix-13625] Modify systemd-run from MemoryMax to MemoryLimit for more generality ( #13627 )
...
Co-authored-by: xiecao <haolong.zhl@alibaba-inc.com>
2 years ago
4351a25f2a
Add execute function to handle the workflow instance operation ( #13610 )
2 years ago
7bf3e3cdd6
[improve-#13201] update pid during running ( #13206 )
2 years ago
047fa2f65e
[Feature-13511] Submit Spark task directly on Kubernetes ( #13550 )
2 years ago
Wenjun Ruan
Eric Gao
Aaron Wang
Rick Cheng
旺阳
fuchanghai
JieguangZhou
HQFGit
haibingtown
Alex Ting
Kerwin
Rick Cheng
Jeremy
huangxinan1990
QuantumXiecao