zhangbaipeng
7640c29495
[Improvement] [Task] Change Seatunnel task run-mode to deploy-mode ( #14800 )
1 year ago
Wenjun Ruan
55cc77226f
[RPC] Refactor the remote module. ( #14776 )
...
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
1 year ago
Aaron Wang
a491e3a849
fix: null app-info-path cause NPE ( #14752 )
1 year ago
caishunfeng
59998f4c75
[Improvement] add dispatch fail times when compare ( #14761 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
1 year ago
KingsleyY
b3af374fe6
[Fix-14646][Worker] Fix match wrong pid on linux server ( #14652 )
...
* [DS-14646][Worker] Fix match wrong pid on linux server
This closes #14646
---------
Co-authored-by: yangxin3 <yangxin3@newhope.cn>
1 year ago
Edison Catto
25480ae9e7
[Feature][Resource Center] Add support for Huawei Cloud OBS ( #14643 )
...
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center
* add license and add doc
* add 3-party dependency license
* Update LICENSE
* fix
* Update pom.xml
* fix
* fix
---------
Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
1 year ago
Aaron Wang
cb55476235
[Fix-14721] [K8S Task] Handle job delete event when user manually delete the job in k8s cluster ( #14722 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
1 year ago
Aaron Wang
5ec9085113
fix: do not override k8sConfigYaml in ShShellInterceptorBuilder ( #14682 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
Wenjun Ruan
9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
1 year ago
KingsleyY
512f4c7021
[Fix-10795][plugin] sqoop task query columns bug ( #10776 )
...
* [Improvement][Api] add @Override on the Override method
* [FIX]sqoop task query columns bug
* merge two if
* update checkstyle problem
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/SqoopColumnType.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/generator/sources/MySQLSourceGenerator.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* code format
* code format
---------
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
Wenjun Ruan
4aab0b234f
Use AdHoc datasource client in sqlTask ( #14631 )
...
* Use AdHoc datasource client in sqlTask
* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
1 year ago
Eric Gao
d6a4e99e19
fix process utils comment typo ( #14613 )
1 year ago
Aaron Wang
04800a48d1
[Feature-13763][K8S Task] collect real-time log ( #14379 )
...
* [Improvement-13763][K8S Task] collect real-time log
* fix codesmell
* get pod watcher until pod is ready
* fix codesmell
* specify container name & loop waiting pod creation
* sleep when pod is not initialized
---------
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
1 year ago
Kerwin
9664bef933
Fix wrong judgment condition when building parameters of seatunnel task node. ( #14594 )
1 year ago
Jay Chung
de2d215aee
impv: Correct datax execute and python execute script name ( #14523 )
...
* impv: Correct datax execute and python execute script name
we use PYTHON_LAUNCHER for python script execute and
DATAX_LAUNCHER for datax script name
* Add pr number
* fix ut
* style
---------
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
Wenjun Ruan
c30cca9d9a
Support execute shell in different interceptor ( #14582 )
1 year ago
Rick Cheng
9778652f17
[Improvement-14136][task] Support submitting spark-sql task with the sql file in resource center ( #14527 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
FlechazoW
027d66d200
[Typo] standardized license header. ( #14548 )
...
Co-authored-by: wentz <wentz@getui.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
1 year ago
Wenjun Ruan
e6d94632b9
Add WorkflowExecuteContext ( #14544 )
1 year ago
Rick Cheng
09afa75a3d
[Improvement][Worker] No need to kill remote app when appid is empty ( #14534 )
1 year ago
Rick Cheng
c1cffba349
[Improvement] Remove the unused newDaemonSingleThreadExecutor ( #14520 )
1 year ago
zyfjrx
4111acb58e
[Bug] [Seatunnel-Task] Run Mode defaults to run, and you cannot select a null value ( #14267 )
...
Co-authored-by: zhang <zhangyfcoder@163.com>
1 year ago
旺阳
175d976cd2
[Feature][Data Quality] Data Quality Support Choose Database ( #14406 )
...
* add dataquality database api
* change ui
* api change
* update
* fix spotless
* fix h2
* fix pg
* fix-dead-line
* update
* fix-spotless
* update pg sql
* add ut
* fix ut
1 year ago
jegger
75d29f6101
[improvement] DataQuality module improve ( #14463 )
1 year ago
Aaron Wang
0b69236b04
[Feature-14251][Task] Support yarn queue definition in yarn task ( #14310 )
1 year ago
fuchanghai
742a2dc780
[Fix-14447][K8S Task] Fix IndexOutofBoundsException for pod name without dash ( #14448 )
1 year ago
Rick Cheng
cff53feaeb
[Improvement-14464][Task] Remove the useless taskResultString ( #14465 )
1 year ago
Rick Cheng
68a3a75211
[Improvement-14457][UT] Improve the expected and actual value of Assertions ( #14459 )
1 year ago
Aaron Wang
65b6a4b097
[Feature-14421][K8S Task] Configurable image pull policy ( #14426 )
1 year ago
Rick Cheng
13dfa859b5
[Improvement-14371][Hivecli] Add support for hivevar setting in hivecli task ( #14423 )
1 year ago
Jeremy
9c92b4b240
fix: default datax plugin ( #14204 )
1 year ago
Rick Cheng
b49b91c45d
[Improvement-14331][common] Remove the duplicate ParameterUtils ( #14332 )
1 year ago
Wenjun Ruan
a55612e4f7
Fix switch task use boolean type error ( #14326 )
1 year ago
Rick Cheng
1cd5070b5d
[Improvement-14329][common] Remove the duplicate BusinessTimeUtils ( #14330 )
1 year ago
Rick Cheng
4fa0ab4d7e
[feature][Resource] Add support for Azure Blob Storage in the resource center ( #14082 )
1 year ago
ORuteMa
de2cc0e235
[bug][plugin]Fix: Correct the way to determine the yarn queue in Flink CommandLine and SQL mode ( #14237 )
...
* Fix: Correct the way to determine the yarn queue in Flink CommandLine
* fix the yarn queue in sql mode && refine the code
* refine code
* remove unnecessary comment
* fix yarn queue properties
* remove redundant variable
1 year ago
Aaron Wang
b7fb393eed
fix: trim node selector requirement value ( #14307 )
1 year ago
Rick Cheng
66e09dc66e
[Improvement-14295][common] Remove the duplicate FileUtils ( #14301 )
1 year ago
JieguangZhou
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters ( #14127 )
...
* add dynamic task plugin
* fix ui
* update code
* add DOC
* ADD UT
* fix doc
* fix sanity check
* add upgrade ddl
* fix sql
1 year ago
Rick Cheng
2c057d6a8f
[Improvement-14292][Common] Remove the duplicate OSUtils ( #14293 )
1 year ago
Rick Cheng
4d798e774e
[Improvement-14280][Task] update the task execution status check ( #14281 )
1 year ago
HQFGit
d587b5f03d
[Bug][Master][Fixes 14216]workflow state keep READY_STOP when sub_process state … ( #14221 )
...
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes #14216
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes mvn spotless:apply#14216
* Update testFile.txt.crc
* Update testFile.txt.crc
* Delete testFile.txt.crc
1 year ago
Aaron Wang
9e0991145e
[Improvement-14146][K8S Task] print complete error logs in task log file ( #14147 )
1 year ago
Rick Cheng
1011852ab2
[Improvement-14195][Parameter] remove misleading comment of combining local and global parameter ( #14196 )
1 year ago
Aaron Wang
4e78248238
[Feature][K8S Task] support node selector ( #14126 )
...
* feat: support K8S node selector
* fix CI
1 year ago
haibingtown
af7da0630e
[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables ( #14099 )
...
* [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables
1 year ago
Alex Ting
606b56403e
[Fix-13381][plugin] fix error while parsing hive load sql ( #13378 )
...
* [fix][task plugin][task sql] fix parsing error while using variables sometimes parsing exception happened if task sql contains a variable followed by \" or \' .
for example:
input sql: load inpath '/tmp/test_table/dt=${dt}'
into table test_table partition(dt=${dt})
after replace: preparing : load inpath '/tmp/test_table/dt=?
into table test_table partition(dt=?)
1 year ago
Kerwin
8315279620
[Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes ( #14150 )
...
* Added a dependent task failure policy in the dependent task node
1 year ago
Jay Chung
56b0f91f6e
fix: Save task seatunnel error ( #14129 )
1 year ago
fuchanghai
67f50e3904
[fix-#13940] add status of ``` stop``` in class of TaskExecutionStatus ( #14016 )
1 year ago