bf2cdc5300
Use Druid to split sql ( #15367 )
1 year ago
93ea5f64c3
[Improvement][K8S] Custom label of a K8S task can be passed to the pod ( #15369 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
1 year ago
d6dea4633c
Optimize server startup log ( #15362 )
1 year ago
5523a62825
Remove taskQueue and looper in worker ( #15292 )
1 year ago
921253109a
[Bug] Close SSH session after remote shell finish ( #15348 )
...
* Close SSH session after remote shell finish
---------
Co-authored-by: David Zollo <davidzollo365@gmail.com>
1 year ago
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) ( #15355 )
1 year ago
f921457a55
Set maxRows in SqlTask ( #15342 )
...
(cherry picked from commit b6b88e3e62d8245e08408fb60c48d23c0bd0bb46)
1 year ago
ce11674668
Fix k8sTaskExecutionContext setting configYaml ( #15116 )
...
* fixed the issue of obtaining kubeConfig of k8s tasks
* removed excess files
* removed excess files
* formatted the code
---------
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
1 year ago
159179ac95
[Feature-15146][dolphinscheduler-task-sqoop] add sqoop source/target type ( #15146 )
...
* task list: sgoop node params optimize
* security.ts add alarm_instance params
* 1 add SqoopTask params
2 add alert plugin aliyun-voice
* add license header
* commit sqhoop optimize
* pnpm-locl.yaml supplement annotation
* remove irrelevent commit.
* Code specification optimization
* optimize sqoop task ui
* Merge Code
* add the license header to pnpm-locl.yaml
* format the code
* format the code
* Fix sqoop task echo error
---------
Co-authored-by: xujiaqiang <xujiaqiang@aimatech.com>
Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
1 year ago
0bb48f3429
fail-fast for dependent check ( #15197 )
...
Co-authored-by: Leoric Yue <leoric.yue@zoom.us>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
1 year ago
2119e41800
[Improvement] Move delay calculation to Master ( #15278 )
1 year ago
db3d84b73d
[HotFix] Fix TaskOutputParameterParser might OOM if meed a bad output param expression ( #15264 )
1 year ago
ef2e9c9b67
Set the tenant as the owner in final stage ( #15256 )
1 year ago
71ee1f0c3d
Support parse task output params under multiple log ( #15244 )
1 year ago
df656a7671
Fix home page workflow instance miss status ( #15193 )
1 year ago
5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue ( #14992 )
...
* change data quality field name
* fix-dao
* fix-dao
* fix-dao
* fix-dao
* update
* test
* update
* update
1 year ago
c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center ( #14977 )
1 year ago
5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks ( #14976 )
...
* Refactoring `Sagemaker` task plugin with connections managed in connection center.
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
1 year ago
e99c5950b9
[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks ( #14434 )
...
* Refactoring zeppelin task plugin with connections managed in connection center
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
1 year ago
99214d0fc3
Fix SqlTask cannot split the given sql when browser in windows ( #15062 )
1 year ago
688f84452d
[Feature-14678][Master][UI]Dependent task parameter passing ( #14702 )
...
* feat: dependent task parameter passing
* style: code format
* refactor: remove inappropriate log
* refactor: Modify unreasonable code structure
* refactor: Modify unreasonable code structure
* refactor: Solve conflicts caused by merge
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: BaoLiang <29528966+lenboo@users.noreply.github.com>
1 year ago
049c1511fa
[Improvement][K8S] K8S task support passing context parameters to downstream task ( #14934 )
...
* [Improvement][K8S] K8S task support passing context parameters to downstream task
Signed-off-by: Gallardot <gallardot@apache.org>
* [Improvement][K8S] K8S task support passing context parameters to downstream task
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: update doc img
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: update doc
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: add UT
Signed-off-by: Gallardot <gallardot@apache.org>
* merge suggestion
Co-authored-by: 旺阳 <wang@lqwang.net>
* merge suggestion
Co-authored-by: 旺阳 <wang@lqwang.net>
* merge suggestion
Co-authored-by: 旺阳 <wang@lqwang.net>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: David Zollo <dailidong66@gmail.com>
Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
f90eadd40f
[feature-#14449] add secret in k8s Task ( #14930 )
...
* [feature-#14449] add secret
* [feature-#14449] upgrade UT and rename parameter ```secret```
* [feature-#14449] fix UT
* [feature-#14449] fix UT
* [feature-#14449] improvement
* [feature-#14449] format
---------
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
1 year ago
6de1e2c1b3
Fix the error of using shell task to obtain Home variable in Ubuntu system. ( #14964 )
1 year ago
e748c2eb9a
fix the #14729 problem ( #14902 )
...
Co-authored-by: 刘阳 <liuy2590@chinaunicom.cn>
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
9083f9d0e0
[Fix-14546] seatunnel unable submit spark job to spark ( #14617 )
1 year ago
f6a6794289
[Bug][Worker] Global and local parameters are not replaced in SQL components ( #14926 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
1 year ago
3fa69d2db8
[Bug][Worker] Fix sql/shell task doesn't replace parameter ( #14876 )
...
Fix parameter assign error, fix #14869 which including
* shell task param not work, botch local and global
* others task chain assign not work
1 year ago
7640c29495
[Improvement] [Task] Change Seatunnel task run-mode to deploy-mode ( #14800 )
1 year ago
55cc77226f
[RPC] Refactor the remote module. ( #14776 )
...
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
1 year ago
a491e3a849
fix: null app-info-path cause NPE ( #14752 )
1 year ago
59998f4c75
[Improvement] add dispatch fail times when compare ( #14761 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
1 year ago
b3af374fe6
[Fix-14646][Worker] Fix match wrong pid on linux server ( #14652 )
...
* [DS-14646][Worker] Fix match wrong pid on linux server
This closes #14646
---------
Co-authored-by: yangxin3 <yangxin3@newhope.cn>
1 year ago
25480ae9e7
[Feature][Resource Center] Add support for Huawei Cloud OBS ( #14643 )
...
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center
* add license and add doc
* add 3-party dependency license
* Update LICENSE
* fix
* Update pom.xml
* fix
* fix
---------
Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
1 year ago
cb55476235
[Fix-14721] [K8S Task] Handle job delete event when user manually delete the job in k8s cluster ( #14722 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
1 year ago
5ec9085113
fix: do not override k8sConfigYaml in ShShellInterceptorBuilder ( #14682 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
1 year ago
512f4c7021
[Fix-10795][plugin] sqoop task query columns bug ( #10776 )
...
* [Improvement][Api] add @Override on the Override method
* [FIX]sqoop task query columns bug
* merge two if
* update checkstyle problem
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/SqoopColumnType.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/generator/sources/MySQLSourceGenerator.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* code format
* code format
---------
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
4aab0b234f
Use AdHoc datasource client in sqlTask ( #14631 )
...
* Use AdHoc datasource client in sqlTask
* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
1 year ago
d6a4e99e19
fix process utils comment typo ( #14613 )
1 year ago
04800a48d1
[Feature-13763][K8S Task] collect real-time log ( #14379 )
...
* [Improvement-13763][K8S Task] collect real-time log
* fix codesmell
* get pod watcher until pod is ready
* fix codesmell
* specify container name & loop waiting pod creation
* sleep when pod is not initialized
---------
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
1 year ago
9664bef933
Fix wrong judgment condition when building parameters of seatunnel task node. ( #14594 )
1 year ago
de2d215aee
impv: Correct datax execute and python execute script name ( #14523 )
...
* impv: Correct datax execute and python execute script name
we use PYTHON_LAUNCHER for python script execute and
DATAX_LAUNCHER for datax script name
* Add pr number
* fix ut
* style
---------
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
c30cca9d9a
Support execute shell in different interceptor ( #14582 )
1 year ago
9778652f17
[Improvement-14136][task] Support submitting spark-sql task with the sql file in resource center ( #14527 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
027d66d200
[Typo] standardized license header. ( #14548 )
...
Co-authored-by: wentz <wentz@getui.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
1 year ago
e6d94632b9
Add WorkflowExecuteContext ( #14544 )
1 year ago
09afa75a3d
[Improvement][Worker] No need to kill remote app when appid is empty ( #14534 )
1 year ago
c1cffba349
[Improvement] Remove the unused newDaemonSingleThreadExecutor ( #14520 )
1 year ago
4111acb58e
[Bug] [Seatunnel-Task] Run Mode defaults to run, and you cannot select a null value ( #14267 )
...
Co-authored-by: zhang <zhangyfcoder@163.com>
1 year ago
Wenjun Ruan
Gallardot
chenrj
xujiaqiang
Dyqer
旺阳
Orange Summer
fuchanghai
Kerwin
LiuCanWu
Yhr-N
Jay Chung
zhangbaipeng
Aaron Wang
caishunfeng
KingsleyY
Edison Catto
Eric Gao
Rick Cheng
FlechazoW
zyfjrx