047fa2f65e
[Feature-13511] Submit Spark task directly on Kubernetes ( #13550 )
2 years ago
2bd65fb2df
[Feature][Remote Logging] Add support for writing task logs to OSS ( #13332 )
2 years ago
37939add0c
[fix-13492]add single quotes ( #13534 )
...
* [fix-13492]add single quotes
* [improve-#13201] fix test error
---------
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2 years ago
16b193454b
[Improvement][Task Plugin] Support Task Plugin output response as a parameter ( #13529 )
...
* support http output response as param
* add UT
* fix sqoop UT
2 years ago
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2 years ago
8fa90c2d8f
feature flinksql support standalone deploy mode ( #13495 )
2 years ago
5a9cb7ee64
Fix task log might delay until task finished due to batch flush ( #13502 )
2 years ago
78ba71bdbd
[Feature][task]add azure datafactory task type ( #13417 )
...
* init data factory
* add task-type-config.yaml
* add data factory ui
* submit doc
* submit license
---------
Co-authored-by: devosend <devosend@gmail.com>
2 years ago
a1f5675012
[Bug][TaskPlugin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string ( #13162 )
...
* [Bug] [TaskPlugin-Zeppelin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string. #13161
2 years ago
ba0a253f09
fix k8s task run fail ( #13348 )
2 years ago
6634ff98af
Add source env list in common.properties ( #13317 )
2 years ago
df491f78cc
fix sagemaker sync status ( #13264 )
2 years ago
14ec4a2398
Remove dao module in worker ( #13242 )
2 years ago
b2336b0ce9
Remove datasource in procedure task ( #13198 )
2 years ago
042ec74a24
[refactor] cache compiled regex pattern ( #13208 )
...
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
2 years ago
43e6ac32dd
[Fix-13187] refactor OpenmldbParameters ( #13190 )
2 years ago
838a0df783
Remove logic task plugin module ( #13171 )
2 years ago
70ccffeee2
Format task parameter as pretty json ( #13173 )
2 years ago
ab96a3d0c0
Fix datasource jdbc url param doesn't work ( #13098 )
2 years ago
560d232cbd
[Bug][Worker] Fix duplicated key bug when have out parameters of same name ( #12930 )
2 years ago
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 ( #13002 )
2 years ago
169168ef34
Add plugin-all module ( #13079 )
2 years ago
c2ae76c0bd
Expand all static imports and add CI check to block wildcard static imports ( #13067 )
2 years ago
32a8241ef2
Don't source env file in task, this will override tenant's environment ( #13028 )
2 years ago
ae670e6e1c
[Bug-13010] [Task] The Flink SQL task page selects the pre-job deployment mode, but the task executed by the worker is the Flink local mode ( #13011 )
2 years ago
d480c271ec
[Bug-12994] [Worker] Fix kill process does not take effect ( #12995 )
2 years ago
a2ff140f43
Fix sql task will send alert if we don't choose the send email ( #12984 )
2 years ago
38b876733c
[Feature-10498] Mask the password in the log of sqoop task ( #11589 )
2 years ago
6e224d08a2
[fix] Python task can not pass the parameters to downstream task. ( #12961 )
2 years ago
7c90bf01bc
[Improvement][Task Plugin] Improvement Kubeflow task plugin ( #12928 )
...
add example
check phase in status:conditions
2 years ago
2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis ( #12693 )
2 years ago
92a9b7e243
Use System#lineSeparator to replace /n ( #12889 )
2 years ago
229c554912
[feature][task] Add Kubeflow task plugin for MLOps scenario ( #12843 )
2 years ago
43d69ae157
Upgrade clickhouse jdbc driver ( #12639 )
2 years ago
c4fb555883
Modify shell env to bash ( #12742 )
2 years ago
2c3cc31a4c
Fix flink sql cannot run due to missing main jar ( #12705 )
2 years ago
eaef68856b
Change command file permission to 755 ( #12678 )
...
(cherry picked from commit 95484d930762712f4ab68ece27ca3b65a2073ab2)
2 years ago
a47b4d0672
[Feature][Task] Transfer files between tasks ( #12552 )
...
* add data transfer between tasks
* add delete DATA_TRANSFER API
* convert Result to DeleteDataTransferResponse
* add api UT
* fix final
* fix doc
2 years ago
b71d4983dd
[Feature-6586][Server]add some ds process definition demo when init ( #11759 )
...
* [Feature-6586][Server]add some ds process definition demo when init
1.add some ds process definition demo when init, to display what task type can run and make user easy to
use ds.
2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service
3.need modify the tenant information in it
2 years ago
3c31ddfd9f
[Improvement-12536][k8s] Support the command for the container in k8s task plugin ( #12538 )
2 years ago
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2 years ago
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2 years ago
dedff70f90
[Improvement] Merge spi.utils into common.utils ( #12502 )
...
* improve JSONUtil
* merge spi.utils into common.utils
* split constants && remove some copy utils method
* remove StringUtils
2 years ago
f0a34cddec
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE ( #12449 )
2 years ago
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2 years ago
1e11034049
[Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly ( #12253 )
...
* [Fix][Task] dependent task can not predicate the status of the corresponding task correctly
2 years ago
ba538067f2
[fix] Change the default unix shell executor from sh to bash ( #12180 )
...
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2 years ago
3bef85f546
fix datax NPE issue ( #12388 )
2 years ago
55004bebe0
[Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function ( #12392 )
...
* Workflow definitions that contain logical task nodes support the copy function.
2 years ago
b7bd8d7803
[Improvement] add presto in datax ( #12371 )
2 years ago
Aaron Wang
Rick Cheng
fuchanghai
JieguangZhou
seedscoder
shenyun
Wenjun Ruan
Tq
LucasClt
caishunfeng
youzipi
HuangWei
Yann Ann
Eric Gao
Kerwin
Assert
simsicon
amao
HanayoZz
insist777