2c93dc6041
cherry-pick ( #12608 )
...
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE
#12449
Co-authored-by: fuchanghai <33984497+fuchanghai@users.noreply.github.com>
2 years ago
94ef02e9de
cherry-pick [Improvement] Merge spi.utils into common.utils ( #12607 )
...
#12502
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2 years ago
eb7b48596c
cherry-pick Reorganize some classes in common module, remove duplicate classes
...
#12321
2 years ago
7ca4438682
Resolve AbstractTask code conflicts
2 years ago
f3277277f0
cherry-pick Refactor LogServiceClient Singleton to avoid repeat creation of NettyClient #11777 ( #12542 )
...
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2 years ago
344293102d
[3.1.1-prepare][cherry-pick][Improvement] Add remote task model #11767 ( #12541 )
2 years ago
3a14fd07b3
[Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly ( #12253 )
...
* [Fix][Task] dependent task can not predicate the status of the corresponding task correctly
2 years ago
f0ad67f992
cherry-pick source is not available in sh ( #12413 )
...
Co-authored-by: simsicon <simsicon@gmail.com>
2 years ago
5cfe6a96b4
[Improvement-12372][k8s] Update the deprecated k8s api ( #12373 )
2 years ago
771dd67b2e
cherry-pick [Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function
2 years ago
7e39396a76
[maven-release-plugin] prepare for next development iteration
2 years ago
ae33ba5947
[maven-release-plugin] prepare release 3.1.0
2 years ago
e7b12bf205
[Cherry-Pick][3.1.0-prepare] Cherry-Pick OssOperator for Resource Center to 3.1.0 ( #12094 )
...
* Cherry OssOperator for Resource Center to 3.1.0
* Fix doc dead link
2 years ago
fcc75ef1c6
Bump up dependencies to fix cves ( #11765 )
2 years ago
8d1c2d3eeb
cherry-pick [Bug] Fix procedure task param pass #11919 ( #11926 )
2 years ago
f95cdd350a
reduce task dependencies ( #11731 )
2 years ago
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task ( #11701 )
...
* Fix kill job may cause worker oom
2 years ago
1b120e3a59
Refactor worker execute task process ( #11540 )
...
* Refactor worker execute task process
2 years ago
b96d69701a
[Improvement-#11613] Add spi priority factory ( #11614 )
...
* Add spi priority factory
* Add doc
* Add override log
* Use lombok
* Add comment
2 years ago
a41c6824fc
Add dependent task instance log ( #11541 )
...
* Add dependent task instance log
* Optimize log
* Fix dependent task initialize failed will throw exception
2 years ago
13e4720aa2
Upgrade spring boot and swagger ( #11574 )
2 years ago
f6ef628030
Support reading application properties from kubernetes config map ( #11402 )
2 years ago
b79bad19e5
[Fix-11051][Task]Fix the process exitValue exception when the process times out and th… ( #11099 )
2 years ago
5fa3e7b1ed
Add pause to task instance status ( #11390 )
2 years ago
6345a13cdc
[Fix-10392][SqlTask]: Fix list param error when use sql task ( #11285 )
...
* FIX: fix list param in.
* FIX: code bug
* FIX: fix one value
* FIX: header
* FIX: ci
* FIX: fix ci
* MOD: for review
* MOD: for review
2 years ago
0464123c2b
[Feature-11223] support stream task ( #11350 )
...
* add task execute type
* update task definition list paging
* update task instance list paging
* stream task start
* [Feature][UI] Some changes to execute task.
* Set the connection edge to dashed line.
* Add FLINK_STREAM task.
* add stream task
* flink savepoint and cancel
* fix query bug
* add stream task definition
* add task instance for stream task
* delete stream task definition state
* update api for stream task definition edit
* modify search for stream task instance
* add language
* delete task type search for stream task definition
* change task type search for stream task instance
* add jump button
* add savepoint
* add down log for stream task instance
* ui test
* stream task start
* run DAG
* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.
* [Feature][UI] Remove some fields for FLINK_STREAM.
* add start modal
* add dryRun column for stream task instance
* fix duration
* fix pon
* fix build error
* Add success tip
* add auto sync for stream task instance
* remove forgien key for task instance
* license header
* UT fix
* modify locales
* recover common config
* fix UT
* add doc
Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2 years ago
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus ( #11340 )
2 years ago
9a53c6ac34
fix work oom when task logs's size is very large ( #11224 )
2 years ago
bfff3a7c5d
fix error ( #11206 )
2 years ago
818648df7d
[Bug-11071] fix local params ( #11184 )
...
* fix local params
* add CI case
2 years ago
052ceaacd0
Add is success in loop task status ( #11169 )
2 years ago
c7789bf049
Add base loop task execotor and http template parser ( #11137 )
...
* Add base loop task execotor and http template parser
* Add JsonPathUtils
2 years ago
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version ( #11025 )
2 years ago
a38fa34d43
[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks ( #10925 )
...
* [Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#9814 )
2 years ago
427c58546c
[fix][plugin]: fix spark sql use custom parma. ( #10821 )
...
* [fix][plugin]: fix spark sql use custom parma.
* [fix][plugin] roll back part of the code
Co-authored-by: huaangcg <huangcg@getui.com>
2 years ago
8f621ff98b
[Optimization] Calculate global parameter and local parameter at master. ( #10704 )
...
* Global parameter and local parameter calculation external expansion.
* k8s task ut fix.
* TimePlaceholderUtils import DateUtils fix
* follow the review comments to fix.
* follow the review comments to fix.
* e2e rerun
2 years ago
66624c5c86
[Bug] [Master] Worker failover will cause task cannot be failover ( #10631 )
...
* fix worker failover may lose event
2 years ago
db595b3eff
Optimize master log, use MDC to inject workflow instance id and task instance id in log ( #10516 )
...
* Optimize master log, add workflow instance id and task instance id in log
* Use MDC to set the workflow info in log4j
* Add workflowInstanceId and taskInstanceId in MDC
2 years ago
2c227ab384
[Feature-10363][server] S3 Resource center supports bucket customization ( #10433 )
...
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify the configuration. Sort out HDFS and S3 configurations to distinguish them
* Modify the configuration comment to clarify the configuration meaning
* Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage
* Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties
* Modified code to support bucket customization
* Check that the bucket on personal permission, Otherwise, the startup fails
* S3 increase support resource. Storage. Upload. Base. The base path path
* Change S3Utils implementation style to be consistent with HadoopUtils
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify code review comments
2 years ago
bbba20d39a
change to common lang3 and fix checkstyle ( #10380 )
2 years ago
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup ( #10373 )
2 years ago
d80cf21456
Clean up unused dependencies and packaging issues ( #9944 )
2 years ago
5bb1eb04fc
[Feature][plugin] Add k8s task in task plugin ( #9425 )
...
* [Feature][plugin] Add k8s task in task plugin
* [Feature][plugin] fix dos and code problems
* [Feature][plugin] refactor some code based on sonar
* [Feature][UI] front-end for k8s task plugin
* [Feature][plugin] delete some front files
* [Feature][plugin] update document
* Update docs/docs/zh/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* [Feature][UI] front-end change from review comments
* [Feature][UI] replace get namespace list api
* [Feature][plugin] change file name
* Add kubernetes to zh task list
Co-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
3 years ago
80369363c9
[feat][task plugin] Add Jupyter task plugin ( #9872 )
...
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
3 years ago
0e7c98d465
[Feature-9177][Task] The sql task supports configuring segmentation notation to provide execution of multiple statements ( #9917 )
...
* Support sql segmentation to execute multiple functions.
3 years ago
ed425d2d2d
[Feature-9772][plugin/ui] support SparkSQL Task ( #9790 )
...
* [refactor] Dolphinscheduler sparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Refactor ui code and add sparksql test cases
* [refactor] refactor dolphinscheduler SparkSQL
* [refactor] refactor dolphinscheduler plugin-sparkSQL
* [refactor] refactor dolphinscheduler plugin-SparkSQL
* [refactor] dolphinscheduler plugin-SparkTaskTest
* [refactor] dolphinscheduler plugin-SparkTask
* [refactor] dolphinscheduler plugin-Spark
* [refactor] dolphinscheduler plugin-SparkTask-SparkSQL
* [refactor] dolphinscheduler plugin-spark-SparkTask
* [refactor] dolphinscheduler plugin-spark-SparkTask redefine code
3 years ago
36f01155b5
[Improvement][server] varPool support syntax #{setValue(key=value)} ( #9586 )
3 years ago
c5b7e5adff
[Bug] [API-9558]fix homepage task instance count method to use submit time to recount ( #9559 )
...
* fix homepage task instance count method to use submit time to recount
* fix homepage task instance count method to use submit time to recount
* fix homepage task instance count method to use submit time to recount
* fix homepage task instance count method JUNIT
* fix homepage task instance count method JUNIT
* fix homepage task instance count method JUNIT
3 years ago
a5bbf7852d
[Feature][Task-Plugin]Add zeppelin task-plugin to support Apache Zeppelin ( #9327 )
3 years ago
04d4e4e0c4
fix the OUT direct param could pass to the current script ( #9463 )
3 years ago
Kerwin
kezhenxu94
zhuangchong
insist777
rickchengx
caishunfeng
Eric Gao
Wenjun Ruan
Stalary
microeastcowboy
xuhhui
chuxing
WangJPLeo
guodong
xiangzihao
He Zhao
sq-q
exmy
Tq