7b44612f28
[Improvement-12372][k8s] Update the deprecated k8s api ( #12373 )
2 years ago
db1d319159
Reorganize some classes in common module, remove duplicate classes ( #12321 )
2 years ago
875682d267
[Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example ( #12299 )
...
* Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example
2 years ago
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones ( #12239 )
...
* fix DS support cross time zone, use timestamp replace date
2 years ago
779a0b9c75
[Improvement-#11608][task-plugin] New time variables are added to facilitate business development ( #11667 )
...
Add new time expressions for users' convenience
Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
Co-authored-by: David <dailidong66@gmail.com>
2 years ago
e6832220c3
[Improvement][SQL Task]use default sql segment separator ( #10869 )
2 years ago
08a4c7981f
[improve] Remove the spark version of spark task ( #11860 )
2 years ago
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center ( #11708 )
...
* Add support for Alibaba Cloud OSS as storage of resource center
* Add dependency licenses
* Refactor code, add unit tests and related docs
2 years ago
5b384f3fab
[Feature-11473][Task]Support test task ( #11670 )
...
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* Update TaskExecuteThread.java
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update DataSource.java
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages_zh_CN.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages.properties
* Update messages_en_US.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
2 years ago
511149f644
[fix-10938]: use dot to replace source and make the default env work in shell ( #11625 )
...
* [fix-10938]: use the dot to replace source and make the default env work in the sh
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2 years ago
d3a77c68e6
Make LogServiceClient Singleton ( #11777 )
2 years ago
0deafba12c
fix procedure task param pass ( #11919 )
2 years ago
5d9331f4ee
[Feature-10683][Task Plugin] Add Java Task Plugin. ( #10689 )
2 years ago
2e61c76c22
[Improvement] Add remote task model ( #11767 )
2 years ago
efedc9a53c
Bump up dependencies to fix cves ( #11765 )
2 years ago
f95cdd350a
reduce task dependencies ( #11731 )
2 years ago
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task ( #11701 )
...
* Fix kill job may cause worker oom
2 years ago
1b120e3a59
Refactor worker execute task process ( #11540 )
...
* Refactor worker execute task process
2 years ago
b96d69701a
[Improvement-#11613] Add spi priority factory ( #11614 )
...
* Add spi priority factory
* Add doc
* Add override log
* Use lombok
* Add comment
2 years ago
a41c6824fc
Add dependent task instance log ( #11541 )
...
* Add dependent task instance log
* Optimize log
* Fix dependent task initialize failed will throw exception
2 years ago
13e4720aa2
Upgrade spring boot and swagger ( #11574 )
2 years ago
f6ef628030
Support reading application properties from kubernetes config map ( #11402 )
2 years ago
b79bad19e5
[Fix-11051][Task]Fix the process exitValue exception when the process times out and th… ( #11099 )
2 years ago
5fa3e7b1ed
Add pause to task instance status ( #11390 )
2 years ago
6345a13cdc
[Fix-10392][SqlTask]: Fix list param error when use sql task ( #11285 )
...
* FIX: fix list param in.
* FIX: code bug
* FIX: fix one value
* FIX: header
* FIX: ci
* FIX: fix ci
* MOD: for review
* MOD: for review
2 years ago
0464123c2b
[Feature-11223] support stream task ( #11350 )
...
* add task execute type
* update task definition list paging
* update task instance list paging
* stream task start
* [Feature][UI] Some changes to execute task.
* Set the connection edge to dashed line.
* Add FLINK_STREAM task.
* add stream task
* flink savepoint and cancel
* fix query bug
* add stream task definition
* add task instance for stream task
* delete stream task definition state
* update api for stream task definition edit
* modify search for stream task instance
* add language
* delete task type search for stream task definition
* change task type search for stream task instance
* add jump button
* add savepoint
* add down log for stream task instance
* ui test
* stream task start
* run DAG
* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.
* [Feature][UI] Remove some fields for FLINK_STREAM.
* add start modal
* add dryRun column for stream task instance
* fix duration
* fix pon
* fix build error
* Add success tip
* add auto sync for stream task instance
* remove forgien key for task instance
* license header
* UT fix
* modify locales
* recover common config
* fix UT
* add doc
Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2 years ago
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus ( #11340 )
2 years ago
9a53c6ac34
fix work oom when task logs's size is very large ( #11224 )
2 years ago
bfff3a7c5d
fix error ( #11206 )
2 years ago
818648df7d
[Bug-11071] fix local params ( #11184 )
...
* fix local params
* add CI case
2 years ago
052ceaacd0
Add is success in loop task status ( #11169 )
2 years ago
c7789bf049
Add base loop task execotor and http template parser ( #11137 )
...
* Add base loop task execotor and http template parser
* Add JsonPathUtils
2 years ago
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version ( #11025 )
2 years ago
a38fa34d43
[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks ( #10925 )
...
* [Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#9814 )
2 years ago
427c58546c
[fix][plugin]: fix spark sql use custom parma. ( #10821 )
...
* [fix][plugin]: fix spark sql use custom parma.
* [fix][plugin] roll back part of the code
Co-authored-by: huaangcg <huangcg@getui.com>
2 years ago
8f621ff98b
[Optimization] Calculate global parameter and local parameter at master. ( #10704 )
...
* Global parameter and local parameter calculation external expansion.
* k8s task ut fix.
* TimePlaceholderUtils import DateUtils fix
* follow the review comments to fix.
* follow the review comments to fix.
* e2e rerun
2 years ago
66624c5c86
[Bug] [Master] Worker failover will cause task cannot be failover ( #10631 )
...
* fix worker failover may lose event
2 years ago
db595b3eff
Optimize master log, use MDC to inject workflow instance id and task instance id in log ( #10516 )
...
* Optimize master log, add workflow instance id and task instance id in log
* Use MDC to set the workflow info in log4j
* Add workflowInstanceId and taskInstanceId in MDC
2 years ago
2c227ab384
[Feature-10363][server] S3 Resource center supports bucket customization ( #10433 )
...
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify the configuration. Sort out HDFS and S3 configurations to distinguish them
* Modify the configuration comment to clarify the configuration meaning
* Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage
* Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties
* Modified code to support bucket customization
* Check that the bucket on personal permission, Otherwise, the startup fails
* S3 increase support resource. Storage. Upload. Base. The base path path
* Change S3Utils implementation style to be consistent with HadoopUtils
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify code review comments
2 years ago
bbba20d39a
change to common lang3 and fix checkstyle ( #10380 )
3 years ago
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup ( #10373 )
3 years ago
d80cf21456
Clean up unused dependencies and packaging issues ( #9944 )
3 years ago
5bb1eb04fc
[Feature][plugin] Add k8s task in task plugin ( #9425 )
...
* [Feature][plugin] Add k8s task in task plugin
* [Feature][plugin] fix dos and code problems
* [Feature][plugin] refactor some code based on sonar
* [Feature][UI] front-end for k8s task plugin
* [Feature][plugin] delete some front files
* [Feature][plugin] update document
* Update docs/docs/zh/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* [Feature][UI] front-end change from review comments
* [Feature][UI] replace get namespace list api
* [Feature][plugin] change file name
* Add kubernetes to zh task list
Co-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
3 years ago
80369363c9
[feat][task plugin] Add Jupyter task plugin ( #9872 )
...
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
3 years ago
0e7c98d465
[Feature-9177][Task] The sql task supports configuring segmentation notation to provide execution of multiple statements ( #9917 )
...
* Support sql segmentation to execute multiple functions.
3 years ago
ed425d2d2d
[Feature-9772][plugin/ui] support SparkSQL Task ( #9790 )
...
* [refactor] Dolphinscheduler sparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Dolphinscheduler SparkSQL
* [refactor] Refactor ui code and add sparksql test cases
* [refactor] refactor dolphinscheduler SparkSQL
* [refactor] refactor dolphinscheduler plugin-sparkSQL
* [refactor] refactor dolphinscheduler plugin-SparkSQL
* [refactor] dolphinscheduler plugin-SparkTaskTest
* [refactor] dolphinscheduler plugin-SparkTask
* [refactor] dolphinscheduler plugin-Spark
* [refactor] dolphinscheduler plugin-SparkTask-SparkSQL
* [refactor] dolphinscheduler plugin-spark-SparkTask
* [refactor] dolphinscheduler plugin-spark-SparkTask redefine code
3 years ago
36f01155b5
[Improvement][server] varPool support syntax #{setValue(key=value)} ( #9586 )
3 years ago
c5b7e5adff
[Bug] [API-9558]fix homepage task instance count method to use submit time to recount ( #9559 )
...
* fix homepage task instance count method to use submit time to recount
* fix homepage task instance count method to use submit time to recount
* fix homepage task instance count method to use submit time to recount
* fix homepage task instance count method JUNIT
* fix homepage task instance count method JUNIT
* fix homepage task instance count method JUNIT
3 years ago
a5bbf7852d
[Feature][Task-Plugin]Add zeppelin task-plugin to support Apache Zeppelin ( #9327 )
3 years ago
04d4e4e0c4
fix the OUT direct param could pass to the current script ( #9463 )
3 years ago
rickchengx
kezhenxu94
Eric Gao
Yann Ann
冯剑
zhuxt2015
insist777
itzhang89
Wenjun Ruan
caishunfeng
juzimao
Kerwin
Stalary
microeastcowboy
xuhhui
chuxing
WangJPLeo
guodong
xiangzihao
He Zhao
sq-q
exmy
Tq