67e7f88d8b
Refactor heart beat task, use json to serialize/deserialize ( #11702 )
...
* Refactor heart beat task, use json to serialize/deserialize
2 years ago
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task ( #11701 )
...
* Fix kill job may cause worker oom
2 years ago
b96d69701a
[Improvement-#11613] Add spi priority factory ( #11614 )
...
* Add spi priority factory
* Add doc
* Add override log
* Use lombok
* Add comment
2 years ago
4d13a5104b
Fix kill task failed will cause the taskGroup cannot release and add taskGroup log ( #11469 )
2 years ago
f6ef628030
Support reading application properties from kubernetes config map ( #11402 )
2 years ago
13d400ceb6
[Fix-11465] Serial first, the last executing instance will be stopped ( #11466 )
...
* Serial first, the last executing instance will be stopped
* ut fix
2 years ago
7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost ( #11368 )
...
* Add waiting strategy to support master/worker can recover from registry lost
* throw exception when zookeeper registry start failed due to interrupted
2 years ago
be66035dee
[Improvement] Avoid using search in for and start using testSaveTaskDefine ( #11383 )
...
* [Improvement] Avoid using search in for and start using testSaveTaskDefine()
* [Improvement] not import *
Co-authored-by: zhangshunmin <zhangshunmin@kezaihui.com>
2 years ago
5fa3e7b1ed
Add pause to task instance status ( #11390 )
2 years ago
0464123c2b
[Feature-11223] support stream task ( #11350 )
...
* add task execute type
* update task definition list paging
* update task instance list paging
* stream task start
* [Feature][UI] Some changes to execute task.
* Set the connection edge to dashed line.
* Add FLINK_STREAM task.
* add stream task
* flink savepoint and cancel
* fix query bug
* add stream task definition
* add task instance for stream task
* delete stream task definition state
* update api for stream task definition edit
* modify search for stream task instance
* add language
* delete task type search for stream task definition
* change task type search for stream task instance
* add jump button
* add savepoint
* add down log for stream task instance
* ui test
* stream task start
* run DAG
* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.
* [Feature][UI] Remove some fields for FLINK_STREAM.
* add start modal
* add dryRun column for stream task instance
* fix duration
* fix pon
* fix build error
* Add success tip
* add auto sync for stream task instance
* remove forgien key for task instance
* license header
* UT fix
* modify locales
* recover common config
* fix UT
* add doc
Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2 years ago
8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus ( #11340 )
2 years ago
acd3d3fab1
Fix TaskGroup cannot work and will cause master dead loop ( #11254 )
...
* Fix TaskGroup cannot work and will cause master dead loop
* Remove acquireTaskGroupAgain in ProcessServiceImpl
2 years ago
8e21c38c00
Write alert result into db ( #11221 )
2 years ago
e5cca0e79b
[Fix-11007] [Master] fix forced_success bug ( #11088 )
...
* fix forced_success bug
* add comments
* add transactional
* refactor code
Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
2 years ago
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version ( #11025 )
2 years ago
65d2b8f14e
[Bug] fix task params pass ( #11053 )
2 years ago
d7df899909
[Fix-11003]Task group queue is not updated to final state ( #11004 )
...
* fix after the task group is forced to start the task. Task group status has not changed
2 years ago
083ab2b5c9
Remove dao in worker ( #10994 )
2 years ago
2be1d4bf0a
Fix worker cannot shutdown due to resource close failed or heart beat check failed ( #10979 )
...
* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed
* Move heartbeat error threshold to applicaiton.yml
2 years ago
2397423eb6
Solve transaction problems under multi-threading ( #10917 )
2 years ago
2a67866718
Catch exception when check state in StateWheelExecuteThread ( #10908 )
...
* Catch exception when check state
2 years ago
f639a2eed4
[Fix-10854] Fix database restart may lost task instance status ( #10866 )
...
* Fix database update error doesn't rollback the task instance status
* Fix database error may cause workflow dead with running status
2 years ago
3f69ec8f28
[Fix-10842] Fix master/worker failover will cause status incorrect ( #10839 )
...
* Fix master failover will not update task instance status
* Add some failover log
* Fix worker failover will rerun task more than once
* Fix workflowInstance failover may rerun already success taskInstance
2 years ago
426567348e
Remove quartz in service ( #10748 )
...
* Remove quartz in service
2 years ago
67d14fb7b3
[Fix-10785] Fix state event handle error will not retry ( #10786 )
...
* Fix state event handle error will not retry
* Use state event handler to deal with the event
2 years ago
b7fbf6c4ad
[API] Do not specify rollback for ensure in case of any exception or error. ( #10793 )
2 years ago
182b9c9b26
[Feature][Master] Failover workflow instance base on priority ( #10754 )
...
* failover workflow instance base on priority
2 years ago
35a10d092f
[Fix-10666] Workflow submit failed will still in memory and never retry ( #10667 )
...
* Workflow submit failed will still in memory and never retry
2 years ago
f8650b7b7c
[Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun。 ( #10727 )
...
* [Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun.
* [Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun.
* [Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun.
Co-authored-by: houshitao <shitaohou@163.com>
2 years ago
8f621ff98b
[Optimization] Calculate global parameter and local parameter at master. ( #10704 )
...
* Global parameter and local parameter calculation external expansion.
* k8s task ut fix.
* TimePlaceholderUtils import DateUtils fix
* follow the review comments to fix.
* follow the review comments to fix.
* e2e rerun
2 years ago
247ca4ae8a
[Improvement-10617] Add comment in slot check ( #10618 )
2 years ago
75581b7543
[Improvement]Delete the getSession in the Registry interface ( #10633 )
2 years ago
35b25da863
Validate master/worker config ( #10649 )
2 years ago
66624c5c86
[Bug] [Master] Worker failover will cause task cannot be failover ( #10631 )
...
* fix worker failover may lose event
2 years ago
b5184138fa
[Feature] Time function analysis extension. ( #10624 )
...
* Time function analysis extension.
* param add.
* clear useless logs and update method notes
* permission omission fix.
* extending time functions to optimize static methods
* e2e rerun.
2 years ago
db595b3eff
Optimize master log, use MDC to inject workflow instance id and task instance id in log ( #10516 )
...
* Optimize master log, add workflow instance id and task instance id in log
* Use MDC to set the workflow info in log4j
* Add workflowInstanceId and taskInstanceId in MDC
2 years ago
f46faa02c3
[Fix-10452] Serial wait for policy recovery ( #10453 )
...
* Serial wait for policy recovery
* processInstance state check null
* add sendresult(Host host, Command command) method and replace the original.
2 years ago
cefcb6d556
[Fix-10522] [API] Fix get executable time null pointer exception ( #10523 )
2 years ago
cc06eaaf54
[Improvement][Metrics] Apply micrometer naming convention to metrics ( #10477 )
...
* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards
2 years ago
0bdfa0cff9
Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check ( #10491 )
2 years ago
ad2646ff1f
Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe ( #10479 )
...
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe
2 years ago
f6fea06f10
[Improve] Enhance complement function transformation ( #10376 )
2 years ago
b0d9d3f9ab
Add some warning log in master ( #10383 )
...
* Add some warn log in master
* fix may skip sleep
2 years ago
e21d7b1551
[Feature][metrics] Add master, worker metrics ( #10326 )
...
* Add mater metrics
* fix UT
* Add url to mysql profile
* Add worker metrics
* Update grafana config
* Add system metrics doc
* Add process failover counter
* Add metrics image
* Change jpg to png
* Add command insert metrics
* Fix UT
* Revert UT
2 years ago
3ecbee3885
Optimize MasterServer, add MasterRPCService ( #10371 )
...
* Optimize MasterServer, avoid NPE
2 years ago
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup ( #10373 )
2 years ago
2d3be6b36c
Add dolphinscheduler-scheduler module ( #10360 )
...
* Add dolphinscheduler-scheduler module
2 years ago
022e4886be
Remove quartz at WorkerServer ( #10358 )
...
* Remove quartz at WorkerServer
* move k8s and permission from dolphinscheduler-service to dolphinscheduler-api
2 years ago
0bf4043d5f
Move datasource params to datasouce plugin module ( #10353 )
...
* Move datasource params to datasouce plugin module
2 years ago
e3e39cbdea
[Fix] Privilege Reconstruction Data Query Fix & Resource Creation Post Oper… ( #10313 )
...
* Privilege Reconstruction Data Query Fix & Resource Creation Post Operations.
* Resource Retry
* e2e retry
* create project log add
* project list query log add
* clean project log
* delete delay
* delete delay
* remove post handle
* project e2e
* browser refresh
* browser refresh
* e2e fix
* e2e browser refresh
* rowk flow e2e fix
* mapper deduplication
* udf e2e
* e2e
2 years ago
Wenjun Ruan
kezhenxu94
JiPeng Wang
longtb
caishunfeng
JinYong Li
Mr.An
zhuxt2015
houshitao
wjf
Kerwin
Eric Gao
xiangzihao