397e51fa63
dependent node log output optimization and fix checkstyle ( #10593 )
2 years ago
1111371c9a
add datasource health check to the healthcheck endpoint ( #10588 )
2 years ago
0f38217b12
fix_10514 ( #10568 )
2 years ago
db595b3eff
Optimize master log, use MDC to inject workflow instance id and task instance id in log ( #10516 )
...
* Optimize master log, add workflow instance id and task instance id in log
* Use MDC to set the workflow info in log4j
* Add workflowInstanceId and taskInstanceId in MDC
2 years ago
f46faa02c3
[Fix-10452] Serial wait for policy recovery ( #10453 )
...
* Serial wait for policy recovery
* processInstance state check null
* add sendresult(Host host, Command command) method and replace the original.
2 years ago
cc06eaaf54
[Improvement][Metrics] Apply micrometer naming convention to metrics ( #10477 )
...
* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards
2 years ago
b100f6c489
Remove the schedule thread in LowerWeightHostManager ( #10310 )
2 years ago
117f78ec4b
[Fix-10413] Fix Master startup failure the server still hang ( #10500 )
...
* Fix Master startup failure the server still hang
2 years ago
0bdfa0cff9
Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check ( #10491 )
2 years ago
ad2646ff1f
Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe ( #10479 )
...
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe
2 years ago
78c5fcc6ac
Add mysql registry plugin ( #10406 )
...
* Add mysql registry plugin
2 years ago
f6fea06f10
[Improve] Enhance complement function transformation ( #10376 )
2 years ago
b0d9d3f9ab
Add some warning log in master ( #10383 )
...
* Add some warn log in master
* fix may skip sleep
2 years ago
e21d7b1551
[Feature][metrics] Add master, worker metrics ( #10326 )
...
* Add mater metrics
* fix UT
* Add url to mysql profile
* Add worker metrics
* Update grafana config
* Add system metrics doc
* Add process failover counter
* Add metrics image
* Change jpg to png
* Add command insert metrics
* Fix UT
* Revert UT
2 years ago
3ecbee3885
Optimize MasterServer, add MasterRPCService ( #10371 )
...
* Optimize MasterServer, avoid NPE
2 years ago
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup ( #10373 )
2 years ago
2d3be6b36c
Add dolphinscheduler-scheduler module ( #10360 )
...
* Add dolphinscheduler-scheduler module
3 years ago
d80cf21456
Clean up unused dependencies and packaging issues ( #9944 )
3 years ago
49979c658e
[Fix-8828] [Master] Assign tasks to worker optimization ( #9919 )
...
* fix 9584
* master recall
* fix ut
* update logger
* update delay queue
* fix ut
* remove sleep
Co-authored-by: 进勇 <lijinyong@cai-inc.com>
Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
3 years ago
aba5f8a40e
[improve] Change Mysql Driver ( #10220 )
3 years ago
ee2b855ced
[BUG][TaskGroup] Task group does not take effect ( #10093 )
...
* fix 10092: Task group does not take effect
* fix 10092: Task group does not take effect
* fix 10092: Task group does not take effect
3 years ago
5bb1eb04fc
[Feature][plugin] Add k8s task in task plugin ( #9425 )
...
* [Feature][plugin] Add k8s task in task plugin
* [Feature][plugin] fix dos and code problems
* [Feature][plugin] refactor some code based on sonar
* [Feature][UI] front-end for k8s task plugin
* [Feature][plugin] delete some front files
* [Feature][plugin] update document
* Update docs/docs/zh/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* Update docs/docs/en/guide/task/k8s.md
* [Feature][UI] front-end change from review comments
* [Feature][UI] replace get namespace list api
* [Feature][plugin] change file name
* Add kubernetes to zh task list
Co-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
3 years ago
225cb332d1
[Fix-10049] Conditions Task branch flow failed ( #10077 )
3 years ago
0cc0ee77fa
[Bug][Master] fix master task failover ( #10065 )
...
* fix master task failover
* ui
3 years ago
dbdbfeaeee
[Fix-9975] The selected task instance was recreated when the Master service fail… ( #9976 )
...
* The selected task instance was recreated when the Master service failed over.
* Returns the expression result directly.
* Use Recovery to determine whether to use the old task instance.
3 years ago
d4aeee16e5
[Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement ( #9952 )
...
* fix cmd param to overwrite global param when executing complement
* fix cmd param to overwrite global param when executing complement
3 years ago
8562f6a878
[Feature][Log]Add timezone information in log output ( #9913 )
3 years ago
a1b6b033ad
[Fix-9906] After the serial wait execution strategy stops the running workflow instance, the instance will be woken up and executed if there is a wait instance. ( #9907 )
...
* After the serial wait execution strategy stops the running workflow instance, the instance will be woken up and executed if there is a wait instance.
* clear logic
* Resource overloading
3 years ago
fb0f96ed94
[Fix-9868] A task flow definition isolates the runs of different execution strategies by version numbers. ( #9869 )
...
* The thread cache task flow definition should get the latest version.
* Coverage on New Code
* Coverage on New Code
* Coverage on New Code
* use an existing method.
* Increase unit test coverage.
* Task flow definitions enforce policy isolation.
3 years ago
31cd1b5e61
Serial wait for subsequent execution ( #9847 )
3 years ago
3cea039239
Task queue status update. ( #9832 )
3 years ago
5c0be8a3d7
A task instance that normally queries the serial wait state. ( #9777 )
...
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
3 years ago
897d7cb555
Add the host address of the execution server to the sub task task instance. ( #9758 )
...
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
3 years ago
de50f43de6
[common] Make dolphinscheduler_env.sh work when start server ( #9726 )
...
* [common] Make dolphinscheduler_env.sh work
* Change dist tarball `dolphinscheduler_env.sh` location
from `bin/` to `conf/`, which users could finish their
change configuration operation in one single directory.
and we only need to add `$DOLPHINSCHEDULER_HOME/conf`
when we start our sever instead of adding both
`$DOLPHINSCHEDULER_HOME/conf` and `$DOLPHINSCHEDULER_HOME/bin`
* Change the `start.sh`'s path of `dolphinscheduler_env.sh`
* Change the setting order of `dolphinscheduler_env.sh`
* `bin/env/dolphinscheduler_env.sh` will overwrite the `<server>/conf/dolphinscheduler_env.sh`
when start the server using `bin/dolphinsceduler_daemon.sh` or `bin/install.sh`
* Change the related docs
3 years ago
7bcec7115a
[Fix-9717] The failure policy of the task flow takes effect ( #9718 )
...
* Failure policy takes effect.
* Coverage on New Code
* correct description logic
* Compatible with all scenarios
* clearer logic
Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>
3 years ago
5657cb9aec
[Bug-9719][Master] fix failover fail because task plugins has not been loaded ( #9720 )
3 years ago
ebc4253d50
[fix][Service] BusinessTime should format with schedule timezone ( #9714 )
...
* BusinessTime should format with schedule timezone
* fix test error
* fix test error
* fix test error
3 years ago
88d2803fe1
fix task dispatch error overload resource pool of task group ( #9667 )
3 years ago
63638601b0
fix process pause and rerun ( #9568 )
3 years ago
508ed9769a
[Fix][Master Server] handle warn+failed timeout strategy in workflow execute thread of master server ( #8077 ) ( #9485 )
...
Co-authored-by: xudong.zhang <xudong.zhang@nio.com>
3 years ago
3815a86a3b
[Improvement][Master] Fix typo for MasterTaskExecThreadTest ( #9513 )
3 years ago
66d148872d
[Bug-9501][Worker] fix kill task error before running ( #9509 )
3 years ago
706cdb6a8e
[Fix-9174] [Alert] Fix deduplication of alarm information ( #9371 )
...
* feat(issue #9174 ):
Fix-9174
3 years ago
69923546a1
[DS-9263][Improvement][master]optimize failover ( #9281 )
...
- add FailoverService.java
- move failover method from MasterRegistryClient to FailoverService
- move failover code from FailoverExecuteThread to FailoverService
This closes #9263
3 years ago
b285ccf930
[Future-9396]Support output parameters transfer from parent workflow to child work flow ( #9410 )
...
* [Future-9396]Support output parameters transfer from parent workflow to child work flow
* fix note
3 years ago
dce3c132ca
[DS-9387][refactor]Remove the lock in the start method of the MasterRegistryClient class ( #9389 )
3 years ago
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT ( #9223 ) ( #9299 )
3 years ago
40e44f6bc4
fix repeated submit task ( #9323 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
3e5781b6e1
[Fix-9065] [master] when task submit failed, remove from queue and add state event to thread ( #9186 )
...
* [Fix-9065] [master] when task submit failed, remove from queue and add state event to thread
This closes #9065
* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java
yes, your idea is better
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* [Bug-9065] [master] update logger info
Co-authored-by: guoshupei <guoshupei@lixiang.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
3 years ago
1073fcae44
[Bug-9295][Master] fix repeated submit task ( #9304 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
xiangzihao
Wenjun Ruan
WangJPLeo
Eric Gao
hstdream
kezhenxu94
JinYong Li
旺阳
BaoLiang
He Zhao
caishunfeng
Tq
Paul Zhang
Jiajie Zhong
gaojun2048
sparklezzz
czeming
worry
guoshupei