wind
e6239e808b
[DS-6829][WorkerServer] skip create log dir and print log in dryRun model ( #6852 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
wind
28872d8706
[DS-6891][MasterServer] reduce db operation when process instance running ( #6904 )
...
* remove taskInstanceCacheManager
* remove taskInstanceCacheManager
* [DS-6891][MasterServer] reduce db operation when process instance running
* [DS-6891][MasterServer] reduce db operation when process instance running
* fix test
* fix Transactional method call
* checkstyle
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
wind
595e4843d0
[DS-6849][MasterServer] fetch more commands and handle in parallel ( #6850 )
...
* [DS-6849][MasterServer] fetch more commands and handle in parallel
* add return
* handle command with check
* add test
* delete master prefix
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
OS
653eae2419
[BUG][Master] process cannot finish and its status is always running ( #6886 )
...
* fix 6882: process cannot finish and its status is always running
* fix 6882: process cannot finish and its status is always running
3 years ago
chouc
f564687a89
[Improvement][Log] optimize task log path to reduce task running time ( #6731 )
...
* [Improvement][Log] optimize task log path to reduce task running time
* Reduce the creation of log directories
Co-authored-by: Kirs <acm_master@163.com>
3 years ago
KyoYang
94f6c5c27e
fix complement data retry bug; issues: [Bug] [Master] A bug on task retry mechanism #6613 ( #6843 )
...
Co-authored-by: yangqiyu <yangqiyu@deepexi.com>
3 years ago
kezhenxu94
4114cb07f6
Add actuator and enable metrics, add Grafana configs ( #6840 )
3 years ago
Jiajie Zhong
6045728428
[ci] Add checkstyle prohibit rule System.out.println ( #6778 )
...
* [ci] Add checkstyle prohibit rule System.out.println
* Correct RegexpSingleline place
3 years ago
wind
5741c758b3
[DS-6771][MasterServer] fix failover worker interrupt ( #6801 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
3 years ago
mask
e36d18b588
[Fix-6792] resolve API and alert port conflicts ( #6792 ) ( #6815 )
...
* [Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792 )
3 years ago
kezhenxu94
11844415b0
Refactor datasource and task plugins and simplify its usage ( #6767 )
3 years ago
mask
0aea5ae11a
[feature-6071][plugin] add datasource plugin. ( #6071 ) ( #6361 )
...
* [feature-6071][plugin] add datasource plugin. (#6071 )
* remove the dependency of datasource in common and task-plugin module.
* the new datasource-api module provides data source connection acquisition.
* add sqlserver data source plug-in.
* add ds license
* fix check style
* update spi pom
* frx conflict resolution
* fix check style
* fix datasource ut
* fix code check style
* fix code check style
* fix datasource ut
* add pg hive mysql oracle clickhouse db2 datasource
* fix datasource package
* update datasource pom
* update datasource pom
* update datasource pom
* update datasource pom
* update pom
* update pom dep
* update pom dep
* fix license
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* add oracle package
* modify comments
* conflict resolution
* rename log comment
* rename log comment
3 years ago
kezhenxu94
05bae0b226
Refactor alert plugins and simplify its usage ( #6723 )
3 years ago
kezhenxu94
02350ea37b
Fix datasource not work in tests and standalone server, and run all tests by default ( #6743 )
3 years ago
kezhenxu94
63602bf77c
Clean up the utility codes ( #6732 )
3 years ago
wind
b6824b4741
[DS-6737][MasterServer] fix event handle twice ( #6738 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
OS
24bd8a16e1
[Bug][Master] The task of dependent should use taskCode ( #6730 )
...
* fix #6682 The task of dependent should use taskCode
* fix #6682 The task of dependent should use taskCode
3 years ago
Wenjun Ruan
0afb780718
[Improvement-5807][Dao] Use h2 in UT ( #6500 )
...
* Use H2 in UT
* change scope
* Change profile name to h2
* Add datasource-h2.properties, datasource-postgresql.properties
* Fix UT
* Resolve conflict
* Remove startDatabase
3 years ago
kezhenxu94
5400abc74f
Refactor registry plugin and simplify its usage ( #6712 )
3 years ago
Yao WANG
73f20b7553
[Fix-#6641]fix npe when the LowerWeightHostManager can not get the workNodeInfo ( #6656 )
...
* change the return type of getHostWeight
Co-authored-by: ywang46 <ywang46@paypal.com>
3 years ago
wind
72daba58cd
[DS-6694][Master] Reduce the refresh resource interval of LowerWeight ( #6695 )
...
* [DS-6694][Master] Reduce the refresh resource interval of LowerWeight
* add server node update event handle
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
wangxj3
dc8b87e473
[Feature-#6268][server-master] Serial execte process ( #6185 )
...
* add serial processInstance
* fix bug
* add test
* fix code style
* fix style code
* add sql
* fix sql error
* add api
* add test
* fix code style
* modify api
* delete column , fix mapper
* fix unimport
* fix test
* fix bug of missing param for Python
* fix code style
* fix test
* fix test
Co-authored-by: wangxj <wangxj31>
3 years ago
xuhhui
2b90d90873
improve install_config.conf description ( #6681 )
...
* improve install_config.conf description
* Update install_config.conf
update accroding review ideas
3 years ago
wind
ae0b18f3a3
[DS-6638][MasterServer] fix task state no change when failover worker ( #6639 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
wind
8850baff07
[DS-6640][WorkerServer] support process update host command type ( #6642 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
wind
06e8e24708
[DS-6616][WorkerServer] fix worker stop fail and fakes death ( #6621 )
...
* [DS-6616][WorkerServer] fix worker stop fail and fakes death
* remove unuse test
* just add try catch to remove zk worker path
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
wind
b642e9ca54
[DS-6582][Master] fix condition task state ( #6583 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
mask
9b1984c244
[fix-6570][task] add sql task udf resource load ( #6570 ) ( #6571 )
...
* [fix-6570][task] add sql task udf resource load (#6570 )
-add sql task udf resource load
-Fix UdfFuncRequest serialization
* fix checkstyle
* add defaultFS in sqlTaskContext
* fix checkstyle
3 years ago
ououtt
a3982f5025
[Improvement-6540][server] Optimize the code related to the processInstanceExecMaps object ( #6556 )
...
* [DS-6540][refactor-server] Optimize the code related to the processInstanceExecMaps object
3 years ago
Kirs
de61d65af2
[Task]fix business log using error ( #6538 )
3 years ago
OS
52a550b6ae
[BUG-6543][Master] process instance state is always running when failure task ( #6547 )
...
* fix-6543: process instance state is always running when failure task exists
* code style
* code style
3 years ago
Jiajie Zhong
6495a204f3
Fix install.sh for DS 2.0 and add comment to install_config.conf ( #6506 )
...
* Fix install.sh for DS 2.0
* Use #* make it work in macos
* Add comment for properties change
* Add JAVA HOME for only one file change to deploy
* Reduce config change by using default value of: plugin registry, alert, task as
* Change javaHome default value
* Delete config param zkQuorum
* Reorder param
* Add comment for install_config.conf
3 years ago
wind
50f1766903
[DS-6499][WorkerServer] report busy state when worker is overload ( #6512 )
...
* [DS-6499][WorkerServer] report busy state when worker is overload
3 years ago
OS
db04a5b04d
[Feature-6471]Cache Process definition in master ( #6485 )
...
* feature-6471 Cache Process definition in master
3 years ago
wind
e091801e05
[Improvement-6474] [MasterServer] schedule time for process instance optimization ( #6477 )
...
* [DS-6474][MasterServer] change to handle schedule time for process instance in WorkflowExecuteThread
* delete all the valid tasks when complement data if id is not null
* checkstyle
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
OS
a8baa9553f
[Bug-6455][Master]fix bug 6455: cannot stop sub-task ( #6458 )
...
* fix bug: cannot stop the task.
* fix bug: cannot stop the task.
* remove the check thread number
3 years ago
Hua Jiang
fac6b4afd3
[Fix-5875][API] When I saved the task that had the same name task in another flow ,the service would throw DuplicateKeyException ( #6430 )
...
* using the task code in the dag
3 years ago
Kirs
ea2a8d26a0
[Improvement#6438][Worker]remove meaningless DB query ( #6440 )
...
org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread
In the process of task kill, it will query the DB to obtain the task instance object, but this judgment does not make any sense. In fact, the relevant context is already clear, so there is no need to query from DB.
3 years ago
百岁
d284e44d52
Rename tis to pigeon from backend layer ( #6437 )
...
* add mock getTaskLogName() for UT test pass
* make task-tis rename to task-pigeon
* rename module name 'tis' to pigeon
* rename module name 'tis' to pigeon,make sanity check pass
* remove execute console.log()
3 years ago
Shukun Zhang
6e453de241
[Feature] workflow level task dry run ( #6104 )
...
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
* feat workflow level task dry run
3 years ago
caishunfeng
ea6503b997
[DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue ( #6406 )
...
* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue
* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue
* [DS-6403][fix] Put DB queries ahead before submit to TaskPriorityQueue
* checkstyle
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
RichardStark
d184ebda1e
issue-6298-added-the-task-plugin ( #6299 )
3 years ago
OS
c013b49e72
[Fix-6347][Master-API]fix bug #6347 the first schedule_time is error in complement data ( #6389 )
...
* fix bug #6347 complement data errors
3 years ago
Kirs
67508d8d1b
update project version to 2.0.0-SNAPSHOT ( #6383 )
3 years ago
caishunfeng
67094d0f7f
[BUG-6353][MasterServer] fix event handle slowly ( #6375 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
Wenjun Ruan
ebb56b2620
fix dead loop in StateEventResponseWorker ( #6281 )
3 years ago
OS
20a3741b24
fix bug #6347 the first schedule_time is error in complement data ( #6351 )
3 years ago
jon-qj
dee191f2bb
provide the default value(task.plugin.dir) ( #6340 )
3 years ago
Kirs
be158328fb
[Bug #6306 ][Worker] task log path generated error. ( #6338 )
3 years ago
caishunfeng
89cc045961
[DS-6323][fix] EventExecuteService run loop without sleep which make cpu high ( #6324 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago