* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
* rebase dev
* change task state to dispatch if dispatch success
* update task host when dispatch
* add dispatch task event
* test
* check tenant after enable auto create
* handle dispatch state
Co-authored-by: caishunfeng <534328519@qq.com>
* date convert of timezone
* remove @JsonFormat
* add unit test
* fix time preview in scheduler
* optimization & add env config
Co-authored-by: caishunfeng <534328519@qq.com>
* add task group
* modify task group
* pull dev
* add license header
* modify code style
* fix code style
* fix sql error
* fix error
* fix test
* fix test
* fix test
* fix test
* fix code style
* fix ut
* code style
* fix unit test
* test ut
* ut
* add unittest
* test ut
* modify back ut
* majorization code
* fix conflict
* fix ut
Co-authored-by: wangxj <wangxj31>
* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)
* [Bug][WorkerServer] SqlTask NullPointerException #5549
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428
* fix the readme typing issue (#5998)
* Fix unchecked type conversions
* Use indentation level reported by checkstyle
* Reorganize CI workflows to fasten the wasted time and resources (#6011)
* Add standalone server module to make it easier to develop (#6022)
* Task node of SWITCH (#5939)
* [Feature-#5273][server-master] Task node of SWITCH (#5922)
Co-authored-by: wangxj <wangxj31>
* remove description of bonecp (#6030)
Co-authored-by: shaojwu <shaojwu@ebay.com>
* [Improvement][Api Module]split alert group list-paging interface (#5941)
* [Improvement][Api Module]split alert group list-paging interface
* [FIX-#6007]Wrong complement date (#6026)
* [FIX-#6007]Wrong complement date
* [style]Wrong complement date
* [Improvement-6024][dist] Remove useless packaging commands (#6029)
·Remove useless packaging commands in dolphinscheduler-bin.xml
This closes#6024
Co-authored-by: mask <liuhu@zhiyoutec.com>
* [FIX-5908][MasterServer] When executing an compensation task, the execution thread would have a NPE (#5909)
* fix the npe in MasterExec
* fix the compile error
* Add `.asf.yaml` to easily set the GitHub metadata (#6035)
* fix dead server cannot stop (#6046)
* Enhancement Translation (#6042)
* replaced Loading... with i18n
* modified Edit zh_CN translation
* Delete zh_CN.js
Co-authored-by: David <dailidong66@gmail.com>
* fix bug #6053 zh_CN.js is lost
* [Fix-6038][ui] width of "SQL Statement" in Dag FormLineModal will be shrunk if sql line is too long (#6040)
This closes#6038
* [Improvement] Fix inefficient map iterator (#6004)
* Fix inefficient map iterator
* Use forEach and remove call to valueOf
* Modify AbstractParameters
* Enhance `StandaloneServer` so that we don't need to update the version number manually (#6074)
* Remove invalid character in `.asf.yaml` (#6075)
* Remove invalid character `\n` in `.asf.yaml` (#6077)
It turns out that the invalid character is `\n`
* Add alert server into standalone-server as well and some minor polish (#6087)
* Support starting standalone server in Docker image (#6102)
Also remove unused class
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095)
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)
* master refactor:
1. spi for task submit and other actions(pause, kill)
2. remove threads for process instance and task instance.
3. add events for process instance and task instance
* ut npe
* add try catch
* code style
* fix critical bugs
* fix critical bugs
* fix critical bugs
* fix critical bugs
* Remove unused params in SwitchTaskTest (#6109)
* [Feature-5987][Server] Support to set multiple environment configs for a certain worker. (#6082)
* support multi environments
* add some test cases
* add an environment vue component
* improve environment form
* improve environment form
* add environment worker group relation
* add environment worker group relation
* add the environment choice for formModel
* set an environment for the task
* modify the modal form of starting process
* add the environment config to TaskExecutionContext
* add the environment config to the timing form
* fix conflicts
* fix issues of the code style
* fix some issues of the code style
* fix some issues of the code style
* fix some issues of the code style
* fix some issues of the code style
* fix some issues of the code style
* fix some bugs in the code review
* add the same table and columns to support H2
* fix some bugs
* [Plugin][Task]Task Spi
* [Plugin][Task]Task Spi
* [Plugin][Task]Task Spi
* [Plugin][Task]Task Spi
* [Plugin][Task]Task Spi
* [Plugin][Task]Task Spi
Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: myangle1120 <942542838@qq.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: shaojwu <shaojwu@ebay.com>
Co-authored-by: Shukun Zhang <60541766+andream7@users.noreply.github.com>
Co-authored-by: linquan <1175687813@qq.com>
Co-authored-by: mask <39329477+Narcasserun@users.noreply.github.com>
Co-authored-by: mask <liuhu@zhiyoutec.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: RichardStark <49977764+RichardStark@users.noreply.github.com>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: lenboo <baoliang.leon@gmail.com>
Co-authored-by: lilyzhou <lj_zhou@outlook.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
Co-authored-by: junfan.zhang <zuston.shacha@gmail.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
* [BUG-#5678][Registry]fix registry init node miss (#5686)
* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)
* improve
edit the userinfo success, but the updatetime is not the latest.
* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)
Co-authored-by: shenglm <shenglm840722@126.com>
* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)
* add globalParams new plan with varPool
* add unit test
* add python task varPoolParams
Co-authored-by: wangxj <wangxj31>
* Issue robot translation judgment changed to Chinese (#5694)
Co-authored-by: chenxingchun <438044805@qq.com>
* the update function should use post instead of get (#5703)
* enhance form verify (#5696)
* checkState only supports %s not {} (#5711)
* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)
* update
* fix the codestyle error
* fix the compile error
* support rollback
* [Fix-5699][UI] Fix update user error in user information (#5700)
* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)
* bug fix
the auto generated spi service can't be recongized
* include a new method
* [Improvement-5622][project management] Modify the title (#5723)
* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)
* add a new init method.
* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)
* add the necessary parameters
* openapi improve
* fix code style error
* [FIX-#5721][master-server] Global params parameter missing (#5757)
Co-authored-by: wangxj <wangxj31>
* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move` doesn't work. (#5739)
* Update relatedItems.vue
* Update relatedItems.vue
* [Improvement#5741][Worker] Improve task process status log (#5776)
* [Improvement-5773][server] need to support two parameters related to task (#5774)
* add some new parameter for task
* restore official properties
* improve imports
* modify a variable's name
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE
* [Improvement][Worker] Task log may be lost #5775 (#5783)
* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)
* [Imporvement #5725][CheckStyle] upgrade checkstyle file
Upgrade checkstyle.xml to support checkstyle version 8.24+
* change ci checkstyle version
* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected. (#5796)
* improve timestamp format
make the startime in the log of httptask to be easier to read.
* fix bad code smell and update the note.
* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)
·the list of workflow instances is sorted by start time and end time
·This closes#5621
* fix (#5803)
Co-authored-by: shuangbofu <fusb@tuya.com>
* fix: Remove duplicate "registryClient.close" method calls (#5805)
Co-authored-by: wen-hemin <wenhemin@apache.com>
* [Improvement][SPI] support load single plugin (#5794)
change load operation of 'registry.plugin.dir'
* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)
* fix: refactor registry client, remove spring annotation
* fix UT
* fix UT
* fix checkstyle
* fix UT
* fix UT
* fix UT
* fix: Rename RegistryCenterUtils method name
Co-authored-by: wen-hemin <wenhemin@apache.com>
* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)
* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)
* enhance the validation in UI
* enchance form verifaction
* simplify disable condition
* fix: Remove unused class (#5833)
Co-authored-by: wen-hemin <wenhemin@apache.com>
* [fix-5737] [Bug][Datasource] datsource other param check error (#5835)
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
* [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes
[Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes
* [Fix-5825][BUG][WEB] the resource tree in the process definition of latest dev branch can't display correctly (#5826)
* resoures-shows-error
* fix codestyle error
* add license header for new js
* fix codesmell
* [Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867)
* provide two system parameters to support the rest of type of tasks
* provide two system parameters to support the rest of type of tasks
* improve test conversion
* [Improvement][Fix-5769][UI]When we try to delete the existing dag, the console in web browser would shows exception (#5770)
* fix bug
* cache the this variable
* Avoid self name
* fix code style compile error
* [Fix-5781][UT] Fix test coverage in sonar (#5817)
* build(UT): make jacoco running in offline-instrumentation
issue: #5781
* build(UT): remove the jacoco agent dependency in microbench
issue: #5781
* [Fix-5808][Server] When we try to transfer data using datax between different types of data sources, the worker will exit with ClassCastException (#5809)
* bug fix
* fix bug
* simplify the code format
* add a new parameter to make it easier to understand.
* [Fix-5830][Improvement][UI] Improve the selection style in dag edit dialog (#5829)
* improve the selection style
* update another file
* remove unnecessary css part.
* [Fix-5904][upgrade]fix dev branch upgrade mysql sql script error (#5821)
* fix dev branch upgrade mysql sql script error.
* Update naming convention.
* [Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832)
* fix: refactor api utils class, remove spring annotation.
* fix: Optimization comments
Co-authored-by: wen-hemin <wenhemin@apache.com>
* correct the wrong annotion from zk queue implemented to java priority blocking queue (#5906)
Co-authored-by: ywang46 <ywang46@paypal.com>
* Add a Gitter chat badge to README.md (#5883)
* Add Gitter badge
* Update README.md
Co-authored-by: David <dailidong66@gmail.com>
* ci: improve maven connection in CI builds (#5924)
issue: #5921
* [Improvement][Master]fix typo (#5934)
·fix typo in MasterBaseTaskExecThread
* [Fix-5886][server] Enhanced scheduler delete check (#5936)
* Add:Name verification remove the first and last spaces.
* Update: wrong word: 'WAITTING' ->'WAITING'
* Add: Strengthen verification
Co-authored-by: Squid <2824638304@qq.com>
* [Improvement-5880][api] Optimized data structure of pagination query API results (#5895)
* [5880][refactor]Optimized data structure of pagination query API results
- refactor PageInfo and delete returnDataListPaging in API
- modify the related Controller and Service and the corresponding Test
* Merge branch 'dev' of github.com:apache/dolphinscheduler into dev
Conflicts:
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
Co-authored-by: 蔡泽华 <sorea1k@163.com>
* [IMPROVEMENT]fix mysql comment error (#5959)
* [Improvement][Api]fix typo (#5960)
* [Imporvement #5621][job instance] start-time and end-time (#5621)
·the list of workflow instances is sorted by start time and end time
·This closes#5621
* [FIX-5975]queryLastRunningProcess sql in ProcessInstanceMapper.xml (#5980)
* [NEW FEATURE][FIX-4385] compensation task add the ability to configure parallelism (#5912)
* update
* web improved
* improve the ui
* add the ability to configure the parallelism
* update variables
* enhance the ut and add necessary note
* fix code style
* fix code style issue
* ensure the complation task in parallel mode can run the right numbers of tasks.
* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)
* [Bug][WorkerServer] SqlTask NullPointerException #5549
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428
* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428
* fix the readme typing issue (#5998)
* Fix unchecked type conversions
* Use indentation level reported by checkstyle
* Reorganize CI workflows to fasten the wasted time and resources (#6011)
* Add standalone server module to make it easier to develop (#6022)
* Task node of SWITCH (#5939)
* [Feature-#5273][server-master] Task node of SWITCH (#5922)
Co-authored-by: wangxj <wangxj31>
* remove description of bonecp (#6030)
Co-authored-by: shaojwu <shaojwu@ebay.com>
* [Improvement][Api Module]split alert group list-paging interface (#5941)
* [Improvement][Api Module]split alert group list-paging interface
* [FIX-#6007]Wrong complement date (#6026)
* [FIX-#6007]Wrong complement date
* [style]Wrong complement date
* [Improvement-6024][dist] Remove useless packaging commands (#6029)
·Remove useless packaging commands in dolphinscheduler-bin.xml
This closes#6024
Co-authored-by: mask <liuhu@zhiyoutec.com>
* [FIX-5908][MasterServer] When executing an compensation task, the execution thread would have a NPE (#5909)
* fix the npe in MasterExec
* fix the compile error
* Add `.asf.yaml` to easily set the GitHub metadata (#6035)
* fix dead server cannot stop (#6046)
* Enhancement Translation (#6042)
* replaced Loading... with i18n
* modified Edit zh_CN translation
* Delete zh_CN.js
Co-authored-by: David <dailidong66@gmail.com>
* fix bug #6053 zh_CN.js is lost
* [Fix-6038][ui] width of "SQL Statement" in Dag FormLineModal will be shrunk if sql line is too long (#6040)
This closes#6038
* [Improvement] Fix inefficient map iterator (#6004)
* Fix inefficient map iterator
* Use forEach and remove call to valueOf
* Modify AbstractParameters
* Enhance `StandaloneServer` so that we don't need to update the version number manually (#6074)
* Remove invalid character in `.asf.yaml` (#6075)
* Remove invalid character `\n` in `.asf.yaml` (#6077)
It turns out that the invalid character is `\n`
* Add alert server into standalone-server as well and some minor polish (#6087)
* Support starting standalone server in Docker image (#6102)
Also remove unused class
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095)
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)
* master refactor:
1. spi for task submit and other actions(pause, kill)
2. remove threads for process instance and task instance.
3. add events for process instance and task instance
* ut npe
* add try catch
* code style
* fix critical bugs
* fix critical bugs
* fix critical bugs
* fix critical bugs
* Remove unused params in SwitchTaskTest (#6109)
* fix ut
* fix ut
* fix ut
* fix ut
* fix server module ut
* fix ut
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
Co-authored-by: AzureCN <colorazure@163.com>
Co-authored-by: 深刻 <tsund@qq.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Yao WANG <Yao.MR.CN@gmail.com>
Co-authored-by: ywang46 <ywang46@paypal.com>
Co-authored-by: The Gitter Badger <badger@gitter.im>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: Squidyu <1297554122@qq.com>
Co-authored-by: Squid <2824638304@qq.com>
Co-authored-by: soreak <60459867+soreak@users.noreply.github.com>
Co-authored-by: 蔡泽华 <sorea1k@163.com>
Co-authored-by: yimaixinchen <yimaixinchen@163.com>
Co-authored-by: atai-555 <74188560+atai-555@users.noreply.github.com>
Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: myangle1120 <942542838@qq.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: shaojwu <shaojwu@ebay.com>
Co-authored-by: Shukun Zhang <60541766+andream7@users.noreply.github.com>
Co-authored-by: linquan <1175687813@qq.com>
Co-authored-by: mask <39329477+Narcasserun@users.noreply.github.com>
Co-authored-by: mask <liuhu@zhiyoutec.com>
Co-authored-by: RichardStark <49977764+RichardStark@users.noreply.github.com>
Co-authored-by: lenboo <baoliang.leon@gmail.com>
Co-authored-by: lilyzhou <lj_zhou@outlook.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
Co-authored-by: junfan.zhang <zuston.shacha@gmail.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
* [BUG-#5678][Registry]fix registry init node miss (#5686)
* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)
* improve
edit the userinfo success, but the updatetime is not the latest.
* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)
Co-authored-by: shenglm <shenglm840722@126.com>
* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)
* add globalParams new plan with varPool
* add unit test
* add python task varPoolParams
Co-authored-by: wangxj <wangxj31>
* Issue robot translation judgment changed to Chinese (#5694)
Co-authored-by: chenxingchun <438044805@qq.com>
* the update function should use post instead of get (#5703)
* enhance form verify (#5696)
* checkState only supports %s not {} (#5711)
* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)
* update
* fix the codestyle error
* fix the compile error
* support rollback
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
Aaron Wang
caishunfeng
Eric Gao
Yann Ann
kezhenxu94
sgw
Wenjun Ruan
zhuxt2015
WangJPLeo
JinYong Li
guoshupei
Kerwin
wangyang
yimaixinchen
xuhhui
wind
wangxj3
lyq
Kirs
OS
ruanwenjun
QuakeWang