* [Feature-3134]Refactor to use a shared singleton Curator Zookeeper Client
* autowire CuratorZookeeperClient to ZookeeperOperator
* [Improvement] Add the issue specifications reference (#3221)
* log zookeeper address when the connect state change
* resume the operation of add connect state listener in MasterRegistry
Co-authored-by: Yichao Yang <1048262223@qq.com>
* [Feature-2815]cherry pick commit from dev for one worker can belong to different worker groups
* fix 2975:download resource need find the tenant code of resource owner
* change NetUtils to OSUtils because NetUtils didn't exist in the 1.3.2-release branch
Co-authored-by: Yichao Yang <1048262223@qq.com>
* fix the bug #2923 (Hive JDBC connection parameter ignored) , the hive jdbc connection url ;
* fix the bug index out of bound and add some test case
* add the Licensed
* fix the checkstyle and import StringBuilder
* put HiveDataSourceTest.java to root pom maven-surefire-plugin to avoiding the sonar 0.0% Coverage.
* Update HiveDataSource.java
* fix the bug #3240 [worker] Cannot find a (Map) Key deserializer for type
* [DS-3245][fix] change toString use JsonString
- change the method of UdfFunc.toString use JsonString
This closes#3245
* [DS-3245][fix] change toString use JsonString
- fix the junit test for full
This closes#3245
* [DS-3245][fix] change toString use JsonString
- fix the sonarcloud merge error, Swap these 2 arguments so they are in the correct order: expected value, actual value.
This closes#3245
* [DS-3245][fix] move the Class UdfFuncDeserializer to UdfFunc.java
move the class UdfFuncDeserializer.java to UdfFunc.java
This closes#3245
* [DS-3245][fix] move the Class UdfFuncDeserializer to UdfFunc.java
add blank line
This closes#3245
* [DS-3245][fix] add the unit test of UdfFunc
add the unit test of UdfFunc.java
This closes#3245
* [DS-3245][fix] add the unit test of UdfFunc
remove the UdfFuncTest.java from maven-surefire-plugin list
This closes#3245
* [DS-3245][fix] add the unit test of UdfFunc
add the UdfFuncTest to the root pom maven-surefire-plugin
This closes#3245
Co-authored-by: dailidong <dailidong66@gmail.com>
* [Feature-2815][server] One worker can belong to different workergroups
* Feature: Add test cases
* Update MonitorService.java
* Update MonitorService.java
Co-authored-by: dailidong <dailidong66@gmail.com>
* merge from 1.3.1-release
* [bugfix] remove Conflicting configuration sudo and user (#3038)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* remove some standard notices in order to cut down the NOTICE file. (#3101)
* #3084 remove some standard notices in order to cut down the NOTICE file.
* update exclude element sql/soft_version of rat plugin
* update soft_version
* merge from 1.3.1-release
* fix 3112:download udf resource need find the tenant code of resource owner (#3114)
* fix 3112:download udf resource need find the tenant code of resource owner
* fix 2975:download resource need find the tenant code of resource owner
* remove useless code
* merge from 1.3.1-release
* remove some notes (#3127)
Co-authored-by: lenboo <baoliang@analysys.com.cn>
* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)
* fix Permission denied if not found dolphinscheder-env.sh
* update some comments on method
* add PropertyPlaceholderHelper on license (#3129)
* update license header location
* add PropertyPlaceholderHelper on license
* merge from 1.3.1-release
* change from 1.3.0 to 1.3.2
* add Set import
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
* In the case of multiple network cards, a valid host is returned
* In the case of multiple network cards, a valid host is returned
* In the case of multiple network cards, a valid host is returned
* In the case of multiple network cards, a valid host is returned
* fix LOCAL_ADDRESS not assigned value
* fix LOCAL_ADDRESS not assigned value
* fix LOCAL_ADDRESS not assigned value
* fix lock error
* fix lock error
* end
* end
* end
* end
* code smell
* code smell
* code smell
* code smell
* code smell
* code smell
* code smell
* lowering
* messy
* small change
* compile error
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
* fix 3112:download udf resource need find the tenant code of resource owner
* fix 2975:download resource need find the tenant code of resource owner
* remove useless code
* add DolphinScheduler slack channel
* Update README.md
Co-Authored-By: Jiajie Zhong <zhongjiajie955@hotmail.com>
* modify soft_version 1.2.2 to 1.3.0
* Repair worklist cannot be deleted
* update schema version (#2657)
* change schema version from 1.2.2 to 1.3.0
* change version value of table t_ds_version from 2.0.0 to 1.3.0
* fix bug: sub worker flow in complement data would run start date (#2581) (#2656)
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* Fix conditions node save exception
* add unique key for t_ds_process_definition
* add unique key for t_ds_process_definition
* script variable has "processDefinitionId" is error #2664 (#2666)
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* update filename
* Update dolphinscheduler_ddl.sql
* Delete dolphinscheduler_ddl.sql
* Delete dolphinscheduler_ddl.sql
* add unique key for t_ds_process_definition
add unique key for t_ds_process_definition
* add unique key for t_ds_process_definition
add unique key for t_ds_process_definition
* Modify the alert log file
* blank in eamil and left font align #2648 (#2673)
* script variable has "processDefinitionId" is error #2664
* blank in eamil and left font align #2648
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* fix zookeeper health check and merge volumes into logs volume (#2674)
* fix zookeeper health check and merge volumes into logs volume
1. Fix zookeeper health check
2. Merge volumes into logs volume
3. Modify `dockerfile/hooks/build` script
* Modify ci_e2e and docker image registry
* Modify ci_e2e
* Modify ci_e2e
* Modify ci_e2e and move ./dockerfile/hooks/check to ./docker/docker-swarm/
* migrate docker composer and move dockerfile dir to docker/build
* ut docker
* alter dockerfile to docker/build dir
* delete docker-compose old
* delete dockerfile zk
* delete dockerfile zk
* delete dockerfile zk
* [bug fix] some bugs from 1.2.0 to 1.3.0 (#2676)
* fix bug: some bugs about worker group from 1.2.0 to 1.3.0
* fix bug: scheduler from 1.2.0 to 1.3.0 do not work
* fix bug: scheduler from 1.2.0 to 1.3.0 do not work
* fix ut bug
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* add worker group config (#2681)
* update README about DolphinScheduler
* Update issue templates
* update
* regularize api pom xml
update rpc maven compile to 1.8
* regularize api pom xml
* change commons.lang3.StringUtils to common.utils.StringUtils
* update pom.xml
* update
* correct equals method
* jasper-runtime is needed when api server start
* jasper-runtime jar is needed when api server start
* combine logback config of master/worker/alert/api server to one logback.xml
* remove tomcat runtime jar
* add UT
* add license
* remove jasper-runtime jar, not need anymore
* sovle jar conflict， remove servlet-api 2.5 jar
* remove servlet 2.5
* add embedded database h2 and update embedded zookeeper
* delete logs
* add license
* change log level to INFO
* simplify and optimize config
* add development state
* simplify common config
* add worker group config
* add worker group config
Co-authored-by: lidongdai <escheduler@outlook.com>
Co-authored-by: dailidong <dolphinschedule@gmail.com>
* delete log files while deleting process instances #2463 (#2693)
* script variable has "processDefinitionId" is error #2664
* blank in eamil and left font align #2648
* delete log files while deleting process instances #2463
* delete log files while deleting process instances #2463
* delete log files while deleting process instances #2463
* delete log files while deleting process instances #2463
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* Update dolphinscheduler_ddl.sql
* Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml (#2692)
* Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml
1. add initdb volume for pg service in docker-compose.yml
2. modify ci_ut.yml
* add sudo in ci_ut.yml
* Query the metadata of indexes is not correct in postgresql/dolphinscheduler_ddl.sql
1. replace information_schema.STATISTICS to pg_stat_all_indexes
2. modify ProcessDefinitionMapperTest.java
* Remove inner zk and pg
* add inner pg
* conditions The task node adds a background color to distinguish successful and failed connections
* add LoggerServerTest UT (#2713)
* add LoggerServerTest UT
* add LoggerServerTest UT
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* add UT (#2717)
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* fix worker group config no effect (#2719)
* Conditional judgment adds text
* fix
* fix #2449:It's should successful to cancel authorize a resource of a user if all the online process definitions created by this user didn't use this resource. (#2725)
* Underline the list of workflow operations
* Modify HDFS/S3 path (#2736)
* Modify HDFS/S3 path
* Modify HDFS/S3 path
* Update common.properties
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
* Modify ip configuration
* set worker group (#2743)
* simple config
* fix worker group config no effect
* fix #2598::allow to update resource suffix,but if it is authorized to other users,it is not allowed (#2732)
* fix #2598:allow to update resource suffix,but if it is authorized to other users,it is not allowed.
* add RESOURCE_IS_AUTHORIZED status
* verify whether the suffix is empty
* remove extra variables
* fix code smell
* Update known-dependencies.txt
* update worker group config (#2746)
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* Update datasource.properties
* remove codehaus janino jar (#2742)
the license about janino maybe not compatiable with Apache v2
* [bug fix] fix bug:2634:Process shows success,when the task of the process is killed in the terminal (#2731)
* fix bug: some bugs about worker group from 1.2.0 to 1.3.0
* fix bug: scheduler from 1.2.0 to 1.3.0 do not work
* fix bug: scheduler from 1.2.0 to 1.3.0 do not work
* fix ut bug
* fix bug: Process shows success,when the task of the process is killed in the termina
* fix bug #2634: Process shows success,when the task of the process is killed in the terminal
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* Update known-dependencies.txt (#2748)
* fix getWorkers bug on RunConfig (#2747)
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* master select worker filter high load worker #2704 (#2733)
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* Fix rename check parameter
* fix
* fix 2598:change getId to getUserId (#2757)
* fix #2598:allow to update resource suffix,but if it is authorized to other users,it is not allowed.
* add RESOURCE_IS_AUTHORIZED status
* verify whether the suffix is empty
* remove extra variables
* fix code smell
* change getId to getUserId
* Fix MySQL autoDeserialize bug (#2728)
* Fix MySQL autoDeserialize bug
* add MySQLDataSourceTest
* Fix workflow definition history node data cannot be opened
* fix
* delete normalize
* update e2e chrome version (#2775)
Co-authored-by: chenxingchun <438044805@qq.com>
* Modify the background color of conditions task node connection
* [BUF FIX]fix: the master would be blocked when worker group not exists (#2764)
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* [BUG FIX] fix bug: dependent task failed when conditions task exists (#2768)
* fix bug 2464: change dependent task for process.
* remove unused code
* add ut
* add ut
* update comments
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* Hourly dependency configuration， 'currentHour' and 'last24Hours' (#2696)
* [feature] Add hourly dependency scenarios
* [Feature] #2680
add DependentUtilsTest UT
Co-authored-by: changqun.xu <changqun.xu@17zuoye.com>
Co-authored-by: xuchangqun <changqun.x@gmail.com>
* fix sql task bug #2739 (#2741)
* Solve the failure to execute non-query sql
* fix sql task bug
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: lgcareer <18610854716@163.com>
* Worker group adds check
* fix
* Worker group adds check (#2787)
* Worker group adds check
* fix
* add not worker log and remove worker invalid property (#2784)
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* add not worker log and remove worker invalid property
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory (#2786)
* fixed#2788 resolve the bug #2788 (#2789)
* Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory
* fixed#2788
* [BUG FIX] fix bug:cannot pause work flow when task state is "submit success" (#2783)
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
* fix bug(2781): cannot pause work flow when task state is "submit success"
* fix code smell
* add mysql other param blank judge
* test
* update comments
* update comments
* add ut
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* fix bug:#2735 (#2770)
1. store process id in zk.
2. resolve host and port through zk path.
3. modify ui key.
* Modify startup parameter worker group value
* Fix [Bug] process definition json worker group convert #2794 (#2795)
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* add not worker log and remove worker invalid property
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* [BUG FIX]fix bug: Restart the worker service again, the previously submitted successful tasks are not executed bug (#2800)
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
* fix bug(2781): cannot pause work flow when task state is "submit success"
* fix code smell
* add mysql other param blank judge
* test
* update comments
* update comments
* add ut
* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
* update comments
* add sleep
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* delete unnecessary configuration items (#2803)
* rename worker_group_id to worker_group and set the value of worker_group_name (#2801)
* process definition json worker group convert #2794 (#2809)
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* add not worker log and remove worker invalid property
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* compatible old version host not port delete log file (#2811)
* add LoggerServerTest UT
* add LoggerServerTest UT
* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* master select worker filter high load worker #2704
* add not worker log and remove worker invalid property
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* process definition json worker group convert #2794
* compatible old version host not port delete log file
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
* Added instructions for ambari to add and remove Dolphin Schdumer components on nodes (#2806)
* delete unnecessary configuration items
* Added instructions for ambari to add and remove Dolphin Schdumer components on nodes
* Added instructions for ambari to add and remove Dolphin Schdumer components on nodes
* fix oracle connect failure (#2812)
* use method getJdbcUrl of parent
* Remove the incorrect print message
* Modify process group workerGroup value display and filter child node duplicate values
* Repair process definition bulk delete function
* when init workflow, the tenantId linked by the tenantId of user
* dag download does not refresh, modify the task instance operation width value
* fix
* [bug fix]fix bug: After the master is fault-tolerant, it cannot resume operation (#2813)
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
* fix bug(2781): cannot pause work flow when task state is "submit success"
* fix code smell
* add mysql other param blank judge
* test
* update comments
* update comments
* add ut
* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
* update comments
* add sleep
* add null point check
* fix bug:After the master is fault-tolerant, it cannot resume operation
* fix bug: do not failover the host is 'NULL' process
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* When update the data source,the host and port show error (#2821)
* When update the data source,the host and port show error
* update DataSourceServiceTest
* update DataSourceServiceTest
* update DataSourceServiceTest
* Fix ORACLE service name value display
* Add tips for download workflow
* fix
* [bug fix] fix: The workflow is fault-tolerant and 2 task instances are generated (#2833)
* feature: add number configuration for master dispatch tasks
* fix bug(#2762) the master would be blocked when worker group not exists
* fix bug(#2762) the master would be blocked when worker group not exists
* fix ut
* fix ut
* fix bug(2781): cannot pause work flow when task state is "submit success"
* fix code smell
* add mysql other param blank judge
* test
* update comments
* update comments
* add ut
* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed
* update comments
* add sleep
* add null point check
* fix bug:After the master is fault-tolerant, it cannot resume operation
* fix bug: do not failover the host is 'NULL' process
* fix bug:worker failover error.
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* add maven Xmx to 5g
* fix bug: send task result to master until success. (#2839)
* fix bug: send task result to master until success.
* fix bug: send task result to master until success.
* add sleep
* set null if send result failed
* set null if send result failed
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* Fix errors when running on win10 (#2842)
1. DS cant run bat script
2. DS cant run python script
3. DS cant run datax script
* Fix abnormal display of regular worker parameters
* add source license full text into source license (#2840)
* add source license full text into source license
* exclude ScriptRunner.java on rat check
* fix table not exist when init db metadata (#2838)
* fix table not exist when init db metadata
* fix db init error
* fix ds-1.3.0 init db
* init db modify process instance json
Co-authored-by: eights <yelli.hl@foxmail.com>
* change notice copyright date from 2019 to 2019-2020 (#2847)
* add source license full text into source license
* exclude ScriptRunner.java on rat check
* change notice copyright date from 2019 to 2019-2020
* fix [BUG] TaskExecutionContextCacheManagerImpl Do not execute removeByTaskInstanceId #2745 (#2754)
* mr resource display and create user username length cannot be less than 3
* fix
* fix
* fix
* fix table T_DS_ALERT (#2854)
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* remove param monitor server state
* fix table T_DS_ALERT
* fix
* fix
* modify t_ds_task_instance app_link change sql (#2861)
* change max length of user name to 39 (#2856)
* fix bug: send task result to master until success.
* fix bug: send task result to master until success.
* add sleep
* set null if send result failed
* set null if send result failed
* change max length of username to 39
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* copy resources need get top directory first (#2866)
* copy resources need get top directory first
* copy resources need get top directory first
* fix #2860:copy files and create empty directory (#2876)
* copy resources need get top directory first
* copy resources need get top directory first
* fix #2860:copy files and create empty directory
* Fix bug:If user didn't set the value of tenant,release process definition fail (#2885)
* copy resources need get top directory first
* copy resources need get top directory first
* fix #2860:copy files and create empty directory
* Fix bug:If user didn't set the value of tenant,release process definition online fail
* merge method updateuser if and fix bug#2892
fix [BUG] When editing user information, clear the phone number, the database table t_ds_user.phone is not cleared #2892
* fix bug: sqoop mysql import params error (#2896)
* fix bug: sqoop mysql import params error
* fix bug: sqoop mysql import params error
* Fixed selection of tenant value and defalut changed to lower case
* update how to build and remove docs link (#2906)
* Fix tasks not execute in sub-process when rerun scheduled process. #2903 (#2908)
* fix create table sql exception (#2912)
fix create table sql exception
* fix#2910 master server will show exception for some time when it restart (#2913)
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* remove param monitor server state
* fix table T_DS_ALERT
* update h2 database
* fix#2910 master server will show exception for some time when it restart
* Fixbug datax task (#2909)
* fix: local param bug
* fix: UT bug
Co-authored-by: Rubik-W <whm_777@163.com>
* [license] add license and notice (#2916)
* fix bug: send task result to master until success.
* fix bug: send task result to master until success.
* add sleep
* set null if send result failed
* set null if send result failed
* change max length of username to 39
* test for gc
* change version to 1.3.0
* check and add some license files
* check and add some license files
* revert
Co-authored-by: baoliang <baoliang@analysys.com.cn>
* fix oom when no master is active in dev-1.3.0 (#2918)
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* remove param monitor server state
* fix table T_DS_ALERT
* update h2 database
* fix#2910 master server will show exception for some time when it restart
* fix oom when no master is active
* fix worker oom when master server restart
* fix oom
* fix
* add UT
* fix worker group config no effect
* Python, sql, sqoop,datax task node editor adds zoom function #2929 (#2933)
* Python, sql, sqoop task node editor adds zoom function
* fix
* change 1.3.0 to 1.3.0-snapshot
* add janino-3.0.12.jar
* delete duplicate codes
* add commons-compiler-3.0.12.jar
* change delete status according to delete action result
* Delete the pom files animation.scss, _normalize.scss and _animation.scss under the common folder
* Change email to default (#2981)
* add mybatis-3 notice in the top-level notice (#3003)
* fix download resource file that under the directory error (#3007)
* add mybatis-3 notice in the top-level notice
* fix download resource file that under the directory error
* Fix unfiltered jar files in resource directory and UDF upload resource parameters (#3008)
* Change email to default
* Fix unfiltered jar files in resource directory and UDF upload resource parameters
* fix singe resource manager will NPE (#3013)
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* remove param monitor server state
* fix table T_DS_ALERT
* update h2 database
* fix#2910 master server will show exception for some time when it restart
* fix oom when no master is active
* fix worker oom when master server restart
* fix oom
* fix
* add UT
* fix worker group config no effect
* fix singe resource manager will NPE
* Update HadoopUtilsTest.java
* merge from dev-1.3.0
* update ut
* remove duplicate code
* remove exception ut
* remove exception ut
* remove exception ut
* remove exception ut
* remove exception ut
* [bugfix] remove Conflicting configuration sudo and user (#3038)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* [bugfix] 1. fix alert.json includes repeated tag 'DOLPHIN_ALERT' 2. metainfo.xml The version number of the RPM package is no longer specified in the metainfo.xml (#3032)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* [feature] readme.md adds instructions for changing environment variables (#3031)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* merge from dev
* Change favicon.ico icon introduction method and package compilation to increase version number (#3030)
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: Simon <3656562@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: liwenhe1993 <liwenhe@apache.org>
Co-authored-by: xiaochun.liu <xiaochun.liu@centurygame.com>
Co-authored-by: lidongdai <escheduler@outlook.com>
Co-authored-by: dailidong <dolphinschedule@gmail.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: xuchangqun <58328021+xuchangqun@users.noreply.github.com>
Co-authored-by: changqun.xu <changqun.xu@17zuoye.com>
Co-authored-by: xuchangqun <changqun.x@gmail.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: tlhhup <137273278@qq.com>
Co-authored-by: yelli.hl@foxmail.com <amarantine@my.com>
Co-authored-by: eights <yelli.hl@foxmail.com>
Co-authored-by: yaoyao <42244568+1445103710@users.noreply.github.com>
Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: Rubik-W <39549317+Rubik-W@users.noreply.github.com>
Co-authored-by: Rubik-W <whm_777@163.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
* fix worker group config no effect
* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2
* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0
# Conflicts:
# dolphinscheduler-server/src/main/resources/config/install_config.conf
* datasource config
* Update datasource.properties
* fix RunConfig bug
* remove param monitor server state
* fix table T_DS_ALERT
* update h2 database
* fix#2910 master server will show exception for some time when it restart
* fix oom when no master is active
* fix worker oom when master server restart
* fix oom
* fix
* add UT
* fix worker group config no effect
* fix singe resource manager will NPE
* Update HadoopUtilsTest.java
* let quartz use the same datasource
* move master/worker config from dao.properties to each config
add master/worker registry test
* move mybatis config from application.properties to SpringConnectionFactory
* move mybatis-plus config from application.properties to SpringConnectionFactory
* refactor TaskCallbackService
* add ZookeeperNodeManagerTest
* add create resource directory
* add create resource directory
* update the resource test
* add upgrade sql in version 1.2.2
* Adding request parameter id to update queryResourceListPaging
* set isDirectory value is false default
* add full name to update updateResource
* remove request parameter isDirectory to update createResource method
* update queryResourceListPaging with change get to post
* update updateResource method with remove fullName
* File management list modification (#1976)
* add resource component
* add resource tree visitor
* return json string
* update queryResourceList
* upload file need fullName
* add method rootNode
* Shell task resources and authorization resources (#1989)
* File management list modification
* Shell task resources and authorization resources
* download resource when execute task
* download resource when execute task
* update authorization type
* download resource when execute task
* Spark task resource changes (#1990)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* download resource when execute task
* update udf function service
* add resource type in ResourceComponent
* UDF resource tree and change DAG style (#2019)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* UDF resource tree and change DAG style
* add deleteIds method in ResourceMapper and ResourceMapperTest
* Add comments on class and method
* add queryResourceByName method in controller
* update verify-name with change name to full name
* update queryResource with add parameter pid
* update queryResource with add parameter pid
* add resource ids in process definition and delete resource need judge whether it is used by any process definition
* Breadcrumb development (#2033)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* UDF resource tree and change DAG style
* Breadcrumb development
* Breadcrumb development
* Resource tree bug fix (#2040)
* File management list modification
* Shell task resources and authorization resources
* Spark task resource changes
* UDF resource tree and change DAG style
* Breadcrumb development
* Breadcrumb development
* Resource tree bug fix
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* Fix github action rerun failed (#2067)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* Change crumb position
* Change crumb position (#2068)
* build resource process definition map
* UDF changed to multiple choice
* UDF changed to multiple choice (#2077)
* Change crumb position
* UDF changed to multiple choice
* build resource process definition map (#2076)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* update resource name also need update all the children full name (#2096)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* Limit customization file content to no more than 3000 lines
* Limit customization file content to no more than 3000 lines(#2128)
* Limit customization file content to no more than 3000 lines(#2128) (#2140)
* Change crumb position
* UDF changed to multiple choice
* Limit customization file content to no more than 3000 lines
* Limit customization file content to no more than 3000 lines(#2128)
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList (#2192)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* Modify the main jar package
* Modify the main jar package (#2200)
* Change crumb position
* UDF changed to multiple choice
* Limit customization file content to no more than 3000 lines
* Limit customization file content to no more than 3000 lines(#2128)
* Modify the main jar package
* add resource filter in order to get filtered resource
* add comments of resource filter
* update list children by resource
* choose main jar with resource tree (#2220)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* add resource filter in order to get filtered resource
* add comments of resource filter
* update list children by resource
* Return null if query resource list is empty
* update queryResource method change parameter pid to id
* getResouDelete checksum and modify parameter namerceId
* revert .env
* remove parameter pid
* Delete request interface
* go back to the last page
* jar interface call
* Fix issue #2234 and #2228
* change resource name with full name
* Fix issue #2234 and #2228 (#2246)
* update resource service test
* Fix github action rerun failed
* add status of PARENT_RESOURCE_NOT_EXIST
* build resource process definition map
* update resource name also need update all the children full name
* need add queryResource
* add queryResourceJarList
* add queryResourceJarList
* add queryResourceJarList
* add resource filter in order to get filtered resource
* add comments of resource filter
* update list children by resource
* Return null if query resource list is empty
* update queryResource method change parameter pid to id
* revert .env
* remove parameter pid
* Fix issue #2234 and #2228
* change resource name with full name
* Fix list query value error
* remove unauth-file with authorize-resource-tree
* Repair data cannot be echoed
* Repair data cannot be echoed
* execute mr and spark task need query resource name before
* Authorized resource interface replacement
* Authorized resource interface replacement
* Filter UDF resources
* Change parameters
* need query all authorized directory children when create task
* Change normalize.scss import method and animation.scss license modification
* Delete file list update processing
* It's fixed that resource not deleted in hdfs when delete it.
* add tooltips
* add tooltips (#2310)
* Echo workflow name and modify udf management name
* [new feature]add resource tree function
* revert front code in order to be same as dev branch
* revert front code in order to be same as dev branch
* revert common.properties and application.properties
* add super method
* update flink parameter test
* update flink parameter and unit test
* update resource service test
* If resource list is empty,need init it
* update flink parameter test
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
* add release notes in 1.2.1 (#1654)
* fix Monitor bug (#1656)
* fix Monitor bug
* fix zk monitor bug
* fix api url (#1673)
* get root path from zookeeper config
* Fixed DAG zoom in and zoom out nodes separated from arrows #1679 (#1685)
* fix api url
* Fixed DAG zoom in and zoom out nodes separated from arrows #1679
* check license and update (#1725)
* Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 (#1734)
Co-authored-by: elonlo <gamelifedong@gmail.com>
* upgrade jackson from 2.9.8 to 2.10.1 (#1767)
* add out put log when master/worker server start (#1769)
* merge hadoop.properties into common.properties
* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties
* change db user/pwd to test/test
* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh
* add out put log when master/worker server start...
* add start log when servers start
* add check download resource permission in order to fix issues 1770 (#1788)
* merge dev branch front-end code (#1786)
* fix#1775 bug，delete process definition when process instance is running (#1790)
* fix#1775 bug，delete process definition when process instance is running
* revert CONTRIBUTING.md
* fix udfs assignment and task instance page jump #1789 (#1791)
* merge dev branch front-end code
* fix udfs assignment
* Fix task instance page jump
* fix udfs assignment and task instance page jump #1789
* update method checkDownloadPermission in order to fix issues 1770 (#1794)
* add check download resource permission in order to fix issues 1770
* update method checkDownloadPermission in order to fix issues 1770
* update method listUnauthorizedResource in order to fix issues 1770
* update method listUnauthorizedResource in order to fix issues 1770 (#1797)
* if login user is admin,it will has permission to query all udf function (#1799)
* Fixed space and icon display issues before and after the input box (#1798)
* merge dev branch front-end code
* fix udfs assignment
* Fix task instance page jump
* fix udfs assignment and task instance page jump #1789
* Fixed space and icon display issues before and after the input box
* add license
* add license
* donot submit task to queue if sub process (#1793)
* dockerfile modify (#1800)
* fix#1775 bug，delete process definition when process instance is running
* revert CONTRIBUTING.md
* dockerfile modify
* dockerfile modify
* dockerfile modify
* Fix data echo, style and popup cannot be closed (#1801)
* merge dev branch front-end code
* fix udfs assignment
* Fix task instance page jump
* fix udfs assignment and task instance page jump #1789
* Fixed space and icon display issues before and after the input box
* add license
* add license
* Fix data echo, style and popup cannot be closed
* Fix page number loading issue and dag not getting value(#1810) (#1815)
* merge dev branch front-end code
* fix udfs assignment
* Fix task instance page jump
* fix udfs assignment and task instance page jump #1789
* Fixed space and icon display issues before and after the input box
* add license
* add license
* Fix data echo, style and popup cannot be closed
* Fix page number loading issue and dag not getting value
* [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1817)
* if login user is admin,it will has permission to query all udf function
* check udf and data source in order to fix issue 1770
* check udf and data source in order to fix issue 1770
* check udf and data source in order to fix issue 1770
* check udf and data source in order to fix issue 1770
* check udf and data source in order to fix issue 1770
* revert common.properties
* update the test method name
* remove useless code and import in unit test
* refactor has permission and check permission
* #1813 remove "_001" after the master/server register path in zookeepe (#1824)
* donot submit task to queue if sub process
* [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820)
* change master/worker register path.
* remove "_" from register path.
* remove install.sh server.servlet.context-path replace (#1823)
* fix#1775 bug，delete process definition when process instance is running
* revert CONTRIBUTING.md
* dockerfile modify
* dockerfile modify
* dockerfile modify
* remove install.sh server.servlet.context-path replace
* Fix DAG add dependent project value exception # 1816 and keep requesting task interface (#1827)
* merge dev branch front-end code
* fix udfs assignment
* Fix task instance page jump
* fix udfs assignment and task instance page jump #1789
* Fixed space and icon display issues before and after the input box
* add license
* add license
* Fix data echo, style and popup cannot be closed
* Fix page number loading issue and dag not getting value
* Fix DAG add dependent project value exception # 1816 and keep requesting task interface
* rpm package add UI (#1846)
* [fix#1828] when the executor of process instance is not the owner of udf resouce, the path of the read resource file is incorrect (#1847)
* fix issue 1828:get the udf resource path error when create udf function
* update grantResources
* first verify whether udf resource is bound by udf function
* update grantResources
* update testListAuthorizedUdfFunc
* update getUserInfo in order to run success
* add LoggerServer RPC PORT modified. #1848 (#1849)
* fix#1775 bug，delete process definition when process instance is running
* revert CONTRIBUTING.md
* dockerfile modify
* dockerfile modify
* dockerfile modify
* remove install.sh server.servlet.context-path replace
* add LoggerServer RPC PORT modified. #1848
* LoggerService Logger RPC PORT get Error modify . #1848 (#1857)
* fix#1775 bug，delete process definition when process instance is running
* revert CONTRIBUTING.md
* dockerfile modify
* dockerfile modify
* dockerfile modify
* remove install.sh server.servlet.context-path replace
* add LoggerServer RPC PORT modified. #1848
* LoggerService Logger RPC PORT get Error modify . #1848
* Fix UDF function list delete data without refresh and page data display widened #1851 (#1852)
* merge dev branch front-end code
* fix udfs assignment
* Fix task instance page jump
* fix udfs assignment and task instance page jump #1789
* Fixed space and icon display issues before and after the input box
* add license
* add license
* Fix data echo, style and popup cannot be closed
* Fix page number loading issue and dag not getting value
* Fix DAG add dependent project value exception # 1816 and keep requesting task interface
* Fix UDF function list delete data without refresh and page data display widened
* [Fix#1828]check whether has permission to download udf file or delete udf function (#1858)
* fix issue 1828:get the udf resource path error when create udf function
* update grantResources
* first verify whether udf resource is bound by udf function
* update grantResources
* update testListAuthorizedUdfFunc
* update getUserInfo in order to run success
* check whether has permission to download udf file or delete udf file
* update listAuthorizedResourceById in ResourceMapper.xml
* add log, don't swallow exception info (#1877)
* Added hints in Resource Center (#1891)
* update jackson version from 2.9.8 to 2.10.1 (#1892)
* update release notes and modify the plugin of rat (#1899)
* modify how to build (#1902)
* update release notes and modify the plugin of rat
* modify how to build
* [maven-release-plugin] prepare release 1.2.1
* [maven-release-plugin] prepare for next development iteration
* get authorized udf resources need exclude all created by self (#1905)
* get authorized udf resources need exclude all created by self
* reset pom version to 1.2.1-SNAPSHOT
* Update ReleaseNotes.md
add detail info
* Update ReleaseNotes.md
* update notice year form 2019 to 2019-2020 (#1907)
* [maven-release-plugin] prepare release 1.2.1
* [maven-release-plugin] prepare for next development iteration
* 当手动重跑历史活着补数的时候对shell和sql脚本中的[YYYYmmddd...]变量赋值与传递的日期，而不是当前日期
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: elonlo <gamelifedong@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: zhangchunyang <18910529250@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: 谷雨 <lixun@guyudeMacBook-Pro.local>
lgcareer
lenboo
Yichao Yang
dailidong
tswstarplanet
LEI SHENG
amenglu
hcz
bao liang
gabry.wu
CalvinKirs
muzhongjiang
iceblue510
lenboo
gaojun2048
Yelli
743294668
songgg
Tboy
zixi0825
Rubik-W
t1mon
gabry.wu
Simon
ruson