e53369318b
[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal ( #4722 )
...
* fix the shell starts the yarn task in the background
* update StringUtils code style.
* solve code smell.
* add method comment in StringUtils class.
* update AlertGroupMapper code style.
* update AlertGroupMapperTest
* update sql script test.
4 years ago
689106d114
transform taskCode from long to string ( #5174 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
a467fffd7b
[Feature][JsonSplit] fix taskId in json ( #5165 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
* fix json taskId
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
a6a1b94df4
[Feature-#5128]Support turning off sudo permissions ( #5129 )
...
* [Feature-#5128]Support turning off sudo permissions
* [Feature-#5128] without sudo permission do not create tenant user
4 years ago
492b318bd3
[Fix][Server] Fix clear task execute path is related to master ( #5123 )
4 years ago
cd19507271
[feature][jsonsplit]fix copy/export/import processDefinition bugs ( #5146 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
* fix copy/export/import processDefinition bugs
* 1
* codestyle
4 years ago
1eaf1f6234
[Feature][JsonSplit] fix taskId in processDefinitionJson ( #5136 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
5d264c9f20
[Improvement][Api] Optimize query processInstanceList #5134 ( #5135 )
...
* Batch query user rather than several query
4 years ago
0d065c0712
[Feature-5062][*] Create/Edit/Delete Worker Group in Web UI ( #5113 )
...
* [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE
* [Feature][Api&Dao] Support api and dao of worker group
* [Feature][UI] Support to create, edit and delete worker group
* [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management
* [Feature][API&Dao] Support long ip list and check request parameters when saving worker group
* [Feature][*] Refactor ipList to addrList
* [Feature][Server] Support host manager to select worker host by worker group in database
* [Improvement][Server] Improve the config of master and worker
* [Improvement][*] Rename zkRegistered to systemDefault
* [Improvement][Test] Fix unit test and improve ui
* [Improvement][Server] Improve getServerMaps
Co-authored-by: dailidong <dailidong66@gmail.com>
4 years ago
d99934d0da
[Feature][JsonSplit] Fix api bug ( #5102 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
8ac72e80e6
[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask ( #5088 )
...
* [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask
* [Feature-5087][SqlTask] Add SqlParametersTest
4 years ago
086e716441
[Improvement][Dao] SQL optimization - check exist #5069 ( #5070 )
4 years ago
f94cfc620d
[Feature][Worker] Add a configuration item to set whether the tenant is automatically created on Worker ( #5007 )
4 years ago
29d42fd92d
[Feature-#3805][server-master] Task parameter transfer ( #5077 )
...
* fix out param format bug
Co-authored-by: wangxj <wangxj31>
4 years ago
5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance ( #4996 )
...
* [Improvement][Remote] Improve unit tests
* [Improvement][Worker] Improve worker load balance
* [Improvement][Worker] Fix code smells
* [Improvement][Worker] Rename weight to hostWeight
4 years ago
6216817861
[Improvement-4971][alert|dao|service]Remove redundant PropertyUtils.class ( #5020 )
...
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
4 years ago
1cd62b4a5e
[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py ( #5019 )
4 years ago
d98c2c5cfe
[Feature][JsonSplit] Fix bug in interface of queryProcessDefinitionList ( #5043 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
6bf5031f45
[FIX] fix MySQL datasource jdbc connect parameters ( #4851 )
...
* [FIX-CVE-2020-11974] fix MySQLDataSource Security
* remove useless code
* add ut
* A more reasonable way
* code style
* fix test error
* code style
* update exception msg
4 years ago
ae608e024a
[Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping ( #5038 )
...
* [Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping
* [Improvement][*] Replace commons.lang.StringUtils with common.utils.StringUtils
4 years ago
c8e31ad58e
[Improvement-4884][Common] fix the subsequent processing of getting the YARN ResourceManager address to be empty ( #4885 )
...
* fix yarn application url generate empty.
* removing Excess Processing
* update hadoop utils test class .
* add base exception class.
* update hadoop utils class code style.
4 years ago
ea6b1de120
[Improvement][worker] Optimize KillYarnJob (4939) ( #4943 )
4 years ago
beb5adb985
[Feature][MR] Support mapreduce name ( #5029 )
4 years ago
e74932cfc2
[Feature-4960][Spark] Add name input for Spark and improve Spark & MR args ( #4959 )
...
* [Improvement][Spark] Improve spark args and add name input
* [Improvement][MR] Improve map reduce args
* [Improvement][Spark] Fix check style
* [Improvement][Spark] Fix check style for spark, flink and mr
4 years ago
7f0678d3be
[Feature][JsonSplit] Fix save process bug ( #5001 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
e68e994d46
when Kerberos authentication is enabled,cannot get application status. ( #4980 )
4 years ago
39a9c0c392
[Feature][jsonspilt]modify process instance for project home page ( #4991 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
01b71f1a0d
[Fix-4839][Server] Process cannot stop when killed with retry ( #4841 )
...
* [Fix-4839][Server] Process cannot stop when killed with retry
* [Fix-4839][Server] Process cannot stop when killed with retry
4 years ago
d667525913
[Fix-4905][K8s] Fix incorrect host problem in minikube ( #4906 )
...
* [Fix][K8s] Fix incorrect host problem in minikube
* [Improvement][K8s] Rename MASTER_WORKER_STS_PATTERN to STS_PATTERN
4 years ago
24fda3fbb1
[Feature][Flink] Support parallelism input ( #4937 )
...
* [Feature][Flink] Support parallelism input
* [Feature][Flink] Fix check style
* [Feature][Flink] Remove FLINK_DETACH
4 years ago
6d087ccce0
[Feature][JsonSplit] fix processDefinitionService ut and bug ( #4894 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
4 years ago
a388b6853d
Optimize HashMap Initial Capacity. ( #4896 )
4 years ago
a9e1d316f5
[Improvement][SPI]Migrate related classes loaded by SPI classes to public modules ( #4586 )
...
* Migrate related classes loaded by SPI classes to public modules
* Migrate related classes loaded by SPI classes to public modules
* remove guava
* update guava
* update guava
* move to common model
* code style
* code style
* resolved conflicting file
* remove ignore test
* add jar
* fix ut error
* fix test
* fix error
* test
* exclude asm、resolver、aether-api
* code style
* remove default plugin jar config
* add LICENSE file
* add version
* add version
* remove ignore ut
* fix ut error
* add ut
* code style
* add license
4 years ago
c64d73b075
[Fix-#4866][common] Fix loading hdfs-site.xml ( #4867 )
4 years ago
a32b40ee95
[Feature][JsonSplit] add gen taskNodeList method ( #4861 )
...
* add gen taskNodeList method
* modify code style
* modify code style
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
149b0dab4d
[Improvement][Docker] Sync docker conf templates to the latest conf properties ( #4772 )
4 years ago
62e961e3f1
[Feature][JsonSplit]refactor remove the json in process instance and definition ( #4828 )
...
* #4417 [JsonSplit] refactor process definition json
* [#4177 ] refactor json split. remove json in process definition and process instance
* [#4177 ] refactor json split. copy process definition need set task code 0L.
* code style
* code style
* code style
* code style
* update
* update
4 years ago
3110f023af
[BUG][API]Fix format2Duration when parameter has one is null ( #4802 )
...
* fix format2Duration when parameter has one is null
4 years ago
61397b338a
[Improvement][api] Introduce base service interface for clear code and unify all controllers, services and tests of api ( #4798 )
...
* [Improvement][api] Introduce base service interface for clear code
* [Improvement][api] Unify all controllers, services and tests of api
* [Fix][api] Fix unit tests left over from the past
* [Improvement][api] Use RegexUtils.escapeNRT instead of StringUtils.replaceNRTtoUnderline for vulnerable check
4 years ago
9ae29a756f
[Improvement-3369][api] Introduce resources, scheduler and taskinstance service interface for clear code ( #4766 )
...
* [Improvement-3369][api] Introduce resources, scheduler and taskinstance service interface for clear code
4 years ago
dc55b5ba6f
[Improvement-3369][api] Introduce monitor, processinstance and queue service interface for clear code ( #4765 )
...
* [Improvement-3369][api] Introduce monitor, processinstance and queue service interface for clear code
4 years ago
070424fc78
[Improvement-3369][api] Introduce taskrecord, udffunc, workflowlineage and workergroup service interface for clear code ( #4768 )
...
* [Improvement-3369][api] Introduce taskrecord, udffunc, workflowlineage and workergroup service interface for clear code
4 years ago
a53195fa15
[Improvement-3369][api] Introduce alert group and users service interface for clear code ( #4758 )
4 years ago
8ebddc1f5b
[Fix-3298][K8s] Fix wrong host of task instance in k8s ( #4786 )
4 years ago
23591b71a2
[Improvement-4751][API] Improvement the duration field modify the display to a time-based minute-second format (e.g. 1d 10h 20m 1s) ( #4752 )
...
* imp ui instance duration field display content.
* update DateUtils format2Duration method.
4 years ago
18e4087c7e
[Improvement][JSONUtils] Improvement JSONUtils parse object method ( #4781 )
...
* Improvement JSONUtils parse object method.
* update JSONUtils toMap method.
* rerun ut test.
4 years ago
53598fdd18
[Fix-3298][K8s] Fix task log disappear after recreating or upgrading helm release ( #4755 )
...
* [Improvement][K8s] Alter column host varchar(15) to varchar(135) for long host
* [Improvement][K8s] Improve getHost and getAddr in NetUtils
* [Improvement][K8s] Replace getHost with getAddr
4 years ago
a5443f1173
[fix-4128][worker] Dolphin executes the command using sudo to specify the -u parameter ( #4217 )
...
* update sudo cmd.
* update sudo cmd.
* update OSUtils code style.
* add OSUtils test method.
4 years ago
03a2c6b7b7
[Improvement-4450][datasource] Hive/Spark data sources do not support multi-tenancy when Kerberos authentication is enabled ( #4593 )
...
* hive/spark jdbc add kerberos paramter.
* update datasource vue paramter placeholder.
* update code style.
* add datasource ApiImplicitParam in the kerberos paramter.
* update constants class code style.
* add code test and code style.
* update BaseDataSourceTest code style.
* update BaseDataSourceTest class code style.
* update DataSourceController class code style.
* update DataSourceController ApiImplicitParam
* update ui create data source vue.
4 years ago
773909efe2
[Improvement][Common] Add more resource suffix ( #4683 )
4 years ago
zhuangchong
JinyLeeChina
guohaozhang
Shiwen Cheng
Simon
wenjun
wangxj3
Ts686
Kirs
dddyszy
bao liang
felix.wang