d04f4b6053
[Implement][Common] Modify the plug-in loading log to print messages
...
Co-authored-by: caoping <caoping@staff.weibo.com>
4 years ago
9c77faa8ac
[Improvement-5058][Master] The master-server creates the DB data source twice ( #5139 )
...
* reduce the number of data sources created twice by master server.
* update ProcessAlertManagerTest test class to increase coverage.
* add DolphinSchedulerPluginLoaderTest test ignore.
Co-authored-by: zhuang chong
4 years ago
f7b71fb9ca
[Feature][JsonSplit] Fix codeStyle for json_split ( #5432 )
...
* update taskParams/add task delayTime/fix conditionType bug
* update codeStyle for merge to dev
* fix process lineage bug
* fix codeStyle for json_split
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
1646188027
fix code style
4 years ago
86dcddb32b
[Feature-4417]fix code style ( #5395 )
...
* fix code style
* fix ut
* remove unused ut
* remove unused ut
Co-authored-by: baoliang <baoliang@analysys.com.cn>
4 years ago
2a44cb9de6
[Feature][JsonSplit] update codeStyle for merge to dev ( #5392 )
...
* update taskParams/add task delayTime/fix conditionType bug
* update codeStyle for merge to dev
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
c3558965c1
update taskParams/add task delayTime/fix conditionType bug ( #5385 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
e6d8da484f
[Improvement][API] Create Datasource #5083 ( #5116 )
...
* [Improvement][API] Create Datasource #5083
* Refact create/ update datasource API
* fix Code Smells
* fix code smell
* resolve code smell
* use ConnectionParam to replace BaseDatasource
* solve license
* add ut
* code check
* add ut
* fix ut coverage
* fix ut
4 years ago
75d1ad645c
fix bug: 5349, work flow would running if task is killed by manual. ( #5362 )
...
Co-authored-by: baoliang <baoliang@analysys.com.cn>
4 years ago
e57ee658e8
[Improvement] Optimized close of IO ( #5336 )
...
* [improvement]modify all methods that refer to LogClientService in the code.
* delete unused code
* [Improvement] Optimized close of IO
4 years ago
feffd972b5
[Feature][JsonSplit] Fix task logger path ( #5331 )
...
* update SnowFlake
* update processDefinite from processInstance
* update processDefinite from processInstance
* Fix task logger path
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
5f1fc462c9
[Bug][server-api] copy the node ,the preTasks is error #5261 ( #5264 )
4 years ago
49b234609a
update SnowFlake ( #5321 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
837c9ba2cc
fix bug:when shell task raw script has $[${day} - 1],throw java.lang.IllegalArgumentException::Pattern includes reserved character:'{' ( #5313 )
4 years ago
83519bdfc3
[Improvement][Server] Optimize log display ( #5311 )
4 years ago
e92e29ef9a
[Fix-5309][Common] Fix the wrong memory ( #5312 )
...
* [Fix-5309][Common] Fix the wrong memory
* [Fix-5309][Common] Fix unit test and remove unused code
4 years ago
f7a06a033d
[Feature][JsonSplit] Fix master/worker about shellTask ( #5300 )
...
* Fix quoted bug about processDefineId of processInstance
* Fix master/worker about shellTask
* codeStyle
* codeStyle
* update taskType
* codeStyle
* update timeoutNotifyStrategy
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
f3dfaacca2
[Improvement][Service] Optimize query log ( #5205 )
...
* [Improvement][Service] Optimize query log
* remove IPUtils
4 years ago
1c1f6663a6
[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select ( #5182 )
...
* [Improvement][Server] Optimize and reduce database and ZooKeeper query
* [Improvement][WorkerGroup] Replace worker address input with multiple-select
* [Improvement][WorkerGroup] Fix unit test
* [Improvement][API&UI] Optimize long host to short host and long host display in k8s
4 years ago
0eb8c0c234
[Improvement][common]Task status error
4 years ago
73f5a8e0a7
[Feature][JsonSplit] Fix master/worker server ( #5253 )
...
* Fix dependency failure
* Fix taskInstance
* task instance list page
* code review
* fix logger path
* Fix master/worker server
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
31188ffca0
[Feature][JsonSplit] Fix taskInstance ( #5246 )
...
* Fix dependency failure
* Fix taskInstance
* task instance list page
* code review
* fix logger path
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
73e917d3c1
Clean up useless files. ( #5252 )
4 years ago
8c88bfa222
Fix dependency failure ( #5239 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
2f1f193ba8
Update common.properties ( #5223 )
4 years ago
86ce8f133a
[JsonSplit-4417][master/worker/api]json split- remove json in task instance. ( #5160 )
...
* [json split] refactor json in task instance
* add task_pramas in task_instance
* feature #4417 refactor json split in task instance
* code style
* code style
* code style
* code style
* code style
* update
* update
* update unit test
* update
4 years ago
dd36da8d1f
[Feature][JsonSplit] fix process bug ( #5193 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
e53369318b
[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal ( #4722 )
...
* fix the shell starts the yarn task in the background
* update StringUtils code style.
* solve code smell.
* add method comment in StringUtils class.
* update AlertGroupMapper code style.
* update AlertGroupMapperTest
* update sql script test.
4 years ago
689106d114
transform taskCode from long to string ( #5174 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
a467fffd7b
[Feature][JsonSplit] fix taskId in json ( #5165 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
* fix json taskId
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
a6a1b94df4
[Feature-#5128]Support turning off sudo permissions ( #5129 )
...
* [Feature-#5128]Support turning off sudo permissions
* [Feature-#5128] without sudo permission do not create tenant user
4 years ago
492b318bd3
[Fix][Server] Fix clear task execute path is related to master ( #5123 )
4 years ago
cd19507271
[feature][jsonsplit]fix copy/export/import processDefinition bugs ( #5146 )
...
* Modify Project and ProjectUser Mapper
* Modify Project and ProjectUser Mapper
* project_code is bigint(20)
* modify ERROR name
* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation
* modify import/export processdefinition, add genProcessData
* fix ut and bug
* code style
* repalce project_id with code
* conflicts solve
* conflicts solve
* conflicts solve
* bugfix
* modify listResources mothod and remove getResourceIds mothod
* 1
* conflicts solve
* modify listResources mothod and remove getResourceIds mothod
* modify listResources mothod and remove getResourceIds mothod
* replace processDefinitionVersion with processDefinitionLog
* codestyle
* codestyle
* add mapper module ut
* codestyle
* fix ProcessInstanceMapperTest
* codestyle
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* conflicts solve
* fix ProcessInstanceMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest
* fix TaskInstanceMapperTest
* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* codestyle
* codestyle
* Refactor process lineage
* Refactor process lineage
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest
* add ProcessServiceTest
* 1
* 1
* fix copy/export/import processDefinition bugs
* 1
* codestyle
4 years ago
1eaf1f6234
[Feature][JsonSplit] fix taskId in processDefinitionJson ( #5136 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
5d264c9f20
[Improvement][Api] Optimize query processInstanceList #5134 ( #5135 )
...
* Batch query user rather than several query
4 years ago
0d065c0712
[Feature-5062][*] Create/Edit/Delete Worker Group in Web UI ( #5113 )
...
* [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE
* [Feature][Api&Dao] Support api and dao of worker group
* [Feature][UI] Support to create, edit and delete worker group
* [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management
* [Feature][API&Dao] Support long ip list and check request parameters when saving worker group
* [Feature][*] Refactor ipList to addrList
* [Feature][Server] Support host manager to select worker host by worker group in database
* [Improvement][Server] Improve the config of master and worker
* [Improvement][*] Rename zkRegistered to systemDefault
* [Improvement][Test] Fix unit test and improve ui
* [Improvement][Server] Improve getServerMaps
Co-authored-by: dailidong <dailidong66@gmail.com>
4 years ago
d99934d0da
[Feature][JsonSplit] Fix api bug ( #5102 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
8ac72e80e6
[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask ( #5088 )
...
* [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask
* [Feature-5087][SqlTask] Add SqlParametersTest
4 years ago
086e716441
[Improvement][Dao] SQL optimization - check exist #5069 ( #5070 )
4 years ago
f94cfc620d
[Feature][Worker] Add a configuration item to set whether the tenant is automatically created on Worker ( #5007 )
4 years ago
29d42fd92d
[Feature-#3805][server-master] Task parameter transfer ( #5077 )
...
* fix out param format bug
Co-authored-by: wangxj <wangxj31>
4 years ago
5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance ( #4996 )
...
* [Improvement][Remote] Improve unit tests
* [Improvement][Worker] Improve worker load balance
* [Improvement][Worker] Fix code smells
* [Improvement][Worker] Rename weight to hostWeight
4 years ago
6216817861
[Improvement-4971][alert|dao|service]Remove redundant PropertyUtils.class ( #5020 )
...
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971 )
2. Remove useless import package.
4 years ago
1cd62b4a5e
[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py ( #5019 )
4 years ago
d98c2c5cfe
[Feature][JsonSplit] Fix bug in interface of queryProcessDefinitionList ( #5043 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
6bf5031f45
[FIX] fix MySQL datasource jdbc connect parameters ( #4851 )
...
* [FIX-CVE-2020-11974] fix MySQLDataSource Security
* remove useless code
* add ut
* A more reasonable way
* code style
* fix test error
* code style
* update exception msg
4 years ago
ae608e024a
[Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping ( #5038 )
...
* [Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping
* [Improvement][*] Replace commons.lang.StringUtils with common.utils.StringUtils
4 years ago
c8e31ad58e
[Improvement-4884][Common] fix the subsequent processing of getting the YARN ResourceManager address to be empty ( #4885 )
...
* fix yarn application url generate empty.
* removing Excess Processing
* update hadoop utils test class .
* add base exception class.
* update hadoop utils class code style.
4 years ago
ea6b1de120
[Improvement][worker] Optimize KillYarnJob (4939) ( #4943 )
4 years ago
beb5adb985
[Feature][MR] Support mapreduce name ( #5029 )
4 years ago
lcdrs
zhuangchong
JinyLeeChina
baoliang
bao liang
ruanwenjun
luoyuan
roi
Shiwen Cheng
didiaode18
xiaoyu
guohaozhang
Simon
wangxj3
Ts686
Kirs