DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
zhuangchong	9c77faa8ac	[Improvement-5058][Master] The master-server creates the DB data source twice (#5139 ) * reduce the number of data sources created twice by master server. * update ProcessAlertManagerTest test class to increase coverage. * add DolphinSchedulerPluginLoaderTest test ignore. Co-authored-by: zhuang chong	4 years ago
baoliang	c850b85484	Merge remote-tracking branch 'upstream/dev' into json_split # Conflicts: # dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/procedure/ProcedureTask.java	4 years ago
zhuangchong	88a07f7b55	[Feature-4093][server] Support for stored procedures and stored function calls and data source supports DB2 (#4094 ) * Support for stored procedures and stored function calls and data source supports DB2. Co-authored-by: zhuangchong <zhuangchong8@163.com>	4 years ago
JinyLeeChina	c3558965c1	update taskParams/add task delayTime/fix conditionType bug (#5385 ) Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
ruanwenjun	e6d8da484f	[Improvement][API] Create Datasource #5083 (#5116 ) * [Improvement][API] Create Datasource #5083 * Refact create/ update datasource API * fix Code Smells * fix code smell * resolve code smell * use ConnectionParam to replace BaseDatasource * solve license * add ut * code check * add ut * fix ut coverage * fix ut	4 years ago
bao liang	75d1ad645c	fix bug: 5349, work flow would running if task is killed by manual. (#5362 ) Co-authored-by: baoliang <baoliang@analysys.com.cn>	4 years ago
JinyLeeChina	862565a7c7	[Feature][JsonSplit] Fix dependTask bug (#5360 ) * update SnowFlake * update processDefinite from processInstance * update processDefinite from processInstance * Fix task logger path * Fix dependTask bug Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
Wenjun	abdd2337b1	[Bug][worker] Fix sql task query result is empty (#5322 ) (#5324 ) * [Bug][worker] Fix sql task query result is empty (#5322) * add ut * remove duplicate * add ut	4 years ago
luoyuan	e57ee658e8	[Improvement] Optimized close of IO (#5336 ) * [improvement]modify all methods that refer to LogClientService in the code. * delete unused code * [Improvement] Optimized close of IO	4 years ago
JinyLeeChina	feffd972b5	[Feature][JsonSplit] Fix task logger path (#5331 ) * update SnowFlake * update processDefinite from processInstance * update processDefinite from processInstance * Fix task logger path Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
JinyLeeChina	49b234609a	update SnowFlake (#5321 ) Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
luoyuan	513eb76919	[Improvement-5147][Service]This judgment always returns true (#5156 ) * this expression which always evaluates "true" * delete unused variables * LogClientService add AutoCloseable * reformat code * reformat code * reformat code * add log in ProcessService * reformat code * Optimize the code * Optimize the code * [improvement]modify all methods that refer to LogClientService in the code. * delete unused code	4 years ago
Shiwen Cheng	83519bdfc3	[Improvement][Server] Optimize log display (#5311 )	4 years ago
JinyLeeChina	f7a06a033d	[Feature][JsonSplit] Fix master/worker about shellTask (#5300 ) * Fix quoted bug about processDefineId of processInstance * Fix master/worker about shellTask * codeStyle * codeStyle * update taskType * codeStyle * update timeoutNotifyStrategy * codeStyle Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
Shiwen Cheng	e39409a412	[Improvement-5275][Server] Fix wrong excludeFilters and move zk/ZKMasterClient and registry/ServerNodeManager into master (#5278 ) * [Improvement-5275][Server] Move zk/ZKMasterClient and registry/ServerNodeManager into master * [Improvement-5275][Server] Fix code smells * [Improvement-5275][Server] Add ZKMasterClientTest	4 years ago
Shiwen Cheng	1c1f6663a6	[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select (#5182 ) * [Improvement][Server] Optimize and reduce database and ZooKeeper query * [Improvement][WorkerGroup] Replace worker address input with multiple-select * [Improvement][WorkerGroup] Fix unit test * [Improvement][API&UI] Optimize long host to short host and long host display in k8s	4 years ago
wenjun	cc7a4446f5	[Improvement][Task] DataX Node doesn't Support Clickhouse as datasource (#5180 ) (#5243 ) * [Improvement][Module Name] DataX Node doesn't Support Clickhouse as the datasource (#5180) * add ut * add ut * resolve code smell	4 years ago
JinyLeeChina	645847c096	Fix quoted bug about processDefineId of processInstance (#5263 ) Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
JinyLeeChina	73f5a8e0a7	[Feature][JsonSplit] Fix master/worker server (#5253 ) * Fix dependency failure * Fix taskInstance * task instance list page * code review * fix logger path * Fix master/worker server Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
JinyLeeChina	31188ffca0	[Feature][JsonSplit] Fix taskInstance (#5246 ) * Fix dependency failure * Fix taskInstance * task instance list page * code review * fix logger path Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
wenjun	4a1303b625	[Improvement][Worker] Kill task error(#5199 ) (#5212 )	4 years ago
JinyLeeChina	a8f2894b79	[Feature][JsonSplit] Fix master/processInstance bug (#5206 ) * transform taskCode from long to string * fix process bug * code review * code review * code review * Fix master/processInstance bug Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
bao liang	86ce8f133a	[JsonSplit-4417][master/worker/api]json split- remove json in task instance. (#5160 ) * [json split] refactor json in task instance * add task_pramas in task_instance * feature #4417 refactor json split in task instance * code style * code style * code style * code style * code style * update * update * update unit test * update	4 years ago
wenjun	97fb08f0ce	[Improvement][LogServer] Thread pool parameter adjustment #5161 (#5163 ) * [Improvement][LogServer] Thread pool parameter adjustment #5161 * change log request processor thread num 2 * cpuCores + 1	4 years ago
JinyLeeChina	dd36da8d1f	[Feature][JsonSplit] fix process bug (#5193 ) * transform taskCode from long to string * fix process bug * code review * code review * code review Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
zhuangchong	e53369318b	[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal (#4722 ) * fix the shell starts the yarn task in the background * update StringUtils code style. * solve code smell. * add method comment in StringUtils class. * update AlertGroupMapper code style. * update AlertGroupMapperTest * update sql script test.	4 years ago
JinyLeeChina	a467fffd7b	[Feature][JsonSplit] fix taskId in json (#5165 ) * modify checkDAGRing and ProcessService method * merge * modify dagRing * modify process instance for project home page * fix save process bug * codeStyle * Fix logical bug in saving process definition * codeSytle * Fix bug in interface of queryProcessDefinitionList * codeSytle * Fix api bug" * fix taskId in processDefinitionJson * fix json taskId * codeStyle Co-authored-by: JinyLeeChina <297062848@qq.com>	4 years ago
guohaozhang	a6a1b94df4	[Feature-#5128]Support turning off sudo permissions (#5129 ) * [Feature-#5128]Support turning off sudo permissions * [Feature-#5128] without sudo permission do not create tenant user	4 years ago
xiaojingXU	9c04396218	[Bug-5132][server] stop the take immediately if an exception occurs in the taskExecuteThread (#5133 ) * [DS-5132][fix] stop the take immediately if an exception occurs in the taskExecuteThread - let softkill() method return true when the process is stopped,else return false;	4 years ago
wenjun	573252f3e1	[Bug][Server] DB transaction failure #5150 (#5152 )	4 years ago
wenjun	87ed189ef5	[Improvement] print exception when task execute (#5159 )	4 years ago
Shiwen Cheng	492b318bd3	[Fix][Server] Fix clear task execute path is related to master (#5123 )	4 years ago
wenjun	a8b47e1d4f	[Bug][Master] Typo in MasterConfig #5141 (#5142 )	4 years ago
Shiwen Cheng	0d065c0712	[Feature-5062][] Create/Edit/Delete Worker Group in Web UI (#5113 ) [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE * [Feature][Api&Dao] Support api and dao of worker group * [Feature][UI] Support to create, edit and delete worker group * [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management * [Feature][API&Dao] Support long ip list and check request parameters when saving worker group * [Feature][] Refactor ipList to addrList [Feature][Server] Support host manager to select worker host by worker group in database * [Improvement][Server] Improve the config of master and worker * [Improvement][] Rename zkRegistered to systemDefault [Improvement][Test] Fix unit test and improve ui * [Improvement][Server] Improve getServerMaps Co-authored-by: dailidong <dailidong66@gmail.com>	4 years ago
zhuangchong	ba039dc252	[Improvement][dao] Resource features SQL slow query optimization. (#4947 ) * Resourse features SQL slow query optimization. * update resource mapper queryResourcePaging method. * split resource and resource_uesr sql statement. * determine the size of the collection. * update process service class code style. * update ResourceUserMapper java class code style. * add resourceUserMapper in the test config file. * solve code smell * split paging query join statement. * update ResourceUserMapper class. * remove sql join statement. * update resource test class code style. * update ResourcesServiceTest test class. * rerun ut. * update user service test class. * solve query resource list method code smell.	4 years ago
Shiwen Cheng	8ac72e80e6	[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask (#5088 ) * [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask * [Feature-5087][SqlTask] Add SqlParametersTest	4 years ago
Shiwen Cheng	f94cfc620d	[Feature][Worker] Add a configuration item to set whether the tenant is automatically created on Worker (#5007 )	4 years ago
wangxj3	29d42fd92d	[Feature-#3805][server-master] Task parameter transfer (#5077 ) * fix out param format bug Co-authored-by: wangxj <wangxj31>	4 years ago
Shiwen Cheng	5856a12855	[Improvement-4984][Worker] Refactor and Improve worker load balance (#4996 ) * [Improvement][Remote] Improve unit tests * [Improvement][Worker] Improve worker load balance * [Improvement][Worker] Fix code smells * [Improvement][Worker] Rename weight to hostWeight	4 years ago
Shiwen Cheng	1cd62b4a5e	[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py (#5019 )	4 years ago
mrproliu	0ba120e8ab	Fix shell task ${setValue(key,value)} is not working (#5067 ) Co-authored-by: dalekliuhan <dalekliuhan@didiglobal.com>	4 years ago
Kirs	6bf5031f45	[FIX] fix MySQL datasource jdbc connect parameters (#4851 ) * [FIX-CVE-2020-11974] fix MySQLDataSource Security * remove useless code * add ut * A more reasonable way * code style * fix test error * code style * update exception msg	4 years ago
游侠	2d8c4ec86b	[Improvement][Worker-Task] Python Command (#5036 ) * create the interpreter for python command * Update README.md Co-authored-by: zhouhong <zhouhong@zhangyue.com>	4 years ago
guohaozhang	a3153b5e44	[Bug][Master] Fix First master fault tolerance when startup (#5045 ) * [Fix-#5044] Fix First master fault tolerance when startup * [Fix-#5044] Fix spell error	4 years ago
Shiwen Cheng	ae608e024a	[Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping (#5038 ) * [Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping * [Improvement][*] Replace commons.lang.StringUtils with common.utils.StringUtils	4 years ago
wenjun	ea6b1de120	[Improvement][worker] Optimize KillYarnJob (4939) (#4943 )	4 years ago
Shiwen Cheng	beb5adb985	[Feature][MR] Support mapreduce name (#5029 )	4 years ago
dddyszy	28dfde7002	[Fix-4862][Server] Kill yarn application command won't be execute whe… (#4936 ) * [Fix-4862][Server] Kill yarn application command won't be execute when kill error	4 years ago
wenjun	fd4b59ad6c	[Improvement][Worker] Rename worker.weight to worker.host.weight (#4997 ) (#5014 )	4 years ago
Shiwen Cheng	e74932cfc2	[Feature-4960][Spark] Add name input for Spark and improve Spark & MR args (#4959 ) * [Improvement][Spark] Improve spark args and add name input * [Improvement][MR] Improve map reduce args * [Improvement][Spark] Fix check style * [Improvement][Spark] Fix check style for spark, flink and mr	4 years ago

... 3 4 5 6 7 ...

767 Commits (35a1d732c64077580b109a49cb5de8a6246a2452)