DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
Kerwin	22c5292b93	[Feature][SPI] Task (#6044 )	3 years ago
Kerwin	79952d926f	[Feature][SPI] Task (#5996 )	3 years ago
kyoty	bca92157a0	[Fix-5808][Server] When we try to transfer data using datax between different types of data sources, the worker will exit with ClassCastException (#5809 ) * bug fix * fix bug * simplify the code format * add a new parameter to make it easier to understand.	3 years ago
Hua Jiang	4a68bfbe1c	[Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867 ) * provide two system parameters to support the rest of type of tasks * provide two system parameters to support the rest of type of tasks * improve test conversion	3 years ago
kyoty	7bffe0ac85	[Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected. (#5796 ) * improve timestamp format make the startime in the log of httptask to be easier to read. * fix bad code smell and update the note.	3 years ago
Wenjun Ruan	30af55b82a	[Improvement][Worker] Task log may be lost #5775 (#5783 )	3 years ago
Hua Jiang	9fd5145b66	[Improvement-5773][server] need to support two parameters related to task (#5774 ) * add some new parameter for task * restore official properties * improve imports * modify a variable's name Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>	3 years ago
Tanvi Moharir	ab527a5e5a	[Improvement#5741][Worker] Improve task process status log (#5776 )	3 years ago
wangxj3	6df58e8d78	[Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603 ) * add globalParams new plan with varPool * add unit test * add python task varPoolParams Co-authored-by: wangxj <wangxj31>	3 years ago
ji04xiaogang	3b80760c42	Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691 ) Co-authored-by: shenglm <shenglm840722@126.com>	3 years ago
LOUKHNATI Mohamed Khalil	86ffc1f3f2	[Bug][WorkerServer] SqlTask NullPointerException (#5556 ) * [Bug][WorkerServer] add null checks to sqlParamsMap and sqlParamsMap.get(i), Add test to verify it * Change Test Name * Fix Code style issues, Modify checking null for Sql params in sqlParamsMap with clearer logging.	3 years ago
kyoty	67711442d5	[Improvement][SQL] Query return number In SQL should be configurable (#5632 ) * [Improvement][SQL] Query return number should be configurable #5580 * Update SqlTaskTest.java * Update sql.vue	3 years ago
Shiwen Cheng	8bf042ae6e	[Fix-5596][Python] Fix conflict between python_home and datax_home configuration in dolphinscheduler_env.sh (#5612 )	3 years ago
ruanwenjun	9d0c816cee	[Bug][worker] task throw ConcurrentModifiedException #5528 (#5530 ) * [Bug][worker] task throw ConcurrentModifiedException #5528 * fix code smell	3 years ago
Shiwen Cheng	b537f4eb26	[Improvement][Docker/K8s] Support more configurations and service accesses, reduce the duplication configurations, improve image for python and the compatibility of python home (#5393 ) * [Improvement][Docker] Support more configurations and improve image for python * [Improvement][K8s] Support more configurations, more service accesses, and reduce the duplication configurations * [Improvement][Python] Improve the compatibility of python home * [Improvement][Install] Fix install config and install.sh * [Improvement][Install] Fix workflow ut * [Improvement][Docker] Optimize docker-swarm/check script * [Improvement][DB] Update default username and password of database * Update comments in master/worker.properties * Specify version for all images * [Improvement][Docker] Optimize PS1 and WORKDIR * [Improvement][SQL] Reduce heap size to 64m in create-dolphinscheduler.sh and upgrade-dolphinscheduler.sh * [Fix-5431][K8s] Fix master and worker cannot get the right address with custom DNS	3 years ago
Yao WANG	bc22ae7c91	[Improvement][Task] Remove deprecated TaskRecordDao and simply the after() method in the AbstractTask class (#5492 ) * remove the depreated data quality module * fix lint Co-authored-by: Yao <ywang46@paypal.com>	3 years ago
baoliang	dd80f98196	update sleep exception	3 years ago
baoliang	7aa8b518ac	Merge remote-tracking branch 'upstream/dev' into json_split # Conflicts: # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java	3 years ago
JinyLeeChina	515fa4912e	[Feature][JsonSplit] Fix ProcessService ut (#5435 ) * Fix ProcessService ut * codeStyle * Fix AlertPlugin ut * Fix AlertPlugin ut * Fix server module ut * Fix server module ut * Fix api ut * codeStyle Co-authored-by: JinyLeeChina <297062848@qq.com>	3 years ago
baoliang	c850b85484	Merge remote-tracking branch 'upstream/dev' into json_split # Conflicts: # dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/procedure/ProcedureTask.java	3 years ago
zhuangchong	88a07f7b55	[Feature-4093][server] Support for stored procedures and stored function calls and data source supports DB2 (#4094 ) * Support for stored procedures and stored function calls and data source supports DB2. Co-authored-by: zhuangchong <zhuangchong8@163.com>	3 years ago
ruanwenjun	e6d8da484f	[Improvement][API] Create Datasource #5083 (#5116 ) * [Improvement][API] Create Datasource #5083 * Refact create/ update datasource API * fix Code Smells * fix code smell * resolve code smell * use ConnectionParam to replace BaseDatasource * solve license * add ut * code check * add ut * fix ut coverage * fix ut	3 years ago
Wenjun	abdd2337b1	[Bug][worker] Fix sql task query result is empty (#5322 ) (#5324 ) * [Bug][worker] Fix sql task query result is empty (#5322) * add ut * remove duplicate * add ut	3 years ago
JinyLeeChina	f7a06a033d	[Feature][JsonSplit] Fix master/worker about shellTask (#5300 ) * Fix quoted bug about processDefineId of processInstance * Fix master/worker about shellTask * codeStyle * codeStyle * update taskType * codeStyle * update timeoutNotifyStrategy * codeStyle Co-authored-by: JinyLeeChina <297062848@qq.com>	3 years ago
wenjun	cc7a4446f5	[Improvement][Task] DataX Node doesn't Support Clickhouse as datasource (#5180 ) (#5243 ) * [Improvement][Module Name] DataX Node doesn't Support Clickhouse as the datasource (#5180) * add ut * add ut * resolve code smell	3 years ago
zhuangchong	e53369318b	[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal (#4722 ) * fix the shell starts the yarn task in the background * update StringUtils code style. * solve code smell. * add method comment in StringUtils class. * update AlertGroupMapper code style. * update AlertGroupMapperTest * update sql script test.	3 years ago
guohaozhang	a6a1b94df4	[Feature-#5128]Support turning off sudo permissions (#5129 ) * [Feature-#5128]Support turning off sudo permissions * [Feature-#5128] without sudo permission do not create tenant user	3 years ago
xiaojingXU	9c04396218	[Bug-5132][server] stop the take immediately if an exception occurs in the taskExecuteThread (#5133 ) * [DS-5132][fix] stop the take immediately if an exception occurs in the taskExecuteThread - let softkill() method return true when the process is stopped,else return false;	3 years ago
Shiwen Cheng	8ac72e80e6	[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask (#5088 ) * [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask * [Feature-5087][SqlTask] Add SqlParametersTest	3 years ago
Shiwen Cheng	1cd62b4a5e	[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py (#5019 )	3 years ago
mrproliu	0ba120e8ab	Fix shell task ${setValue(key,value)} is not working (#5067 ) Co-authored-by: dalekliuhan <dalekliuhan@didiglobal.com>	3 years ago
游侠	2d8c4ec86b	[Improvement][Worker-Task] Python Command (#5036 ) * create the interpreter for python command * Update README.md Co-authored-by: zhouhong <zhouhong@zhangyue.com>	3 years ago
Shiwen Cheng	ae608e024a	[Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping (#5038 ) * [Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping * [Improvement][*] Replace commons.lang.StringUtils with common.utils.StringUtils	3 years ago
Shiwen Cheng	beb5adb985	[Feature][MR] Support mapreduce name (#5029 )	3 years ago
Shiwen Cheng	e74932cfc2	[Feature-4960][Spark] Add name input for Spark and improve Spark & MR args (#4959 ) * [Improvement][Spark] Improve spark args and add name input * [Improvement][MR] Improve map reduce args * [Improvement][Spark] Fix check style * [Improvement][Spark] Fix check style for spark, flink and mr	3 years ago
zhuangchong	5bfe3fca67	[Fix][SqlTask] fix hive sqltask invalid connection parameters. (#4917 ) * fix hive sqltask invalid connection parameters. * add setConnParams test method. * update base datasource test class code style. * add BaseDataSourceTest in the pom xml. * update hive conf list jdbc url stitching. * update hive datasource test class.	3 years ago
小清	9154d9b7b5	Update AbstractCommandExecutor.java	3 years ago
wangxj3	874ca07e6e	[Feature-3805][mater-worker-ui]fix function of passing variables between tasks bug (#4811 ) * global initParam and set Param * fix dataFormat error * fix deal outParams bug * fix code style * fix code style * fix code style * fix code style * fix code style * fix code style * fix code style * add test * fix code style (variable name) * fix reset globalParams bug * fix format bug * fix SonarCloud Code Analysis * add test * fix code style * fix code style * modify test * fix test Coverage * fix code style * 4 test * add test pom * add header Co-authored-by: wangxj <wangxj31>	3 years ago
小清	dc7a0a0f0e	Update AbstractCommandExecutor.java	3 years ago
lenboo	90e197a4c5	add return result	3 years ago
Shiwen Cheng	0c26e10f3f	[Fix-4738][Mail] Fix send mail failed instead of execute sql error (#4739 )	3 years ago
fanwq	559efac58d	[Feature-#3805][server-worker]global params of worker (#4679 ) * return shell output to master node * return sql output to master node * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug * fix bug	3 years ago
zhuangchong	a5443f1173	[fix-4128][worker] Dolphin executes the command using sudo to specify the -u parameter (#4217 ) * update sudo cmd. * update sudo cmd. * update OSUtils code style. * add OSUtils test method.	3 years ago
zhuangchong	03a2c6b7b7	[Improvement-4450][datasource] Hive/Spark data sources do not support multi-tenancy when Kerberos authentication is enabled (#4593 ) * hive/spark jdbc add kerberos paramter. * update datasource vue paramter placeholder. * update code style. * add datasource ApiImplicitParam in the kerberos paramter. * update constants class code style. * add code test and code style. * update BaseDataSourceTest code style. * update BaseDataSourceTest class code style. * update DataSourceController class code style. * update DataSourceController ApiImplicitParam * update ui create data source vue.	3 years ago
小清	4597e87970	Update AbstractCommandExecutor.java	3 years ago
小清	f30f4cb3d4	Update AbstractCommandExecutor.java 将inReader.readLine()和logBuffer 分开，改成两个线程来处理。	3 years ago
小清	c61d0a7174	Update AbstractCommandExecutor.java	3 years ago
小清	cd89aed360	Update AbstractCommandExecutor.java	3 years ago
小清	7b345f697a	Update AbstractCommandExecutor.java	3 years ago
小清	d5c4c290d3	Update AbstractCommandExecutor.java	3 years ago

1 2 3 4

168 Commits (5a12e00cbe926b40162b459b94ed74b7e7f25903)