DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
Wenjun Ruan	5b6b0ceb31	Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355 )	12 months ago
Wenjun Ruan	71ee1f0c3d	Support parse task output params under multiple log (#15244 )	1 year ago
Wenjun Ruan	4aab0b234f	Use AdHoc datasource client in sqlTask (#14631 ) * Use AdHoc datasource client in sqlTask * Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient	1 year ago
Jay Chung	de2d215aee	impv: Correct datax execute and python execute script name (#14523 ) * impv: Correct datax execute and python execute script name we use PYTHON_LAUNCHER for python script execute and DATAX_LAUNCHER for datax script name * Add pr number * fix ut * style --------- Co-authored-by: xiangzihao <460888207@qq.com>	1 year ago
Wenjun Ruan	c30cca9d9a	Support execute shell in different interceptor (#14582 )	1 year ago
Jeremy	9c92b4b240	fix: default datax plugin (#14204 )	1 year ago
Rick Cheng	b49b91c45d	[Improvement-14331][common] Remove the duplicate ParameterUtils (#14332 )	1 year ago
Jeremy	db6c061c8b	[Feature][plugin] Add databend datasource in datasource plugin (#13866 ) Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>	2 years ago
fuchanghai	ec29220977	[improve-#13966] mask datax log (#13969 ) Co-authored-by: fuchanghai <‘2875334588@qq.com’>	2 years ago
Aaron Wang	d6fcf60534	[Improvement-13751][Worker] Support real-time pod log collection (#13752 ) * Support real-time pod log collection * datax task test have to mock ShellCommandExecutor with whole dependencies * replace \n with EMPTY_STRING * modify error log msg * Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java Co-authored-by: caishunfeng <caishunfeng2021@gmail.com> * throw exception in pod log collection thread --------- Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	2 years ago
Wenjun Ruan	a1861bc2df	Use DATAX_PYTHON to specify a datax python version (#13849 ) (cherry picked from commit 752452ecbd2f8094b819eae219f3eec0f6a6427b)	2 years ago
fuchanghai	6218f818c0	[improve-#13665] improve datax parser if sql contain * (#13680 ) * [improve-#13665] improve datax parser if sql contain * * Update dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxTask.java Co-authored-by: fuchanghai <‘2875334588@qq.com’> Co-authored-by: Aaron Wang <wangweirao16@gmail.com>	2 years ago
fuchanghai	79d3ed0744	Update DataxUtils.java (#13604 )	2 years ago
fuchanghai	7bf3e3cdd6	[improve-#13201] update pid during running (#13206 )	2 years ago
fuchanghai	37939add0c	[fix-13492]add single quotes (#13534 ) * [fix-13492]add single quotes * [improve-#13201] fix test error --------- Co-authored-by: fuchanghai <‘2875334588@qq.com’>	2 years ago
seedscoder	8d12dc0702	[Improvement-13491] Use lombok @Slf4j annotation to generate logger (#13509 )	2 years ago
Wenjun Ruan	70ccffeee2	Format task parameter as pretty json (#13173 )	2 years ago
Wenjun Ruan	92a9b7e243	Use System#lineSeparator to replace /n (#12889 )	2 years ago
caishunfeng	dedff70f90	[Improvement] Merge spi.utils into common.utils (#12502 ) * improve JSONUtil * merge spi.utils into common.utils * split constants && remove some copy utils method * remove StringUtils	2 years ago
Eric Gao	d46e4659d9	[Improvement][Style] Fix remaining style errors and wildcard imports (#12412 ) * Fix remaining style errors and wildcard imports * Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link * Fix spotless matching scope of markdown files	2 years ago
Yann Ann	3bef85f546	fix datax NPE issue (#12388 )	2 years ago
fuchanghai	b7bd8d7803	[Improvement] add presto in datax (#12371 )	2 years ago
Eric Gao	875682d267	[Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example (#12299 ) * Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example	2 years ago
fuchanghai	e101a5cb2c	[fix-#11815] fix ck column names contain special characters (#11818 ) Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>	2 years ago
caishunfeng	2e61c76c22	[Improvement] Add remote task model (#11767 )	2 years ago
fuchanghai	f8b9aad239	[feature-#11689] change replaceAll to replace in method of tryExecuteSqlResolveColumnNames (#11696 )	2 years ago
Wenjun Ruan	1b120e3a59	Refactor worker execute task process (#11540 ) * Refactor worker execute task process	2 years ago
fuchanghai	d8d5d392a7	[feature-11408] add hive option (#11493 )	2 years ago
lugela	e5d06c40a5	[Feature-10182][task-datax] (#11387 ) In the process of realizing datax synchronization, use custom json for better connection to enable kerberos-authenticated clusters, such as using plug-ins such as hdfs and hbase	2 years ago
caishunfeng	0464123c2b	[Feature-11223] support stream task (#11350 ) * add task execute type * update task definition list paging * update task instance list paging * stream task start * [Feature][UI] Some changes to execute task. * Set the connection edge to dashed line. * Add FLINK_STREAM task. * add stream task * flink savepoint and cancel * fix query bug * add stream task definition * add task instance for stream task * delete stream task definition state * update api for stream task definition edit * modify search for stream task instance * add language * delete task type search for stream task definition * change task type search for stream task instance * add jump button * add savepoint * add down log for stream task instance * ui test * stream task start * run DAG * [Fix][UI] Fix the stream task edgs not to be dashed when filling back. * [Feature][UI] Remove some fields for FLINK_STREAM. * add start modal * add dryRun column for stream task instance * fix duration * fix pon * fix build error * Add success tip * add auto sync for stream task instance * remove forgien key for task instance * license header * UT fix * modify locales * recover common config * fix UT * add doc Co-authored-by: Amy <amywang0104@163.com> Co-authored-by: devosend <devosend@gmail.com>	2 years ago
zhuxt2015	7196b125d9	[Feature][Datax Task]Datax task plugin support custom parameters (#10950 ) * datax task plugin support custom parameters	2 years ago
WangJPLeo	8f621ff98b	[Optimization] Calculate global parameter and local parameter at master. (#10704 ) * Global parameter and local parameter calculation external expansion. * k8s task ut fix. * TimePlaceholderUtils import DateUtils fix * follow the review comments to fix. * follow the review comments to fix. * e2e rerun	2 years ago
zhuxt2015	7e7df1d8ff	[Bug] [Task] SQL task no exception stack trace in task log (#10656 )	2 years ago
xiangzihao	bbba20d39a	change to common lang3 and fix checkstyle (#10380 )	3 years ago
kezhenxu94	d80cf21456	Clean up unused dependencies and packaging issues (#9944 )	3 years ago
zhuxt2015	206b7c1c51	[Fix-9352][datasource-api]data source client uses a cache with an expiration time (#9353 ) * datasource client cache with expiration time * remove unused import * fixed unit test * fix * fix * fix * fix * fix * fix * fix	3 years ago
Eric Gao	fd6b43bc81	[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223 ) (#9299 )	3 years ago
Kerwin	d7d756e7b0	Fix DataX and Sqoop Task execute NullPointerException (#9192 )	3 years ago
Kerwin	25fc1dcb5f	Fix the exception of task getting external resource data during execution (#9134 )	3 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	3 years ago
Kerwin	103ccb01da	change version to 2.0.4-SNAPSHOT (#8188 )	3 years ago
seagle	f1a0215978	[bug] Fix datax task type error parse clickhouse reader sql (#8091 )	3 years ago
wind	e4cdd82b7c	[Improvement][TaskLog] Unified task log (#7831 ) * task logger * update * log format * remove master config task-logger * udpate log level Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Zonglei Dong	0b9792db76	[Refactor]Rename class file, DatasourceUtil to DataSourceUtils. (#7181 ) * Rename class file, DatasourceUtil to DataSourceUtils.	3 years ago
kezhenxu94	11844415b0	Refactor datasource and task plugins and simplify its usage (#6767 )	3 years ago
mask	0aea5ae11a	[feature-6071][plugin] add datasource plugin. (#6071 ) (#6361 ) * [feature-6071][plugin] add datasource plugin. (#6071) * remove the dependency of datasource in common and task-plugin module. * the new datasource-api module provides data source connection acquisition. * add sqlserver data source plug-in. * add ds license * fix check style * update spi pom * frx conflict resolution * fix check style * fix datasource ut * fix code check style * fix code check style * fix datasource ut * add pg hive mysql oracle clickhouse db2 datasource * fix datasource package * update datasource pom * update datasource pom * update datasource pom * update datasource pom * update pom * update pom dep * update pom dep * fix license * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * add oracle package * modify comments * conflict resolution * rename log comment * rename log comment	3 years ago
Kirs	67508d8d1b	update project version to 2.0.0-SNAPSHOT (#6383 )	3 years ago
Kirs	fab4eae923	[SPI][Task]fix Task parse Err (#6244 ) * [SPI][Task]fix Task parse Err	3 years ago
CalvinKirs	ac8fafa343	fix code style	3 years ago
CalvinKirs	2dd5279388	Merge remote-tracking branch 'upstream/dev' into task-end # Conflicts: # .asf.yaml # dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/FuncUtils.java # dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java # dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java # dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java # dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java # dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/AlertGroupControllerTest.java # dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java # dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java # dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/StateEventType.java # dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Command.java # dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ErrorCommand.java # dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java # dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java # dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ScheduleMapper.xml # dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionLogMapper.xml # dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/ConditionsTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/DependentTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/CommonTaskProcessor.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/ConditionTaskProcessor.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/DependentTaskProcessor.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/ConditionsTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/DependentTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/MasterBaseTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/MasterTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/SubProcessTaskExecThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/WorkerManagerThread.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/python/PythonTask.java # dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTask.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/master/WorkflowExecuteThreadTest.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/processor/TaskKillProcessorTest.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThreadTest.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/WorkerManagerThreadTest.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/PythonCommandExecutorTest.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/http/HttpTaskTest.java # dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTaskTest.java # dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java # dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/quartz/cron/CronUtils.java # dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java # dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/DolphinSchedulerPlugin.java # dolphinscheduler-standalone-server/pom.xml # dolphinscheduler-standalone-server/src/main/java/org/apache/dolphinscheduler/server/StandaloneServer.java # dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/ShellCommandExecutor.java # dolphinscheduler-task-plugin/dolphinscheduler-task-procedure/src/main/java/org/apache/dolphinscheduler/plugin/task/procedure/ProcedureTaskChannelFactory.java # dolphinscheduler-task-plugin/dolphinscheduler-task-python/src/main/java/org/apache/dolphinscheduler/plugin/task/python/PythonCommandExecutor.java # dolphinscheduler-task-plugin/dolphinscheduler-task-shell/src/main/java/org/apache/dolphinscheduler/plugin/task/shell/ShellTask.java # dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java # dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/SqoopTask.java # dolphinscheduler-ui/package.json # dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.scss # dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.vue # dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/formModel/formModel.vue # sql/dolphinscheduler_h2.sql	3 years ago

1 2

55 Commits (3bc88e32c76e1ca3526c67b0c3bf77d420967f64)