DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
Bharati Kulkarni	2b2adecd85	[Improvement] Fixed Possible JSON ordering permutations problem in Tests (#13118 ) * Fixed flaky test * Fixed imports order	2 years ago
insist777	b40c63f02d	[Feature][DataSource]Add trino datasource plugin (#12911 )	2 years ago
caishunfeng	dedff70f90	[Improvement] Merge spi.utils into common.utils (#12502 ) * improve JSONUtil * merge spi.utils into common.utils * split constants && remove some copy utils method * remove StringUtils	2 years ago
Eric Gao	d46e4659d9	[Improvement][Style] Fix remaining style errors and wildcard imports (#12412 ) * Fix remaining style errors and wildcard imports * Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link * Fix spotless matching scope of markdown files	2 years ago
Yann Ann	b357781df2	[Migrate][Test] Migrate junit4 -> junit5 test cases in common, service, and spi modules (#12363 )	2 years ago
kezhenxu94	db1d319159	Reorganize some classes in common module, remove duplicate classes (#12321 )	2 years ago
Eric Gao	9ab79e064c	[Improvement][Test] Fully remove the usage of powermock from the whole project (#12244 ) * Fully remove the usage of powermock from the whole project * Upgrade org.reflections to 0.10.12	2 years ago
Yann Ann	c87f6d89a4	[fix#12195] Change date to timestamp to support cross diff time zones (#12239 ) * fix DS support cross time zone, use timestamp replace date	2 years ago
zhuxt2015	e6832220c3	[Improvement][SQL Task]use default sql segment separator (#10869 )	2 years ago
Eric Gao	66958b91d7	[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708 ) * Add support for Alibaba Cloud OSS as storage of resource center * Add dependency licenses * Refactor code, add unit tests and related docs	2 years ago
ZhaoGuodong	86291bb5e6	[Feature-10273][Datasource] Add dataSource for AWS Athena (#11157 )	2 years ago
Wenjun Ruan	b96d69701a	[Improvement-#11613] Add spi priority factory (#11614 ) * Add spi priority factory * Add doc * Add override log * Use lombok * Add comment	2 years ago
Wenjun Ruan	71cf7e1baa	[Fix-10918] Close datasource when expire from guava cache (#11120 ) * Close datasource when expire from guava cache * Remove duplicate datasource in HiveDataSourceClient	2 years ago
Wenjun Ruan	5e9c7dad23	Add dolphinscheduler-bom to manage the dependency version (#11025 )	2 years ago
Eric Gao	71f0168510	[Feature] Enable users to create python env from requirements.txt (#10658 )	2 years ago
HomminLee	dafed572d1	[Fix][dolphinscheduler-spi] json parse support LocalDateTime (#10440 ) * JSONUtils support LocalDateTime * remove extraneous unit tests * remove unused classes import. Co-authored-by: HomminLee <lihongmin@ysstech.com>	2 years ago
kezhenxu94	d80cf21456	Clean up unused dependencies and packaging issues (#9944 )	2 years ago
Tq	592e433b1e	[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276 ) * replace DataSourceController API with string JSON and cast to DTO * add DataSourceProcessorManager and DataSourceProcessorProvider to datasource processor functions	2 years ago
WangJPLeo	31cd1b5e61	Serial wait for subsequent execution (#9847 )	3 years ago
Eric Gao	fd6b43bc81	[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223 ) (#9299 )	3 years ago
caishunfeng	263791a63e	[Bug-9137][Server] fix timezone init (#9219 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	3 years ago
ronyang1985	7b66b66e8f	[Feature-8527][Datasource] Add dataSource for Amazon Redshift (#8564 ) Co-authored-by: Wenjun Ruan <wenjun@apache.org>	3 years ago
ronyang1985	20ee9a37fe	[Feature-8485][task] Add EMR task plugin to support submit task to AWS EMR (#8503 ) Co-authored-by: Wenjun Ruan <wenjun@apache.org> Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	3 years ago
yimaixinchen	a72742b9de	remove unuse import jsonFormat (#8548 )	3 years ago
Molin Wang	e174d7a40e	[Fix-8394][Server]fix when the data is supplemented business time still date+1 (#8532 )	3 years ago
caishunfeng	84f2cfec39	[Feature][Server] Custom timezone - Add timezone handler unified (#8477 ) * date convert of timezone * remove @JsonFormat * add unit test * fix time preview in scheduler * optimization & add env config Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
caishunfeng	f2b9796bc2	[Bug-8110][WorkerServer] kill all running task before worker stop (#8490 ) * kill all running task before worker stop * kill local process * remove kill yarn job * adjust the orders of close Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Ivan0626	60ddede6a6	fixbug: varPool value is error (#7724 )	3 years ago
cheney	703d790415	[Bug] [Parameter Context] The preparameter OUT is null (#8315 ) * fix:param context prop value is null	3 years ago
zixi0825	4ba6d667fc	[Feature][DataQuality] Add Data quality Module #4283 (#6718 ) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA * fix Constants bug * remove the unused class * add unit test * fix code style error * add unit test * refactor data quality common entity * fix code style error * add unit test * close e2e test * fix code smell bug * modify dataquality enum value to 14 in TaskType * add data qualtiy task * update * add getDatasourceOptions interface * fix checkstyle * close e2e test * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * add unit test and alter dataquality task result * fix checkstyle * fix process service test error * add unit test and fix code smells * fix checkstyle error * fix unit test error * fix checkstyle error * change execute sql type name * revert ui pom.xml * fix data quality task error * fix checkstyle error * fix dq task src_connector_type ui select bug * fix spark rw postgresql bug * change mysql driver scope * fix form-create json bug * fix code smell * fix DolphinException Bug * fix ui validate rule and Alert title * fix target connection param bug * fix threshold validate change * add rule input entry index * change statistic_comparison_check logic * remove check type change * add DateExpressionReplaceUtil * fix null point expetion * fix null point expetion * fix test error * add more sql driver * fix test error and remove DateExprReplaceUtil * add get datasource tables and columns * add get datasource tables and columns * remove hive-jdbc in pom.xml * fix code smells * update sql * change the pom.xml * optimize multi_table_accuracy ui * fix v-show error * fix code smells * update sql * [Feature][DataQuality] Add data quality task ui (#5054) * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * fix dq task src_connector_type ui select bug * fix threshold validate change * remove check type change * add get datasource tables and columns * optimize multi_table_accuracy ui * fix v-show error * fix code smells Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality module (#4830) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * update * close e2e test * fix spark rw postgresql bug * change mysql driver scope * add more sql driver * remove hive-jdbc in pom.xml * change the pom.xml Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality task backend (#4883) * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * refactor data_quality_module * add header license * data quality module refactor * fix unit test error * fix checkstyle error * fix unit test error * fix checkstyle error * fix unit test error * fix code smell * fix check style * fix unit test error * task statistics value add unique code * fix unit test error * fix checkstyle error * fix checkstyle * fix security hotspot * fix unit test error * fix security hotspot * fix check * add data quality task error handling * fix unit test error * add unit test * add unit test * optimize data quality result alert * fix unit test * fix sql script error * fix bug * update sql script * fix checkstyle * add license * fix checkstyle * fix checkstyle * fix unit test * add jacoco dependencies * fix unit test * fix unit test * add jacoco dependencies * add unit test * add unit test * add license * fix checkstyle * fix pom * fix checkstyle * fix checkstyle * merge dev * fix ui error * fix pom error * fix pom error * fix test error * fix test error * mssql-jdbc exclude azure-keyvault * fix test error * merge dev and add unit test * add notes * rollback the CollectionUtils * fix * update sql * fix * fix * fix query rule page error * change dq.jar path * fix sql error * fix ui error * fix(dq): jar path&task enum description * add notes on DataQualityApplication * fix dq result jump error * fix(ui): page condition * feat(ui): add show error output path * change version * remove all chinese word in sql * merge Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>	3 years ago
Kerwin	103ccb01da	change version to 2.0.4-SNAPSHOT (#8188 )	3 years ago
mask	898727709f	[fix-7671][plugin] Supports whether SQL is placed in the same session for configuration(fix-7671) (#7675 ) -Supports whether SQL is placed in the same session for configuration	3 years ago
wind	e4cdd82b7c	[Improvement][TaskLog] Unified task log (#7831 ) * task logger * update * log format * remove master config task-logger * udpate log level Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
mask	8e62dd8246	[Fix-7291][datasource] Compatible with different versions of MySQL data source loading (#7291 ) (#7305 )	3 years ago
kezhenxu94	67cc260d52	Split the components into individual package, refactor configurations to use Spring properties (#7160 ) * Split the components into individual package A follow-up PR will be made to build dedicated Docker images for each component, so that every component Docker image has minimal jars, which is easy to maintain and good for security fixes. * Split the components into individual package A follow-up PR will be made to build dedicated Docker images for each component, so that every component Docker image has minimal jars, which is easy to maintain and good for security fixes. * Split the components into individual package A follow-up PR will be made to build dedicated Docker images for each component, so that every component Docker image has minimal jars, which is easy to maintain and good for security fixes.	3 years ago
Kerwin	802fc498b5	[Fix-#6605][Common] Fix the situation where the obtained string property is empty. (#6661 ) * Fix the situation where the obtained string property is empty. * recovery test * update code style. * megre dev.	3 years ago
yihong	861aaaf971	[FIX-#6505][Dao] upgrade the MySQL driver package for building MySQL jdbcUrl (#6708 ) * replace com.mysql.jdbc.Driver everywhere with com.mysql.cj.jdbc.Driver	3 years ago
kezhenxu94	4114cb07f6	Add actuator and enable metrics, add Grafana configs (#6840 )	3 years ago
kezhenxu94	11844415b0	Refactor datasource and task plugins and simplify its usage (#6767 )	3 years ago
mask	0aea5ae11a	[feature-6071][plugin] add datasource plugin. (#6071 ) (#6361 ) * [feature-6071][plugin] add datasource plugin. (#6071) * remove the dependency of datasource in common and task-plugin module. * the new datasource-api module provides data source connection acquisition. * add sqlserver data source plug-in. * add ds license * fix check style * update spi pom * frx conflict resolution * fix check style * fix datasource ut * fix code check style * fix code check style * fix datasource ut * add pg hive mysql oracle clickhouse db2 datasource * fix datasource package * update datasource pom * update datasource pom * update datasource pom * update datasource pom * update pom * update pom dep * update pom dep * fix license * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * fix ut * add oracle package * modify comments * conflict resolution * rename log comment * rename log comment	3 years ago
kezhenxu94	05bae0b226	Refactor alert plugins and simplify its usage (#6723 )	3 years ago
kezhenxu94	5400abc74f	Refactor registry plugin and simplify its usage (#6712 )	3 years ago
kezhenxu94	5e923971bf	Migrate http to https in maven address (#6698 )	3 years ago
wind	72daba58cd	[DS-6694][Master] Reduce the refresh resource interval of LowerWeight (#6695 ) * [DS-6694][Master] Reduce the refresh resource interval of LowerWeight * add server node update event handle Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
LiuBodong	71047db39e	[FIX][Standalone Server] check if maven settings.xml not exists (#6691 )	3 years ago
LiuBodong	61a8726ccc	[Fix][Standalone Server] fix plugin load failed when local maven repository path is not default value (#6625 ) (#6626 ) * [Fix][Standalone Server] fix plugin load failed when local maven repository path is not default value	3 years ago
jon-qj	86c85114b2	make alert instance page display more friendly (#6635 ) * make alert instance page display more friendly * Update EmailAlertChannelTest.java	3 years ago
mask	9b1984c244	[fix-6570][task] add sql task udf resource load (#6570 ) (#6571 ) * [fix-6570][task] add sql task udf resource load (#6570) -add sql task udf resource load -Fix UdfFuncRequest serialization * fix checkstyle * add defaultFS in sqlTaskContext * fix checkstyle	3 years ago
Hua Jiang	a80fca70f7	[Fix-6497][Server] Fix the missing code about the feature of supporting multiple environments. (#6509 )	3 years ago

1 2

94 Commits (6ef74073cc21b283673605a1f5dd6cb200ef1c4c)