DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
chenrj	c532fea00e	[Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977 )	1 year ago
chenrj	5a3827eef4	[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks (#14976 ) * Refactoring `Sagemaker` task plugin with connections managed in connection center. --------- Co-authored-by: Eric Gao <ericgao.apache@gmail.com>	1 year ago
chenrj	e99c5950b9	[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks (#14434 ) * Refactoring zeppelin task plugin with connections managed in connection center --------- Co-authored-by: Eric Gao <ericgao.apache@gmail.com>	1 year ago
Wenjun Ruan	4aab0b234f	Use AdHoc datasource client in sqlTask (#14631 ) * Use AdHoc datasource client in sqlTask * Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient	1 year ago
Xinxing	1d0d85b988	[Feature] doris add datasource (#14391 ) * Added doris data source plug-in * Update DbType.java * XML configuration * doc and ui * Missing catch of NumberFormatException * code format --------- Co-authored-by: xiangzihao <460888207@qq.com>	1 year ago
c3Vu	50195c46aa	[Improvement-11913] Mask password when creating/editing alert instances (#14415 ) * [Improvement-11913] Mask password when creating/editing alert instances	1 year ago
IT-Kwj	7a24be0672	[Feature-14404][datasource-plugin] add HANA datasource (#14404 ) --------- Co-authored-by: WeiJian Kong <> Co-authored-by: xujiaqiang <Xjq634828243>	1 year ago
Zinway	18d176c5f4	feat: Introduce new datasource vertica (#14232 ) Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>	2 years ago
He Zhao	f1e78627fc	[feature][plugin] Add snowflake datasource in datasource plugin (#13729 ) --------- Co-authored-by: weitong <weitong@cisco.com>	2 years ago
Jeremy	db6c061c8b	[Feature][plugin] Add databend datasource in datasource plugin (#13866 ) Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>	2 years ago
He Zhao	c0126b7f30	[Feature][plugin] Support Kyuubi datasource (#13642 ) --------- Co-authored-by: weitong <weitong@cisco.com> Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>	2 years ago
Aaron Wang	4bf097ee40	[Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath (#13848 ) * [Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath * remove semicolon * set null in some specific scenario * remove useless console.log	2 years ago
JieguangZhou	273a5e78eb	[FEATURE][Task Plugin]Add remote-shell task plugin (#13801 )	2 years ago
Jay Chung	6a33d2bec0	fix: Alarm instance port verification (#13827 ) close: #13532	2 years ago
He Wang	35d8e8ac17	[Feature][DataSource] add oceanbase datasource (#13675 ) Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>	2 years ago
lenian	bcf03ad4d1	[Feature-9467] add DAMENG DataSource (#12860 ) * [Feature-9467][DataSource] add DAMENG （达梦） data source * [Feature-9467] add DM datasource doc (Issue #9467) * [Feature-9467] add DM IDataBaseOptionKeys * [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method * [Feature-9467] modification package dm to dameng; * [Feature-9467] add Dameng DmJdbcDriver18 License * [Feature-9467] add DM DataSource: remove DruidDataSourceClient Co-authored-by: lenian <yuenian@mananacare.com>	2 years ago
Tq	f0dd4f081f	[Feature][task-plugins-12820] azure azure-sql datasource plugins (#13209 ) * submit azure sql code * change mssql-jdbc version * merge changes to delete other * merge changes to delete other * add dependencies	2 years ago
Wenjun Ruan	5c75d312fa	Add StarRocks datasource (#13204 )	2 years ago
Wenjun Ruan	ab96a3d0c0	Fix datasource jdbc url param doesn't work (#13098 )	2 years ago
Bharati Kulkarni	2b2adecd85	[Improvement] Fixed Possible JSON ordering permutations problem in Tests (#13118 ) * Fixed flaky test * Fixed imports order	2 years ago
insist777	b40c63f02d	[Feature][DataSource]Add trino datasource plugin (#12911 )	2 years ago
caishunfeng	dedff70f90	[Improvement] Merge spi.utils into common.utils (#12502 ) * improve JSONUtil * merge spi.utils into common.utils * split constants && remove some copy utils method * remove StringUtils	2 years ago
Eric Gao	d46e4659d9	[Improvement][Style] Fix remaining style errors and wildcard imports (#12412 ) * Fix remaining style errors and wildcard imports * Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link * Fix spotless matching scope of markdown files	2 years ago
Yann Ann	b357781df2	[Migrate][Test] Migrate junit4 -> junit5 test cases in common, service, and spi modules (#12363 )	2 years ago
kezhenxu94	db1d319159	Reorganize some classes in common module, remove duplicate classes (#12321 )	2 years ago
Eric Gao	9ab79e064c	[Improvement][Test] Fully remove the usage of powermock from the whole project (#12244 ) * Fully remove the usage of powermock from the whole project * Upgrade org.reflections to 0.10.12	2 years ago
Yann Ann	c87f6d89a4	[fix#12195] Change date to timestamp to support cross diff time zones (#12239 ) * fix DS support cross time zone, use timestamp replace date	2 years ago
zhuxt2015	e6832220c3	[Improvement][SQL Task]use default sql segment separator (#10869 )	2 years ago
Eric Gao	66958b91d7	[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708 ) * Add support for Alibaba Cloud OSS as storage of resource center * Add dependency licenses * Refactor code, add unit tests and related docs	2 years ago
ZhaoGuodong	86291bb5e6	[Feature-10273][Datasource] Add dataSource for AWS Athena (#11157 )	2 years ago
Wenjun Ruan	b96d69701a	[Improvement-#11613] Add spi priority factory (#11614 ) * Add spi priority factory * Add doc * Add override log * Use lombok * Add comment	2 years ago
Wenjun Ruan	71cf7e1baa	[Fix-10918] Close datasource when expire from guava cache (#11120 ) * Close datasource when expire from guava cache * Remove duplicate datasource in HiveDataSourceClient	2 years ago
Eric Gao	71f0168510	[Feature] Enable users to create python env from requirements.txt (#10658 )	2 years ago
HomminLee	dafed572d1	[Fix][dolphinscheduler-spi] json parse support LocalDateTime (#10440 ) * JSONUtils support LocalDateTime * remove extraneous unit tests * remove unused classes import. Co-authored-by: HomminLee <lihongmin@ysstech.com>	2 years ago
Tq	592e433b1e	[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276 ) * replace DataSourceController API with string JSON and cast to DTO * add DataSourceProcessorManager and DataSourceProcessorProvider to datasource processor functions	3 years ago
WangJPLeo	31cd1b5e61	Serial wait for subsequent execution (#9847 )	3 years ago
caishunfeng	263791a63e	[Bug-9137][Server] fix timezone init (#9219 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	3 years ago
ronyang1985	7b66b66e8f	[Feature-8527][Datasource] Add dataSource for Amazon Redshift (#8564 ) Co-authored-by: Wenjun Ruan <wenjun@apache.org>	3 years ago
ronyang1985	20ee9a37fe	[Feature-8485][task] Add EMR task plugin to support submit task to AWS EMR (#8503 ) Co-authored-by: Wenjun Ruan <wenjun@apache.org> Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>	3 years ago
yimaixinchen	a72742b9de	remove unuse import jsonFormat (#8548 )	3 years ago
Molin Wang	e174d7a40e	[Fix-8394][Server]fix when the data is supplemented business time still date+1 (#8532 )	3 years ago
caishunfeng	84f2cfec39	[Feature][Server] Custom timezone - Add timezone handler unified (#8477 ) * date convert of timezone * remove @JsonFormat * add unit test * fix time preview in scheduler * optimization & add env config Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
caishunfeng	f2b9796bc2	[Bug-8110][WorkerServer] kill all running task before worker stop (#8490 ) * kill all running task before worker stop * kill local process * remove kill yarn job * adjust the orders of close Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Ivan0626	60ddede6a6	fixbug: varPool value is error (#7724 )	3 years ago
cheney	703d790415	[Bug] [Parameter Context] The preparameter OUT is null (#8315 ) * fix:param context prop value is null	3 years ago
zixi0825	4ba6d667fc	[Feature][DataQuality] Add Data quality Module #4283 (#6718 ) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA * fix Constants bug * remove the unused class * add unit test * fix code style error * add unit test * refactor data quality common entity * fix code style error * add unit test * close e2e test * fix code smell bug * modify dataquality enum value to 14 in TaskType * add data qualtiy task * update * add getDatasourceOptions interface * fix checkstyle * close e2e test * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * add unit test and alter dataquality task result * fix checkstyle * fix process service test error * add unit test and fix code smells * fix checkstyle error * fix unit test error * fix checkstyle error * change execute sql type name * revert ui pom.xml * fix data quality task error * fix checkstyle error * fix dq task src_connector_type ui select bug * fix spark rw postgresql bug * change mysql driver scope * fix form-create json bug * fix code smell * fix DolphinException Bug * fix ui validate rule and Alert title * fix target connection param bug * fix threshold validate change * add rule input entry index * change statistic_comparison_check logic * remove check type change * add DateExpressionReplaceUtil * fix null point expetion * fix null point expetion * fix test error * add more sql driver * fix test error and remove DateExprReplaceUtil * add get datasource tables and columns * add get datasource tables and columns * remove hive-jdbc in pom.xml * fix code smells * update sql * change the pom.xml * optimize multi_table_accuracy ui * fix v-show error * fix code smells * update sql * [Feature][DataQuality] Add data quality task ui (#5054) * add data quality task ui * update skywalking-eyes SHA * fix style * fix eslint error * fix eslint error * test e2e * fix dq task src_connector_type ui select bug * fix threshold validate change * remove check type change * add get datasource tables and columns * optimize multi_table_accuracy ui * fix v-show error * fix code smells Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality module (#4830) * add data quality module * add license * add package configuration in dist pom * fix license and jar import bug * replace apache/skywalking-eyes@9bd5feb SHA * refacotr jbdc-connector and writer * modify parameter name in HiveConnector * fix checkstyle error * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix checkstyle error in dolphinschesuler-dist * fix duplicate code bug * fix code style bug * fix code smells * update * close e2e test * fix spark rw postgresql bug * change mysql driver scope * add more sql driver * remove hive-jdbc in pom.xml * change the pom.xml Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * [Feature][DataQuality] Add data quality task backend (#4883) * add dq relevant enums and parameter * replace apache/skywalking-eyes@9bd5feb SHA Co-authored-by: sunchaohe <sunzhaohe@linklogis.com> * refactor data_quality_module * add header license * data quality module refactor * fix unit test error * fix checkstyle error * fix unit test error * fix checkstyle error * fix unit test error * fix code smell * fix check style * fix unit test error * task statistics value add unique code * fix unit test error * fix checkstyle error * fix checkstyle * fix security hotspot * fix unit test error * fix security hotspot * fix check * add data quality task error handling * fix unit test error * add unit test * add unit test * optimize data quality result alert * fix unit test * fix sql script error * fix bug * update sql script * fix checkstyle * add license * fix checkstyle * fix checkstyle * fix unit test * add jacoco dependencies * fix unit test * fix unit test * add jacoco dependencies * add unit test * add unit test * add license * fix checkstyle * fix pom * fix checkstyle * fix checkstyle * merge dev * fix ui error * fix pom error * fix pom error * fix test error * fix test error * mssql-jdbc exclude azure-keyvault * fix test error * merge dev and add unit test * add notes * rollback the CollectionUtils * fix * update sql * fix * fix * fix query rule page error * change dq.jar path * fix sql error * fix ui error * fix(dq): jar path&task enum description * add notes on DataQualityApplication * fix dq result jump error * fix(ui): page condition * feat(ui): add show error output path * change version * remove all chinese word in sql * merge Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>	3 years ago
mask	898727709f	[fix-7671][plugin] Supports whether SQL is placed in the same session for configuration(fix-7671) (#7675 ) -Supports whether SQL is placed in the same session for configuration	3 years ago
wind	e4cdd82b7c	[Improvement][TaskLog] Unified task log (#7831 ) * task logger * update * log format * remove master config task-logger * udpate log level Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
mask	8e62dd8246	[Fix-7291][datasource] Compatible with different versions of MySQL data source loading (#7291 ) (#7305 )	3 years ago

1 2

100 Commits (c3ffb1330de88e9fa8cd0368a0d0991d076ae53c)