c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center ( #14977 )
1 year ago
5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks ( #14976 )
...
* Refactoring `Sagemaker` task plugin with connections managed in connection center.
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
1 year ago
e99c5950b9
[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks ( #14434 )
...
* Refactoring zeppelin task plugin with connections managed in connection center
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
1 year ago
4aab0b234f
Use AdHoc datasource client in sqlTask ( #14631 )
...
* Use AdHoc datasource client in sqlTask
* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
1 year ago
1d0d85b988
[Feature] doris add datasource ( #14391 )
...
* Added doris data source plug-in
* Update DbType.java
* XML configuration
* doc and ui
* Missing catch of NumberFormatException
* code format
---------
Co-authored-by: xiangzihao <460888207@qq.com>
1 year ago
50195c46aa
[Improvement-11913] Mask password when creating/editing alert instances ( #14415 )
...
* [Improvement-11913] Mask password when creating/editing alert instances
1 year ago
7a24be0672
[Feature-14404][datasource-plugin] add HANA datasource ( #14404 )
...
---------
Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
1 year ago
18d176c5f4
feat: Introduce new datasource vertica ( #14232 )
...
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
2 years ago
f1e78627fc
[feature][plugin] Add snowflake datasource in datasource plugin ( #13729 )
...
---------
Co-authored-by: weitong <weitong@cisco.com>
2 years ago
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin ( #13866 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2 years ago
c0126b7f30
[Feature][plugin] Support Kyuubi datasource ( #13642 )
...
---------
Co-authored-by: weitong <weitong@cisco.com>
Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>
2 years ago
4bf097ee40
[Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath ( #13848 )
...
* [Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath
* remove semicolon
* set null in some specific scenario
* remove useless console.log
2 years ago
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin ( #13801 )
2 years ago
6a33d2bec0
fix: Alarm instance port verification ( #13827 )
...
close : #13532
2 years ago
35d8e8ac17
[Feature][DataSource] add oceanbase datasource ( #13675 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2 years ago
bcf03ad4d1
[Feature-9467] add DAMENG DataSource ( #12860 )
...
* [Feature-9467][DataSource] add DAMENG （达梦） data source
* [Feature-9467] add DM datasource doc (Issue #9467 )
* [Feature-9467] add DM IDataBaseOptionKeys
* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method
* [Feature-9467] modification package dm to dameng;
* [Feature-9467] add Dameng DmJdbcDriver18 License
* [Feature-9467] add DM DataSource: remove DruidDataSourceClient
Co-authored-by: lenian <yuenian@mananacare.com>
2 years ago
f0dd4f081f
[Feature][task-plugins-12820] azure azure-sql datasource plugins ( #13209 )
...
* submit azure sql code
* change mssql-jdbc version
* merge changes to delete other
* merge changes to delete other
* add dependencies
2 years ago
5c75d312fa
Add StarRocks datasource ( #13204 )
2 years ago
ab96a3d0c0
Fix datasource jdbc url param doesn't work ( #13098 )
2 years ago
2b2adecd85
[Improvement] Fixed Possible JSON ordering permutations problem in Tests ( #13118 )
...
* Fixed flaky test
* Fixed imports order
2 years ago
b40c63f02d
[Feature][DataSource]Add trino datasource plugin ( #12911 )
2 years ago
dedff70f90
[Improvement] Merge spi.utils into common.utils ( #12502 )
...
* improve JSONUtil
* merge spi.utils into common.utils
* split constants && remove some copy utils method
* remove StringUtils
2 years ago
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2 years ago
b357781df2
[Migrate][Test] Migrate junit4 -> junit5 test cases in common, service, and spi modules ( #12363 )
2 years ago
db1d319159
Reorganize some classes in common module, remove duplicate classes ( #12321 )
2 years ago
9ab79e064c
[Improvement][Test] Fully remove the usage of powermock from the whole project ( #12244 )
...
* Fully remove the usage of powermock from the whole project
* Upgrade org.reflections to 0.10.12
2 years ago
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones ( #12239 )
...
* fix DS support cross time zone, use timestamp replace date
2 years ago
e6832220c3
[Improvement][SQL Task]use default sql segment separator ( #10869 )
2 years ago
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center ( #11708 )
...
* Add support for Alibaba Cloud OSS as storage of resource center
* Add dependency licenses
* Refactor code, add unit tests and related docs
2 years ago
86291bb5e6
[Feature-10273][Datasource] Add dataSource for AWS Athena ( #11157 )
2 years ago
b96d69701a
[Improvement-#11613] Add spi priority factory ( #11614 )
...
* Add spi priority factory
* Add doc
* Add override log
* Use lombok
* Add comment
2 years ago
71cf7e1baa
[Fix-10918] Close datasource when expire from guava cache ( #11120 )
...
* Close datasource when expire from guava cache
* Remove duplicate datasource in HiveDataSourceClient
2 years ago
5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version ( #11025 )
2 years ago
71f0168510
[Feature] Enable users to create python env from requirements.txt ( #10658 )
2 years ago
dafed572d1
[Fix][dolphinscheduler-spi] json parse support LocalDateTime ( #10440 )
...
* JSONUtils support LocalDateTime
* remove extraneous unit tests
* remove unused classes import.
Co-authored-by: HomminLee <lihongmin@ysstech.com>
2 years ago
d80cf21456
Clean up unused dependencies and packaging issues ( #9944 )
3 years ago
592e433b1e
[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO ( #10276 )
...
* replace DataSourceController API with string JSON and cast to DTO
* add DataSourceProcessorManager and DataSourceProcessorProvider to datasource processor functions
3 years ago
31cd1b5e61
Serial wait for subsequent execution ( #9847 )
3 years ago
fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT ( #9223 ) ( #9299 )
3 years ago
263791a63e
[Bug-9137][Server] fix timezone init ( #9219 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module ( #8584 )
...
* Task plug-in optimization, task-related classes are migrated to the task plugin module
* remove tasktype enum class.
* fix code style.
* fix code style
* fix code style
* fix ProcessServiceTest.testUpdateResourceInfo test class
* fix code style
* fix test task plugin manager check parameters null
* fix test ut
* update aws license
* change TaskTimeoutStrategy class
3 years ago
7b66b66e8f
[Feature-8527][Datasource] Add dataSource for Amazon Redshift ( #8564 )
...
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
3 years ago
20ee9a37fe
[Feature-8485][task] Add EMR task plugin to support submit task to AWS EMR ( #8503 )
...
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
3 years ago
a72742b9de
remove unuse import jsonFormat ( #8548 )
3 years ago
e174d7a40e
[Fix-8394][Server]fix when the data is supplemented business time still date+1 ( #8532 )
3 years ago
84f2cfec39
[Feature][Server] Custom timezone - Add timezone handler unified ( #8477 )
...
* date convert of timezone
* remove @JsonFormat
* add unit test
* fix time preview in scheduler
* optimization & add env config
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
f2b9796bc2
[Bug-8110][WorkerServer] kill all running task before worker stop ( #8490 )
...
* kill all running task before worker stop
* kill local process
* remove kill yarn job
* adjust the orders of close
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
60ddede6a6
fixbug: varPool value is error ( #7724 )
3 years ago
703d790415
[Bug] [Parameter Context] The preparameter OUT is null ( #8315 )
...
* fix:param context prop value is null
3 years ago
4ba6d667fc
[Feature][DataQuality] Add Data quality Module #4283 ( #6718 )
...
* add data quality module
* add license
* add package configuration in dist pom
* fix license and jar import bug
* replace apache/skywalking-eyes@9bd5feb SHA
* refacotr jbdc-connector and writer
* modify parameter name in HiveConnector
* fix checkstyle error
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix duplicate code bug
* fix code style bug
* fix code smells
* add dq relevant enums and parameter
* replace apache/skywalking-eyes@9bd5feb SHA
* fix Constants bug
* remove the unused class
* add unit test
* fix code style error
* add unit test
* refactor data quality common entity
* fix code style error
* add unit test
* close e2e test
* fix code smell bug
* modify dataquality enum value to 14 in TaskType
* add data qualtiy task
* update
* add getDatasourceOptions interface
* fix checkstyle
* close e2e test
* add data quality task ui
* update skywalking-eyes SHA
* fix style
* fix eslint error
* fix eslint error
* test e2e
* add unit test and alter dataquality task result
* fix checkstyle
* fix process service test error
* add unit test and fix code smells
* fix checkstyle error
* fix unit test error
* fix checkstyle error
* change execute sql type name
* revert ui pom.xml
* fix data quality task error
* fix checkstyle error
* fix dq task src_connector_type ui select bug
* fix spark rw postgresql bug
* change mysql driver scope
* fix form-create json bug
* fix code smell
* fix DolphinException Bug
* fix ui validate rule and Alert title
* fix target connection param bug
* fix threshold validate change
* add rule input entry index
* change statistic_comparison_check logic
* remove check type change
* add DateExpressionReplaceUtil
* fix null point expetion
* fix null point expetion
* fix test error
* add more sql driver
* fix test error and remove DateExprReplaceUtil
* add get datasource tables and columns
* add get datasource tables and columns
* remove hive-jdbc in pom.xml
* fix code smells
* update sql
* change the pom.xml
* optimize multi_table_accuracy ui
* fix v-show error
* fix code smells
* update sql
* [Feature][DataQuality] Add data quality task ui (#5054 )
* add data quality task ui
* update skywalking-eyes SHA
* fix style
* fix eslint error
* fix eslint error
* test e2e
* fix dq task src_connector_type ui select bug
* fix threshold validate change
* remove check type change
* add get datasource tables and columns
* optimize multi_table_accuracy ui
* fix v-show error
* fix code smells
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* [Feature][DataQuality] Add data quality module (#4830 )
* add data quality module
* add license
* add package configuration in dist pom
* fix license and jar import bug
* replace apache/skywalking-eyes@9bd5feb SHA
* refacotr jbdc-connector and writer
* modify parameter name in HiveConnector
* fix checkstyle error
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix duplicate code bug
* fix code style bug
* fix code smells
* update
* close e2e test
* fix spark rw postgresql bug
* change mysql driver scope
* add more sql driver
* remove hive-jdbc in pom.xml
* change the pom.xml
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* [Feature][DataQuality] Add data quality task backend (#4883 )
* add dq relevant enums and parameter
* replace apache/skywalking-eyes@9bd5feb SHA
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* refactor data_quality_module
* add header license
* data quality module refactor
* fix unit test error
* fix checkstyle error
* fix unit test error
* fix checkstyle error
* fix unit test error
* fix code smell
* fix check style
* fix unit test error
* task statistics value add unique code
* fix unit test error
* fix checkstyle error
* fix checkstyle
* fix security hotspot
* fix unit test error
* fix security hotspot
* fix check
* add data quality task error handling
* fix unit test error
* add unit test
* add unit test
* optimize data quality result alert
* fix unit test
* fix sql script error
* fix bug
* update sql script
* fix checkstyle
* add license
* fix checkstyle
* fix checkstyle
* fix unit test
* add jacoco dependencies
* fix unit test
* fix unit test
* add jacoco dependencies
* add unit test
* add unit test
* add license
* fix checkstyle
* fix pom
* fix checkstyle
* fix checkstyle
* merge dev
* fix ui error
* fix pom error
* fix pom error
* fix test error
* fix test error
* mssql-jdbc exclude azure-keyvault
* fix test error
* merge dev and add unit test
* add notes
* rollback the CollectionUtils
* fix
* update sql
* fix
* fix
* fix query rule page error
* change dq.jar path
* fix sql error
* fix ui error
* fix(dq): jar path&task enum description
* add notes on DataQualityApplication
* fix dq result jump error
* fix(ui): page condition
* feat(ui): add show error output path
* change version
* remove all chinese word in sql
* merge
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
3 years ago
chenrj
Wenjun Ruan
Xinxing
c3Vu
IT-Kwj
Zinway
He Zhao
Jeremy
Aaron Wang
JieguangZhou
Jay Chung
He Wang
lenian
Tq
Bharati Kulkarni
insist777
caishunfeng
Eric Gao
Yann Ann
kezhenxu94
zhuxt2015
ZhaoGuodong
HomminLee
WangJPLeo
Kerwin
ronyang1985
yimaixinchen
Molin Wang
Ivan0626
cheney
zixi0825