25fc1dcb5f
Fix the exception of task getting external resource data during execution ( #9134 )
2 years ago
852597c9bf
[Bug-9070][Server]fix taskExecutionContext json convert error ( #9072 )
...
* [Bug-9070][Server]fix taskExecutionContext json convert error
* test
* add license header
Co-authored-by: caishunfeng <534328519@qq.com>
2 years ago
63a10037a6
fix package bug ( #9051 )
2 years ago
a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module ( #8584 )
...
* Task plug-in optimization, task-related classes are migrated to the task plugin module
* remove tasktype enum class.
* fix code style.
* fix code style
* fix code style
* fix ProcessServiceTest.testUpdateResourceInfo test class
* fix code style
* fix test task plugin manager check parameters null
* fix test ut
* update aws license
* change TaskTimeoutStrategy class
2 years ago
6a02b15e40
[Fix][data-quality] change data-quality task main jar path ( #8752 )
...
* feat(data-quality) change data-quality task main jar path
* feat(data-quality) change data-quality task main jar path
Co-authored-by: zixi0825 <sunzhaohe@linklogis.com>
2 years ago
20ee9a37fe
[Feature-8485][task] Add EMR task plugin to support submit task to AWS EMR ( #8503 )
...
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2 years ago
a72742b9de
remove unuse import jsonFormat ( #8548 )
2 years ago
84f2cfec39
[Feature][Server] Custom timezone - Add timezone handler unified ( #8477 )
...
* date convert of timezone
* remove @JsonFormat
* add unit test
* fix time preview in scheduler
* optimization & add env config
Co-authored-by: caishunfeng <534328519@qq.com>
2 years ago
dd1397aaed
[Bug] [Task-plugin] ShellTask add the parentfolder exist check and create if not exit ( #8526 )
...
* check the parent dir exist and create it
* fix lint
Co-authored-by: ywang46 <ywang46@paypal.com>
2 years ago
60ddede6a6
fixbug: varPool value is error ( #7724 )
2 years ago
806333a11c
fix sudo.enable=false Is invalid ( #8388 )
2 years ago
4ba6d667fc
[Feature][DataQuality] Add Data quality Module #4283 ( #6718 )
...
* add data quality module
* add license
* add package configuration in dist pom
* fix license and jar import bug
* replace apache/skywalking-eyes@9bd5feb SHA
* refacotr jbdc-connector and writer
* modify parameter name in HiveConnector
* fix checkstyle error
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix duplicate code bug
* fix code style bug
* fix code smells
* add dq relevant enums and parameter
* replace apache/skywalking-eyes@9bd5feb SHA
* fix Constants bug
* remove the unused class
* add unit test
* fix code style error
* add unit test
* refactor data quality common entity
* fix code style error
* add unit test
* close e2e test
* fix code smell bug
* modify dataquality enum value to 14 in TaskType
* add data qualtiy task
* update
* add getDatasourceOptions interface
* fix checkstyle
* close e2e test
* add data quality task ui
* update skywalking-eyes SHA
* fix style
* fix eslint error
* fix eslint error
* test e2e
* add unit test and alter dataquality task result
* fix checkstyle
* fix process service test error
* add unit test and fix code smells
* fix checkstyle error
* fix unit test error
* fix checkstyle error
* change execute sql type name
* revert ui pom.xml
* fix data quality task error
* fix checkstyle error
* fix dq task src_connector_type ui select bug
* fix spark rw postgresql bug
* change mysql driver scope
* fix form-create json bug
* fix code smell
* fix DolphinException Bug
* fix ui validate rule and Alert title
* fix target connection param bug
* fix threshold validate change
* add rule input entry index
* change statistic_comparison_check logic
* remove check type change
* add DateExpressionReplaceUtil
* fix null point expetion
* fix null point expetion
* fix test error
* add more sql driver
* fix test error and remove DateExprReplaceUtil
* add get datasource tables and columns
* add get datasource tables and columns
* remove hive-jdbc in pom.xml
* fix code smells
* update sql
* change the pom.xml
* optimize multi_table_accuracy ui
* fix v-show error
* fix code smells
* update sql
* [Feature][DataQuality] Add data quality task ui (#5054 )
* add data quality task ui
* update skywalking-eyes SHA
* fix style
* fix eslint error
* fix eslint error
* test e2e
* fix dq task src_connector_type ui select bug
* fix threshold validate change
* remove check type change
* add get datasource tables and columns
* optimize multi_table_accuracy ui
* fix v-show error
* fix code smells
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* [Feature][DataQuality] Add data quality module (#4830 )
* add data quality module
* add license
* add package configuration in dist pom
* fix license and jar import bug
* replace apache/skywalking-eyes@9bd5feb SHA
* refacotr jbdc-connector and writer
* modify parameter name in HiveConnector
* fix checkstyle error
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix duplicate code bug
* fix code style bug
* fix code smells
* update
* close e2e test
* fix spark rw postgresql bug
* change mysql driver scope
* add more sql driver
* remove hive-jdbc in pom.xml
* change the pom.xml
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* [Feature][DataQuality] Add data quality task backend (#4883 )
* add dq relevant enums and parameter
* replace apache/skywalking-eyes@9bd5feb SHA
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* refactor data_quality_module
* add header license
* data quality module refactor
* fix unit test error
* fix checkstyle error
* fix unit test error
* fix checkstyle error
* fix unit test error
* fix code smell
* fix check style
* fix unit test error
* task statistics value add unique code
* fix unit test error
* fix checkstyle error
* fix checkstyle
* fix security hotspot
* fix unit test error
* fix security hotspot
* fix check
* add data quality task error handling
* fix unit test error
* add unit test
* add unit test
* optimize data quality result alert
* fix unit test
* fix sql script error
* fix bug
* update sql script
* fix checkstyle
* add license
* fix checkstyle
* fix checkstyle
* fix unit test
* add jacoco dependencies
* fix unit test
* fix unit test
* add jacoco dependencies
* add unit test
* add unit test
* add license
* fix checkstyle
* fix pom
* fix checkstyle
* fix checkstyle
* merge dev
* fix ui error
* fix pom error
* fix pom error
* fix test error
* fix test error
* mssql-jdbc exclude azure-keyvault
* fix test error
* merge dev and add unit test
* add notes
* rollback the CollectionUtils
* fix
* update sql
* fix
* fix
* fix query rule page error
* change dq.jar path
* fix sql error
* fix ui error
* fix(dq): jar path&task enum description
* add notes on DataQualityApplication
* fix dq result jump error
* fix(ui): page condition
* feat(ui): add show error output path
* change version
* remove all chinese word in sql
* merge
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
2 years ago
a1cc2fd11a
[task] Python task support PYTHON_HOME ( #8196 )
2 years ago
103ccb01da
change version to 2.0.4-SNAPSHOT ( #8188 )
2 years ago
0057cf3fe7
[Improvement][task-plugin] Remove unreasonable log ( #8115 )
2 years ago
f1a0215978
[bug] Fix datax task type error parse clickhouse reader sql ( #8091 )
2 years ago
e4cdd82b7c
[Improvement][TaskLog] Unified task log ( #7831 )
...
* task logger
* update
* log format
* remove master config task-logger
* udpate log level
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
5172862a0a
Add seatunnel task plugin ( #7131 )
3 years ago
a4168ae24e
[Improvement] The spark version is associated with the spark command ( #7544 )
...
* [Improvement] The spark version is associated with the spark command
3 years ago
c58dbefaa5
[Fix-7459][Sqoop] running error in parallel ( #7752 )
...
* fix bug_7459
* fix bug_7459
* fix bug_7459
* fix bug_7459
* fix bug_7459
Co-authored-by: SbloodyS <sbloodys@qq.com>
3 years ago
cc77963522
[FIX-#6727][worker-server] Fix procedure params bug ( #7680 )
...
* fix Procedure param error
* code style
Co-authored-by: wangxj <wangxj31>
3 years ago
6ae46f2c1b
[Improvement] Merge redundant codes ( #7550 )
3 years ago
7e0010de35
[Refactor] Rename class file, unified word spelling. ( #7261 )
3 years ago
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties ( #7160 )
...
* Split the components into individual package
A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
* Split the components into individual package
A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
* Split the components into individual package
A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
3 years ago
0b9792db76
[Refactor]Rename class file, DatasourceUtil to DataSourceUtils. ( #7181 )
...
* Rename class file, DatasourceUtil to DataSourceUtils.
3 years ago
463fc76bb7
[Fix-6809][dolphinscheduler-task-sql] Fix sql task can't send alert ( #7089 )
3 years ago
2ed3c0c3a7
Recover unit tests that have not been run for some time ( #7054 )
3 years ago
4114cb07f6
Add actuator and enable metrics, add Grafana configs ( #6840 )
3 years ago
11844415b0
Refactor datasource and task plugins and simplify its usage ( #6767 )
3 years ago
0aea5ae11a
[feature-6071][plugin] add datasource plugin. ( #6071 ) ( #6361 )
...
* [feature-6071][plugin] add datasource plugin. (#6071 )
* remove the dependency of datasource in common and task-plugin module.
* the new datasource-api module provides data source connection acquisition.
* add sqlserver data source plug-in.
* add ds license
* fix check style
* update spi pom
* frx conflict resolution
* fix check style
* fix datasource ut
* fix code check style
* fix code check style
* fix datasource ut
* add pg hive mysql oracle clickhouse db2 datasource
* fix datasource package
* update datasource pom
* update datasource pom
* update datasource pom
* update datasource pom
* update pom
* update pom dep
* update pom dep
* fix license
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* fix ut
* add oracle package
* modify comments
* conflict resolution
* rename log comment
* rename log comment
3 years ago
9b1984c244
[fix-6570][task] add sql task udf resource load ( #6570 ) ( #6571 )
...
* [fix-6570][task] add sql task udf resource load (#6570 )
-add sql task udf resource load
-Fix UdfFuncRequest serialization
* fix checkstyle
* add defaultFS in sqlTaskContext
* fix checkstyle
3 years ago
09657d8576
[ #6337 ]When the SQL result reaches the limit value, increase the log result prompt ( #6528 )
3 years ago
8200bc1452
[Fix-6529] Improve the python task to solve the issue about getting the environment variables. ( #6531 )
...
* modify config
* improve python task
* improve python task
3 years ago
bddf3e6f2c
[FIX-#6514]Task Class Load Err ( #6513 )
...
* [FIX-#6412]Task Class Load Err
* Task Class Load Err
3 years ago
a80fca70f7
[Fix-6497][Server] Fix the missing code about the feature of supporting multiple environments. ( #6509 )
3 years ago
d284e44d52
Rename tis to pigeon from backend layer ( #6437 )
...
* add mock getTaskLogName() for UT test pass
* make task-tis rename to task-pigeon
* rename module name 'tis' to pigeon
* rename module name 'tis' to pigeon,make sanity check pass
* remove execute console.log()
3 years ago
1843e986f5
[BUG][Task-API]load codec class fail ( #6411 )
3 years ago
67508d8d1b
update project version to 2.0.0-SNAPSHOT ( #6383 )
3 years ago
a62a7a7216
[Fix][Task] sql limit param no default value ( #6337 ) ( #6387 )
...
* add default query sql limit
3 years ago
545173faa3
add mock getTaskLogName() for UT test pass ( #6369 )
3 years ago
34352c1bf8
Enhance using experience of DataX by introduce TIS ( #6229 )
...
* Enhance using experience of DataX by introduce TIS [Feature-5992]
* Enhance using experience of DataX by introduce TIS [Feature-5992]
* fix stylecheck error
* make testCase pass
* make dev testCase pass
* add new java dependency Java-WebSocket
* modfiy TISParameters.java avoid to name confliction
* add InterruptedException checking
* reAdd async-http-client annotation
* in order to staisfy the coverage degree add test which has been removed
* make testCase pass
* make testCase pass
* add jacoco dependency
* make code duplications be more lower
* add Java-WebSocket dependency in root pom
* remove useless code comment
* make tis http apply path and post body configurable ,the params save in config.properites
* Remove the dangerous instance of double-checked locking
3 years ago
be158328fb
[Bug #6306 ][Worker] task log path generated error. ( #6338 )
3 years ago
61c6461994
[BUG-6331][WorkerServer] fix preparedStatement auto close by try with resource ( #6334 )
...
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
163a0696a4
[Bug #6306 ][Worker] task log path generated error. ( #6311 )
...
* [Bug #6306 ][Worker] task log path generated error.
* [Bug #6306 ][Worker] task log path generated error.
3 years ago
9115062838
[Plugin][Task]Fix Task log may be lost ( #6294 )
3 years ago
552fd17878
[Plugin][Task]Fix Yarn Task Kill Not valid ( #6293 )
3 years ago
7e8febed4d
[Plugin][Task]Fix Sql Task NPE ( #6290 )
3 years ago
fab4eae923
[SPI][Task]fix Task parse Err ( #6244 )
...
* [SPI][Task]fix Task parse Err
3 years ago
fced3892ee
[Fix][Task] Wrong complement date ( #6009 ) ( #6186 )
...
* [Fix][Task] Wrong complement date (#6009 )
3 years ago
b90152fc2e
[SPI][Task]add SQL Task ( #6237 )
...
Add Sql Task And Fix Task Parse Err
3 years ago
Kerwin
caishunfeng
zixi0825
zixi0825
ronyang1985
yimaixinchen
Yao WANG
Ivan0626
ShuiMuNianHuaLP
Wenjun Ruan
Shiwen Cheng
seagle
wind
J·Y
xiangzihao
wangxj3
Zonglei Dong
kezhenxu94
张俊杰
mask
Kirs
Hua Jiang
百岁