65744d4624
[Cherry-pick-dev][API] fix relation api bug ( #8207 )
...
* pick 8111
* fix relation delete (#8190 )
* pick 8190
3 years ago
4ba6d667fc
[Feature][DataQuality] Add Data quality Module #4283 ( #6718 )
...
* add data quality module
* add license
* add package configuration in dist pom
* fix license and jar import bug
* replace apache/skywalking-eyes@9bd5feb SHA
* refacotr jbdc-connector and writer
* modify parameter name in HiveConnector
* fix checkstyle error
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix duplicate code bug
* fix code style bug
* fix code smells
* add dq relevant enums and parameter
* replace apache/skywalking-eyes@9bd5feb SHA
* fix Constants bug
* remove the unused class
* add unit test
* fix code style error
* add unit test
* refactor data quality common entity
* fix code style error
* add unit test
* close e2e test
* fix code smell bug
* modify dataquality enum value to 14 in TaskType
* add data qualtiy task
* update
* add getDatasourceOptions interface
* fix checkstyle
* close e2e test
* add data quality task ui
* update skywalking-eyes SHA
* fix style
* fix eslint error
* fix eslint error
* test e2e
* add unit test and alter dataquality task result
* fix checkstyle
* fix process service test error
* add unit test and fix code smells
* fix checkstyle error
* fix unit test error
* fix checkstyle error
* change execute sql type name
* revert ui pom.xml
* fix data quality task error
* fix checkstyle error
* fix dq task src_connector_type ui select bug
* fix spark rw postgresql bug
* change mysql driver scope
* fix form-create json bug
* fix code smell
* fix DolphinException Bug
* fix ui validate rule and Alert title
* fix target connection param bug
* fix threshold validate change
* add rule input entry index
* change statistic_comparison_check logic
* remove check type change
* add DateExpressionReplaceUtil
* fix null point expetion
* fix null point expetion
* fix test error
* add more sql driver
* fix test error and remove DateExprReplaceUtil
* add get datasource tables and columns
* add get datasource tables and columns
* remove hive-jdbc in pom.xml
* fix code smells
* update sql
* change the pom.xml
* optimize multi_table_accuracy ui
* fix v-show error
* fix code smells
* update sql
* [Feature][DataQuality] Add data quality task ui (#5054 )
* add data quality task ui
* update skywalking-eyes SHA
* fix style
* fix eslint error
* fix eslint error
* test e2e
* fix dq task src_connector_type ui select bug
* fix threshold validate change
* remove check type change
* add get datasource tables and columns
* optimize multi_table_accuracy ui
* fix v-show error
* fix code smells
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* [Feature][DataQuality] Add data quality module (#4830 )
* add data quality module
* add license
* add package configuration in dist pom
* fix license and jar import bug
* replace apache/skywalking-eyes@9bd5feb SHA
* refacotr jbdc-connector and writer
* modify parameter name in HiveConnector
* fix checkstyle error
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix checkstyle error in dolphinschesuler-dist
* fix duplicate code bug
* fix code style bug
* fix code smells
* update
* close e2e test
* fix spark rw postgresql bug
* change mysql driver scope
* add more sql driver
* remove hive-jdbc in pom.xml
* change the pom.xml
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* [Feature][DataQuality] Add data quality task backend (#4883 )
* add dq relevant enums and parameter
* replace apache/skywalking-eyes@9bd5feb SHA
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
* refactor data_quality_module
* add header license
* data quality module refactor
* fix unit test error
* fix checkstyle error
* fix unit test error
* fix checkstyle error
* fix unit test error
* fix code smell
* fix check style
* fix unit test error
* task statistics value add unique code
* fix unit test error
* fix checkstyle error
* fix checkstyle
* fix security hotspot
* fix unit test error
* fix security hotspot
* fix check
* add data quality task error handling
* fix unit test error
* add unit test
* add unit test
* optimize data quality result alert
* fix unit test
* fix sql script error
* fix bug
* update sql script
* fix checkstyle
* add license
* fix checkstyle
* fix checkstyle
* fix unit test
* add jacoco dependencies
* fix unit test
* fix unit test
* add jacoco dependencies
* add unit test
* add unit test
* add license
* fix checkstyle
* fix pom
* fix checkstyle
* fix checkstyle
* merge dev
* fix ui error
* fix pom error
* fix pom error
* fix test error
* fix test error
* mssql-jdbc exclude azure-keyvault
* fix test error
* merge dev and add unit test
* add notes
* rollback the CollectionUtils
* fix
* update sql
* fix
* fix
* fix query rule page error
* change dq.jar path
* fix sql error
* fix ui error
* fix(dq): jar path&task enum description
* add notes on DataQualityApplication
* fix dq result jump error
* fix(ui): page condition
* feat(ui): add show error output path
* change version
* remove all chinese word in sql
* merge
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
3 years ago
fb9f4a1339
[Feature-5822][api] Add audit log ( #6322 )
...
* rebase
* update audit log type name
* fix the quoted property error
* fix bug
* update audit global switch default value
* fix wrong ddl comment
* fix wrong code style
* update audit log search sql
* add licenses
* add unit test
* fix wrong code style
* fix bugs
* fix wrong code style
* add test case
* add test case
* add license
* Add unit test in pom.xml
* resolve code smell problem
* mysql and postgresql ddl update
* update audit log path and audit page query
* update audit configuration
* update audit log schema, add resource type and resource id
* update audit query and list page
* resolve audit log front page bugs and log query error
* update audit configuration
* resolve wrong configuration
* update AuditLogControllerTest
* update application.yaml
* resolve code smell
* update standalone configuration
* resolve bugs
Co-authored-by: yc322 <1393242780@qq.com>
3 years ago
b5631d5281
[Fix-8119][API] Update ProcessInstance error and without reason output ( #8122 )
...
* fix bug_8119
* fix bug_8119
* fix bug_8119
* fix bug_8119
* fix bug_8119
3 years ago
3456904af8
add unit test of queryWorkAddressLest ( #8054 )
...
Co-authored-by: brian <flyunrain@163.com>
3 years ago
82d04f1924
[DS-6820][feat] Ordinary users can also share resource data source projects with others ( #7929 )
...
Admin's entire contents of non-selected users
Non-admins can only authorize content they create
This closes #6820
3 years ago
d198a9051e
[Bug-7950] Add unit test for AlertPluginInstanceController ( #7952 )
3 years ago
23068e9d21
[Fix-7977] Rectify the wrong prompt in the class of 'TaskGroupServiceImpl'. ( #7978 )
3 years ago
6dcf53c4d9
[Fix-7984] Add Unit Test for UiPluginController ( #7985 )
3 years ago
8cac0d8f94
[Fix-7288] Not allow to delete the default alarm group. ( #7932 )
3 years ago
a81093d6cb
[Fix-7935] Rectify this issue with running test cases in the class of AlertGroupControllerTest. ( #7954 )
...
* fix a few issues
3 years ago
24a97fda75
[Bug] [dolphinscheduler-api]support workflow instance to definition ( #7930 )
...
* add task save and binds workflow
* add task update with upstream
* support workflow instance to definition
* fix ut
3 years ago
3d67edafbe
[Improvement][API] Add and fix unit test for ExecutorController，Fix A… ( #7877 )
3 years ago
4c2f77ee9c
[DS-7016][feat] Auto create workflow while import sql script with specific hint ( #7214 )
...
* [DS-7016][feat] Auto create workflow while import sql script with specific hint
* datasource : datasource name
* name: task name
* upstream: pre task names
* [DS-7016][feat] Auto create workflow while import sql script with specific hint
* remove excess blank lines
* [DS-7016][feat] Auto create workflow while import sql script with specific hint
* datasource : datasource name
* name: task name
* upstream: pre task names
* [DS-7016][feat] Auto create workflow while import sql script with specific hint
* datasource : datasource name
* name: task name
* upstream: pre task names
* Code optimization
* for Zip Bomb Attack
Co-authored-by: eye <eye.gu@aloudata.com>
3 years ago
73993e98ee
[Fix-7713] Handling the sensitive data in the log ( #7728 )
...
* add a feature to handle sensitive data
3 years ago
5ab0135d18
[Fix-7345] Support the whole operations for the task group and the task group queue. ( #7678 )
...
* add task group
* modify task group
* add task group queue
* add task group queue
* add task group queue
* add task group queue
* add task group priority
* fix a few issue with api
* add task group queue
* modify task group queue
* modify task group
* fix a little bit issue
* fix a little bug in the page of managing task group
3 years ago
605767e47b
[Fix-7654]Dependent node onChangeProjectCode error ( #7672 )
...
* fix #7654 Dependent node onChangeProjectCode error
* fix #7654 Dependent node onChangeProjectCode error
* fix #7654 Dependent node onChangeProjectCode error
* update ut
3 years ago
3d9d91ccc3
[Feature-7576][Master] Optimize complement task's date ( #7585 )
...
* feature_7576
* feature_7576
* feature 7576
* feature 7576
3 years ago
9e763ad0d1
[Feature-#6422] [api-server] task group queue ( #7491 )
...
* add task group
* modify task group
* pull dev
* add license header
* modify code style
* fix code style
* fix sql error
* fix error
* fix test
* fix test
* fix test
* fix test
* fix code style
* fix ut
* code style
* fix unit test
* test ut
* ut
* add unittest
* test ut
* modify back ut
* majorization code
* fix conflict
* fix ut
* add task group api
* reset file
* fix ut
* fix lost column
* fix ut
* fix ut
* fix ut
* fix ut
* delete duplicate code
* fix code style 、name
* fix ut
* fix mapper
Co-authored-by: wangxj <wangxj31>
3 years ago
6cad119806
[Fix-7340] Improve the prompt content for deleting a project that didn't exist. ( #7449 )
...
* improve the prompt
3 years ago
801e6dd6bb
to feature 7418 ( #7420 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
6b5c393449
[Feature-7408][dolphinscheduler-api] Built GenerateToken into the UpdateToken API ( #7411 )
...
* to feature 7408
* trigger e2e test
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
7e0010de35
[Refactor] Rename class file, unified word spelling. ( #7261 )
3 years ago
14343864bf
to feature 7401 ( #7404 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
57697602c5
[Feature-7110][dolphinscheduler-api] support grant project by code ( #7390 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
8c3fa47903
[Fix-7375][datasource] remove excess hive jdbc principal ( #7375 ) ( #7381 )
...
* [Fix-7375][datasource] remove excess hive jdbc principal (#7375 )
* fix ut
3 years ago
f03b0e8c7b
to feature-7301 ( #7308 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
67cc260d52
Split the components into individual package, refactor configurations to use Spring properties ( #7160 )
...
* Split the components into individual package
A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
* Split the components into individual package
A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
* Split the components into individual package
A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
3 years ago
29414a080f
Fix homepage user wrong statistics result for new user ( #7242 )
...
* Fix homepage user wrong statistics result for new user
For now, when we create a new user, it would see homepage
statistic info as admin user like, cause `projectCodeArray`
would not filter when we render SQL. So I add prefix conditions
to handle this situation.
Also, I do some minimal refactor here. including change function
name no quit fit
fix : #7182
* Fix UT
3 years ago
0b9792db76
[Refactor]Rename class file, DatasourceUtil to DataSourceUtils. ( #7181 )
...
* Rename class file, DatasourceUtil to DataSourceUtils.
3 years ago
544b015f76
[Bug] [API] task definition release/delete bug ( #7232 )
...
* fix worker group display
* fix workflow query bug
* fix workflow query bug
* fix task definition release/delete bug
3 years ago
8f77fbbeb8
[Fix-7154][API] Fix MultipartFile resource [file] cannot be resolved to absolute file… ( #7155 )
...
* fix MultipartFile resource [file] cannot be resolved to absolute file path
* update testCopyFile.
3 years ago
3c8d1c6da9
to feature #7191 ( #7194 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
05aef279c6
[Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153 ( #7192 )
...
* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153
* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153
* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
3 years ago
3361d763c4
to feature #7180 ( #7185 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
27ea43d5df
to feature #7110 ( #7111 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
db090685a8
to feature #7169 ( #7170 )
...
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
3 years ago
ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow ( #7090 )
...
* add cache manager for workflow
* [DS-6988][MasterServer] add cache manager for workflow
* cache evict code optimization
* test
Co-authored-by: caishunfeng <534328519@qq.com>
3 years ago
1f184440ca
[UT#7157]Fix Ut Run Error ( #7158 )
...
* to #7157
* to #7157
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
3 years ago
8ebe060658
[Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project #7073 ( #7080 )
...
* to #7073
* to #7073
* to #7073
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
3 years ago
2b519f31e9
[Feature-#6422] task group queue ( #6722 )
...
* add task group
* modify task group
* pull dev
* add license header
* modify code style
* fix code style
* fix sql error
* fix error
* fix test
* fix test
* fix test
* fix test
* fix code style
* fix ut
* code style
* fix unit test
* test ut
* ut
* add unittest
* test ut
* modify back ut
* majorization code
* fix conflict
* fix ut
Co-authored-by: wangxj <wangxj31>
3 years ago
b265297fc8
[Improvement][API] fix 6772 new api bug ( #7098 )
...
* add processDefinition releaseWorkflowAndSchedule
* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation
* add dependency and subprocess judgement
* fix api bug
* fix codestyle
3 years ago
2ed3c0c3a7
Recover unit tests that have not been run for some time ( #7054 )
3 years ago
6e55394783
[Improvement][API][num-9] add ProcessTaskRelationServiceImpl.moveTaskProcessRelation ( #7053 )
...
* add processDefinition releaseWorkflowAndSchedule
* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation
* add dependency and subprocess judgement
3 years ago
b815da6158
to #7001 : delete certain edge in processDefinition ( #7038 )
...
* to #7001 : delete certain edge in processDefinition
* to #7001 : delete certain edge in processDefinition
* to #7001 : delete certain edge in processDefinition
* to #7001 : delete certain edge in processDefinition
* to #7001 : fix checkstyle
* to #7001 : delete edge by projectCode,processDefinitionCode,preTaskCode and postTaskCode
* to #7001 : add UT
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
3 years ago
f865c26322
to #7000 : implement deleteTaskProcessRelation ( #7043 )
...
* to #7000 : implement deleteTaskProcessRelation
* to #7000 : ! CollectionUtils.isEmpty -> CollectionUtils.isNotEmpty
* to #7000 : fix bug
* to #7000 : add UT
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
3 years ago
5a04b8d49a
update schdule api ( #6977 )
3 years ago
1edd2bd0f6
to #6957 implement interface deleteUpstreamRelation deleteDownstreamRelation ( #6966 )
...
* to #6957
* to #6957
* to #6957
* to #6957 : group by process definition code
* to #6957 : fix sonar
* to #6957 : fix sonar
* to #6957 : add ut , add preTaskVersion
* to #6957 : fix style
* to #6957 : use org.apache.dolphinscheduler.spi.utils.StringUtils
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
3 years ago
e51a2a1642
[Improvement][API][num-8] add ProcessTaskRelationServiceImpl.createProcessTaskRelation ( #6970 )
...
* add ProcessTaskRelationServiceImpl.createProcessTaskRelation
* code style
3 years ago
360464e279
[Improvement][API][num-6] TaskDefinition releaseTaskDefinition ( #6965 )
...
* finished releaseTaskDefinition
* add releaseTaskDefinition unit test
* change import
* add import
* bug fix
* change import
Co-authored-by: 时光 <caizhedong@cai-inc.com>
3 years ago
JinYong Li
zixi0825
yc322
xiangzihao
XiaochenNan
Sunny Lei
springmonster
calvin
ShuoTiann
Frank Chen
ououtt
BaoLiang
wangxj3
ouyangyewei
Zonglei Dong
mask
kezhenxu94
Jiajie Zhong
Kerwin
zwZjut
wind
时光