github
/
DolphinScheduler
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git
1
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

Merge dev to alert_plugin_design (#4351) 

* fixed queryProcessInstanceListPaging query slow (#3893)

because queryProcessInstanceListPaging has contain longtext type field by instance.*,so queryProcessInstanceListPaging query slow , but longtext field not used.

* fixed  queryTaskInstanceListPaging query slow (#3892)

because queryTaskInstanceListPaging  has contain longtext  type field  by instance.*,so  queryTaskInstanceListPaging query slow , but longtext  field not used.

* add verify tenant name cannot contain special characters. (#3865)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes. (#3895)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes.(#3894)

Signed-off-by: YoungMan <bushengquan@gmail.com>

* Update Dockerfile

update the  comment
and delete the maridb client

Co-authored-by: dailidong <dailidong66@gmail.com>

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* [CodeClean][DAO]Remove redundant comments (#3939)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [Fix][api] Add queryProjectCreatedAndAuthorizedByUser function (#3658)

* fix getLoginUserCreated bug

* fix getLoginUserCreated bug 1

* fix checkstyle

* modify function name

* modify comment

* fix unit test error

* rollback sqoop task hive target file

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog(). (#3963)

* Enhance user experience, add close button to file details page

* [Test-3557][API] full cover DataAnalysisServiceImpl (#3605)

* generate equals and hashCode method

* make up for the state of oblivion

* simplify countQueueState method

* delete try catch, because this never throw exception

* full cover DataAnalysisServiceImpl

* improve test coverage

* adjust code style

* simplify countCommandState method

* reduce duplication

* simplify countTaskDtos method

* Update DataAnalysisServiceImpl.java

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* the resources API document does not match the interface parameters

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* fix bug

* reset code

* Update udfs.vue

* [Feature-3941][ui] Remove tenant name  (#3949)

* FeatureUI-3941: Removed Tenant

* FeatureUI-3941: Removed tenantName in the suggested pages

* FeatureUI-3941: Updated the suggested files

* FeatureUI-3941: Updated the TenantController

* FeatureUI-3941: Updated the TenantController and the pom.xml

* FeatureUI-3941: Updated the TenantControllerTest

* [Improvement][script] daemon script usage (#3968)

* Improvement daemon script usage

* don't remove status command

don't remove status command

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix #3688, Optimization of table building statements (#3689)

* merge from 1.3.3-release

* merge from 1.3.3-release

* [Improvement-4012][common/remote] Json util code integration,remove the remote module json util (#4013)

* Json util code integration,remove the remote module json util.

* update code checkstyle.

* update code checkstyle.

* add code checkstyle.

* add test class.

* update JSONUtils class.

* update JSONUtils class.

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* fix #3900 kill multi yarn app in one job

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* add ProcessUtils UT

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* [Draft][Merge][133-dev]133 merge dev (#4031)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>

* Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)

This reverts commit ad2d9f99d0.

* [Fix][API] Condition task null pointer exception (#4056)

* add ProcessUtils UT

* modify ProcessUtils & ProcessUtilsTest

* Enhance user experience, add close button, return to the previous page (#4006)

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067)

* [Fix][api] Fix build parameter error of sqlserver when create. (#4015)

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog().

* [Fix][api] Fix build parameter error of sqlserver when create.

* [Feature-3985][Datax] Datax supports setting up running memory (#3986)

* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit

* [Improvement-3767][api] Task instance supports query by process instance name (#3825)

* Task instance supports query by process instance name.

* add test code checkstyle.

* add test param.

* resolve the sonar duplication check.

* solve logger single-line string exceeds 200 characters.

* resolve the sonar check.

* Resolve code conflicts.

Co-authored-by: zhuangchong <zhuangchong8@163.com>

* deal with magic value

* [Feature-4050][server] Spark task support udv inject (#4061)

* #4050 spark task support udv inject

* modify spark task UT

* modify sparkTaskExecutionCtx

* add exp for spark task get main jar method

* deal with magic value

* [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481)

* JSONUtils.toMap call improvement.

* [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error

* 解决单独执行子节点空指针的问题

* using OSUtils.execCmd when kill yarn app

* 解决单独执行子节点空指针的问题(增加checkstyle)

* 解决单独执行子节点空指针的问题(增加checkstyle)

* [FIX-#4083][server]fix taskInstance state change error
Concurrent processing of ack message and result message causes the execution sequence to be wrong

# this close # 4083

* code style

* fix replaceNRTtoUnderline NullPointerException #4098 (#4100)

* fix replaceNRTtoUnderline NullPointerException

* add  unit Test

* add taskResponseTest

* add taskResponseTest

* code smell

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* [FIX-4034][server] fix sqoop import fail (#4036)

* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* test

* remove assert

* test

* test

* fix task instance status judgment error

* fix: security page disappear delay problem when force refresh under GENERAL_USER

* improvement: resovle download url with resolveURL to prevent change of apiPrefix

* fix sqoop task jdbc string contains special char (#4105)

* [Bug][Common] read file garbled (#3479)

* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [Improvement-3933][db operation] Improve the performance of sql query (#3940)

* optimize select * case

* emove redundancy

* bug fixed

* Update en_US.js

* Update startup.sh

* optimize

* optimize code

* optimize

* bug fixed

* add ut

* bug fixed

* bug fixed

* bug fixed

* bug fixed

* Delete WorkFlowLineageMapper.xml

* Delete createTenement.vue

* recove wrongly deleted file

* Update WorkFlowLineageMapper.xml

* Update createTenement.vue

* [Feature][API]enable response resources gzip compression (#4121)

* enable response compression

* add server.compression.mime-types with default value explicitly

* [Improvement][API] ignore noNodeException when get worker groups (#4120)

* ignore noNodeException when get worker groups

* add ut

* [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)

* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* [FIX#4033] $[] conflicts with mysql keywords (#4111)

* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error

* split sqoop import hive database and table (#4141)

* upgrade quartz version to 2.3.0

* add HikariCP-java6,c3p0,mchange-commons-java license

* upgrade jackson version to 2.9.10

* remove c3p0,mchange-commons-java license

* update Check code style

* update  code style

* Update pom.xml

* Update pom.xml

* Update LICENSE

not need to add HikariCP-java6

* Delete LICENSE-HikariCP-java6.txt

* Update known-dependencies.txt

* Update TaskInstanceMapper.xml

optimize page of [TaskInstance] load data slow

* [FIX-PR-4097][server-master]task ack miss (#4189)

When the message of successful execution arrives earlier than
the message of ack,
the message of ack will be discarded,
resulting in some information missing

* fix bug #4125 (#4127)

* fix bug #4125

* code style!

* code style.

Co-authored-by: chengp <chengp@chengp.net>

* [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201)

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-3457][flink] fix flink args build problem (#4166)

* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local

* [common]del windows file( not support windows ) (#4204)

* del support win

* del support win

* [Feature-4138][Master] dispatch workgroup error add sleep time (#4139)

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* fix code smell & code style

* fix code smell & code style

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX-#4172][server-worker] kill task NPE (#4182)

* [FIX-#4172][server-worker] kill task NPE

The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear
Modification method: write into the cache when the task is received, and mark it as preData
If the task is killed before the Process is generated, delete the cache directly at this time
It will be judged before the process is generated. If the task has been killed, it will not be executed.
After the new process is created, write it into the cache, and judge again, if kill, then kill the process.

this closes #4172

* Delete the commented out code
Add spring beans

* code smell

* add test

* add test

* fix error

* test

* test

* revert

* fix error

* [Feature-3878]Replace the page with element-ui (#4065)

* [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui

* Change node version

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Replace the page with element-ui

* Replace user management with elment-ui

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Repair the last Sunday of each month

* support auto eslint for .js, .vue file on save

* update .eslintrc.yml and license check exclude .eslintignore

* fix eslint: fix syntax by npm run lint automatically

* fix eslint: fatal syntax errors

* fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '=='

* fix eslint: Unexpected side effect in 'cacheParams' computed property

* fix eslint: assigned a value but never used

* fix eslint: component has been registered but not used

* fix eslint: unexpected mutation of prop

* fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219)

* deleted invalid code as assigned in issue 4215 (#4221)

deleted the code at following two places:
org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue
org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo

* [FIX-#3177]Task time parameter parsing error (#4224)

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* fix sql error (#4227)

* [FIX-3177]Task time parameter parsing error (#4228)

* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import

* fix date_convert null

* fix time cover

* fix time cover

* reformat

* add ut

* [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162)

* fix(ui): re-login problem in new tab

* refresh page in new tab automatically

* support to reload router view in lightweight

* optimize visibility code

* [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225)

Add no-var rule for eslint, and fix related files
Add npm run lint:fix command, which distinguished from the default command npm run lint
The behavior of command npm run lint and command npm run lint:fix should be distinguished
The first command is used to lint only, and the second one is used to lint and fix problems automatically

* [FIX][UI ]  security user state  tenantCode User Type Display abnormal (#4255)

* fix user state error

* fix userType  error

* fix tenantCode  error

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250)

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Remove useless import

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest

* [Fix-4222][Master]Fix code smell

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Fix code smell

Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* update bug template (#4260)

* [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)

* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.

* remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262)

* remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters()

* remove unused imports in AbstractTask.java

* reformat

* [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272)

* fix version funcation delete. (#4265)

* [improvement][config] Update datasource.properties，add mysql meta data template (#4266)

* Update datasource.properties

[新增]元数据mysql连接模版

* Update datasource.properties

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix taskInstance submitTime is empty. (#4274)

* [Improvement-3878]Tenant list delete user name (#4278)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html

* Fix checkstyle error

* Update swagger-models in known-dependencies.txt

* [Improvement-3878][ui]Fix the list style (#4280)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [FIX] [UI] fix create project cancel button invalid (#4282)

* fix create project cancel button invalid

* [Improvement] Refactor code to support distributed tracing (#4270)

* Refactor code to support tracing

* Extension network protocol, support context and version
* Extension master asynchronous queue support context
* Extract scan task method from MasterSchedulerService for tracing

* fix

* fix

* add test case

* fix

* fix

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Improvement] Add Flink job name (#4285)

* [Improvement] Add Flink job name

* fix typo

* [Improvement][ui]List vacancy optimization and icon icon repair (#4286)

* [Improvement][ui] List vacancy optimization and icon icon repair

* [FEATURE-736] integrate ldap authentication (#3743)

* [FEATURE-736] integrate ldap authentication

* add ldap authentication type
* refactor authentication with ldap and password
* add  createUser for ldap in user service
* remove duplicate password authenticator

* [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290)

* fix flink name not display in process definition editor

* fix flink name not escape problem

* simplify escape method

* fix database is the mysql DB keyword. (#4295)

* fix the token management list does not display the user name. (#4302)

* Dev imp server process utils (#4263)


* Adds the comment on the WINDOWSATTERN

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>

* fix version close function does not take effect. (#4307)

* fix user management authorization operation exceptions do not prompt exception information. (#4292)

* [Fix][UI] Fix ui style problem and refactor form style (#4329)

* rename from-model to form-model

* [UI] remove duplicated css style code

* [UI] refactor css style of form model

* fix form model overflow problem in ie

* Align each item one by one in locale

* fix el-dialog width is too wide

* fix locale

* [Improvement][UI] Improve script box and dialog css style (#4331)

* [Fix-4335][UI] Fix IE 9-11 not supported

* [FIx-4338][UI] Fix invalid date problem in IE

* [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341)

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Fix][common] only two yarns can be selected(#4314) (#4344)

* Handling conflicts manually

* code style

* ut skip ui

* fix ut error

* fix ut error

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* ci skip ui

* ignore AlertPluginManagerTest.java and DolphinPluginLoaderTest.java

* fix ut error

* fix ut error

* test

* code style

* replace ui

* replace ui

* replace ui

* replace ui

* fix license

* fix license

Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: BoYiZhang <zhangboyi_mx@163.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: 孙继峰 <sun.jifeng@outlook.com>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com>
Co-authored-by: huzekang <1040080742@qq.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>
data_quality_design
Kirs 3 years ago committed by GitHub
parent
5808a0a8b1
commit
4d553603ee
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
332 changed files with 11693 additions and 9331 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 5
      .github/ISSUE_TEMPLATE/bug_report.md
  2. 5
      .github/ISSUE_TEMPLATE/question.md
  3. 24
      .github/ISSUE_TEMPLATE/test.md
  4. 3
      .github/workflows/ci_frontend.yml
  5. 128
      .gitignore
  6. BIN
      .mvn/wrapper/maven-wrapper.jar
  7. 467
      ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-application.xml
  8. 6
      ambari_plugin/common-services/DOLPHIN/1.3.3/alerts.json
  9. 7
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-alert.xml
  10. 16
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-application-api.xml
  11. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-common.xml
  12. 206
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-datasource.xml
  13. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-env.xml
  14. 88
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-master.xml
  15. 23
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-quartz.xml
  16. 67
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-worker.xml
  17. 76
      ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-zookeeper.xml
  18. 6
      ambari_plugin/common-services/DOLPHIN/1.3.3/metainfo.xml
  19. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py
  20. 3
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_alert_service.py
  21. 3
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_api_service.py
  22. 46
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_env.py
  23. 4
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_logger_service.py
  24. 3
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_master_service.py
  25. 3
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_worker_service.py
  26. 85
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/params.py
  27. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/service_check.py
  28. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/status_params.py
  29. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/alert.properties.j2
  30. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/application-api.properties.j2
  31. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/common.properties.j2
  32. 20
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/datasource.properties.j2
  33. 13
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/dolphin-daemon.sh.j2
  34. 20
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/master.properties.j2
  35. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/quartz.properties.j2
  36. 20
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/worker.properties.j2
  37. 20
      ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/zookeeper.properties.j2
  38. 0
      ambari_plugin/common-services/DOLPHIN/1.3.3/quicklinks/quicklinks.json
  39. 84
      ambari_plugin/common-services/DOLPHIN/1.3.3/themes/theme.json
  40. 4
      docker/build/startup.sh
  41. 13
      docker/kubernetes/dolphinscheduler/requirements.yaml
  42. 2
      dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/pom.xml
  43. 2
      dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml
  44. 2
      dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/pom.xml
  45. 2
      dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/pom.xml
  46. 2
      dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml
  47. 2
      dolphinscheduler-alert-plugin/pom.xml
  48. 2
      dolphinscheduler-alert/pom.xml
  49. 2
      dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManagerTest.java
  50. 2
      dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoaderTest.java
  51. 438
      dolphinscheduler-api/pom.xml
  52. 156
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/configuration/AppConfiguration.java
  53. 6
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AccessTokenController.java
  54. 2
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java
  55. 4
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
  56. 17
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProjectController.java
  57. 244
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ResourcesController.java
  58. 131
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java
  59. 42
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TaskInstanceController.java
  60. 25
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TenantController.java
  61. 22
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UsersController.java
  62. 14
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/WorkFlowLineageController.java
  63. 28
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/CommandStateCount.java
  64. 81
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/TaskCountDto.java
  65. 24
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/TaskStateCount.java
  66. 12
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
  67. 2
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/AuthenticationType.java
  68. 5
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/Authenticator.java
  69. 9
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/SecurityConfig.java
  70. 35
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/AbstractAuthenticator.java
  71. 45
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticator.java
  72. 133
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapService.java
  73. 34
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/pwd/PasswordAuthenticator.java
  74. 7
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AccessTokenService.java
  75. 19
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/BaseService.java
  76. 30
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/DataSourceService.java
  77. 171
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java
  78. 4
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
  79. 7
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProjectService.java
  80. 181
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ResourcesService.java
  81. 90
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java
  82. 7
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskInstanceService.java
  83. 5
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TenantService.java
  84. 20
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UdfFuncService.java
  85. 135
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UsersService.java
  86. 76
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/WorkerGroupService.java
  87. 24
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/AccessTokenServiceImpl.java
  88. 169
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/DataAnalysisServiceImpl.java
  89. 1
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/LoggerServiceImpl.java
  90. 172
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
  91. 22
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
  92. 32
      dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TenantServiceImpl.java
  93. 20
      dolphinscheduler-api/src/main/resources/application-api.properties
  94. 2
      dolphinscheduler-api/src/main/resources/i18n/messages.properties
  95. 2
      dolphinscheduler-api/src/main/resources/i18n/messages_en_US.properties
  96. 489
      dolphinscheduler-api/src/main/resources/i18n/messages_zh_CN.properties
  97. 1
      dolphinscheduler-api/src/main/resources/logback-api.xml
  98. 4
      dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
  99. 72
      dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TaskInstanceControllerTest.java
  100. 8
      dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TenantControllerTest.java
  101. Some files were not shown because too many files have changed in this diff Show More

5
.github/ISSUE_TEMPLATE/bug_report.md
Unescape View File

@ -7,10 +7,7 @@ assignees: ''
---
*For better global communication, please give priority to using English description, thx! *
*Please review https://dolphinscheduler.apache.org/en-us/docs/development/issue.html when describe an issue.*
**For better global communication, Please describe it in English. If you feel the description in English is not clear, then you can append description in Chinese(just for Mandarin(CN)), thx! **
**Describe the bug**
A clear and concise description of what the bug is.

5
.github/ISSUE_TEMPLATE/question.md
Unescape View File

@ -7,10 +7,7 @@ assignees: ''
---
*For better global communication, please give priority to using English description, thx! *
*Please review https://dolphinscheduler.apache.org/en-us/docs/development/issue.html when describe an issue.*
**For better global communication, Please describe it in English. If you feel the description in English is not clear, then you can append description in Chinese(just for Mandarin(CN)), thx! **
**Describe the question**
A clear and concise description of what the question is.

24
.github/ISSUE_TEMPLATE/test.md
Unescape View File

@ -1,24 +0,0 @@
---
name: Test
about: Test to enhance the robustness of this project
title: "[Test][Module Name] Test title"
labels: test
assignees: ''
---
*For better global communication, please give priority to using English description, thx! *
*Please review https://dolphinscheduler.apache.org/en-us/docs/development/issue.html when describe an issue.*
**Describe the question**
A clear and concise description of what the test part is.
**What are the current deficiencies and the benefits of changing or adding this test**
- A clear and concise description of the current deficiencies, the benefits of changing or adding this test, and the scope involved.
**Which version of DolphinScheduler:**
-[1.1.0-preview]
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions you've considered.

3
.github/workflows/ci_frontend.yml
Unescape View File

@ -52,7 +52,6 @@ jobs:
npm install
npm run lint
npm run build
License-check:
runs-on: ubuntu-latest
steps:
@ -68,4 +67,4 @@ jobs:
with:
java-version: 1.8
- name: Check
run: mvn -B apache-rat:check
run: mvn -B apache-rat:check

128
.gitignore vendored
Unescape View File

@ -4,25 +4,27 @@
.zip
.gz
.DS_Store
.idea
.idea/
dist/
all-dependencies.txt
self-modules.txt
third-party-dependencies.txt
**/target/
.idea/*
.target
.target/
**/**/target/**
target/*
*/target
*/target/*
.settings
.nbproject
.classpath
.project
**/*.iml
*.iml
*.ipr
*.iws
*.tgz
.*.swp
.factorypath
.vim
.tmp
**/node_modules
node_modules
npm-debug.log
.vscode
logs/*
@ -39,10 +41,110 @@ dolphinscheduler-alert/logs/
dolphinscheduler-alert/src/main/resources/alert.properties_bak
dolphinscheduler-alert/src/main/resources/logback.xml
dolphinscheduler-server/src/main/resources/logback.xml
dolphinscheduler-ui/dist/
dolphinscheduler-ui/dist
dolphinscheduler-ui/node
dolphinscheduler-dao/src/main/resources/dao/data_source.properties
dolphinscheduler-ui/dist/css/common.16ac5d9.css
dolphinscheduler-ui/dist/css/home/index.b444b91.css
dolphinscheduler-ui/dist/css/login/index.5866c64.css
dolphinscheduler-ui/dist/js/0.ac94e5d.js
dolphinscheduler-ui/dist/js/0.ac94e5d.js.map
dolphinscheduler-ui/dist/js/1.0b043a3.js
dolphinscheduler-ui/dist/js/1.0b043a3.js.map
dolphinscheduler-ui/dist/js/10.1bce3dc.js
dolphinscheduler-ui/dist/js/10.1bce3dc.js.map
dolphinscheduler-ui/dist/js/11.79f04d8.js
dolphinscheduler-ui/dist/js/11.79f04d8.js.map
dolphinscheduler-ui/dist/js/12.420daa5.js
dolphinscheduler-ui/dist/js/12.420daa5.js.map
dolphinscheduler-ui/dist/js/13.e5bae1c.js
dolphinscheduler-ui/dist/js/13.e5bae1c.js.map
dolphinscheduler-ui/dist/js/14.f2a0dca.js
dolphinscheduler-ui/dist/js/14.f2a0dca.js.map
dolphinscheduler-ui/dist/js/15.45373e8.js
dolphinscheduler-ui/dist/js/15.45373e8.js.map
dolphinscheduler-ui/dist/js/16.fecb0fc.js
dolphinscheduler-ui/dist/js/16.fecb0fc.js.map
dolphinscheduler-ui/dist/js/17.84be279.js
dolphinscheduler-ui/dist/js/17.84be279.js.map
dolphinscheduler-ui/dist/js/18.307ea70.js
dolphinscheduler-ui/dist/js/18.307ea70.js.map
dolphinscheduler-ui/dist/js/19.144db9c.js
dolphinscheduler-ui/dist/js/19.144db9c.js.map
dolphinscheduler-ui/dist/js/2.8b4ef29.js
dolphinscheduler-ui/dist/js/2.8b4ef29.js.map
dolphinscheduler-ui/dist/js/20.4c527e9.js
dolphinscheduler-ui/dist/js/20.4c527e9.js.map
dolphinscheduler-ui/dist/js/21.831b2a2.js
dolphinscheduler-ui/dist/js/21.831b2a2.js.map
dolphinscheduler-ui/dist/js/22.2b4bb2a.js
dolphinscheduler-ui/dist/js/22.2b4bb2a.js.map
dolphinscheduler-ui/dist/js/23.81467ef.js
dolphinscheduler-ui/dist/js/23.81467ef.js.map
dolphinscheduler-ui/dist/js/24.54a00e4.js
dolphinscheduler-ui/dist/js/24.54a00e4.js.map
dolphinscheduler-ui/dist/js/25.8d7bd36.js
dolphinscheduler-ui/dist/js/25.8d7bd36.js.map
dolphinscheduler-ui/dist/js/26.2ec5e78.js
dolphinscheduler-ui/dist/js/26.2ec5e78.js.map
dolphinscheduler-ui/dist/js/27.3ab48c2.js
dolphinscheduler-ui/dist/js/27.3ab48c2.js.map
dolphinscheduler-ui/dist/js/28.363088a.js
dolphinscheduler-ui/dist/js/28.363088a.js.map
dolphinscheduler-ui/dist/js/29.6c5853a.js
dolphinscheduler-ui/dist/js/29.6c5853a.js.map
dolphinscheduler-ui/dist/js/3.a0edb5b.js
dolphinscheduler-ui/dist/js/3.a0edb5b.js.map
dolphinscheduler-ui/dist/js/30.940fdd3.js
dolphinscheduler-ui/dist/js/30.940fdd3.js.map
dolphinscheduler-ui/dist/js/31.168a460.js
dolphinscheduler-ui/dist/js/31.168a460.js.map
dolphinscheduler-ui/dist/js/32.8df6594.js
dolphinscheduler-ui/dist/js/32.8df6594.js.map
dolphinscheduler-ui/dist/js/33.4480bbe.js
dolphinscheduler-ui/dist/js/33.4480bbe.js.map
dolphinscheduler-ui/dist/js/34.b407fe1.js
dolphinscheduler-ui/dist/js/34.b407fe1.js.map
dolphinscheduler-ui/dist/js/35.f340b0a.js
dolphinscheduler-ui/dist/js/35.f340b0a.js.map
dolphinscheduler-ui/dist/js/36.8880c2d.js
dolphinscheduler-ui/dist/js/36.8880c2d.js.map
dolphinscheduler-ui/dist/js/37.ea2a25d.js
dolphinscheduler-ui/dist/js/37.ea2a25d.js.map
dolphinscheduler-ui/dist/js/38.98a59ee.js
dolphinscheduler-ui/dist/js/38.98a59ee.js.map
dolphinscheduler-ui/dist/js/39.a5e958a.js
dolphinscheduler-ui/dist/js/39.a5e958a.js.map
dolphinscheduler-ui/dist/js/4.4ca44db.js
dolphinscheduler-ui/dist/js/4.4ca44db.js.map
dolphinscheduler-ui/dist/js/40.e187b1e.js
dolphinscheduler-ui/dist/js/40.e187b1e.js.map
dolphinscheduler-ui/dist/js/41.0e89182.js
dolphinscheduler-ui/dist/js/41.0e89182.js.map
dolphinscheduler-ui/dist/js/42.341047c.js
dolphinscheduler-ui/dist/js/42.341047c.js.map
dolphinscheduler-ui/dist/js/43.27b8228.js
dolphinscheduler-ui/dist/js/43.27b8228.js.map
dolphinscheduler-ui/dist/js/44.e8869bc.js
dolphinscheduler-ui/dist/js/44.e8869bc.js.map
dolphinscheduler-ui/dist/js/45.8d54901.js
dolphinscheduler-ui/dist/js/45.8d54901.js.map
dolphinscheduler-ui/dist/js/5.e1ed7f3.js
dolphinscheduler-ui/dist/js/5.e1ed7f3.js.map
dolphinscheduler-ui/dist/js/6.241ba07.js
dolphinscheduler-ui/dist/js/6.241ba07.js.map
dolphinscheduler-ui/dist/js/7.ab2e297.js
dolphinscheduler-ui/dist/js/7.ab2e297.js.map
dolphinscheduler-ui/dist/js/8.83ff814.js
dolphinscheduler-ui/dist/js/8.83ff814.js.map
dolphinscheduler-ui/dist/js/9.39cb29f.js
dolphinscheduler-ui/dist/js/9.39cb29f.js.map
dolphinscheduler-ui/dist/js/common.733e342.js
dolphinscheduler-ui/dist/js/common.733e342.js.map
dolphinscheduler-ui/dist/js/home/index.78a5d12.js
dolphinscheduler-ui/dist/js/home/index.78a5d12.js.map
dolphinscheduler-ui/dist/js/login/index.291b8e3.js
dolphinscheduler-ui/dist/js/login/index.291b8e3.js.map
dolphinscheduler-ui/dist/lib/external/
dolphinscheduler-ui/src/js/conf/home/pages/projects/pages/taskInstance/index.vue
/dolphinscheduler-dao/src/main/resources/dao/data_source.properties
.mvn/wrapper/*.jar
!/zookeeper_data/

BIN
.mvn/wrapper/maven-wrapper.jar vendored
View File

Binary file not shown.

467
ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-application.xml
Unescape View File

@ -1,467 +0,0 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<configuration>
<property>
<name>spring.datasource.initialSize</name>
<value>5</value>
<description>
Init connection number
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.minIdle</name>
<value>5</value>
<description>
Min connection number
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.maxActive</name>
<value>50</value>
<description>
Max connection number
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.maxWait</name>
<value>60000</value>
<description>
Max wait time for get a connection in milliseconds.
If configuring maxWait, fair locks are enabled by default and concurrency efficiency decreases.
If necessary, unfair locks can be used by configuring the useUnfairLock attribute to true.
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.timeBetweenEvictionRunsMillis</name>
<value>60000</value>
<description>
Milliseconds for check to close free connections
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.timeBetweenConnectErrorMillis</name>
<value>60000</value>
<description>
The Destroy thread detects the connection interval and closes the physical connection in milliseconds
if the connection idle time is greater than or equal to minEvictableIdleTimeMillis.
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.minEvictableIdleTimeMillis</name>
<value>300000</value>
<description>
The longest time a connection remains idle without being evicted, in milliseconds
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.validationQuery</name>
<value>SELECT 1</value>
<description>
The SQL used to check whether the connection is valid requires a query statement.
If validation Query is null, testOnBorrow, testOnReturn, and testWhileIdle will not work.
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.validationQueryTimeout</name>
<value>3</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
Check whether the connection is valid for timeout, in seconds
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.testWhileIdle</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
When applying for a connection,
if it is detected that the connection is idle longer than time Between Eviction Runs Millis,
validation Query is performed to check whether the connection is valid
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.testOnBorrow</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
Execute validation to check if the connection is valid when applying for a connection
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.testOnReturn</name>
<value>false</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
Execute validation to check if the connection is valid when the connection is returned
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.defaultAutoCommit</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.keepAlive</name>
<value>false</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.poolPreparedStatements</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
Open PSCache, specify count PSCache for every connection
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.maxPoolPreparedStatementPerConnectionSize</name>
<value>20</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.spring.datasource.filters</name>
<value>stat,wall,log4j</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.connectionProperties</name>
<value>druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.mapper-locations</name>
<value>classpath*:/org.apache.dolphinscheduler.dao.mapper/*.xml</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.typeEnumsPackage</name>
<value>org.apache.dolphinscheduler.*.enums</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.typeAliasesPackage</name>
<value>org.apache.dolphinscheduler.dao.entity</value>
<description>
Entity scan, where multiple packages are separated by a comma or semicolon
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.global-config.db-config.id-type</name>
<value>AUTO</value>
<value-attributes>
<type>value-list</type>
<entries>
<entry>
<value>AUTO</value>
<label>AUTO</label>
</entry>
<entry>
<value>INPUT</value>
<label>INPUT</label>
</entry>
<entry>
<value>ID_WORKER</value>
<label>ID_WORKER</label>
</entry>
<entry>
<value>UUID</value>
<label>UUID</label>
</entry>
</entries>
<selection-cardinality>1</selection-cardinality>
</value-attributes>
<description>
Primary key type AUTO:" database ID AUTO ",
INPUT:" user INPUT ID",
ID_WORKER:" global unique ID (numeric type unique ID)",
UUID:" global unique ID UUID";
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.global-config.db-config.field-strategy</name>
<value>NOT_NULL</value>
<value-attributes>
<type>value-list</type>
<entries>
<entry>
<value>IGNORED</value>
<label>IGNORED</label>
</entry>
<entry>
<value>NOT_NULL</value>
<label>NOT_NULL</label>
</entry>
<entry>
<value>NOT_EMPTY</value>
<label>NOT_EMPTY</label>
</entry>
</entries>
<selection-cardinality>1</selection-cardinality>
</value-attributes>
<description>
Field policy IGNORED:" ignore judgment ",
NOT_NULL:" not NULL judgment "),
NOT_EMPTY:" not NULL judgment"
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.global-config.db-config.column-underline</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.global-config.db-config.logic-delete-value</name>
<value>1</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.global-config.db-config.logic-not-delete-value</name>
<value>0</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.global-config.db-config.banner</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.configuration.map-underscore-to-camel-case</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.configuration.cache-enabled</name>
<value>false</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.configuration.call-setters-on-nulls</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>mybatis-plus.configuration.jdbc-type-for-null</name>
<value>null</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.exec.threads</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.exec.task.num</name>
<value>20</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.heartbeat.interval</name>
<value>10</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.task.commit.retryTimes</name>
<value>5</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.task.commit.interval</name>
<value>1000</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.max.cpuload.avg</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.reserved.memory</name>
<value>0.1</value>
<value-attributes>
<type>float</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.exec.threads</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.heartbeat.interval</name>
<value>10</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.fetch.task.num</name>
<value>3</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.max.cpuload.avg</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.reserved.memory</name>
<value>0.1</value>
<value-attributes>
<type>float</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
</configuration>

6
ambari_plugin/common-services/DOLPHIN/1.2.1/alerts.json → ambari_plugin/common-services/DOLPHIN/1.3.3/alerts.json
Unescape View File

@ -65,7 +65,7 @@
"enabled": true,
"source": {
"type": "SCRIPT",
"path": "DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py",
"path": "DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py",
"parameters": [
{
@ -98,7 +98,7 @@
"enabled": true,
"source": {
"type": "SCRIPT",
"path": "DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py",
"path": "DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py",
"parameters": [
{
@ -131,7 +131,7 @@
"enabled": true,
"source": {
"type": "SCRIPT",
"path": "DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py",
"path": "DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py",
"parameters": [
{

7
ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-alert.xml → ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-alert.xml
Unescape View File

@ -90,13 +90,6 @@
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>xls.file.path</name>
<value>/tmp/xls</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>enterprise.wechat.enable</name>
<value>false</value>

16
ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-application-api.xml → ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-application-api.xml
Unescape View File

@ -34,6 +34,12 @@
<description>
</description>
</property>
<property>
<name>server.servlet.context-path</name>
<value>/dolphinscheduler/</value>
<description>
</description>
</property>
<property>
<name>spring.servlet.multipart.max-file-size</name>
<value>1024</value>
@ -68,4 +74,14 @@
<value>UTF-8</value>
<description></description>
</property>
<property>
<name>spring.messages.basename</name>
<value>i18n/messages</value>
<description></description>
</property>
<property>
<name>security.authentication.type</name>
<value>PASSWORD</value>
<description></description>
</property>
</configuration>

0
ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-common.xml → ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-common.xml
Unescape View File

206
ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-datasource.xml
Unescape View File

@ -0,0 +1,206 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<configuration>
<property>
<name>spring.datasource.initialSize</name>
<value>5</value>
<description>
Init connection number
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.minIdle</name>
<value>5</value>
<description>
Min connection number
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.maxActive</name>
<value>50</value>
<description>
Max connection number
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.maxWait</name>
<value>60000</value>
<description>
Max wait time for get a connection in milliseconds.
If configuring maxWait, fair locks are enabled by default and concurrency efficiency decreases.
If necessary, unfair locks can be used by configuring the useUnfairLock attribute to true.
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.timeBetweenEvictionRunsMillis</name>
<value>60000</value>
<description>
Milliseconds for check to close free connections
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.timeBetweenConnectErrorMillis</name>
<value>60000</value>
<description>
The Destroy thread detects the connection interval and closes the physical connection in milliseconds
if the connection idle time is greater than or equal to minEvictableIdleTimeMillis.
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.minEvictableIdleTimeMillis</name>
<value>300000</value>
<description>
The longest time a connection remains idle without being evicted, in milliseconds
</description>
<value-attributes>
<type>int</type>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.validationQuery</name>
<value>SELECT 1</value>
<description>
The SQL used to check whether the connection is valid requires a query statement.
If validation Query is null, testOnBorrow, testOnReturn, and testWhileIdle will not work.
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.validationQueryTimeout</name>
<value>3</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
Check whether the connection is valid for timeout, in seconds
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.testWhileIdle</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
When applying for a connection,
if it is detected that the connection is idle longer than time Between Eviction Runs Millis,
validation Query is performed to check whether the connection is valid
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.testOnBorrow</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
Execute validation to check if the connection is valid when applying for a connection
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.testOnReturn</name>
<value>false</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
Execute validation to check if the connection is valid when the connection is returned
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.defaultAutoCommit</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.keepAlive</name>
<value>false</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.poolPreparedStatements</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description>
Open PSCache, specify count PSCache for every connection
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.maxPoolPreparedStatementPerConnectionSize</name>
<value>20</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.spring.datasource.filters</name>
<value>stat,wall,log4j</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>spring.datasource.connectionProperties</name>
<value>druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000</value>
<description></description>
<on-ambari-upgrade add="true"/>
</property>
</configuration>

0
ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-env.xml → ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-env.xml
Unescape View File

88
ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-master.xml
Unescape View File

@ -0,0 +1,88 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<configuration>
<property>
<name>master.exec.threads</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>master execute thread num</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.exec.task.num</name>
<value>20</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>master execute task number in parallel</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.heartbeat.interval</name>
<value>10</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>master heartbeat interval</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.task.commit.retryTimes</name>
<value>5</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>master commit task retry times</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.task.commit.interval</name>
<value>1000</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>master commit task interval</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.max.cpuload.avg</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>only less than cpu avg load, master server can work. default value : the number of cpu cores * 2</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.reserved.memory</name>
<value>0.3</value>
<description>only larger than reserved memory, master server can work. default value : physical memory * 1/10, unit is G.</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>master.listen.port</name>
<value>5678</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>master listen port</description>
<on-ambari-upgrade add="true"/>
</property>
</configuration>

23
ambari_plugin/common-services/DOLPHIN/1.2.1/configuration/dolphin-quartz.xml → ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-quartz.xml
Unescape View File

@ -106,26 +106,21 @@
<description></description>
</property>
<property>
<name>org.quartz.jobStore.dataSource</name>
<value>myDs</value>
<description></description>
</property>
<property>
<name>org.quartz.dataSource.myDs.connectionProvider.class</name>
<value>org.apache.dolphinscheduler.server.quartz.DruidConnectionProvider</value>
<name>org.quartz.jobStore.acquireTriggersWithinLock</name>
<value>true</value>
<value-attributes>
<type>boolean</type>
</value-attributes>
<description></description>
</property>
<property>
<name>org.quartz.dataSource.myDs.maxConnections</name>
<value>10</value>
<value-attributes>
<type>int</type>
</value-attributes>
<name>org.quartz.jobStore.dataSource</name>
<value>myDs</value>
<description></description>
</property>
<property>
<name>org.quartz.dataSource.myDs.validationQuery</name>
<value>select 1</value>
<name>org.quartz.dataSource.myDs.connectionProvider.class</name>
<value>org.apache.dolphinscheduler.service.quartz.DruidConnectionProvider</value>
<description></description>
</property>
</configuration>

67
ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-worker.xml
Unescape View File

@ -0,0 +1,67 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<configuration>
<property>
<name>worker.exec.threads</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>worker execute thread num</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.heartbeat.interval</name>
<value>10</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>worker heartbeat interval</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.max.cpuload.avg</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>only less than cpu avg load, worker server can work. default value : the number of cpu cores * 2</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.reserved.memory</name>
<value>0.3</value>
<description>only larger than reserved memory, worker server can work. default value : physical memory * 1/10, unit is G.</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.listen.port</name>
<value>1234</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>worker listen port</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>worker.groups</name>
<value>default</value>
<description>default worker group</description>
<on-ambari-upgrade add="true"/>
</property>
</configuration>

76
ambari_plugin/common-services/DOLPHIN/1.3.3/configuration/dolphin-zookeeper.xml
Unescape View File

@ -0,0 +1,76 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<configuration>
<property>
<name>zookeeper.dolphinscheduler.root</name>
<value>/dolphinscheduler</value>
<description>
dolphinscheduler root directory
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>zookeeper.session.timeout</name>
<value>300</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>zookeeper.connection.timeout</name>
<value>300</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>zookeeper.retry.base.sleep</name>
<value>100</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>zookeeper.retry.max.sleep</name>
<value>30000</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
<property>
<name>zookeeper.retry.maxtime</name>
<value>5</value>
<value-attributes>
<type>int</type>
</value-attributes>
<description>
</description>
<on-ambari-upgrade add="true"/>
</property>
</configuration>

6
ambari_plugin/common-services/DOLPHIN/1.2.1/metainfo.xml → ambari_plugin/common-services/DOLPHIN/1.3.3/metainfo.xml
Unescape View File

@ -22,7 +22,7 @@
<name>DOLPHIN</name>
<displayName>Dolphin Scheduler</displayName>
<comment>分布式易扩展的可视化DAG工作流任务调度系统</comment>
<version>1.2.1</version>
<version>1.3.3</version>
<components>
<component>
<name>DOLPHIN_MASTER</name>
@ -103,7 +103,7 @@
<osFamily>any</osFamily>
<packages>
<package>
<name>apache-dolphinscheduler-incubating-1.2.1*</name>
<name>apache-dolphinscheduler-incubating*</name>
</package>
</packages>
</osSpecific>
@ -134,4 +134,4 @@
</quickLinksConfigurations>
</service>
</services>
</metainfo>
</metainfo>

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py
Unescape View File

3
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/dolphin_alert_service.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_alert_service.py
Unescape View File

@ -26,7 +26,8 @@ class DolphinAlertService(Script):
import params
env.set_params(params)
self.install_packages(env)
Execute(('chmod', '-R', '777', params.dolphin_home), user=params.dolphin_user, sudo=True)
Execute(('chmod', '-R', '777', params.dolphin_home))
Execute(('chown', '-R', params.dolphin_user + ":" + params.dolphin_group, params.dolphin_home))
def configure(self, env):
import params

3
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/dolphin_api_service.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_api_service.py
Unescape View File

@ -26,7 +26,8 @@ class DolphinApiService(Script):
import params
env.set_params(params)
self.install_packages(env)
Execute(('chmod', '-R', '777', params.dolphin_home), user=params.dolphin_user, sudo=True)
Execute(('chmod', '-R', '777', params.dolphin_home))
Execute(('chown', '-R', params.dolphin_user + ":" + params.dolphin_group, params.dolphin_home))
def configure(self, env):
import params

46
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/dolphin_env.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_env.py
Unescape View File

@ -42,31 +42,12 @@ def dolphin_env():
create_parents=True
)
Directory(params.dolphin_alert_map['xls.file.path'],
mode=0777,
owner=params.dolphin_user,
group=params.dolphin_group,
create_parents=True
)
Directory(params.dolphin_common_map['data.basedir.path'],
mode=0777,
owner=params.dolphin_user,
group=params.dolphin_group,
create_parents=True
)
Directory(params.dolphin_common_map['data.download.basedir.path'],
mode=0777,
owner=params.dolphin_user,
group=params.dolphin_group,
create_parents=True
)
Directory(params.dolphin_common_map['process.exec.basepath'],
mode=0777,
owner=params.dolphin_user,
group=params.dolphin_group,
create_parents=True
)
File(format(params.dolphin_env_path),
@ -79,11 +60,25 @@ def dolphin_env():
File(format(params.dolphin_bin_dir + "/dolphinscheduler-daemon.sh"),
mode=0755,
content=Template("dolphin-daemon.j2"),
content=Template("dolphin-daemon.sh.j2"),
owner=params.dolphin_user,
group=params.dolphin_group
)
File(format(params.dolphin_conf_dir + "/master.properties"),
mode=0755,
content=Template("master.properties.j2"),
owner=params.dolphin_user,
group=params.dolphin_group
)
File(format(params.dolphin_conf_dir + "/worker.properties"),
mode=0755,
content=Template("worker.properties.j2"),
owner=params.dolphin_user,
group=params.dolphin_group
)
File(format(params.dolphin_conf_dir + "/alert.properties"),
mode=0755,
@ -92,9 +87,9 @@ def dolphin_env():
group=params.dolphin_group
)
File(format(params.dolphin_conf_dir + "/application.properties"),
File(format(params.dolphin_conf_dir + "/datasource.properties"),
mode=0755,
content=Template("application.properties.j2"),
content=Template("datasource.properties.j2"),
owner=params.dolphin_user,
group=params.dolphin_group
)
@ -119,3 +114,10 @@ def dolphin_env():
owner=params.dolphin_user,
group=params.dolphin_group
)
File(format(params.dolphin_conf_dir + "/zookeeper.properties"),
mode=0755,
content=Template("zookeeper.properties.j2"),
owner=params.dolphin_user,
group=params.dolphin_group
)

4
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/dolphin_logger_service.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_logger_service.py
Unescape View File

@ -26,8 +26,8 @@ class DolphinLoggerService(Script):
import params
env.set_params(params)
self.install_packages(env)
Execute(('chmod', '-R', '777', params.dolphin_home), user=params.dolphin_user, sudo=True)
Execute(('chmod', '-R', '777', params.dolphin_home))
Execute(('chown', '-R', params.dolphin_user + ":" + params.dolphin_group, params.dolphin_home))
def configure(self, env):
import params
params.pika_slave = True

3
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/dolphin_master_service.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_master_service.py
Unescape View File

@ -27,7 +27,8 @@ class DolphinMasterService(Script):
import params
env.set_params(params)
self.install_packages(env)
Execute(('chmod', '-R', '777', params.dolphin_home), user=params.dolphin_user, sudo=True)
Execute(('chmod', '-R', '777', params.dolphin_home))
Execute(('chown', '-R', params.dolphin_user + ":" + params.dolphin_group, params.dolphin_home))
def configure(self, env):
import params

3
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/dolphin_worker_service.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/dolphin_worker_service.py
Unescape View File

@ -26,7 +26,8 @@ class DolphinWorkerService(Script):
import params
env.set_params(params)
self.install_packages(env)
Execute(('chmod', '-R', '777', params.dolphin_home), user=params.dolphin_user, sudo=True)
Execute(('chmod', '-R', '777', params.dolphin_home))
Execute(('chown', '-R', params.dolphin_user + ":" + params.dolphin_group, params.dolphin_home))
def configure(self, env):
import params

85
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/params.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/params.py
Unescape View File

@ -54,11 +54,8 @@ dolphin_env_content = dolphin_env_map['dolphinscheduler-env-content']
# database config
dolphin_database_config = {}
dolphin_database_config['dolphin_database_type'] = dolphin_env_map['dolphin.database.type']
dolphin_database_config['dolphin_database_host'] = dolphin_env_map['dolphin.database.host']
dolphin_database_config['dolphin_database_port'] = dolphin_env_map['dolphin.database.port']
dolphin_database_config['dolphin_database_username'] = dolphin_env_map['dolphin.database.username']
dolphin_database_config['dolphin_database_password'] = dolphin_env_map['dolphin.database.password']
if 'mysql' == dolphin_database_config['dolphin_database_type']:
dolphin_database_config['dolphin_database_driver'] = 'com.mysql.jdbc.Driver'
dolphin_database_config['driverDelegateClass'] = 'org.quartz.impl.jdbcjobstore.StdJDBCDelegate'
@ -72,6 +69,10 @@ else:
+ ':' + dolphin_env_map['dolphin.database.port'] \
+ '/dolphinscheduler'
# application-alert.properties
dolphin_alert_map = {}
wechat_push_url = 'https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=$token'
@ -79,27 +80,22 @@ wechat_token_url = 'https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid=$corpId&
wechat_team_send_msg = '{\"toparty\":\"{toParty}\",\"agentid\":\"{agentId}\",\"msgtype\":\"text\",\"text\":{\"content\":\"{msg}\"},\"safe\":\"0\"}'
wechat_user_send_msg = '{\"touser\":\"{toUser}\",\"agentid\":\"{agentId}\",\"msgtype\":\"markdown\",\"markdown\":{\"content\":\"{msg}\"}}'
dolphin_alert_map['enterprise.wechat.push.ur'] = wechat_push_url
dolphin_alert_map['enterprise.wechat.token.url'] = wechat_token_url
dolphin_alert_map['enterprise.wechat.team.send.msg'] = wechat_team_send_msg
dolphin_alert_map['enterprise.wechat.user.send.msg'] = wechat_user_send_msg
dolphin_alert_map.update(config['configurations']['dolphin-alert'])
dolphin_alert_config_map = config['configurations']['dolphin-alert']
if dolphin_alert_config_map['enterprise.wechat.enable']:
dolphin_alert_map['enterprise.wechat.push.ur'] = wechat_push_url
dolphin_alert_map['enterprise.wechat.token.url'] = wechat_token_url
dolphin_alert_map['enterprise.wechat.team.send.msg'] = wechat_team_send_msg
dolphin_alert_map['enterprise.wechat.user.send.msg'] = wechat_user_send_msg
dolphin_alert_map.update(dolphin_alert_config_map)
# application-api.properties
dolphin_app_api_map = {}
dolphin_app_api_map['logging.config'] = 'classpath:apiserver_logback.xml'
dolphin_app_api_map['spring.messages.basename'] = 'i18n/messages'
dolphin_app_api_map['server.servlet.context-path'] = '/dolphinscheduler/'
dolphin_app_api_map.update(config['configurations']['dolphin-application-api'])
# application-dao.properties
dolphin_application_map = {}
dolphin_application_map['spring.datasource.type'] = 'com.alibaba.druid.pool.DruidDataSource'
dolphin_application_map['spring.datasource.driver-class-name'] = dolphin_database_config['dolphin_database_driver']
dolphin_application_map['spring.datasource.url'] = dolphin_database_config['dolphin_database_url']
dolphin_application_map['spring.datasource.username'] = dolphin_database_config['dolphin_database_username']
dolphin_application_map['spring.datasource.password'] = dolphin_database_config['dolphin_database_password']
dolphin_application_map.update(config['configurations']['dolphin-application'])
# common.properties
dolphin_common_map = {}
@ -118,33 +114,42 @@ else:
dolphin_common_map_tmp = config['configurations']['dolphin-common']
data_basedir_path = dolphin_common_map_tmp['data.basedir.path']
process_exec_basepath = data_basedir_path + '/exec'
data_download_basedir_path = data_basedir_path + '/download'
dolphin_common_map['process.exec.basepath'] = process_exec_basepath
dolphin_common_map['data.download.basedir.path'] = data_download_basedir_path
dolphin_common_map['dolphinscheduler.env.path'] = dolphin_env_path
dolphin_common_map.update(config['configurations']['dolphin-common'])
zookeeperHosts = default("/clusterHostInfo/zookeeper_hosts", [])
if len(zookeeperHosts) > 0 and "clientPort" in config['configurations']['zoo.cfg']:
clientPort = config['configurations']['zoo.cfg']['clientPort']
zookeeperPort = ":" + clientPort + ","
dolphin_common_map['zookeeper.quorum'] = zookeeperPort.join(zookeeperHosts) + ":" + clientPort
# datasource.properties
dolphin_datasource_map = {}
dolphin_datasource_map['spring.datasource.type'] = 'com.alibaba.druid.pool.DruidDataSource'
dolphin_datasource_map['spring.datasource.driver-class-name'] = dolphin_database_config['dolphin_database_driver']
dolphin_datasource_map['spring.datasource.url'] = dolphin_database_config['dolphin_database_url']
dolphin_datasource_map['spring.datasource.username'] = dolphin_database_config['dolphin_database_username']
dolphin_datasource_map['spring.datasource.password'] = dolphin_database_config['dolphin_database_password']
dolphin_datasource_map.update(config['configurations']['dolphin-datasource'])
dolphin_common_map.update(config['configurations']['dolphin-common'])
# master.properties
dolphin_master_map = config['configurations']['dolphin-master']
# quartz.properties
dolphin_quartz_map = {}
dolphin_quartz_map['org.quartz.jobStore.driverDelegateClass'] = dolphin_database_config['driverDelegateClass']
dolphin_quartz_map['org.quartz.dataSource.myDs.driver'] = dolphin_database_config['dolphin_database_driver']
dolphin_quartz_map['org.quartz.dataSource.myDs.URL'] = dolphin_database_config['dolphin_database_url']
dolphin_quartz_map['org.quartz.dataSource.myDs.user'] = dolphin_database_config['dolphin_database_username']
dolphin_quartz_map['org.quartz.dataSource.myDs.password'] = dolphin_database_config['dolphin_database_password']
dolphin_quartz_map.update(config['configurations']['dolphin-quartz'])
# if 'ganglia_server_host' in config['clusterHostInfo'] and \
# len(config['clusterHostInfo']['ganglia_server_host'])>0:
# ganglia_installed = True
# ganglia_server = config['clusterHostInfo']['ganglia_server_host'][0]
# ganglia_report_interval = 60
# else:
# ganglia_installed = False
# worker.properties
dolphin_worker_map = config['configurations']['dolphin-worker']
# zookeeper.properties
dolphin_zookeeper_map={}
zookeeperHosts = default("/clusterHostInfo/zookeeper_hosts", [])
if len(zookeeperHosts) > 0 and "clientPort" in config['configurations']['zoo.cfg']:
clientPort = config['configurations']['zoo.cfg']['clientPort']
zookeeperPort = ":" + clientPort + ","
dolphin_zookeeper_map['zookeeper.quorum'] = zookeeperPort.join(zookeeperHosts) + ":" + clientPort
dolphin_zookeeper_map.update(config['configurations']['dolphin-zookeeper'])
if 'spring.servlet.multipart.max-file-size' in dolphin_app_api_map:
file_size = dolphin_app_api_map['spring.servlet.multipart.max-file-size']
dolphin_app_api_map['spring.servlet.multipart.max-file-size'] = file_size + "MB"
if 'spring.servlet.multipart.max-request-size' in dolphin_app_api_map:
request_size = dolphin_app_api_map['spring.servlet.multipart.max-request-size']
dolphin_app_api_map['spring.servlet.multipart.max-request-size'] = request_size + "MB"

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/service_check.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/service_check.py
Unescape View File

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/scripts/status_params.py → ambari_plugin/common-services/DOLPHIN/1.3.3/package/scripts/status_params.py
Unescape View File

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/templates/alert.properties.j2 → ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/alert.properties.j2
Unescape View File

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/templates/application-api.properties.j2 → ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/application-api.properties.j2
Unescape View File

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/templates/common.properties.j2 → ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/common.properties.j2
Unescape View File

20
ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/datasource.properties.j2
Unescape View File

@ -0,0 +1,20 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
{% for key, value in dolphin_datasource_map.iteritems() -%}
{{key}}={{value}}
{% endfor %}

13
ambari_plugin/common-services/DOLPHIN/1.2.1/package/templates/dolphin-daemon.j2 → ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/dolphin-daemon.sh.j2
Unescape View File

@ -48,22 +48,19 @@ pid={{dolphin_pidfile_dir}}/$command.pid
cd $DOLPHINSCHEDULER_HOME
if [ "$command" = "api-server" ]; then
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/apiserver_logback.xml -Dspring.profiles.active=api"
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/logback-api.xml -Dspring.profiles.active=api"
CLASS=org.apache.dolphinscheduler.api.ApiApplicationServer
elif [ "$command" = "master-server" ]; then
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/master_logback.xml -Ddruid.mysql.usePingMethod=false"
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/logback-master.xml -Ddruid.mysql.usePingMethod=false"
CLASS=org.apache.dolphinscheduler.server.master.MasterServer
elif [ "$command" = "worker-server" ]; then
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/worker_logback.xml -Ddruid.mysql.usePingMethod=false"
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/logback-worker.xml -Ddruid.mysql.usePingMethod=false"
CLASS=org.apache.dolphinscheduler.server.worker.WorkerServer
elif [ "$command" = "alert-server" ]; then
LOG_FILE="-Dlogback.configurationFile={{dolphin_conf_dir}}/alert_logback.xml"
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/logback-alert.xml"
CLASS=org.apache.dolphinscheduler.alert.AlertServer
elif [ "$command" = "logger-server" ]; then
CLASS=org.apache.dolphinscheduler.server.rpc.LoggerServer
elif [ "$command" = "combined-server" ]; then
LOG_FILE="-Dlogging.config={{dolphin_conf_dir}}/combined_logback.xml -Dspring.profiles.active=api -Dserver.is-combined-server=true"
CLASS=org.apache.dolphinscheduler.api.CombinedApplicationServer
CLASS=org.apache.dolphinscheduler.server.log.LoggerServer
else
echo "Error: No command named \`$command' was found."
exit 1

20
ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/master.properties.j2
Unescape View File

@ -0,0 +1,20 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
{% for key, value in dolphin_master_map.iteritems() -%}
{{key}}={{value}}
{% endfor %}

0
ambari_plugin/common-services/DOLPHIN/1.2.1/package/templates/quartz.properties.j2 → ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/quartz.properties.j2
Unescape View File

20
ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/worker.properties.j2
Unescape View File

@ -0,0 +1,20 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
{% for key, value in dolphin_worker_map.iteritems() -%}
{{key}}={{value}}
{% endfor %}

20
ambari_plugin/common-services/DOLPHIN/1.3.3/package/templates/zookeeper.properties.j2
Unescape View File

@ -0,0 +1,20 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
{% for key, value in dolphin_zookeeper_map.iteritems() -%}
{{key}}={{value}}
{% endfor %}

0
ambari_plugin/common-services/DOLPHIN/1.2.1/quicklinks/quicklinks.json → ambari_plugin/common-services/DOLPHIN/1.3.3/quicklinks/quicklinks.json
Unescape View File

84
ambari_plugin/common-services/DOLPHIN/1.2.1/themes/theme.json → ambari_plugin/common-services/DOLPHIN/1.3.3/themes/theme.json
Unescape View File

@ -151,18 +151,40 @@
"subsection-name": "env-row1-col2"
},
{
"config": "dolphin-common/res.upload.startup.type",
"config": "dolphin-common/resource.storage.type",
"subsection-name": "dynamic-row1-col1"
},
{
"config": "dolphin-common/resource.upload.path",
"subsection-name": "dynamic-row1-col1",
"depends-on": [
{
"configs":[
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/resource.storage.type} === HDFS || ${dolphin-common/resource.storage.type} === S3",
"then": {
"property_value_attributes": {
"visible": true
}
},
"else": {
"property_value_attributes": {
"visible": false
}
}
}
]
},
{
"config": "dolphin-common/hdfs.root.user",
"subsection-name": "dynamic-row1-col1",
"depends-on": [
{
"configs":[
"dolphin-common/res.upload.startup.type"
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/res.upload.startup.type} === HDFS",
"if": "${dolphin-common/resource.storage.type} === HDFS",
"then": {
"property_value_attributes": {
"visible": true
@ -182,9 +204,9 @@
"depends-on": [
{
"configs":[
"dolphin-common/res.upload.startup.type"
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/res.upload.startup.type} === HDFS",
"if": "${dolphin-common/resource.storage.type} === HDFS",
"then": {
"property_value_attributes": {
"visible": true
@ -204,9 +226,9 @@
"depends-on": [
{
"configs":[
"dolphin-common/res.upload.startup.type"
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/res.upload.startup.type} === HDFS",
"if": "${dolphin-common/resource.storage.type} === HDFS",
"then": {
"property_value_attributes": {
"visible": true
@ -226,9 +248,9 @@
"depends-on": [
{
"configs":[
"dolphin-common/res.upload.startup.type"
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/res.upload.startup.type} === S3",
"if": "${dolphin-common/resource.storage.type} === S3",
"then": {
"property_value_attributes": {
"visible": true
@ -248,9 +270,9 @@
"depends-on": [
{
"configs":[
"dolphin-common/res.upload.startup.type"
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/res.upload.startup.type} === S3",
"if": "${dolphin-common/resource.storage.type} === S3",
"then": {
"property_value_attributes": {
"visible": true
@ -270,9 +292,9 @@
"depends-on": [
{
"configs":[
"dolphin-common/res.upload.startup.type"
"dolphin-common/resource.storage.type"
],
"if": "${dolphin-common/res.upload.startup.type} === S3",
"if": "${dolphin-common/resource.storage.type} === S3",
"then": {
"property_value_attributes": {
"visible": true
@ -356,6 +378,28 @@
}
]
},
{
"config": "dolphin-common/kerberos.expire.time",
"subsection-name": "dynamic-row1-col2",
"depends-on": [
{
"configs":[
"dolphin-common/hadoop.security.authentication.startup.state"
],
"if": "${dolphin-common/hadoop.security.authentication.startup.state}",
"then": {
"property_value_attributes": {
"visible": true
}
},
"else": {
"property_value_attributes": {
"visible": false
}
}
}
]
},
{
"config": "dolphin-alert/enterprise.wechat.enable",
"subsection-name": "dynamic-row1-col3"
@ -505,11 +549,17 @@
}
},
{
"config": "dolphin-common/res.upload.startup.type",
"config": "dolphin-common/resource.storage.type",
"widget": {
"type": "combo"
}
},
{
"config": "dolphin-common/resource.upload.path",
"widget": {
"type": "text-field"
}
},
{
"config": "dolphin-common/hdfs.root.user",
"widget": {
@ -570,6 +620,12 @@
"type": "text-field"
}
},
{
"config": "dolphin-common/kerberos.expire.time",
"widget": {
"type": "text-field"
}
},
{
"config": "dolphin-alert/enterprise.wechat.enable",
"widget": {

4
docker/build/startup.sh
Unescape View File

@ -38,13 +38,13 @@ initDatabase() {
echo "connect ${DATABASE_TYPE} service"
if [ ${DATABASE_TYPE} = "mysql" ]; then
v=$(mysql -h${DATABASE_HOST} -P${DATABASE_PORT} -u${DATABASE_USERNAME} --password=${DATABASE_PASSWORD} -D ${DATABASE_DATABASE} -e "select 1" 2>&1)
if [ "$(echo '${v}' | grep 'ERROR' | wc -l)" -eq 1 ]; then
if [ "$(echo ${v} | grep 'ERROR' | wc -l)" -eq 1 ]; then
echo "Error: Can't connect to database...${v}"
exit 1
fi
else
v=$(sudo -u postgres PGPASSWORD=${DATABASE_PASSWORD} psql -h ${DATABASE_HOST} -p ${DATABASE_PORT} -U ${DATABASE_USERNAME} -d ${DATABASE_DATABASE} -tAc "select 1")
if [ "$(echo '${v}' | grep 'FATAL' | wc -l)" -eq 1 ]; then
if [ "$(echo ${v} | grep 'FATAL' | wc -l)" -eq 1 ]; then
echo "Error: Can't connect to database...${v}"
exit 1
fi

13
ambari_plugin/common-services/DOLPHIN/1.2.1/package/templates/application.properties.j2 → docker/kubernetes/dolphinscheduler/requirements.yaml
Unescape View File

@ -14,7 +14,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
{% for key, value in dolphin_application_map.iteritems() -%}
{{key}}={{value}}
{% endfor %}
dependencies:
- name: postgresql
version: 8.x.x
repository: https://charts.bitnami.com/bitnami
condition: postgresql.enabled
- name: zookeeper
version: 5.x.x
repository: https://charts.bitnami.com/bitnami
condition: redis.enabled

2
dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

2
dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

2
dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

2
dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

2
dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

2
dolphinscheduler-alert-plugin/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<artifactId>dolphinscheduler</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

2
dolphinscheduler-alert/pom.xml
Unescape View File

@ -21,7 +21,7 @@
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler</artifactId>
<version>1.3.2-SNAPSHOT</version>
<version>1.3.4-SNAPSHOT</version>
</parent>
<artifactId>dolphinscheduler-alert</artifactId>
<name>${project.artifactId}</name>

2
dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManagerTest.java
Unescape View File

@ -23,6 +23,7 @@ import org.apache.dolphinscheduler.alert.utils.PropertyUtils;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import org.junit.Assert;
import org.junit.Ignore;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -32,6 +33,7 @@ import com.google.common.collect.ImmutableList;
/**
* AlertPluginManager Tester.
*/
@Ignore
public class AlertPluginManagerTest {
private static final Logger logger = LoggerFactory.getLogger(AlertPluginManagerTest.class);

2
dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoaderTest.java
Unescape View File

@ -20,6 +20,7 @@ package org.apache.dolphinscheduler.alert.plugin;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import com.google.common.collect.ImmutableList;
@ -27,6 +28,7 @@ import com.google.common.collect.ImmutableList;
/**
* DolphinPluginLoader Tester.
*/
@Ignore
public class DolphinPluginLoaderTest {
@Before

438
dolphinscheduler-api/pom.xml
Unescape View File

@ -16,242 +16,256 @@
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler</artifactId>
<version>1.3.2-SNAPSHOT</version>
</parent>
<artifactId>dolphinscheduler-api</artifactId>
<name>${project.artifactId}</name>
<packaging>jar</packaging>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler</artifactId>
<version>1.3.4-SNAPSHOT</version>
</parent>
<artifactId>dolphinscheduler-api</artifactId>
<name>${project.artifactId}</name>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-alert</artifactId>
</dependency>
<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-alert</artifactId>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-dao</artifactId>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-dao</artifactId>
</dependency>
<!--springboot-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
<exclusion>
<artifactId>log4j-to-slf4j</artifactId>
<groupId>org.apache.logging.log4j</groupId>
</exclusion>
</exclusions>
</dependency>
<!--springboot-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
<exclusion>
<artifactId>log4j-to-slf4j</artifactId>
<groupId>org.apache.logging.log4j</groupId>
</exclusion>
</exclusions>
</dependency>
<!-- use jetty -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
<exclusions>
<exclusion>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>javax-websocket-server-impl</artifactId>
</exclusion>
<exclusion>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>websocket-server</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- use jetty -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
<exclusions>
<exclusion>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>javax-websocket-server-impl</artifactId>
</exclusion>
<exclusion>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>websocket-server</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<exclusions>
<exclusion>
<artifactId>c3p0</artifactId>
<groupId>c3p0</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<exclusions>
<exclusion>
<groupId>com.mchange</groupId>
<artifactId>c3p0</artifactId>
</exclusion>
<exclusion>
<groupId>com.mchange</groupId>
<artifactId>mchange-commons-java</artifactId>
</exclusion>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP-java6</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz-jobs</artifactId>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz-jobs</artifactId>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-service</artifactId>
</dependency>
<dependency>
<groupId>io.swagger</groupId>
<artifactId>swagger-models</artifactId>
</dependency>
<dependency>
<groupId>com.github.xiaoymin</groupId>
<artifactId>swagger-bootstrap-ui</artifactId>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-service</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</dependency>
<dependency>
<groupId>com.github.xiaoymin</groupId>
<artifactId>swagger-bootstrap-ui</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</dependency>
<!-- hadoop -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.curator</groupId>
<artifactId>curator-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- hadoop -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.curator</groupId>
<artifactId>curator-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<exclusions>
<exclusion>
<artifactId>servlet-api</artifactId>
<groupId>javax.servlet</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jsp-2.1</artifactId>
<exclusions>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>servlet-api-2.5</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
</dependency>
<!-- just for test -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.ow2.asm</groupId>
<artifactId>asm</artifactId>
</exclusion>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot</artifactId>
</exclusion>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jsp-2.1</artifactId>
<exclusions>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>servlet-api-2.5</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<!-- just for test -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.ow2.asm</groupId>
<artifactId>asm</artifactId>
</exclusion>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot</artifactId>
</exclusion>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>

156
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/configuration/AppConfiguration.java
Unescape View File

@ -14,6 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.configuration;
import org.apache.dolphinscheduler.api.interceptor.LoginHandlerInterceptor;
@ -33,85 +34,78 @@ import java.util.Locale;
@Configuration
public class AppConfiguration implements WebMvcConfigurer {
public static final String LOGIN_INTERCEPTOR_PATH_PATTERN = "/**/*";
public static final String LOGIN_PATH_PATTERN = "/login";
public static final String REGISTER_PATH_PATTERN = "/users/register";
public static final String PATH_PATTERN = "/**";
public static final String LOCALE_LANGUAGE_COOKIE = "language";
public static final int COOKIE_MAX_AGE = 3600;
@Bean
public LoginHandlerInterceptor loginInterceptor() {
return new LoginHandlerInterceptor();
}
/**
* Cookie
* @return local resolver
*/
@Bean(name = "localeResolver")
public LocaleResolver localeResolver() {
CookieLocaleResolver localeResolver = new CookieLocaleResolver();
localeResolver.setCookieName(LOCALE_LANGUAGE_COOKIE);
/** set default locale **/
localeResolver.setDefaultLocale(Locale.US);
/** set cookie max age **/
localeResolver.setCookieMaxAge(COOKIE_MAX_AGE);
return localeResolver;
}
@Bean
public LocaleChangeInterceptor localeChangeInterceptor() {
LocaleChangeInterceptor lci = new LocaleChangeInterceptor();
/** **/
lci.setParamName("language");
return lci;
}
@Override
public void addInterceptors(InterceptorRegistry registry) {
//i18n
registry.addInterceptor(localeChangeInterceptor());
registry.addInterceptor(loginInterceptor()).addPathPatterns(LOGIN_INTERCEPTOR_PATH_PATTERN).excludePathPatterns(LOGIN_PATH_PATTERN, REGISTER_PATH_PATTERN, "/swagger-resources/**", "/webjars/**", "/v2/**", "/doc.html", "*.html", "/ui/**");
}
@Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/static/**").addResourceLocations("classpath:/static/");
registry.addResourceHandler("doc.html").addResourceLocations("classpath:/META-INF/resources/");
registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:/META-INF/resources/webjars/");
registry.addResourceHandler("/ui/**").addResourceLocations("file:ui/");
}
@Override
public void addViewControllers(ViewControllerRegistry registry) {
registry.addViewController("/ui/").setViewName("forward:/ui/index.html");
registry.addViewController("/").setViewName("forward:/ui/index.html");
}
@Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping(PATH_PATTERN).allowedOrigins("*").allowedMethods("*");
}
/**
* Turn off suffix-based content negotiation
*
* @param configurer configurer
*/
@Override
public void configureContentNegotiation(final ContentNegotiationConfigurer configurer) {
configurer.favorPathExtension(false);
}
public static final String LOGIN_INTERCEPTOR_PATH_PATTERN = "/**/*";
public static final String LOGIN_PATH_PATTERN = "/login";
public static final String REGISTER_PATH_PATTERN = "/users/register";
public static final String PATH_PATTERN = "/**";
public static final String LOCALE_LANGUAGE_COOKIE = "language";
public static final int COOKIE_MAX_AGE = 3600;
@Bean
public LoginHandlerInterceptor loginInterceptor() {
return new LoginHandlerInterceptor();
}
/**
* Cookie
* @return local resolver
*/
@Bean(name = "localeResolver")
public LocaleResolver localeResolver() {
CookieLocaleResolver localeResolver = new CookieLocaleResolver();
localeResolver.setCookieName(LOCALE_LANGUAGE_COOKIE);
// set default locale
localeResolver.setDefaultLocale(Locale.US);
// set cookie max age
localeResolver.setCookieMaxAge(COOKIE_MAX_AGE);
return localeResolver;
}
@Bean
public LocaleChangeInterceptor localeChangeInterceptor() {
LocaleChangeInterceptor lci = new LocaleChangeInterceptor();
lci.setParamName("language");
return lci;
}
@Override
public void addInterceptors(InterceptorRegistry registry) {
// i18n
registry.addInterceptor(localeChangeInterceptor());
registry.addInterceptor(loginInterceptor())
.addPathPatterns(LOGIN_INTERCEPTOR_PATH_PATTERN)
.excludePathPatterns(LOGIN_PATH_PATTERN, REGISTER_PATH_PATTERN,
"/swagger-resources/**", "/webjars/**", "/v2/**",
"/doc.html", "/swagger-ui.html", "*.html", "/ui/**");
}
@Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/static/**").addResourceLocations("classpath:/static/");
registry.addResourceHandler("doc.html").addResourceLocations("classpath:/META-INF/resources/");
registry.addResourceHandler("swagger-ui.html").addResourceLocations("classpath:/META-INF/resources/");
registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:/META-INF/resources/webjars/");
registry.addResourceHandler("/ui/**").addResourceLocations("file:ui/");
}
@Override
public void addViewControllers(ViewControllerRegistry registry) {
registry.addViewController("/ui/").setViewName("forward:/ui/index.html");
registry.addViewController("/").setViewName("forward:/ui/index.html");
}
@Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping(PATH_PATTERN).allowedOrigins("*").allowedMethods("*");
}
/**
* Turn off suffix-based content negotiation
*
* @param configurer configurer
*/
@Override
public void configureContentNegotiation(final ContentNegotiationConfigurer configurer) {
configurer.favorPathExtension(false);
}
}

6
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AccessTokenController.java
Unescape View File

@ -86,7 +86,7 @@ public class AccessTokenController extends BaseController {
logger.info("login user {}, create token , userId : {} , token expire time : {} , token : {}", loginUser.getUserName(),
userId, expireTime, token);
Map<String, Object> result = accessTokenService.createToken(userId, expireTime, token);
Map<String, Object> result = accessTokenService.createToken(loginUser, userId, expireTime, token);
return returnDataList(result);
}
@ -106,7 +106,7 @@ public class AccessTokenController extends BaseController {
@RequestParam(value = "userId") int userId,
@RequestParam(value = "expireTime") String expireTime) {
logger.info("login user {}, generate token , userId : {} , token expire time : {}", loginUser, userId, expireTime);
Map<String, Object> result = accessTokenService.generateToken(userId, expireTime);
Map<String, Object> result = accessTokenService.generateToken(loginUser, userId, expireTime);
return returnDataList(result);
}
@ -185,7 +185,7 @@ public class AccessTokenController extends BaseController {
logger.info("login user {}, update token , userId : {} , token expire time : {} , token : {}", loginUser.getUserName(),
userId, expireTime, token);
Map<String, Object> result = accessTokenService.updateToken(id, userId, expireTime, token);
Map<String, Object> result = accessTokenService.updateToken(loginUser, id, userId, expireTime, token);
return returnDataList(result);
}

2
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java
Unescape View File

@ -131,8 +131,8 @@ public class AlertGroupController extends BaseController {
@ResponseStatus(HttpStatus.OK)
@ApiException(LIST_PAGING_ALERT_GROUP_ERROR)
public Result listPaging(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam("pageNo") Integer pageNo,
@RequestParam(value = "searchVal", required = false) String searchVal,
@RequestParam("pageNo") Integer pageNo,
@RequestParam("pageSize") Integer pageSize) {
logger.info("login user {}, list paging, pageNo: {}, searchVal: {}, pageSize: {}",
loginUser.getUserName(), pageNo, searchVal, pageSize);

4
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
Unescape View File

@ -142,7 +142,7 @@ public class ProcessDefinitionController extends BaseController {
@ApiOperation(value = "copyProcessDefinition", notes = "COPY_PROCESS_DEFINITION_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "processDefinitionIds", value = "PROCESS_DEFINITION_IDS", required = true, dataType = "String", example = "3,4"),
@ApiImplicitParam(name = "targetProjectId", value = "TARGET_PROJECT_ID", required = true, type = "Integer")
@ApiImplicitParam(name = "targetProjectId", value = "TARGET_PROJECT_ID", required = true, dataType = "Int", example = "10")
})
@PostMapping(value = "/copy")
@ResponseStatus(HttpStatus.OK)
@ -173,7 +173,7 @@ public class ProcessDefinitionController extends BaseController {
@ApiOperation(value = "moveProcessDefinition", notes = "MOVE_PROCESS_DEFINITION_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "processDefinitionIds", value = "PROCESS_DEFINITION_IDS", required = true, dataType = "String", example = "3,4"),
@ApiImplicitParam(name = "targetProjectId", value = "TARGET_PROJECT_ID", required = true, type = "Integer")
@ApiImplicitParam(name = "targetProjectId", value = "TARGET_PROJECT_ID", required = true, dataType = "Int", example = "10")
})
@PostMapping(value = "/move")
@ResponseStatus(HttpStatus.OK)

17
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProjectController.java
Unescape View File

@ -228,21 +228,20 @@ public class ProjectController extends BaseController {
}
/**
* query user created project
* query authorized and user created project
*
* @param loginUser login user
* @return projects which the user create
* @return projects which the user create and authorized
*/
@ApiOperation(value = "queryProjectCreatedByUser", notes = "QUERY_USER_CREATED_PROJECT_NOTES")
@GetMapping(value = "/login-user-created-project")
@ApiOperation(value = "queryProjectCreatedAndAuthorizedByUser", notes = "QUERY_AUTHORIZED_AND_USER_CREATED_PROJECT_NOTES")
@GetMapping(value = "/created-and-authorized-project")
@ResponseStatus(HttpStatus.OK)
@ApiException(QUERY_USER_CREATED_PROJECT_ERROR)
public Result queryProjectCreatedByUser(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser) {
logger.info("login user {}, query authorized project by user id: {}.",
@ApiException(QUERY_AUTHORIZED_AND_USER_CREATED_PROJECT_ERROR)
public Result queryProjectCreatedAndAuthorizedByUser(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser) {
logger.info("login user {}, query authorized and user created project by user id: {}.",
StringUtils.replaceNRTtoUnderline(loginUser.getUserName()),
StringUtils.replaceNRTtoUnderline(String.valueOf(loginUser.getId())));
Map<String, Object> result = projectService.queryProjectCreatedByUser(loginUser);
Map<String, Object> result = projectService.queryProjectCreatedAndAuthorizedByUser(loginUser);
return returnDataList(result);
}

244
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ResourcesController.java
Unescape View File

@ -14,23 +14,48 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.controller;
import static org.apache.dolphinscheduler.api.enums.Status.AUTHORIZED_FILE_RESOURCE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.AUTHORIZED_UDF_FUNCTION_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.AUTHORIZE_RESOURCE_TREE;
import static org.apache.dolphinscheduler.api.enums.Status.CREATE_RESOURCE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.CREATE_RESOURCE_FILE_ON_LINE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.CREATE_UDF_FUNCTION_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.DELETE_RESOURCE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.DELETE_UDF_FUNCTION_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.DOWNLOAD_RESOURCE_FILE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.EDIT_RESOURCE_FILE_ON_LINE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_DATASOURCE_BY_TYPE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_RESOURCES_LIST_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_RESOURCES_LIST_PAGING;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_UDF_FUNCTION_LIST_PAGING_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.RESOURCE_FILE_IS_EMPTY;
import static org.apache.dolphinscheduler.api.enums.Status.RESOURCE_NOT_EXIST;
import static org.apache.dolphinscheduler.api.enums.Status.UNAUTHORIZED_UDF_FUNCTION_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_RESOURCE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_UDF_FUNCTION_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.VERIFY_RESOURCE_BY_NAME_AND_TYPE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.VERIFY_UDF_FUNCTION_NAME_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.VIEW_RESOURCE_FILE_ON_LINE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.VIEW_UDF_FUNCTION_ERROR;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.exceptions.ApiException;
import org.apache.dolphinscheduler.api.service.ResourcesService;
import org.apache.dolphinscheduler.api.service.UdfFuncService;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.ProgramType;
import org.apache.dolphinscheduler.common.enums.ResourceType;
import org.apache.dolphinscheduler.common.enums.UdfType;
import org.apache.dolphinscheduler.common.utils.ParameterUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.dao.entity.User;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import org.apache.commons.lang.StringUtils;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
@ -38,13 +63,21 @@ import org.springframework.core.io.Resource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import springfox.documentation.annotations.ApiIgnore;
import java.util.Map;
import static org.apache.dolphinscheduler.api.enums.Status.*;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import springfox.documentation.annotations.ApiIgnore;
/**
* resources controller
@ -56,37 +89,28 @@ public class ResourcesController extends BaseController {
private static final Logger logger = LoggerFactory.getLogger(ResourcesController.class);
@Autowired
private ResourcesService resourceService;
@Autowired
private UdfFuncService udfFuncService;
/**
* create directory
*
* @param loginUser login user
* @param alias alias
* @param description description
* @param type type
* @return create result code
*/
/**
* @param loginUser login user
* @param type type
* @param alias alias
* @param description description
* @param pid parent id
* @param currentDir current directory
* @return
* @return create result code
*/
@ApiOperation(value = "createDirctory", notes = "CREATE_RESOURCE_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "type", value = "RESOURCE_TYPE", required = true, dataType = "ResourceType"),
@ApiImplicitParam(name = "name", value = "RESOURCE_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "description", value = "RESOURCE_DESC", dataType = "String"),
@ApiImplicitParam(name = "file", value = "RESOURCE_FILE", required = true, dataType = "MultipartFile")
@ApiImplicitParam(name = "pid", value = "RESOURCE_PID", required = true, dataType = "Int", example = "10"),
@ApiImplicitParam(name = "currentDir", value = "RESOURCE_CURRENTDIR", required = true, dataType = "String")
})
@PostMapping(value = "/directory/create")
@ApiException(CREATE_RESOURCE_ERROR)
@ -103,12 +127,13 @@ public class ResourcesController extends BaseController {
/**
* create resource
*
* @param loginUser login user
* @param alias alias
* @param description description
* @param type type
* @param file file
* @param loginUser
* @param type
* @param alias
* @param description
* @param file
* @param pid
* @param currentDir
* @return create result code
*/
@ApiOperation(value = "createResource", notes = "CREATE_RESOURCE_NOTES")
@ -116,7 +141,9 @@ public class ResourcesController extends BaseController {
@ApiImplicitParam(name = "type", value = "RESOURCE_TYPE", required = true, dataType = "ResourceType"),
@ApiImplicitParam(name = "name", value = "RESOURCE_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "description", value = "RESOURCE_DESC", dataType = "String"),
@ApiImplicitParam(name = "file", value = "RESOURCE_FILE", required = true, dataType = "MultipartFile")
@ApiImplicitParam(name = "file", value = "RESOURCE_FILE", required = true, dataType = "MultipartFile"),
@ApiImplicitParam(name = "pid", value = "RESOURCE_PID", required = true, dataType = "Int", example = "10"),
@ApiImplicitParam(name = "currentDir", value = "RESOURCE_CURRENTDIR", required = true, dataType = "String")
})
@PostMapping(value = "/create")
@ApiException(CREATE_RESOURCE_ERROR)
@ -135,11 +162,12 @@ public class ResourcesController extends BaseController {
/**
* update resource
*
* @param loginUser login user
* @param alias alias
* @param resourceId resource id
* @param type resource type
* @param loginUser login user
* @param alias alias
* @param resourceId resource id
* @param type resource type
* @param description description
* @param file resource file
* @return update result code
*/
@ApiOperation(value = "updateResource", notes = "UPDATE_RESOURCE_NOTES")
@ -147,7 +175,8 @@ public class ResourcesController extends BaseController {
@ApiImplicitParam(name = "id", value = "RESOURCE_ID", required = true, dataType = "Int", example = "100"),
@ApiImplicitParam(name = "type", value = "RESOURCE_TYPE", required = true, dataType = "ResourceType"),
@ApiImplicitParam(name = "name", value = "RESOURCE_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "description", value = "RESOURCE_DESC", dataType = "String")
@ApiImplicitParam(name = "description", value = "RESOURCE_DESC", dataType = "String"),
@ApiImplicitParam(name = "file", value = "RESOURCE_FILE", required = true, dataType = "MultipartFile")
})
@PostMapping(value = "/update")
@ApiException(UPDATE_RESOURCE_ERROR)
@ -155,17 +184,18 @@ public class ResourcesController extends BaseController {
@RequestParam(value = "id") int resourceId,
@RequestParam(value = "type") ResourceType type,
@RequestParam(value = "name") String alias,
@RequestParam(value = "description", required = false) String description) {
logger.info("login user {}, update resource, type: {}, resource alias: {}, desc: {}",
loginUser.getUserName(), type, alias, description);
return resourceService.updateResource(loginUser, resourceId, alias, description, type);
@RequestParam(value = "description", required = false) String description,
@RequestParam(value = "file" ,required = false) MultipartFile file) {
logger.info("login user {}, update resource, type: {}, resource alias: {}, desc: {}, file: {}",
loginUser.getUserName(), type, alias, description, file);
return resourceService.updateResource(loginUser, resourceId, alias, description, type, file);
}
/**
* query resources list
*
* @param loginUser login user
* @param type resource type
* @param type resource type
* @return resource list
*/
@ApiOperation(value = "queryResourceList", notes = "QUERY_RESOURCE_LIST_NOTES")
@ -187,16 +217,16 @@ public class ResourcesController extends BaseController {
* query resources list paging
*
* @param loginUser login user
* @param type resource type
* @param type resource type
* @param searchVal search value
* @param pageNo page number
* @param pageSize page size
* @param pageNo page number
* @param pageSize page size
* @return resource list page
*/
@ApiOperation(value = "queryResourceListPaging", notes = "QUERY_RESOURCE_LIST_PAGING_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "type", value = "RESOURCE_TYPE", required = true, dataType = "ResourceType"),
@ApiImplicitParam(name = "id", value = "RESOURCE_ID", required = true, dataType = "int"),
@ApiImplicitParam(name = "id", value = "RESOURCE_ID", required = true, dataType = "int", example = "10"),
@ApiImplicitParam(name = "searchVal", value = "SEARCH_VAL", dataType = "String"),
@ApiImplicitParam(name = "pageNo", value = "PAGE_NO", dataType = "Int", example = "1"),
@ApiImplicitParam(name = "pageSize", value = "PAGE_SIZE", dataType = "Int", example = "20")
@ -227,7 +257,7 @@ public class ResourcesController extends BaseController {
/**
* delete resource
*
* @param loginUser login user
* @param loginUser login user
* @param resourceId resource id
* @return delete result code
*/
@ -251,8 +281,8 @@ public class ResourcesController extends BaseController {
* verify resource by alias and type
*
* @param loginUser login user
* @param fullName resource full name
* @param type resource type
* @param fullName resource full name
* @param type resource type
* @return true if the resource name not exists, otherwise return false
*/
@ApiOperation(value = "verifyResourceName", notes = "VERIFY_RESOURCE_NAME_NOTES")
@ -277,10 +307,10 @@ public class ResourcesController extends BaseController {
* query resources jar list
*
* @param loginUser login user
* @param type resource type
* @param type resource type
* @return resource list
*/
@ApiOperation(value = "queryResourceJarList", notes = "QUERY_RESOURCE_LIST_NOTES")
@ApiOperation(value = "queryResourceByProgramType", notes = "QUERY_RESOURCE_LIST_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "type", value = "RESOURCE_TYPE", required = true, dataType = "ResourceType")
})
@ -288,10 +318,14 @@ public class ResourcesController extends BaseController {
@ResponseStatus(HttpStatus.OK)
@ApiException(QUERY_RESOURCES_LIST_ERROR)
public Result queryResourceJarList(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam(value = "type") ResourceType type
@RequestParam(value = "type") ResourceType type,
@RequestParam(value = "programType",required = false) ProgramType programType
) {
logger.info("query resource list, login user:{}, resource type:{}", loginUser.getUserName(), type.toString());
Map<String, Object> result = resourceService.queryResourceJarList(loginUser, type);
String programTypeName = programType == null ? "" : programType.name();
String userName = loginUser.getUserName();
userName = userName.replaceAll("[\n|\r|\t]", "_");
logger.info("query resource list, login user:{}, resource type:{}, program type:{}", userName,programTypeName);
Map<String, Object> result = resourceService.queryResourceByProgramType(loginUser, type,programType);
return returnDataList(result);
}
@ -299,14 +333,16 @@ public class ResourcesController extends BaseController {
* query resource by full name and type
*
* @param loginUser login user
* @param fullName resource full name
* @param type resource type
* @param fullName resource full name
* @param type resource type
* @param id resource id
* @return true if the resource name not exists, otherwise return false
*/
@ApiOperation(value = "queryResource", notes = "QUERY_BY_RESOURCE_NAME")
@ApiImplicitParams({
@ApiImplicitParam(name = "type", value = "RESOURCE_TYPE", required = true, dataType = "ResourceType"),
@ApiImplicitParam(name = "fullName", value = "RESOURCE_FULL_NAME", required = true, dataType = "String")
@ApiImplicitParam(name = "fullName", value = "RESOURCE_FULL_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "id", value = "RESOURCE_ID", required = false, dataType = "Int", example = "10")
})
@GetMapping(value = "/queryResource")
@ResponseStatus(HttpStatus.OK)
@ -325,10 +361,10 @@ public class ResourcesController extends BaseController {
/**
* view resource file online
*
* @param loginUser login user
* @param resourceId resource id
* @param loginUser login user
* @param resourceId resource id
* @param skipLineNum skip line number
* @param limit limit
* @param limit limit
* @return resource content
*/
@ApiOperation(value = "viewResource", notes = "VIEW_RESOURCE_BY_ID_NOTES")
@ -352,13 +388,14 @@ public class ResourcesController extends BaseController {
/**
* create resource file online
*
* @param loginUser login user
* @param type resource type
* @param fileName file name
* @param fileSuffix file suffix
* @param description description
* @param content content
* @param loginUser
* @param type
* @param fileName
* @param fileSuffix
* @param description
* @param content
* @param pid
* @param currentDir
* @return create result code
*/
@ApiOperation(value = "onlineCreateResource", notes = "ONLINE_CREATE_RESOURCE_NOTES")
@ -367,7 +404,9 @@ public class ResourcesController extends BaseController {
@ApiImplicitParam(name = "fileName", value = "RESOURCE_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "suffix", value = "SUFFIX", required = true, dataType = "String"),
@ApiImplicitParam(name = "description", value = "RESOURCE_DESC", dataType = "String"),
@ApiImplicitParam(name = "content", value = "CONTENT", required = true, dataType = "String")
@ApiImplicitParam(name = "content", value = "CONTENT", required = true, dataType = "String"),
@ApiImplicitParam(name = "pid", value = "RESOURCE_PID", required = true, dataType = "Int", example = "10"),
@ApiImplicitParam(name = "currentDir", value = "RESOURCE_CURRENTDIR", required = true, dataType = "String")
})
@PostMapping(value = "/online-create")
@ApiException(CREATE_RESOURCE_FILE_ON_LINE_ERROR)
@ -392,9 +431,9 @@ public class ResourcesController extends BaseController {
/**
* edit resource file online
*
* @param loginUser login user
* @param loginUser login user
* @param resourceId resource id
* @param content content
* @param content content
* @return update result code
*/
@ApiOperation(value = "updateResourceContent", notes = "UPDATE_RESOURCE_NOTES")
@ -420,7 +459,7 @@ public class ResourcesController extends BaseController {
/**
* download resource file
*
* @param loginUser login user
* @param loginUser login user
* @param resourceId resource id
* @return resource content
*/
@ -449,14 +488,14 @@ public class ResourcesController extends BaseController {
/**
* create udf function
*
* @param loginUser login user
* @param type udf type
* @param funcName function name
* @param argTypes argument types
* @param database database
* @param loginUser login user
* @param type udf type
* @param funcName function name
* @param argTypes argument types
* @param database database
* @param description description
* @param className class name
* @param resourceId resource id
* @param className class name
* @param resourceId resource id
* @return create result code
*/
@ApiOperation(value = "createUdfFunc", notes = "CREATE_UDF_FUNCTION_NOTES")
@ -490,12 +529,12 @@ public class ResourcesController extends BaseController {
* view udf function
*
* @param loginUser login user
* @param id resource id
* @param id resource id
* @return udf function detail
*/
@ApiOperation(value = "viewUIUdfFunction", notes = "VIEW_UDF_FUNCTION_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "resourceId", value = "RESOURCE_ID", required = true, dataType = "Int", example = "100")
@ApiImplicitParam(name = "id", value = "RESOURCE_ID", required = true, dataType = "Int", example = "100")
})
@GetMapping(value = "/udf-func/update-ui")
@ -512,26 +551,27 @@ public class ResourcesController extends BaseController {
/**
* update udf function
*
* @param loginUser login user
* @param type resource type
* @param funcName function name
* @param argTypes argument types
* @param database data base
* @param loginUser login user
* @param type resource type
* @param funcName function name
* @param argTypes argument types
* @param database data base
* @param description description
* @param resourceId resource id
* @param className class name
* @param udfFuncId udf function id
* @param resourceId resource id
* @param className class name
* @param udfFuncId udf function id
* @return update result code
*/
@ApiOperation(value = "updateUdfFunc", notes = "UPDATE_UDF_FUNCTION_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "id", value = "UDF_ID", required = true, dataType = "Int"),
@ApiImplicitParam(name = "type", value = "UDF_TYPE", required = true, dataType = "UdfType"),
@ApiImplicitParam(name = "funcName", value = "FUNC_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "suffix", value = "CLASS_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "className", value = "CLASS_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "argTypes", value = "ARG_TYPES", dataType = "String"),
@ApiImplicitParam(name = "database", value = "DATABASE_NAME", dataType = "String"),
@ApiImplicitParam(name = "description", value = "UDF_DESC", dataType = "String"),
@ApiImplicitParam(name = "id", value = "RESOURCE_ID", required = true, dataType = "Int", example = "100")
@ApiImplicitParam(name = "resourceId", value = "RESOURCE_ID", required = true, dataType = "Int", example = "100")
})
@PostMapping(value = "/udf-func/update")
@ -556,8 +596,8 @@ public class ResourcesController extends BaseController {
*
* @param loginUser login user
* @param searchVal search value
* @param pageNo page number
* @param pageSize page size
* @param pageNo page number
* @param pageSize page size
* @return udf function list page
*/
@ApiOperation(value = "queryUdfFuncListPaging", notes = "QUERY_UDF_FUNCTION_LIST_PAGING_NOTES")
@ -569,7 +609,7 @@ public class ResourcesController extends BaseController {
@GetMapping(value = "/udf-func/list-paging")
@ResponseStatus(HttpStatus.OK)
@ApiException(QUERY_UDF_FUNCTION_LIST_PAGING_ERROR)
public Result queryUdfFuncList(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
public Result<Object> queryUdfFuncListPaging(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam("pageNo") Integer pageNo,
@RequestParam(value = "searchVal", required = false) String searchVal,
@RequestParam("pageSize") Integer pageSize
@ -586,23 +626,25 @@ public class ResourcesController extends BaseController {
}
/**
* query resource list by type
* query udf func list by type
*
* @param loginUser login user
* @param type resource type
* @param type resource type
* @return resource list
*/
@ApiOperation(value = "queryResourceList", notes = "QUERY_RESOURCE_LIST_NOTES")
@ApiOperation(value = "queryUdfFuncList", notes = "QUERY_UDF_FUNC_LIST_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "type", value = "UDF_TYPE", required = true, dataType = "UdfType")
})
@GetMapping(value = "/udf-func/list")
@ResponseStatus(HttpStatus.OK)
@ApiException(QUERY_DATASOURCE_BY_TYPE_ERROR)
public Result queryResourceList(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
public Result<Object> queryUdfFuncList(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam("type") UdfType type) {
logger.info("query datasource list, user:{}, type:{}", loginUser.getUserName(), type);
Map<String, Object> result = udfFuncService.queryResourceList(loginUser, type.ordinal());
String userName = loginUser.getUserName();
userName = userName.replaceAll("[\n|\r|\t]", "_");
logger.info("query udf func list, user:{}, type:{}", userName, type);
Map<String, Object> result = udfFuncService.queryUdfFuncList(loginUser, type.ordinal());
return returnDataList(result);
}
@ -610,7 +652,7 @@ public class ResourcesController extends BaseController {
* verify udf function name can use or not
*
* @param loginUser login user
* @param name name
* @param name name
* @return true if the name can user, otherwise return false
*/
@ApiOperation(value = "verifyUdfFuncName", notes = "VERIFY_UDF_FUNCTION_NAME_NOTES")
@ -655,7 +697,7 @@ public class ResourcesController extends BaseController {
* authorized file resource list
*
* @param loginUser login user
* @param userId user id
* @param userId user id
* @return authorized result
*/
@ApiOperation(value = "authorizedFile", notes = "AUTHORIZED_FILE_NOTES")
@ -677,7 +719,7 @@ public class ResourcesController extends BaseController {
* unauthorized file resource list
*
* @param loginUser login user
* @param userId user id
* @param userId user id
* @return unauthorized result code
*/
@ApiOperation(value = "authorizeResourceTree", notes = "AUTHORIZE_RESOURCE_TREE_NOTES")
@ -699,7 +741,7 @@ public class ResourcesController extends BaseController {
* unauthorized udf function
*
* @param loginUser login user
* @param userId user id
* @param userId user id
* @return unauthorized result code
*/
@ApiOperation(value = "unauthUDFFunc", notes = "UNAUTHORIZED_UDF_FUNC_NOTES")
@ -722,7 +764,7 @@ public class ResourcesController extends BaseController {
* authorized udf function
*
* @param loginUser login user
* @param userId user id
* @param userId user id
* @return authorized result code
*/
@ApiOperation(value = "authUDFFunc", notes = "AUTHORIZED_UDF_FUNC_NOTES")

131
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java
Unescape View File

@ -14,8 +14,18 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.controller;
import static org.apache.dolphinscheduler.api.enums.Status.CREATE_SCHEDULE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.DELETE_SCHEDULE_CRON_BY_ID_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.OFFLINE_SCHEDULE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.PREVIEW_SCHEDULE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.PUBLISH_SCHEDULE_ONLINE_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_SCHEDULE_LIST_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_SCHEDULE_LIST_PAGING_ERROR;
import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_SCHEDULE_ERROR;
import static org.apache.dolphinscheduler.common.Constants.SESSION_USER;
import org.apache.dolphinscheduler.api.exceptions.ApiException;
import org.apache.dolphinscheduler.api.service.SchedulerService;
@ -26,19 +36,28 @@ import org.apache.dolphinscheduler.common.enums.ReleaseState;
import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.common.utils.ParameterUtils;
import org.apache.dolphinscheduler.dao.entity.User;
import io.swagger.annotations.*;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.*;
import springfox.documentation.annotations.ApiIgnore;
import java.io.IOException;
import java.util.Map;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;
import static org.apache.dolphinscheduler.api.enums.Status.*;
import static org.apache.dolphinscheduler.common.Constants.SESSION_USER;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import io.swagger.annotations.ApiParam;
import springfox.documentation.annotations.ApiIgnore;
/**
* schedule controller
@ -61,17 +80,15 @@ public class SchedulerController extends BaseController {
/**
* create schedule
*
* @param loginUser login user
* @param projectName project name
* @param processDefinitionId process definition id
* @param schedule scheduler
* @param warningType warning type
* @param warningGroupId warning group id
* @param failureStrategy failure strategy
* @param loginUser login user
* @param projectName project name
* @param processDefinitionId process definition id
* @param schedule scheduler
* @param warningType warning type
* @param warningGroupId warning group id
* @param failureStrategy failure strategy
* @param processInstancePriority process instance priority
* @param receivers receivers
* @param receiversCc receivers cc
* @param workerGroup worker group
* @param workerGroup worker group
* @return create result code
*/
@ApiOperation(value = "createSchedule", notes = "CREATE_SCHEDULE_NOTES")
@ -81,8 +98,6 @@ public class SchedulerController extends BaseController {
@ApiImplicitParam(name = "warningType", value = "WARNING_TYPE", type = "WarningType"),
@ApiImplicitParam(name = "warningGroupId", value = "WARNING_GROUP_ID", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "failureStrategy", value = "FAILURE_STRATEGY", type = "FailureStrategy"),
@ApiImplicitParam(name = "receivers", value = "RECEIVERS", type = "String"),
@ApiImplicitParam(name = "receiversCc", value = "RECEIVERS_CC", type = "String"),
@ApiImplicitParam(name = "workerGroupId", value = "WORKER_GROUP_ID", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "processInstancePriority", value = "PROCESS_INSTANCE_PRIORITY", type = "Priority"),
})
@ -96,16 +111,14 @@ public class SchedulerController extends BaseController {
@RequestParam(value = "warningType", required = false, defaultValue = DEFAULT_WARNING_TYPE) WarningType warningType,
@RequestParam(value = "warningGroupId", required = false, defaultValue = DEFAULT_NOTIFY_GROUP_ID) int warningGroupId,
@RequestParam(value = "failureStrategy", required = false, defaultValue = DEFAULT_FAILURE_POLICY) FailureStrategy failureStrategy,
@RequestParam(value = "receivers", required = false) String receivers,
@RequestParam(value = "receiversCc", required = false) String receiversCc,
@RequestParam(value = "workerGroup", required = false, defaultValue = "default") String workerGroup,
@RequestParam(value = "processInstancePriority", required = false, defaultValue = DEFAULT_PROCESS_INSTANCE_PRIORITY) Priority processInstancePriority) throws IOException {
logger.info("login user {}, project name: {}, process name: {}, create schedule: {}, warning type: {}, warning group id: {}," +
"failure policy: {},receivers : {},receiversCc : {},processInstancePriority : {}, workGroupId:{}",
@RequestParam(value = "processInstancePriority", required = false, defaultValue = DEFAULT_PROCESS_INSTANCE_PRIORITY) Priority processInstancePriority) {
logger.info("login user {}, project name: {}, process name: {}, create schedule: {}, warning type: {}, warning group id: {},"
+ "failure policy: {},processInstancePriority : {}, workGroupId:{}",
loginUser.getUserName(), projectName, processDefinitionId, schedule, warningType, warningGroupId,
failureStrategy, receivers, receiversCc, processInstancePriority, workerGroup);
failureStrategy, processInstancePriority, workerGroup);
Map<String, Object> result = schedulerService.insertSchedule(loginUser, projectName, processDefinitionId, schedule,
warningType, warningGroupId, failureStrategy, receivers, receiversCc, processInstancePriority, workerGroup);
warningType, warningGroupId, failureStrategy, processInstancePriority, workerGroup);
return returnDataList(result);
}
@ -113,17 +126,15 @@ public class SchedulerController extends BaseController {
/**
* updateProcessInstance schedule
*
* @param loginUser login user
* @param projectName project name
* @param id scheduler id
* @param schedule scheduler
* @param warningType warning type
* @param warningGroupId warning group id
* @param failureStrategy failure strategy
* @param receivers receivers
* @param workerGroup worker group
* @param loginUser login user
* @param projectName project name
* @param id scheduler id
* @param schedule scheduler
* @param warningType warning type
* @param warningGroupId warning group id
* @param failureStrategy failure strategy
* @param workerGroup worker group
* @param processInstancePriority process instance priority
* @param receiversCc receivers cc
* @return update result code
*/
@ApiOperation(value = "updateSchedule", notes = "UPDATE_SCHEDULE_NOTES")
@ -133,8 +144,6 @@ public class SchedulerController extends BaseController {
@ApiImplicitParam(name = "warningType", value = "WARNING_TYPE", type = "WarningType"),
@ApiImplicitParam(name = "warningGroupId", value = "WARNING_GROUP_ID", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "failureStrategy", value = "FAILURE_STRATEGY", type = "FailureStrategy"),
@ApiImplicitParam(name = "receivers", value = "RECEIVERS", type = "String"),
@ApiImplicitParam(name = "receiversCc", value = "RECEIVERS_CC", type = "String"),
@ApiImplicitParam(name = "workerGroupId", value = "WORKER_GROUP_ID", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "processInstancePriority", value = "PROCESS_INSTANCE_PRIORITY", type = "Priority"),
})
@ -147,26 +156,24 @@ public class SchedulerController extends BaseController {
@RequestParam(value = "warningType", required = false, defaultValue = DEFAULT_WARNING_TYPE) WarningType warningType,
@RequestParam(value = "warningGroupId", required = false) int warningGroupId,
@RequestParam(value = "failureStrategy", required = false, defaultValue = "END") FailureStrategy failureStrategy,
@RequestParam(value = "receivers", required = false) String receivers,
@RequestParam(value = "receiversCc", required = false) String receiversCc,
@RequestParam(value = "workerGroup", required = false, defaultValue = "default") String workerGroup,
@RequestParam(value = "processInstancePriority", required = false) Priority processInstancePriority) throws IOException {
logger.info("login user {}, project name: {},id: {}, updateProcessInstance schedule: {}, notify type: {}, notify mails: {}, " +
"failure policy: {},receivers : {},receiversCc : {},processInstancePriority : {},workerGroupId:{}",
@RequestParam(value = "processInstancePriority", required = false) Priority processInstancePriority) {
logger.info("login user {}, project name: {},id: {}, updateProcessInstance schedule: {}, notify type: {}, notify mails: {}, "
+ "failure policy: {},processInstancePriority : {},workerGroupId:{}",
loginUser.getUserName(), projectName, id, schedule, warningType, warningGroupId, failureStrategy,
receivers, receiversCc, processInstancePriority, workerGroup);
processInstancePriority, workerGroup);
Map<String, Object> result = schedulerService.updateSchedule(loginUser, projectName, id, schedule,
warningType, warningGroupId, failureStrategy, receivers, receiversCc, null, processInstancePriority, workerGroup);
warningType, warningGroupId, failureStrategy, null, processInstancePriority, workerGroup);
return returnDataList(result);
}
/**
* publish schedule setScheduleState
*
* @param loginUser login user
* @param loginUser login user
* @param projectName project name
* @param id scheduler id
* @param id scheduler id
* @return publish result code
*/
@ApiOperation(value = "online", notes = "ONLINE_SCHEDULE_NOTES")
@ -176,7 +183,7 @@ public class SchedulerController extends BaseController {
@PostMapping("/online")
@ApiException(PUBLISH_SCHEDULE_ONLINE_ERROR)
public Result online(@ApiIgnore @RequestAttribute(value = SESSION_USER) User loginUser,
@ApiParam(name = "projectName", value = "PROJECT_NAME", required = true) @PathVariable("projectName") String projectName,
@ApiParam(name = "projectName", value = "PROJECT_NAME", required = true) @PathVariable String projectName,
@RequestParam("id") Integer id) {
logger.info("login user {}, schedule setScheduleState, project name: {}, id: {}",
loginUser.getUserName(), projectName, id);
@ -187,9 +194,9 @@ public class SchedulerController extends BaseController {
/**
* offline schedule
*
* @param loginUser login user
* @param loginUser login user
* @param projectName project name
* @param id schedule id
* @param id schedule id
* @return operation result code
*/
@ApiOperation(value = "offline", notes = "OFFLINE_SCHEDULE_NOTES")
@ -199,7 +206,7 @@ public class SchedulerController extends BaseController {
@PostMapping("/offline")
@ApiException(OFFLINE_SCHEDULE_ERROR)
public Result offline(@ApiIgnore @RequestAttribute(value = SESSION_USER) User loginUser,
@ApiParam(name = "projectName", value = "PROJECT_NAME", required = true) @PathVariable("projectName") String projectName,
@ApiParam(name = "projectName", value = "PROJECT_NAME", required = true) @PathVariable String projectName,
@RequestParam("id") Integer id) {
logger.info("login user {}, schedule offline, project name: {}, process definition id: {}",
loginUser.getUserName(), projectName, id);
@ -211,12 +218,12 @@ public class SchedulerController extends BaseController {
/**
* query schedule list paging
*
* @param loginUser login user
* @param projectName project name
* @param loginUser login user
* @param projectName project name
* @param processDefinitionId process definition id
* @param pageNo page number
* @param pageSize page size
* @param searchVal search value
* @param pageNo page number
* @param pageSize page size
* @param searchVal search value
* @return schedule list page
*/
@ApiOperation(value = "queryScheduleListPaging", notes = "QUERY_SCHEDULE_LIST_PAGING_NOTES")
@ -245,9 +252,9 @@ public class SchedulerController extends BaseController {
/**
* delete schedule by id
*
* @param loginUser login user
* @param loginUser login user
* @param projectName project name
* @param scheduleId scheule id
* @param scheduleId scheule id
* @return delete result code
*/
@ApiOperation(value = "deleteScheduleById", notes = "OFFLINE_SCHEDULE_NOTES")
@ -270,7 +277,7 @@ public class SchedulerController extends BaseController {
/**
* query schedule list
*
* @param loginUser login user
* @param loginUser login user
* @param projectName project name
* @return schedule list
*/
@ -288,9 +295,9 @@ public class SchedulerController extends BaseController {
/**
* preview schedule
*
* @param loginUser login user
* @param loginUser login user
* @param projectName project name
* @param schedule schedule expression
* @param schedule schedule expression
* @return the next five fire time
*/
@ApiOperation(value = "previewSchedule", notes = "PREVIEW_SCHEDULE_NOTES")

42
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TaskInstanceController.java
Unescape View File

@ -14,8 +14,10 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.controller;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_TASK_LIST_PAGING_ERROR;
import org.apache.dolphinscheduler.api.exceptions.ApiException;
import org.apache.dolphinscheduler.api.service.TaskInstanceService;
@ -23,18 +25,29 @@ import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.ExecutionStatus;
import org.apache.dolphinscheduler.common.utils.ParameterUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.dao.entity.User;
import io.swagger.annotations.*;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.*;
import springfox.documentation.annotations.ApiIgnore;
import java.util.Map;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_TASK_LIST_PAGING_ERROR;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import io.swagger.annotations.ApiParam;
import springfox.documentation.annotations.ApiIgnore;
/**
* task instance controller
@ -69,6 +82,7 @@ public class TaskInstanceController extends BaseController {
@ApiOperation(value = "queryTaskListPaging", notes = "QUERY_TASK_INSTANCE_LIST_PAGING_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "processInstanceId", value = "PROCESS_INSTANCE_ID", required = false, dataType = "Int", example = "100"),
@ApiImplicitParam(name = "processInstanceName", value = "PROCESS_INSTANCE_NAME", required = false, type = "String"),
@ApiImplicitParam(name = "searchVal", value = "SEARCH_VAL", type = "String"),
@ApiImplicitParam(name = "taskName", value = "TASK_NAME", type = "String"),
@ApiImplicitParam(name = "executorName", value = "EXECUTOR_NAME", type = "String"),
@ -85,6 +99,7 @@ public class TaskInstanceController extends BaseController {
public Result queryTaskListPaging(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@ApiParam(name = "projectName", value = "PROJECT_NAME", required = true) @PathVariable String projectName,
@RequestParam(value = "processInstanceId", required = false, defaultValue = "0") Integer processInstanceId,
@RequestParam(value = "processInstanceName", required = false) String processInstanceName,
@RequestParam(value = "searchVal", required = false) String searchVal,
@RequestParam(value = "taskName", required = false) String taskName,
@RequestParam(value = "executorName", required = false) String executorName,
@ -95,11 +110,20 @@ public class TaskInstanceController extends BaseController {
@RequestParam("pageNo") Integer pageNo,
@RequestParam("pageSize") Integer pageSize) {
logger.info("query task instance list, project name:{},process instance:{}, search value:{},task name:{}, executor name: {},state type:{}, host:{}, start:{}, end:{}",
projectName, processInstanceId, searchVal, taskName, executorName, stateType, host, startTime, endTime);
logger.info("query task instance list, projectName:{}, processInstanceId:{}, processInstanceName:{}, search value:{}, taskName:{}, executorName: {}, stateType:{}, host:{}, start:{}, end:{}",
StringUtils.replaceNRTtoUnderline(projectName),
processInstanceId,
StringUtils.replaceNRTtoUnderline(processInstanceName),
StringUtils.replaceNRTtoUnderline(searchVal),
StringUtils.replaceNRTtoUnderline(taskName),
StringUtils.replaceNRTtoUnderline(executorName),
stateType,
StringUtils.replaceNRTtoUnderline(host),
StringUtils.replaceNRTtoUnderline(startTime),
StringUtils.replaceNRTtoUnderline(endTime));
searchVal = ParameterUtils.handleEscapes(searchVal);
Map<String, Object> result = taskInstanceService.queryTaskListPaging(
loginUser, projectName, processInstanceId, taskName, executorName, startTime, endTime, searchVal, stateType, host, pageNo, pageSize);
loginUser, projectName, processInstanceId, processInstanceName, taskName, executorName, startTime, endTime, searchVal, stateType, host, pageNo, pageSize);
return returnDataListPaging(result);
}

25
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TenantController.java
Unescape View File

@ -29,6 +29,7 @@ import org.apache.dolphinscheduler.api.exceptions.ApiException;
import org.apache.dolphinscheduler.api.service.TenantService;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.common.utils.ParameterUtils;
import org.apache.dolphinscheduler.dao.entity.User;
@ -72,7 +73,6 @@ public class TenantController extends BaseController {
*
* @param loginUser login user
* @param tenantCode tenant code
* @param tenantName tenant name
* @param queueId queue id
* @param description description
* @return create result code
@ -80,7 +80,6 @@ public class TenantController extends BaseController {
@ApiOperation(value = "createTenant", notes = "CREATE_TENANT_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "tenantCode", value = "TENANT_CODE", required = true, dataType = "String"),
@ApiImplicitParam(name = "tenantName", value = "TENANT_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "queueId", value = "QUEUE_ID", required = true, dataType = "Int", example = "100"),
@ApiImplicitParam(name = "description", value = "TENANT_DESC", dataType = "String")
@ -90,12 +89,13 @@ public class TenantController extends BaseController {
@ApiException(CREATE_TENANT_ERROR)
public Result createTenant(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam(value = "tenantCode") String tenantCode,
@RequestParam(value = "tenantName") String tenantName,
@RequestParam(value = "queueId") int queueId,
@RequestParam(value = "description", required = false) String description) throws Exception {
logger.info("login user {}, create tenant, tenantCode: {}, tenantName: {}, queueId: {}, desc: {}",
loginUser.getUserName(), tenantCode, tenantName, queueId, description);
Map<String, Object> result = tenantService.createTenant(loginUser, tenantCode, tenantName, queueId, description);
String userReplace = StringUtils.replaceNRTtoUnderline(loginUser.getUserName());
String tenantCodeReplace = StringUtils.replaceNRTtoUnderline(tenantCode);
String descReplace = StringUtils.replaceNRTtoUnderline(description);
logger.info("login user {}, create tenant, tenantCode: {}, queueId: {}, desc: {}", userReplace, tenantCodeReplace, queueId, descReplace);
Map<String, Object> result = tenantService.createTenant(loginUser, tenantCode, queueId, description);
return returnDataList(result);
}
@ -119,8 +119,8 @@ public class TenantController extends BaseController {
@ResponseStatus(HttpStatus.OK)
@ApiException(QUERY_TENANT_LIST_PAGING_ERROR)
public Result queryTenantlistPaging(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam("pageNo") Integer pageNo,
@RequestParam(value = "searchVal", required = false) String searchVal,
@RequestParam("pageNo") Integer pageNo,
@RequestParam("pageSize") Integer pageSize) {
logger.info("login user {}, list paging, pageNo: {}, searchVal: {}, pageSize: {}",
loginUser.getUserName(), pageNo, searchVal, pageSize);
@ -157,7 +157,6 @@ public class TenantController extends BaseController {
* @param loginUser login user
* @param id tennat id
* @param tenantCode tennat code
* @param tenantName tennat name
* @param queueId queue id
* @param description description
* @return update result code
@ -166,7 +165,6 @@ public class TenantController extends BaseController {
@ApiImplicitParams({
@ApiImplicitParam(name = "ID", value = "TENANT_ID", required = true, dataType = "Int", example = "100"),
@ApiImplicitParam(name = "tenantCode", value = "TENANT_CODE", required = true, dataType = "String"),
@ApiImplicitParam(name = "tenantName", value = "TENANT_NAME", required = true, dataType = "String"),
@ApiImplicitParam(name = "queueId", value = "QUEUE_ID", required = true, dataType = "Int", example = "100"),
@ApiImplicitParam(name = "description", value = "TENANT_DESC", type = "String")
@ -177,12 +175,13 @@ public class TenantController extends BaseController {
public Result updateTenant(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam(value = "id") int id,
@RequestParam(value = "tenantCode") String tenantCode,
@RequestParam(value = "tenantName") String tenantName,
@RequestParam(value = "queueId") int queueId,
@RequestParam(value = "description", required = false) String description) throws Exception {
logger.info("login user {}, updateProcessInstance tenant, tenantCode: {}, tenantName: {}, queueId: {}, description: {}",
loginUser.getUserName(), tenantCode, tenantName, queueId, description);
Map<String, Object> result = tenantService.updateTenant(loginUser, id, tenantCode, tenantName, queueId, description);
String userReplace = StringUtils.replaceNRTtoUnderline(loginUser.getUserName());
String tenantCodeReplace = StringUtils.replaceNRTtoUnderline(tenantCode);
String descReplace = StringUtils.replaceNRTtoUnderline(description);
logger.info("login user {}, create tenant, tenantCode: {}, queueId: {}, desc: {}", userReplace, tenantCodeReplace, queueId, descReplace);
Map<String, Object> result = tenantService.updateTenant(loginUser, id, tenantCode, queueId, description);
return returnDataList(result);
}

22
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UsersController.java
Unescape View File

@ -72,9 +72,9 @@ public class UsersController extends BaseController {
@ApiImplicitParam(name = "userName", value = "USER_NAME", type = "String"),
@ApiImplicitParam(name = "userPassword", value = "USER_PASSWORD", type = "String"),
@ApiImplicitParam(name = "tenantId", value = "TENANT_ID", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "queue", value = "QUEUE", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "email", value = "EMAIL", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "phone", value = "PHONE", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "queue", value = "QUEUE", dataType = "String"),
@ApiImplicitParam(name = "email", value = "EMAIL", dataType = "String"),
@ApiImplicitParam(name = "phone", value = "PHONE", dataType = "String"),
@ApiImplicitParam(name = "state", value = "STATE", dataType = "Int", example = "1")
})
@PostMapping(value = "/create")
@ -105,8 +105,8 @@ public class UsersController extends BaseController {
*/
@ApiOperation(value = "queryUserList", notes = "QUERY_USER_LIST_NOTES")
@ApiImplicitParams({
@ApiImplicitParam(name = "pageNo", value = "PAGE_NO", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "pageSize", value = "PAGE_SIZE", type = "String"),
@ApiImplicitParam(name = "pageNo", value = "PAGE_NO", dataType = "Int", example = "1"),
@ApiImplicitParam(name = "pageSize", value = "PAGE_SIZE", dataType = "Int", example = "10"),
@ApiImplicitParam(name = "searchVal", value = "SEARCH_VAL", type = "String")
})
@GetMapping(value = "/list-paging")
@ -114,8 +114,8 @@ public class UsersController extends BaseController {
@ApiException(QUERY_USER_LIST_PAGING_ERROR)
public Result queryUserList(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@RequestParam("pageNo") Integer pageNo,
@RequestParam(value = "searchVal", required = false) String searchVal,
@RequestParam("pageSize") Integer pageSize) {
@RequestParam("pageSize") Integer pageSize,
@RequestParam(value = "searchVal", required = false) String searchVal) {
logger.info("login user {}, list user paging, pageNo: {}, searchVal: {}, pageSize: {}",
loginUser.getUserName(), pageNo, searchVal, pageSize);
Map<String, Object> result = checkPageParams(pageNo, pageSize);
@ -147,9 +147,9 @@ public class UsersController extends BaseController {
@ApiImplicitParam(name = "userName", value = "USER_NAME", type = "String"),
@ApiImplicitParam(name = "userPassword", value = "USER_PASSWORD", type = "String"),
@ApiImplicitParam(name = "tenantId", value = "TENANT_ID", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "queue", value = "QUEUE", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "email", value = "EMAIL", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "phone", value = "PHONE", dataType = "Int", example = "100"),
@ApiImplicitParam(name = "queue", value = "QUEUE", dataType = "String"),
@ApiImplicitParam(name = "email", value = "EMAIL", dataType = "String"),
@ApiImplicitParam(name = "phone", value = "PHONE", dataType = "String"),
@ApiImplicitParam(name = "state", value = "STATE", dataType = "Int", example = "1")
})
@PostMapping(value = "/update")
@ -166,7 +166,7 @@ public class UsersController extends BaseController {
@RequestParam(value = "state", required = false) int state) throws Exception {
logger.info("login user {}, updateProcessInstance user, userName: {}, email: {}, tenantId: {}, userPassword: {}, phone: {}, user queue: {}, state: {}",
loginUser.getUserName(), userName, email, tenantId, Constants.PASSWORD_DEFAULT, phone, queue, state);
Map<String, Object> result = usersService.updateUser(id, userName, userPassword, email, tenantId, phone, queue, state);
Map<String, Object> result = usersService.updateUser(loginUser, id, userName, userPassword, email, tenantId, phone, queue, state);
return returnDataList(result);
}

14
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/WorkFlowLineageController.java
Unescape View File

@ -16,11 +16,14 @@
*/
package org.apache.dolphinscheduler.api.controller;
import io.swagger.annotations.ApiParam;
import org.apache.dolphinscheduler.api.service.WorkFlowLineageService;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.utils.ParameterUtils;
import io.swagger.annotations.ApiParam;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.entity.WorkFlowLineage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
@ -34,6 +37,7 @@ import java.util.Map;
import java.util.Set;
import static org.apache.dolphinscheduler.api.enums.Status.QUERY_WORKFLOW_LINEAGE_ERROR;
import static org.apache.dolphinscheduler.common.Constants.SESSION_USER;
@RestController
@RequestMapping("lineages/{projectId}")
@ -45,7 +49,9 @@ public class WorkFlowLineageController extends BaseController {
@GetMapping(value="/list-name")
@ResponseStatus(HttpStatus.OK)
public Result<List<WorkFlowLineage>> queryWorkFlowLineageByName(@ApiIgnore @RequestParam(value = "searchVal", required = false) String searchVal, @ApiParam(name = "projectId", value = "PROJECT_ID", required = true) @PathVariable int projectId) {
public Result<List<WorkFlowLineage>> queryWorkFlowLineageByName(@ApiIgnore @RequestAttribute(value = SESSION_USER) User loginUser,
@ApiParam(name = "projectId", value = "PROJECT_ID", required = true, example = "1") @PathVariable int projectId,
@ApiIgnore @RequestParam(value = "searchVal", required = false) String searchVal) {
try {
searchVal = ParameterUtils.handleEscapes(searchVal);
Map<String, Object> result = workFlowLineageService.queryWorkFlowLineageByName(searchVal,projectId);
@ -58,7 +64,9 @@ public class WorkFlowLineageController extends BaseController {
@GetMapping(value="/list-ids")
@ResponseStatus(HttpStatus.OK)
public Result<Map<String, Object>> queryWorkFlowLineageByIds(@ApiIgnore @RequestParam(value = "ids", required = false) String ids,@ApiParam(name = "projectId", value = "PROJECT_ID", required = true) @PathVariable int projectId) {
public Result<Map<String, Object>> queryWorkFlowLineageByIds(@ApiIgnore @RequestAttribute(value = SESSION_USER) User loginUser,
@ApiParam(name = "projectId", value = "PROJECT_ID", required = true, example = "1") @PathVariable int projectId,
@ApiIgnore @RequestParam(value = "ids", required = false) String ids) {
try {
ids = ParameterUtils.handleEscapes(ids);

28
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/CommandStateCount.java
Unescape View File

@ -57,4 +57,32 @@ public class CommandStateCount {
public void setCommandState(CommandType commandState) {
this.commandState = commandState;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
CommandStateCount that = (CommandStateCount) o;
if (errorCount != that.errorCount) {
return false;
}
if (normalCount != that.normalCount) {
return false;
}
return commandState == that.commandState;
}
@Override
public int hashCode() {
int result = errorCount;
result = 31 * result + normalCount;
result = 31 * result + (commandState != null ? commandState.hashCode() : 0);
return result;
}
}

81
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/TaskCountDto.java
Unescape View File

@ -19,8 +19,10 @@ package org.apache.dolphinscheduler.api.dto;
import org.apache.dolphinscheduler.common.enums.ExecutionStatus;
import org.apache.dolphinscheduler.dao.entity.ExecuteStatusCount;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
/**
* task count dto
@ -43,77 +45,16 @@ public class TaskCountDto {
}
private void countTaskDtos(List<ExecuteStatusCount> taskInstanceStateCounts) {
int submittedSuccess = 0;
int runningExecution = 0;
int delayExecution = 0;
int readyPause = 0;
int pause = 0;
int readyStop = 0;
int stop = 0;
int failure = 0;
int success = 0;
int needFaultTolerance = 0;
int kill = 0;
int waittingThread = 0;
Map<ExecutionStatus, Integer> statusCountMap = taskInstanceStateCounts.stream()
.collect(Collectors.toMap(ExecuteStatusCount::getExecutionStatus, ExecuteStatusCount::getCount, Integer::sum));
for (ExecuteStatusCount taskInstanceStateCount : taskInstanceStateCounts) {
ExecutionStatus status = taskInstanceStateCount.getExecutionStatus();
totalCount += taskInstanceStateCount.getCount();
switch (status) {
case SUBMITTED_SUCCESS:
submittedSuccess += taskInstanceStateCount.getCount();
break;
case RUNNING_EXECUTION:
runningExecution += taskInstanceStateCount.getCount();
break;
case DELAY_EXECUTION:
delayExecution += taskInstanceStateCount.getCount();
break;
case READY_PAUSE:
readyPause += taskInstanceStateCount.getCount();
break;
case PAUSE:
pause += taskInstanceStateCount.getCount();
break;
case READY_STOP:
readyStop += taskInstanceStateCount.getCount();
break;
case STOP:
stop += taskInstanceStateCount.getCount();
break;
case FAILURE:
failure += taskInstanceStateCount.getCount();
break;
case SUCCESS:
success += taskInstanceStateCount.getCount();
break;
case NEED_FAULT_TOLERANCE:
needFaultTolerance += taskInstanceStateCount.getCount();
break;
case KILL:
kill += taskInstanceStateCount.getCount();
break;
case WAITTING_THREAD:
waittingThread += taskInstanceStateCount.getCount();
break;
taskCountDtos = Arrays.stream(ExecutionStatus.values())
.map(status -> new TaskStateCount(status, statusCountMap.getOrDefault(status, 0)))
.collect(Collectors.toList());
default:
break;
}
}
this.taskCountDtos = new ArrayList<>();
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.SUBMITTED_SUCCESS, submittedSuccess));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.RUNNING_EXECUTION, runningExecution));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.DELAY_EXECUTION, delayExecution));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.READY_PAUSE, readyPause));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.PAUSE, pause));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.READY_STOP, readyStop));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.STOP, stop));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.FAILURE, failure));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.SUCCESS, success));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.NEED_FAULT_TOLERANCE, needFaultTolerance));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.KILL, kill));
this.taskCountDtos.add(new TaskStateCount(ExecutionStatus.WAITTING_THREAD, waittingThread));
totalCount = taskCountDtos.stream()
.mapToInt(TaskStateCount::getCount)
.sum();
}
public List<TaskStateCount> getTaskCountDtos() {

24
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/TaskStateCount.java
Unescape View File

@ -47,4 +47,28 @@ public class TaskStateCount {
public void setTaskStateType(ExecutionStatus taskStateType) {
this.taskStateType = taskStateType;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
TaskStateCount that = (TaskStateCount) o;
if (count != that.count) {
return false;
}
return taskStateType == that.taskStateType;
}
@Override
public int hashCode() {
int result = count;
result = 31 * result + (taskStateType != null ? taskStateType.hashCode() : 0);
return result;
}
}

12
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
Unescape View File

@ -36,7 +36,7 @@ public enum Status {
USER_NAME_NULL(10004, "user name is null", "用户名不能为空"),
HDFS_OPERATION_ERROR(10006, "hdfs operation error", "hdfs操作错误"),
TASK_INSTANCE_NOT_FOUND(10008, "task instance not found", "任务实例不存在"),
TENANT_NAME_EXIST(10009, "tenant code {0} already exists", "租户编码[{0}]已存在"),
TENANT_CODE_EXIST(10009, "tenant code {0} already exists", "租户编码[{0}]已存在"),
USER_NOT_EXIST(10010, "user {0} not exists", "用户[{0}]不存在"),
ALERT_GROUP_NOT_EXIST(10011, "alarm group not found", "告警组不存在"),
ALERT_GROUP_EXIST(10012, "alarm group already exists", "告警组名称已存在"),
@ -133,7 +133,7 @@ public enum Status {
QUERY_TASK_INSTANCE_LOG_ERROR(10103, "view task instance log error", "查询任务实例日志错误"),
DOWNLOAD_TASK_INSTANCE_LOG_FILE_ERROR(10104, "download task instance log file error", "下载任务日志文件错误"),
CREATE_PROCESS_DEFINITION(10105, "create process definition", "创建工作流错误"),
VERIFY_PROCESS_DEFINITION_NAME_UNIQUE_ERROR(10106, "verify process definition name unique error", "工作流名称已存在"),
VERIFY_PROCESS_DEFINITION_NAME_UNIQUE_ERROR(10106, "verify process definition name unique error", "工作流定义名称已存在"),
UPDATE_PROCESS_DEFINITION_ERROR(10107, "update process definition error", "更新工作流定义错误"),
RELEASE_PROCESS_DEFINITION_ERROR(10108, "release process definition error", "上线工作流错误"),
QUERY_DATAIL_OF_PROCESS_DEFINITION_ERROR(10109, "query datail of process definition error", "查询工作流详细信息错误"),
@ -193,13 +193,13 @@ public enum Status {
BATCH_COPY_PROCESS_DEFINITION_ERROR(10159, "batch copy process definition error", "复制工作流错误"),
BATCH_MOVE_PROCESS_DEFINITION_ERROR(10160, "batch move process definition error", "移动工作流错误"),
QUERY_WORKFLOW_LINEAGE_ERROR(10161, "query workflow lineage error", "查询血缘失败"),
DELETE_PROCESS_DEFINITION_BY_ID_FAIL(10162, "delete process definition by id fail, for there are {0} process instances in executing using it", "删除工作流定义失败，有[{0}]个运行中的工作流实例正在使用"),
CHECK_TENANT_CODE_ERROR(10163, "Please enter the English tenant code", "请输入英文租户编码"),
CREATE_ALERT_PLUGIN_INSTANCE_ERROR(10164, "create alert plugin instance error", "创建告警组和告警组插件实例错误"),
QUERY_AUTHORIZED_AND_USER_CREATED_PROJECT_ERROR(10162, "query authorized and user created project error error", "查询授权的和用户创建的项目错误"),
DELETE_PROCESS_DEFINITION_BY_ID_FAIL(10163,"delete process definition by id fail, for there are {0} process instances in executing using it", "删除工作流定义失败，有[{0}]个运行中的工作流实例正在使用"),
CHECK_TENANT_CODE_ERROR(10164, "Please enter the English tenant code", "请输入英文租户编码"),
UPDATE_ALERT_PLUGIN_INSTANCE_ERROR(10165, "update alert plugin instance error", "更新告警组和告警组插件实例错误"),
DELETE_ALERT_PLUGIN_INSTANCE_ERROR(10166, "delete alert plugin instance error", "删除告警组和告警组插件实例错误"),
GET_ALERT_PLUGIN_INSTANCE_ERROR(10167, "get alert plugin instance error", "获取告警组和告警组插件实例错误"),
CREATE_ALERT_PLUGIN_INSTANCE_ERROR(10168, "create alert plugin instance error", "创建告警组和告警组插件实例错误"),
UDF_FUNCTION_NOT_EXIST(20001, "UDF function not found", "UDF函数不存在"),

2
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/AuthenticationType.java
Unescape View File

@ -14,6 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security;
import com.baomidou.mybatisplus.annotation.EnumValue;
@ -24,6 +25,7 @@ import com.baomidou.mybatisplus.annotation.EnumValue;
public enum AuthenticationType {
PASSWORD(0, "verify via user name and password"),
LDAP(1, "verify via LDAP server"),
;
AuthenticationType(int code, String desc) {

5
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/Authenticator.java
Unescape View File

@ -14,13 +14,16 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.dao.entity.User;
import javax.servlet.http.HttpServletRequest;
import java.util.Map;
import javax.servlet.http.HttpServletRequest;
public interface Authenticator {
/**
* Verifying legality via username and password

9
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/SecurityConfig.java
Unescape View File

@ -14,9 +14,13 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security;
import org.apache.commons.lang.StringUtils;
import org.apache.dolphinscheduler.api.security.impl.ldap.LdapAuthenticator;
import org.apache.dolphinscheduler.api.security.impl.pwd.PasswordAuthenticator;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
@ -58,6 +62,9 @@ public class SecurityConfig {
case PASSWORD:
authenticator = new PasswordAuthenticator();
break;
case LDAP:
authenticator = new LdapAuthenticator();
break;
default:
throw new IllegalStateException("Unexpected value: " + authenticationType);
}

35
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/PasswordAuthenticator.java → dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/AbstractAuthenticator.java
Unescape View File

@ -14,9 +14,11 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security;
package org.apache.dolphinscheduler.api.security.impl;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.security.Authenticator;
import org.apache.dolphinscheduler.api.service.SessionService;
import org.apache.dolphinscheduler.api.service.UsersService;
import org.apache.dolphinscheduler.api.utils.Result;
@ -24,26 +26,38 @@ import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.dao.entity.Session;
import org.apache.dolphinscheduler.dao.entity.User;
import java.util.Collections;
import java.util.Map;
import javax.servlet.http.HttpServletRequest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import javax.servlet.http.HttpServletRequest;
import java.util.Collections;
import java.util.Map;
public class PasswordAuthenticator implements Authenticator {
private static final Logger logger = LoggerFactory.getLogger(PasswordAuthenticator.class);
public abstract class AbstractAuthenticator implements Authenticator {
private static final Logger logger = LoggerFactory.getLogger(AbstractAuthenticator.class);
@Autowired
private UsersService userService;
@Autowired
private SessionService sessionService;
/**
* user login and return user in db
*
* @param userId user identity field
* @param password user login password
* @param extra extra user login field
* @return user object in databse
*/
public abstract User login(String userId, String password, String extra);
@Override
public Result<Map<String, String>> authenticate(String username, String password, String extra) {
public Result<Map<String, String>> authenticate(String userId, String password, String extra) {
Result<Map<String, String>> result = new Result<>();
// verify username and password
User user = userService.queryUser(username, password);
User user = login(userId, password, extra);
if (user == null) {
result.setCode(Status.USER_NAME_PASSWD_ERROR.getCode());
result.setMsg(Status.USER_NAME_PASSWD_ERROR.getMsg());
@ -64,7 +78,7 @@ public class PasswordAuthenticator implements Authenticator {
result.setMsg(Status.LOGIN_SESSION_FAILED.getMsg());
return result;
}
logger.info("sessionId : {}" , sessionId);
logger.info("sessionId : {}", sessionId);
result.setData(Collections.singletonMap(Constants.SESSION_ID, sessionId));
result.setCode(Status.SUCCESS.getCode());
result.setMsg(Status.LOGIN_SUCCESS.getMsg());
@ -81,4 +95,5 @@ public class PasswordAuthenticator implements Authenticator {
//get user object from session
return userService.queryUser(session.getUserId());
}
}

45
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticator.java
Unescape View File

@ -0,0 +1,45 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security.impl.ldap;
import org.apache.dolphinscheduler.api.security.impl.AbstractAuthenticator;
import org.apache.dolphinscheduler.api.service.UsersService;
import org.apache.dolphinscheduler.dao.entity.User;
import org.springframework.beans.factory.annotation.Autowired;
public class LdapAuthenticator extends AbstractAuthenticator {
@Autowired
private UsersService usersService;
@Autowired
LdapService ldapService;
@Override
public User login(String userId, String password, String extra) {
User user = null;
String ldapEmail = ldapService.ldapLogin(userId, password);
if (ldapEmail != null) {
//check if user exist
user = usersService.getUserByUserName(userId);
if (user == null) {
user = usersService.createUser(ldapService.getUserType(userId), userId, ldapEmail);
}
}
return user;
}
}

133
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapService.java
Unescape View File

@ -0,0 +1,133 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security.impl.ldap;
import org.apache.dolphinscheduler.common.enums.UserType;
import java.util.Properties;
import javax.naming.Context;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;
import javax.naming.directory.Attribute;
import javax.naming.directory.InitialDirContext;
import javax.naming.directory.SearchControls;
import javax.naming.directory.SearchResult;
import javax.naming.ldap.InitialLdapContext;
import javax.naming.ldap.LdapContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Component;
@Component
@Configuration
public class LdapService {
private static final Logger logger = LoggerFactory.getLogger(LdapService.class);
@Value("${security.authentication.ldap.user.admin:null}")
private String adminUserId;
@Value("${ldap.urls:null}")
private String ldapUrls;
@Value("${ldap.base.dn:null}")
private String ldapBaseDn;
@Value("${ldap.username:null}")
private String ldapSecurityPrincipal;
@Value("${ldap.password:null}")
private String ldapPrincipalPassword;
@Value("${ldap.user.identity.attribute:null}")
private String ldapUserIdentifyingAttribute;
@Value("${ldap.user.email.attribute:null}")
private String ldapEmailAttribute;
/***
* get user type by configured admin userId
* @param userId login userId
* @return user type
*/
public UserType getUserType(String userId) {
return adminUserId.equalsIgnoreCase(userId) ? UserType.ADMIN_USER : UserType.GENERAL_USER;
}
/**
* login by userId and return user email
*
* @param userId user identity id
* @param userPwd user login password
* @return user email
*/
public String ldapLogin(String userId, String userPwd) {
Properties searchEnv = getManagerLdapEnv();
try {
//Connect to the LDAP server and Authenticate with a service user of whom we know the DN and credentials
LdapContext ctx = new InitialLdapContext(searchEnv, null);
SearchControls sc = new SearchControls();
sc.setReturningAttributes(new String[]{ldapEmailAttribute});
sc.setSearchScope(SearchControls.SUBTREE_SCOPE);
String searchFilter = String.format("(%s=%s)", ldapUserIdentifyingAttribute, userId);
//Search for the user you want to authenticate, search him with some attribute
NamingEnumeration<SearchResult> results = ctx.search(ldapBaseDn, searchFilter, sc);
if (results.hasMore()) {
// get the users DN (distinguishedName) from the result
SearchResult result = results.next();
NamingEnumeration attrs = result.getAttributes().getAll();
while (attrs.hasMore()) {
//Open another connection to the LDAP server with the found DN and the password
searchEnv.put(Context.SECURITY_PRINCIPAL, result.getNameInNamespace());
searchEnv.put(Context.SECURITY_CREDENTIALS, userPwd);
try {
new InitialDirContext(searchEnv);
} catch (Exception e) {
logger.warn("invalid ldap credentials or ldap search error", e);
return null;
}
Attribute attr = (Attribute) attrs.next();
if (attr.getID().equals(ldapEmailAttribute)) {
return (String) attr.get();
}
}
}
} catch (NamingException e) {
logger.error("ldap search error", e);
return null;
}
return null;
}
/***
* get ldap env fot ldap server search
* @return Properties
*/
Properties getManagerLdapEnv() {
Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
env.put(Context.SECURITY_AUTHENTICATION, "simple");
env.put(Context.SECURITY_PRINCIPAL, ldapSecurityPrincipal);
env.put(Context.SECURITY_CREDENTIALS, ldapPrincipalPassword);
env.put(Context.PROVIDER_URL, ldapUrls);
return env;
}
}

34
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/pwd/PasswordAuthenticator.java
Unescape View File

@ -0,0 +1,34 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.security.impl.pwd;
import org.apache.dolphinscheduler.api.security.impl.AbstractAuthenticator;
import org.apache.dolphinscheduler.api.service.UsersService;
import org.apache.dolphinscheduler.dao.entity.User;
import org.springframework.beans.factory.annotation.Autowired;
public class PasswordAuthenticator extends AbstractAuthenticator {
@Autowired
private UsersService userService;
@Override
public User login(String userId, String password, String extra) {
return userService.queryUser(userId, password);
}
}

7
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AccessTokenService.java
Unescape View File

@ -44,7 +44,8 @@ public interface AccessTokenService {
* @param token token string
* @return create result code
*/
Map<String, Object> createToken(int userId, String expireTime, String token);
Map<String, Object> createToken(User loginUser, int userId, String expireTime, String token);
/**
* generate token
@ -53,7 +54,7 @@ public interface AccessTokenService {
* @param expireTime token expire time
* @return token string
*/
Map<String, Object> generateToken(int userId, String expireTime);
Map<String, Object> generateToken(User loginUser, int userId, String expireTime);
/**
* delete access token
@ -73,5 +74,5 @@ public interface AccessTokenService {
* @param token token string
* @return update result code
*/
Map<String, Object> updateToken(int id, int userId, String expireTime, String token);
Map<String, Object> updateToken(User loginUser, int id, int userId, String expireTime, String token);
}

19
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/BaseService.java
Unescape View File

@ -95,6 +95,25 @@ public class BaseService {
}
/**
* check
*
* @param result result
* @param bool bool
* @param userNoOperationPerm status
* @return check result
*/
protected boolean check(Map<String, Object> result, boolean bool, Status userNoOperationPerm) {
//only admin can operate
if (bool) {
result.put(Constants.STATUS, userNoOperationPerm);
result.put(Constants.MSG, userNoOperationPerm.getMsg());
return true;
}
return false;
}
/**
* get cookie info by name
*

30
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/DataSourceService.java
Unescape View File

@ -499,7 +499,13 @@ public class DataSourceService extends BaseService {
String address = buildAddress(type, host, port, connectType);
Map<String, Object> parameterMap = new LinkedHashMap<String, Object>(6);
String jdbcUrl = address + "/" + database;
String jdbcUrl;
if (DbType.SQLSERVER == type) {
jdbcUrl = address + ";databaseName=" + database;
} else {
jdbcUrl = address + "/" + database;
}
if (Constants.ORACLE.equals(type.name())) {
parameterMap.put(Constants.ORACLE_DB_CONNECT_TYPE, connectType);
}
@ -533,19 +539,17 @@ public class DataSourceService extends BaseService {
(type == DbType.HIVE || type == DbType.SPARK)) {
parameterMap.put(Constants.PRINCIPAL, principal);
}
if (other != null && !"".equals(other)) {
Map<String, String> map = JSONUtils.toMap(other);
if (map.size() > 0) {
StringBuilder otherSb = new StringBuilder();
for (Map.Entry<String, String> entry : map.entrySet()) {
otherSb.append(String.format("%s=%s%s", entry.getKey(), entry.getValue(), separator));
}
if (!Constants.DB2.equals(type.name())) {
otherSb.deleteCharAt(otherSb.length() - 1);
}
parameterMap.put(Constants.OTHER, otherSb);
}
Map<String, String> map = JSONUtils.toMap(other);
if (map != null) {
StringBuilder otherSb = new StringBuilder();
for (Map.Entry<String, String> entry: map.entrySet()) {
otherSb.append(String.format("%s=%s%s", entry.getKey(), entry.getValue(), separator));
}
if (!Constants.DB2.equals(type.name())) {
otherSb.deleteCharAt(otherSb.length() - 1);
}
parameterMap.put(Constants.OTHER, otherSb);
}
if (logger.isDebugEnabled()) {

171
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java
Unescape View File

@ -14,39 +14,62 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.service;
import static org.apache.dolphinscheduler.common.Constants.CMDPARAM_COMPLEMENT_DATA_END_DATE;
import static org.apache.dolphinscheduler.common.Constants.CMDPARAM_COMPLEMENT_DATA_START_DATE;
import static org.apache.dolphinscheduler.common.Constants.CMD_PARAM_RECOVER_PROCESS_ID_STRING;
import static org.apache.dolphinscheduler.common.Constants.CMD_PARAM_START_NODE_NAMES;
import static org.apache.dolphinscheduler.common.Constants.MAX_TASK_TIMEOUT;
import org.apache.dolphinscheduler.api.enums.ExecuteType;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.*;
import org.apache.dolphinscheduler.common.enums.CommandType;
import org.apache.dolphinscheduler.common.enums.ExecutionStatus;
import org.apache.dolphinscheduler.common.enums.FailureStrategy;
import org.apache.dolphinscheduler.common.enums.Priority;
import org.apache.dolphinscheduler.common.enums.ReleaseState;
import org.apache.dolphinscheduler.common.enums.RunMode;
import org.apache.dolphinscheduler.common.enums.TaskDependType;
import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.common.model.Server;
import org.apache.dolphinscheduler.common.utils.CollectionUtils;
import org.apache.dolphinscheduler.common.utils.DateUtils;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.dao.entity.*;
import org.apache.dolphinscheduler.dao.entity.Command;
import org.apache.dolphinscheduler.dao.entity.ProcessDefinition;
import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
import org.apache.dolphinscheduler.dao.entity.Project;
import org.apache.dolphinscheduler.dao.entity.Schedule;
import org.apache.dolphinscheduler.dao.entity.Tenant;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper;
import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapper;
import org.apache.dolphinscheduler.dao.mapper.ProjectMapper;
import org.apache.dolphinscheduler.service.process.ProcessService;
import org.apache.dolphinscheduler.service.quartz.cron.CronUtils;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.text.ParseException;
import java.util.*;
import static org.apache.dolphinscheduler.common.Constants.*;
/**
* executor service
*/
@Service
public class ExecutorService extends BaseService{
public class ExecutorService extends BaseService {
private static final Logger logger = LoggerFactory.getLogger(ExecutorService.class);
@ -73,20 +96,20 @@ public class ExecutorService extends BaseService{
/**
* execute process instance
*
* @param loginUser login user
* @param projectName project name
* @param processDefinitionId process Definition Id
* @param cronTime cron time
* @param commandType command type
* @param failureStrategy failuer strategy
* @param startNodeList start nodelist
* @param taskDependType node dependency type
* @param warningType warning type
* @param warningGroupId notify group id
* @param loginUser login user
* @param projectName project name
* @param processDefinitionId process Definition Id
* @param cronTime cron time
* @param commandType command type
* @param failureStrategy failuer strategy
* @param startNodeList start nodelist
* @param taskDependType node dependency type
* @param warningType warning type
* @param warningGroupId notify group id
* @param processInstancePriority process instance priority
* @param workerGroup worker group name
* @param runMode run mode
* @param timeout timeout
* @param timeout timeout
* @return execute process instance code
* @throws ParseException Parse Exception
*/
@ -99,23 +122,23 @@ public class ExecutorService extends BaseService{
Map<String, Object> result = new HashMap<>();
// timeout is invalid
if (timeout <= 0 || timeout > MAX_TASK_TIMEOUT) {
putMsg(result,Status.TASK_TIMEOUT_PARAMS_ERROR);
putMsg(result, Status.TASK_TIMEOUT_PARAMS_ERROR);
return result;
}
Project project = projectMapper.queryByName(projectName);
Map<String, Object> checkResultAndAuth = checkResultAndAuth(loginUser, projectName, project);
if (checkResultAndAuth != null){
if (checkResultAndAuth != null) {
return checkResultAndAuth;
}
// check process define release state
ProcessDefinition processDefinition = processDefinitionMapper.selectById(processDefinitionId);
result = checkProcessDefinitionValid(processDefinition, processDefinitionId);
if(result.get(Constants.STATUS) != Status.SUCCESS){
if (result.get(Constants.STATUS) != Status.SUCCESS) {
return result;
}
if (!checkTenantSuitable(processDefinition)){
if (!checkTenantSuitable(processDefinition)) {
logger.error("there is not any valid tenant for the process definition: id:{},name:{}, ",
processDefinition.getId(), processDefinition.getName());
putMsg(result, Status.TENANT_NOT_SUITABLE);
@ -127,14 +150,14 @@ public class ExecutorService extends BaseService{
return result;
}
/**
* create command
*/
int create = this.createCommand(commandType, processDefinitionId,
taskDependType, failureStrategy, startNodeList, cronTime, warningType, loginUser.getId(),
warningGroupId, runMode,processInstancePriority, workerGroup);
if(create > 0 ){
warningGroupId, runMode, processInstancePriority, workerGroup);
if (create > 0) {
processDefinition.setWarningGroupId(warningGroupId);
processDefinitionMapper.updateById(processDefinition);
putMsg(result, Status.SUCCESS);
@ -146,6 +169,7 @@ public class ExecutorService extends BaseService{
/**
* check whether master exists
*
* @param result result
* @return master exists return true , otherwise return false
*/
@ -161,7 +185,6 @@ public class ExecutorService extends BaseService{
return true;
}
/**
* check whether the process definition can be executed
*
@ -169,22 +192,20 @@ public class ExecutorService extends BaseService{
* @param processDefineId process definition id
* @return check result code
*/
public Map<String, Object> checkProcessDefinitionValid(ProcessDefinition processDefinition, int processDefineId){
public Map<String, Object> checkProcessDefinitionValid(ProcessDefinition processDefinition, int processDefineId) {
Map<String, Object> result = new HashMap<>();
if (processDefinition == null) {
// check process definition exists
putMsg(result, Status.PROCESS_DEFINE_NOT_EXIST,processDefineId);
putMsg(result, Status.PROCESS_DEFINE_NOT_EXIST, processDefineId);
} else if (processDefinition.getReleaseState() != ReleaseState.ONLINE) {
// check process definition online
putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE,processDefineId);
}else{
putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE, processDefineId);
} else {
result.put(Constants.STATUS, Status.SUCCESS);
}
return result;
}
/**
* do action to process instance：pause, stop, repeat, recover from pause, recover from stop
*
@ -208,7 +229,6 @@ public class ExecutorService extends BaseService{
return result;
}
ProcessInstance processInstance = processService.findProcessInstanceDetailById(processInstanceId);
if (processInstance == null) {
putMsg(result, Status.PROCESS_INSTANCE_NOT_EXIST, processInstanceId);
@ -216,7 +236,7 @@ public class ExecutorService extends BaseService{
}
ProcessDefinition processDefinition = processService.findProcessDefineById(processInstance.getProcessDefinitionId());
if(executeType != ExecuteType.STOP && executeType != ExecuteType.PAUSE){
if (executeType != ExecuteType.STOP && executeType != ExecuteType.PAUSE) {
result = checkProcessDefinitionValid(processDefinition, processInstance.getProcessDefinitionId());
if (result.get(Constants.STATUS) != Status.SUCCESS) {
return result;
@ -228,7 +248,7 @@ public class ExecutorService extends BaseService{
if (status != Status.SUCCESS) {
return checkResult;
}
if (!checkTenantSuitable(processDefinition)){
if (!checkTenantSuitable(processDefinition)) {
logger.error("there is not any valid tenant for the process definition: id:{},name:{}, ",
processDefinition.getId(), processDefinition.getName());
putMsg(result, Status.TENANT_NOT_SUITABLE);
@ -269,6 +289,7 @@ public class ExecutorService extends BaseService{
/**
* check tenant suitable
*
* @param processDefinition process definition
* @return true if tenant suitable, otherwise return false
*/
@ -309,7 +330,7 @@ public class ExecutorService extends BaseService{
}
break;
case RECOVER_SUSPENDED_PROCESS:
if (executionStatus.typeIsPause()|| executionStatus.typeIsCancel()) {
if (executionStatus.typeIsPause() || executionStatus.typeIsCancel()) {
checkResult = true;
}
break;
@ -317,7 +338,7 @@ public class ExecutorService extends BaseService{
break;
}
if (!checkResult) {
putMsg(result,Status.PROCESS_INSTANCE_STATE_OPERATION_ERROR, processInstance.getName(), executionStatus.toString(), executeType.toString());
putMsg(result, Status.PROCESS_INSTANCE_STATE_OPERATION_ERROR, processInstance.getName(), executionStatus.toString(), executeType.toString());
} else {
putMsg(result, Status.SUCCESS);
}
@ -325,7 +346,7 @@ public class ExecutorService extends BaseService{
}
/**
* prepare to update process instance command type and status
* prepare to update process instance command type and status
*
* @param processInstance process instance
* @param commandType command type
@ -364,11 +385,11 @@ public class ExecutorService extends BaseService{
command.setCommandType(commandType);
command.setProcessDefinitionId(processDefinitionId);
command.setCommandParam(String.format("{\"%s\":%d}",
CMDPARAM_RECOVER_PROCESS_ID_STRING, instanceId));
CMD_PARAM_RECOVER_PROCESS_ID_STRING, instanceId));
command.setExecutorId(loginUser.getId());
if(!processService.verifyIsNeedCreateCommand(command)){
putMsg(result, Status.PROCESS_INSTANCE_EXECUTING_COMMAND,processDefinitionId);
if (!processService.verifyIsNeedCreateCommand(command)) {
putMsg(result, Status.PROCESS_INSTANCE_EXECUTING_COMMAND, processDefinitionId);
return result;
}
@ -385,28 +406,29 @@ public class ExecutorService extends BaseService{
/**
* check if sub processes are offline before starting process definition
*
* @param processDefineId process definition id
* @return check result code
*/
public Map<String, Object> startCheckByProcessDefinedId(int processDefineId) {
Map<String, Object> result = new HashMap<>();
if (processDefineId == 0){
if (processDefineId == 0) {
logger.error("process definition id is null");
putMsg(result,Status.REQUEST_PARAMS_NOT_VALID_ERROR,"process definition id");
putMsg(result, Status.REQUEST_PARAMS_NOT_VALID_ERROR, "process definition id");
}
List<Integer> ids = new ArrayList<>();
processService.recurseFindSubProcessId(processDefineId, ids);
Integer[] idArray = ids.toArray(new Integer[ids.size()]);
if (!ids.isEmpty()){
if (!ids.isEmpty()) {
List<ProcessDefinition> processDefinitionList = processDefinitionMapper.queryDefinitionListByIdList(idArray);
if (processDefinitionList != null){
for (ProcessDefinition processDefinition : processDefinitionList){
if (processDefinitionList != null) {
for (ProcessDefinition processDefinition : processDefinitionList) {
/**
* if there is no online process, exit directly
*/
if (processDefinition.getReleaseState() != ReleaseState.ONLINE){
putMsg(result,Status.PROCESS_DEFINE_NOT_RELEASE, processDefinition.getName());
if (processDefinition.getReleaseState() != ReleaseState.ONLINE) {
putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE, processDefinition.getName());
logger.info("not release process definition id: {} , name : {}",
processDefinition.getId(), processDefinition.getName());
return result;
@ -420,6 +442,7 @@ public class ExecutorService extends BaseService{
/**
* create command
*
* @param commandType commandType
* @param processDefineId processDefineId
* @param nodeDep nodeDep
@ -433,37 +456,36 @@ public class ExecutorService extends BaseService{
* @param processInstancePriority processInstancePriority
* @param workerGroup workerGroup
* @return command id
* @throws ParseException
*/
private int createCommand(CommandType commandType, int processDefineId,
TaskDependType nodeDep, FailureStrategy failureStrategy,
String startNodeList, String schedule, WarningType warningType,
int executorId, int warningGroupId,
RunMode runMode,Priority processInstancePriority, String workerGroup) throws ParseException {
RunMode runMode, Priority processInstancePriority, String workerGroup) throws ParseException {
/**
* instantiate command schedule instance
*/
Command command = new Command();
Map<String,String> cmdParam = new HashMap<>();
if(commandType == null){
Map<String, String> cmdParam = new HashMap<>();
if (commandType == null) {
command.setCommandType(CommandType.START_PROCESS);
}else{
} else {
command.setCommandType(commandType);
}
command.setProcessDefinitionId(processDefineId);
if(nodeDep != null){
if (nodeDep != null) {
command.setTaskDependType(nodeDep);
}
if(failureStrategy != null){
if (failureStrategy != null) {
command.setFailureStrategy(failureStrategy);
}
if(StringUtils.isNotEmpty(startNodeList)){
cmdParam.put(CMDPARAM_START_NODE_NAMES, startNodeList);
if (StringUtils.isNotEmpty(startNodeList)) {
cmdParam.put(CMD_PARAM_START_NODE_NAMES, startNodeList);
}
if(warningType != null){
if (warningType != null) {
command.setWarningType(warningType);
}
command.setCommandParam(JSONUtils.toJsonString(cmdParam));
@ -474,32 +496,32 @@ public class ExecutorService extends BaseService{
Date start = null;
Date end = null;
if(StringUtils.isNotEmpty(schedule)){
if (StringUtils.isNotEmpty(schedule)) {
String[] interval = schedule.split(",");
if(interval.length == 2){
if (interval.length == 2) {
start = DateUtils.getScheduleDate(interval[0]);
end = DateUtils.getScheduleDate(interval[1]);
}
}
// determine whether to complement
if(commandType == CommandType.COMPLEMENT_DATA){
if (commandType == CommandType.COMPLEMENT_DATA) {
runMode = (runMode == null) ? RunMode.RUN_MODE_SERIAL : runMode;
if(null != start && null != end && !start.after(end)){
if(runMode == RunMode.RUN_MODE_SERIAL){
if (null != start && null != end && !start.after(end)) {
if (runMode == RunMode.RUN_MODE_SERIAL) {
cmdParam.put(CMDPARAM_COMPLEMENT_DATA_START_DATE, DateUtils.dateToString(start));
cmdParam.put(CMDPARAM_COMPLEMENT_DATA_END_DATE, DateUtils.dateToString(end));
command.setCommandParam(JSONUtils.toJsonString(cmdParam));
return processService.createCommand(command);
}else if (runMode == RunMode.RUN_MODE_PARALLEL){
} else if (runMode == RunMode.RUN_MODE_PARALLEL) {
List<Schedule> schedules = processService.queryReleaseSchedulerListByProcessDefinitionId(processDefineId);
List<Date> listDate = new LinkedList<>();
if(!CollectionUtils.isEmpty(schedules)){
if (!CollectionUtils.isEmpty(schedules)) {
for (Schedule item : schedules) {
listDate.addAll(CronUtils.getSelfFireDateList(start, end, item.getCrontab()));
}
}
if(!CollectionUtils.isEmpty(listDate)){
if (!CollectionUtils.isEmpty(listDate)) {
// loop by schedule date
for (Date date : listDate) {
cmdParam.put(CMDPARAM_COMPLEMENT_DATA_START_DATE, DateUtils.dateToString(date));
@ -508,10 +530,10 @@ public class ExecutorService extends BaseService{
processService.createCommand(command);
}
return listDate.size();
}else{
} else {
// loop by day
int runCunt = 0;
while(!start.after(end)) {
while (!start.after(end)) {
runCunt += 1;
cmdParam.put(CMDPARAM_COMPLEMENT_DATA_START_DATE, DateUtils.dateToString(start));
cmdParam.put(CMDPARAM_COMPLEMENT_DATA_END_DATE, DateUtils.dateToString(start));
@ -522,11 +544,11 @@ public class ExecutorService extends BaseService{
return runCunt;
}
}
}else{
} else {
logger.error("there is not valid schedule date for the process definition: id:{},date:{}",
processDefineId, schedule);
}
}else{
} else {
command.setCommandParam(JSONUtils.toJsonString(cmdParam));
return processService.createCommand(command);
}
@ -536,11 +558,6 @@ public class ExecutorService extends BaseService{
/**
* check result and auth
*
* @param loginUser
* @param projectName
* @param project
* @return
*/
private Map<String, Object> checkResultAndAuth(User loginUser, String projectName, Project project) {
// check project auth

4
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
Unescape View File

@ -304,10 +304,10 @@ public class ProcessInstanceService extends BaseService {
private void addDependResultForTaskList(List<TaskInstance> taskInstanceList) throws IOException {
for (TaskInstance taskInstance : taskInstanceList) {
if (taskInstance.getTaskType().equalsIgnoreCase(TaskType.DEPENDENT.toString())) {
Result logResult = loggerService.queryLog(
Result<String> logResult = loggerService.queryLog(
taskInstance.getId(), 0, 4098);
if (logResult.getCode() == Status.SUCCESS.ordinal()) {
String log = (String) logResult.getData();
String log = logResult.getData();
Map<String, DependResult> resultMap = parseLogForDependentResult(log);
taskInstance.setDependentResult(JSONUtils.toJsonString(resultMap));
}

7
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProjectService.java
Unescape View File

@ -120,4 +120,11 @@ public interface ProjectService {
*/
Map<String, Object> queryAllProjectList();
/**
* query authorized and user create project list by user id
* @param loginUser
* @return
*/
Map<String, Object> queryProjectCreatedAndAuthorizedByUser(User loginUser);
}

181
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ResourcesService.java
Unescape View File

@ -29,6 +29,7 @@ import org.apache.dolphinscheduler.api.exceptions.ServiceException;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.ProgramType;
import org.apache.dolphinscheduler.common.enums.ResourceType;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.dolphinscheduler.dao.entity.*;
@ -87,7 +88,7 @@ public class ResourcesService extends BaseService {
* @param currentDir current directory
* @return create directory result
*/
@Transactional(rollbackFor = RuntimeException.class)
@Transactional(rollbackFor = Exception.class)
public Result createDirectory(User loginUser,
String name,
String description,
@ -101,8 +102,11 @@ public class ResourcesService extends BaseService {
putMsg(result, Status.HDFS_NOT_STARTUP);
return result;
}
String fullName = "/".equals(currentDir) ? String.format("%s%s",currentDir,name):String.format("%s/%s",currentDir,name);
String fullName = currentDir.equals("/") ? String.format("%s%s",currentDir,name):String.format("%s/%s",currentDir,name);
result = verifyResourceName(fullName,type,loginUser);
if (!result.getCode().equals(Status.SUCCESS.getCode())) {
return result;
}
if (pid != -1) {
Resource parentResource = resourcesMapper.selectById(pid);
@ -165,7 +169,7 @@ public class ResourcesService extends BaseService {
* @param currentDir current directory
* @return create result code
*/
@Transactional(rollbackFor = RuntimeException.class)
@Transactional(rollbackFor = Exception.class)
public Result createResource(User loginUser,
String name,
String desc,
@ -230,7 +234,7 @@ public class ResourcesService extends BaseService {
}
// check resoure name exists
String fullName = "/".equals(currentDir) ? String.format("%s%s",currentDir,name):String.format("%s/%s",currentDir,name);
String fullName = currentDir.equals("/") ? String.format("%s%s",currentDir,name):String.format("%s/%s",currentDir,name);
if (checkResourceExists(fullName, 0, type.ordinal())) {
logger.error("resource {} has exist, can't recreate", name);
putMsg(result, Status.RESOURCE_EXIST);
@ -288,14 +292,16 @@ public class ResourcesService extends BaseService {
* @param name name
* @param desc description
* @param type resource type
* @param file resource file
* @return update result code
*/
@Transactional(rollbackFor = RuntimeException.class)
@Transactional(rollbackFor = Exception.class)
public Result updateResource(User loginUser,
int resourceId,
String name,
String desc,
ResourceType type) {
ResourceType type,
MultipartFile file) {
Result result = new Result();
// if resource upload startup
@ -315,7 +321,7 @@ public class ResourcesService extends BaseService {
return result;
}
if (name.equals(resource.getAlias()) && desc.equals(resource.getDescription())) {
if (file == null && name.equals(resource.getAlias()) && desc.equals(resource.getDescription())) {
putMsg(result, Status.SUCCESS);
return result;
}
@ -331,6 +337,42 @@ public class ResourcesService extends BaseService {
return result;
}
if (file != null) {
// file is empty
if (file.isEmpty()) {
logger.error("file is empty: {}", file.getOriginalFilename());
putMsg(result, Status.RESOURCE_FILE_IS_EMPTY);
return result;
}
// file suffix
String fileSuffix = FileUtils.suffix(file.getOriginalFilename());
String nameSuffix = FileUtils.suffix(name);
// determine file suffix
if (!(StringUtils.isNotEmpty(fileSuffix) && fileSuffix.equalsIgnoreCase(nameSuffix))) {
/**
* rename file suffix and original suffix must be consistent
*/
logger.error("rename file suffix and original suffix must be consistent: {}", file.getOriginalFilename());
putMsg(result, Status.RESOURCE_SUFFIX_FORBID_CHANGE);
return result;
}
//If resource type is UDF, only jar packages are allowed to be uploaded, and the suffix must be .jar
if (Constants.UDF.equals(type.name()) && !JAR.equalsIgnoreCase(FileUtils.suffix(originFullName))) {
logger.error(Status.UDF_RESOURCE_SUFFIX_NOT_JAR.getMsg());
putMsg(result, Status.UDF_RESOURCE_SUFFIX_NOT_JAR);
return result;
}
if (file.getSize() > Constants.MAX_FILE_SIZE) {
logger.error("file size is too large: {}", file.getOriginalFilename());
putMsg(result, Status.RESOURCE_SIZE_EXCEED_LIMIT);
return result;
}
}
// query tenant by user id
String tenantCode = getTenantCode(resource.getUserId(),result);
if (StringUtils.isEmpty(tenantCode)){
@ -380,31 +422,61 @@ public class ResourcesService extends BaseService {
}
// updateResource data
List<Integer> childrenResource = listAllChildren(resource,false);
Date now = new Date();
resource.setAlias(name);
resource.setFullName(fullName);
resource.setDescription(desc);
resource.setUpdateTime(now);
if (file != null) {
resource.setFileName(file.getOriginalFilename());
resource.setSize(file.getSize());
}
try {
resourcesMapper.updateById(resource);
if (resource.isDirectory() && CollectionUtils.isNotEmpty(childrenResource)) {
String matcherFullName = Matcher.quoteReplacement(fullName);
List<Resource> childResourceList = new ArrayList<>();
List<Resource> resourceList = resourcesMapper.listResourceByIds(childrenResource.toArray(new Integer[childrenResource.size()]));
childResourceList = resourceList.stream().map(t -> {
t.setFullName(t.getFullName().replaceFirst(originFullName, matcherFullName));
t.setUpdateTime(now);
return t;
}).collect(Collectors.toList());
resourcesMapper.batchUpdateResource(childResourceList);
if (resource.isDirectory()) {
List<Integer> childrenResource = listAllChildren(resource,false);
if (CollectionUtils.isNotEmpty(childrenResource)) {
String matcherFullName = Matcher.quoteReplacement(fullName);
List<Resource> childResourceList = new ArrayList<>();
Integer[] childResIdArray = childrenResource.toArray(new Integer[childrenResource.size()]);
List<Resource> resourceList = resourcesMapper.listResourceByIds(childResIdArray);
childResourceList = resourceList.stream().map(t -> {
t.setFullName(t.getFullName().replaceFirst(originFullName, matcherFullName));
t.setUpdateTime(now);
return t;
}).collect(Collectors.toList());
resourcesMapper.batchUpdateResource(childResourceList);
if (ResourceType.UDF.equals(resource.getType())) {
List<UdfFunc> udfFuncs = udfFunctionMapper.listUdfByResourceId(childResIdArray);
if (CollectionUtils.isNotEmpty(udfFuncs)) {
udfFuncs = udfFuncs.stream().map(t -> {
t.setResourceName(t.getResourceName().replaceFirst(originFullName, matcherFullName));
t.setUpdateTime(now);
return t;
}).collect(Collectors.toList());
udfFunctionMapper.batchUpdateUdfFunc(udfFuncs);
}
}
}
} else if (ResourceType.UDF.equals(resource.getType())) {
List<UdfFunc> udfFuncs = udfFunctionMapper.listUdfByResourceId(new Integer[]{resourceId});
if (CollectionUtils.isNotEmpty(udfFuncs)) {
udfFuncs = udfFuncs.stream().map(t -> {
t.setResourceName(fullName);
t.setUpdateTime(now);
return t;
}).collect(Collectors.toList());
udfFunctionMapper.batchUpdateUdfFunc(udfFuncs);
}
}
putMsg(result, Status.SUCCESS);
Map<Object, Object> dataMap = new BeanMap(resource);
Map<String, Object> resultMap = new HashMap<>();
Map<String, Object> resultMap = new HashMap<>(5);
for (Map.Entry<Object, Object> entry: dataMap.entrySet()) {
if (!Constants.CLASS.equalsIgnoreCase(entry.getKey().toString())) {
resultMap.put(entry.getKey().toString(), entry.getValue());
@ -415,11 +487,31 @@ public class ResourcesService extends BaseService {
logger.error(Status.UPDATE_RESOURCE_ERROR.getMsg(), e);
throw new ServiceException(Status.UPDATE_RESOURCE_ERROR);
}
// if name unchanged, return directly without moving on HDFS
if (originResourceName.equals(name)) {
if (originResourceName.equals(name) && file == null) {
return result;
}
if (file != null) {
// fail upload
if (!upload(loginUser, fullName, file, type)) {
logger.error("upload resource: {} file: {} failed.", name, file.getOriginalFilename());
putMsg(result, Status.HDFS_OPERATION_ERROR);
throw new RuntimeException(String.format("upload resource: %s file: %s failed.", name, file.getOriginalFilename()));
}
if (!fullName.equals(originFullName)) {
try {
HadoopUtils.getInstance().delete(originHdfsFileName,false);
} catch (IOException e) {
logger.error(e.getMessage(),e);
throw new RuntimeException(String.format("delete resource: %s failed.", originFullName));
}
}
return result;
}
// get the path of dest file in hdfs
String destHdfsFileName = HadoopUtils.getHdfsFileName(resource.getType(),tenantCode,fullName);
@ -449,7 +541,7 @@ public class ResourcesService extends BaseService {
*/
public Map<String, Object> queryResourceListPaging(User loginUser, int direcotryId, ResourceType type, String searchVal, Integer pageNo, Integer pageSize) {
HashMap<String, Object> result = new HashMap<>();
HashMap<String, Object> result = new HashMap<>(5);
Page<Resource> page = new Page(pageNo, pageSize);
int userId = loginUser.getId();
if (isAdmin(loginUser)) {
@ -550,7 +642,7 @@ public class ResourcesService extends BaseService {
*/
public Map<String, Object> queryResourceList(User loginUser, ResourceType type) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
int userId = loginUser.getId();
if(isAdmin(loginUser)){
@ -565,21 +657,33 @@ public class ResourcesService extends BaseService {
}
/**
* query resource list
* query resource list by program type
*
* @param loginUser login user
* @param type resource type
* @return resource list
*/
public Map<String, Object> queryResourceJarList(User loginUser, ResourceType type) {
public Map<String, Object> queryResourceByProgramType(User loginUser, ResourceType type, ProgramType programType) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
String suffix = ".jar";
int userId = loginUser.getId();
if(isAdmin(loginUser)){
userId = 0;
}
if (programType != null) {
switch (programType) {
case JAVA:
break;
case SCALA:
break;
case PYTHON:
suffix = ".py";
break;
}
}
List<Resource> allResourceList = resourcesMapper.queryResourceListAuthored(userId, type.ordinal(),0);
List<Resource> resources = new ResourceFilter(".jar",new ArrayList<>(allResourceList)).filter();
List<Resource> resources = new ResourceFilter(suffix,new ArrayList<>(allResourceList)).filter();
Visitor resourceTreeVisitor = new ResourceTreeVisitor(resources);
result.put(Constants.DATA_LIST, resourceTreeVisitor.visit().getChildren());
putMsg(result,Status.SUCCESS);
@ -829,7 +933,7 @@ public class ResourcesService extends BaseService {
* @param content content
* @return create result code
*/
@Transactional(rollbackFor = RuntimeException.class)
@Transactional(rollbackFor = Exception.class)
public Result onlineCreateResource(User loginUser, ResourceType type, String fileName, String fileSuffix, String desc, String content,int pid,String currentDirectory) {
Result result = new Result();
// if resource upload startup
@ -852,12 +956,25 @@ public class ResourcesService extends BaseService {
}
String name = fileName.trim() + "." + nameSuffix;
String fullName = "/".equals(currentDirectory) ? String.format("%s%s",currentDirectory,name):String.format("%s/%s",currentDirectory,name);
String fullName = currentDirectory.equals("/") ? String.format("%s%s",currentDirectory,name):String.format("%s/%s",currentDirectory,name);
result = verifyResourceName(fullName,type,loginUser);
if (!result.getCode().equals(Status.SUCCESS.getCode())) {
return result;
}
if (pid != -1) {
Resource parentResource = resourcesMapper.selectById(pid);
if (parentResource == null) {
putMsg(result, Status.PARENT_RESOURCE_NOT_EXIST);
return result;
}
if (!hasPerm(loginUser, parentResource.getUserId())) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
}
// save data
Date now = new Date();
@ -891,7 +1008,7 @@ public class ResourcesService extends BaseService {
* @param content content
* @return update result cod
*/
@Transactional(rollbackFor = RuntimeException.class)
@Transactional(rollbackFor = Exception.class)
public Result updateResourceContent(int resourceId, String content) {
Result result = new Result();
@ -1096,7 +1213,7 @@ public class ResourcesService extends BaseService {
* @return unauthorized result code
*/
public Map<String, Object> unauthorizedUDFFunction(User loginUser, Integer userId) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
//only admin can operate
if (checkAdmin(loginUser, result)) {
return result;
@ -1148,7 +1265,7 @@ public class ResourcesService extends BaseService {
* @return authorized result
*/
public Map<String, Object> authorizedFile(User loginUser, Integer userId) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
if (checkAdmin(loginUser, result)){
return result;
}

90
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java
Unescape View File

@ -14,18 +14,22 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.service;
package org.apache.dolphinscheduler.api.service;
import org.apache.dolphinscheduler.api.dto.ScheduleParam;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.exceptions.ServiceException;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.*;
import org.apache.dolphinscheduler.common.enums.FailureStrategy;
import org.apache.dolphinscheduler.common.enums.Priority;
import org.apache.dolphinscheduler.common.enums.ReleaseState;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.common.model.Server;
import org.apache.dolphinscheduler.common.utils.DateUtils;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.dao.entity.ProcessDefinition;
import org.apache.dolphinscheduler.dao.entity.Project;
@ -34,12 +38,18 @@ import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper;
import org.apache.dolphinscheduler.dao.mapper.ProjectMapper;
import org.apache.dolphinscheduler.dao.mapper.ScheduleMapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import org.apache.dolphinscheduler.service.process.ProcessService;
import org.apache.dolphinscheduler.service.quartz.ProcessScheduleJob;
import org.apache.dolphinscheduler.service.quartz.QuartzExecutors;
import org.apache.dolphinscheduler.service.quartz.cron.CronUtils;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.quartz.CronExpression;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -47,8 +57,8 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.text.ParseException;
import java.util.*;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
/**
* scheduler service
@ -86,12 +96,10 @@ public class SchedulerService extends BaseService {
* @param projectName project name
* @param processDefineId process definition id
* @param schedule scheduler
* @param warningType warning type
* @param warningType warning type
* @param warningGroupId warning group id
* @param failureStrategy failure strategy
* @param processInstancePriority process instance priority
* @param receivers receivers
* @param receiversCc receivers cc
* @param workerGroup worker group
* @return create result code
*/
@ -102,12 +110,10 @@ public class SchedulerService extends BaseService {
WarningType warningType,
int warningGroupId,
FailureStrategy failureStrategy,
String receivers,
String receiversCc,
Priority processInstancePriority,
String workerGroup) {
Map<String, Object> result = new HashMap<String, Object>(5);
Map<String, Object> result = new HashMap();
Project project = projectMapper.queryByName(projectName);
@ -132,9 +138,9 @@ public class SchedulerService extends BaseService {
scheduleObj.setProcessDefinitionName(processDefinition.getName());
ScheduleParam scheduleParam = JSONUtils.parseObject(schedule, ScheduleParam.class);
if (DateUtils.differSec(scheduleParam.getStartTime(),scheduleParam.getEndTime()) == 0) {
if (DateUtils.differSec(scheduleParam.getStartTime(), scheduleParam.getEndTime()) == 0) {
logger.warn("The start time must not be the same as the end");
putMsg(result,Status.SCHEDULE_START_TIME_END_TIME_SAME);
putMsg(result, Status.SCHEDULE_START_TIME_END_TIME_SAME);
return result;
}
scheduleObj.setStartTime(scheduleParam.getStartTime());
@ -172,7 +178,6 @@ public class SchedulerService extends BaseService {
return result;
}
/**
* updateProcessInstance schedule
*
@ -185,8 +190,6 @@ public class SchedulerService extends BaseService {
* @param failureStrategy failure strategy
* @param workerGroup worker group
* @param processInstancePriority process instance priority
* @param receiversCc receiver cc
* @param receivers receivers
* @param scheduleStatus schedule status
* @return update result code
*/
@ -198,8 +201,6 @@ public class SchedulerService extends BaseService {
WarningType warningType,
int warningGroupId,
FailureStrategy failureStrategy,
String receivers,
String receiversCc,
ReleaseState scheduleStatus,
Priority processInstancePriority,
String workerGroup) {
@ -239,9 +240,9 @@ public class SchedulerService extends BaseService {
// updateProcessInstance param
if (StringUtils.isNotEmpty(scheduleExpression)) {
ScheduleParam scheduleParam = JSONUtils.parseObject(scheduleExpression, ScheduleParam.class);
if (DateUtils.differSec(scheduleParam.getStartTime(),scheduleParam.getEndTime()) == 0) {
if (DateUtils.differSec(scheduleParam.getStartTime(), scheduleParam.getEndTime()) == 0) {
logger.warn("The start time must not be the same as the end");
putMsg(result,Status.SCHEDULE_START_TIME_END_TIME_SAME);
putMsg(result, Status.SCHEDULE_START_TIME_END_TIME_SAME);
return result;
}
schedule.setStartTime(scheduleParam.getStartTime());
@ -289,7 +290,7 @@ public class SchedulerService extends BaseService {
* @param loginUser login user
* @param projectName project name
* @param id scheduler id
* @param scheduleStatus schedule status
* @param scheduleStatus schedule status
* @return publish result code
*/
@Transactional(rollbackFor = RuntimeException.class)
@ -315,7 +316,7 @@ public class SchedulerService extends BaseService {
return result;
}
// check schedule release state
if(scheduleObj.getReleaseState() == scheduleStatus){
if (scheduleObj.getReleaseState() == scheduleStatus) {
logger.info("schedule release is already {},needn't to change schedule id: {} from {} to {}",
scheduleObj.getReleaseState(), scheduleObj.getId(), scheduleObj.getReleaseState(), scheduleStatus);
putMsg(result, Status.SCHEDULE_CRON_REALEASE_NEED_NOT_CHANGE, scheduleStatus);
@ -327,9 +328,9 @@ public class SchedulerService extends BaseService {
return result;
}
if(scheduleStatus == ReleaseState.ONLINE){
if (scheduleStatus == ReleaseState.ONLINE) {
// check process definition release state
if(processDefinition.getReleaseState() != ReleaseState.ONLINE){
if (processDefinition.getReleaseState() != ReleaseState.ONLINE) {
logger.info("not release process definition id: {} , name : {}",
processDefinition.getId(), processDefinition.getName());
putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE, processDefinition.getName());
@ -339,15 +340,15 @@ public class SchedulerService extends BaseService {
List<Integer> subProcessDefineIds = new ArrayList<>();
processService.recurseFindSubProcessId(scheduleObj.getProcessDefinitionId(), subProcessDefineIds);
Integer[] idArray = subProcessDefineIds.toArray(new Integer[subProcessDefineIds.size()]);
if (subProcessDefineIds.size() > 0){
if (subProcessDefineIds.size() > 0) {
List<ProcessDefinition> subProcessDefinitionList =
processDefinitionMapper.queryDefinitionListByIdList(idArray);
if (subProcessDefinitionList != null && subProcessDefinitionList.size() > 0){
for (ProcessDefinition subProcessDefinition : subProcessDefinitionList){
if (subProcessDefinitionList != null && subProcessDefinitionList.size() > 0) {
for (ProcessDefinition subProcessDefinition : subProcessDefinitionList) {
/**
* if there is no online process, exit directly
*/
if (subProcessDefinition.getReleaseState() != ReleaseState.ONLINE){
if (subProcessDefinition.getReleaseState() != ReleaseState.ONLINE) {
logger.info("not release process definition id: {} , name : {}",
subProcessDefinition.getId(), subProcessDefinition.getName());
putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE, subProcessDefinition.getId());
@ -361,7 +362,6 @@ public class SchedulerService extends BaseService {
// check master server exists
List<Server> masterServers = monitorService.getServerListFromZK(true);
if (masterServers.size() == 0) {
putMsg(result, Status.MASTER_NOT_EXISTS);
return result;
@ -398,8 +398,6 @@ public class SchedulerService extends BaseService {
return result;
}
/**
* query schedule
*
@ -407,7 +405,7 @@ public class SchedulerService extends BaseService {
* @param projectName project name
* @param processDefineId process definition id
* @param pageNo page number
* @param pageSize page size
* @param pageSize page size
* @param searchVal search value
* @return schedule list page
*/
@ -433,9 +431,8 @@ public class SchedulerService extends BaseService {
page, processDefineId, searchVal
);
PageInfo pageInfo = new PageInfo<Schedule>(pageNo, pageSize);
pageInfo.setTotalCount((int)scheduleIPage.getTotal());
pageInfo.setTotalCount((int) scheduleIPage.getTotal());
pageInfo.setLists(scheduleIPage.getRecords());
result.put(Constants.DATA_LIST, pageInfo);
putMsg(result, Status.SUCCESS);
@ -499,8 +496,8 @@ public class SchedulerService extends BaseService {
String jobName = QuartzExecutors.buildJobName(scheduleId);
String jobGroupName = QuartzExecutors.buildJobGroupName(projectId);
if(!QuartzExecutors.getInstance().deleteJob(jobName, jobGroupName)){
logger.warn("set offline failure:projectId:{},scheduleId:{}",projectId,scheduleId);
if (!QuartzExecutors.getInstance().deleteJob(jobName, jobGroupName)) {
logger.warn("set offline failure:projectId:{},scheduleId:{}", projectId, scheduleId);
throw new ServiceException("set offline failure");
}
@ -550,19 +547,18 @@ public class SchedulerService extends BaseService {
}
// Determine if the login user is the owner of the schedule
if (loginUser.getId() != schedule.getUserId() &&
loginUser.getUserType() != UserType.ADMIN_USER) {
if (loginUser.getId() != schedule.getUserId()
&& loginUser.getUserType() != UserType.ADMIN_USER) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
// check schedule is already online
if(schedule.getReleaseState() == ReleaseState.ONLINE){
putMsg(result, Status.SCHEDULE_CRON_STATE_ONLINE,schedule.getId());
if (schedule.getReleaseState() == ReleaseState.ONLINE) {
putMsg(result, Status.SCHEDULE_CRON_STATE_ONLINE, schedule.getId());
return result;
}
int delete = scheduleMapper.deleteById(scheduleId);
if (delete > 0) {
@ -581,7 +577,7 @@ public class SchedulerService extends BaseService {
* @param schedule schedule expression
* @return the next five fire time
*/
public Map<String,Object> previewSchedule(User loginUser, String projectName, String schedule) {
public Map<String, Object> previewSchedule(User loginUser, String projectName, String schedule) {
Map<String, Object> result = new HashMap<>();
CronExpression cronExpression;
ScheduleParam scheduleParam = JSONUtils.parseObject(schedule, ScheduleParam.class);
@ -592,11 +588,11 @@ public class SchedulerService extends BaseService {
try {
cronExpression = CronUtils.parse2CronExpression(scheduleParam.getCrontab());
} catch (ParseException e) {
logger.error(e.getMessage(),e);
putMsg(result,Status.PARSE_TO_CRON_EXPRESSION_ERROR);
logger.error(e.getMessage(), e);
putMsg(result, Status.PARSE_TO_CRON_EXPRESSION_ERROR);
return result;
}
List<Date> selfFireDateList = CronUtils.getSelfFireDateList(startTime, endTime,cronExpression,Constants.PREVIEW_SCHEDULE_EXECUTE_COUNT);
List<Date> selfFireDateList = CronUtils.getSelfFireDateList(startTime, endTime, cronExpression, Constants.PREVIEW_SCHEDULE_EXECUTE_COUNT);
result.put(Constants.DATA_LIST, selfFireDateList.stream().map(t -> DateUtils.dateToString(t)));
putMsg(result, Status.SUCCESS);
return result;

7
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskInstanceService.java
Unescape View File

@ -14,8 +14,8 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.service;
package org.apache.dolphinscheduler.api.service;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.utils.PageInfo;
@ -69,7 +69,6 @@ public class TaskInstanceService extends BaseService {
@Autowired
UsersService usersService;
/**
* query task list by project, process instance, task name, task start time, task end time, task status, keyword paging
*
@ -87,7 +86,7 @@ public class TaskInstanceService extends BaseService {
* @return task list page
*/
public Map<String, Object> queryTaskListPaging(User loginUser, String projectName,
Integer processInstanceId, String taskName, String executorName, String startDate,
Integer processInstanceId, String processInstanceName, String taskName, String executorName, String startDate,
String endDate, String searchVal, ExecutionStatus stateType, String host,
Integer pageNo, Integer pageSize) {
Map<String, Object> result = new HashMap<>();
@ -124,7 +123,7 @@ public class TaskInstanceService extends BaseService {
int executorId = usersService.getUserIdByName(executorName);
IPage<TaskInstance> taskInstanceIPage = taskInstanceMapper.queryTaskInstanceListPaging(
page, project.getId(), processInstanceId, searchVal, taskName, executorId, statusArray, host, start, end
page, project.getId(), processInstanceId, processInstanceName, searchVal, taskName, executorId, statusArray, host, start, end
);
Set<String> exclusionSet = new HashSet<>();
exclusionSet.add(Constants.CLASS);

5
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TenantService.java
Unescape View File

@ -32,7 +32,6 @@ public interface TenantService {
*
* @param loginUser login user
* @param tenantCode tenant code
* @param tenantName tenant name
* @param queueId queue id
* @param desc description
* @return create result code
@ -40,7 +39,6 @@ public interface TenantService {
*/
Map<String, Object> createTenant(User loginUser,
String tenantCode,
String tenantName,
int queueId,
String desc) throws Exception;
@ -61,13 +59,12 @@ public interface TenantService {
* @param loginUser login user
* @param id tennat id
* @param tenantCode tennat code
* @param tenantName tennat name
* @param queueId queue id
* @param desc description
* @return update result code
* @throws Exception exception
*/
Map<String, Object> updateTenant(User loginUser, int id, String tenantCode, String tenantName, int queueId,
Map<String, Object> updateTenant(User loginUser, int id, String tenantCode, int queueId,
String desc) throws Exception;
/**

20
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UdfFuncService.java
Unescape View File

@ -148,7 +148,7 @@ public class UdfFuncService extends BaseService{
*/
public Map<String, Object> queryUdfFuncDetail(int id) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
UdfFunc udfFunc = udfFuncMapper.selectById(id);
if (udfFunc == null) {
putMsg(result, Status.RESOURCE_NOT_EXIST);
@ -244,7 +244,7 @@ public class UdfFuncService extends BaseService{
* @return udf function list page
*/
public Map<String, Object> queryUdfFuncListPaging(User loginUser, String searchVal, Integer pageNo, Integer pageSize) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
PageInfo pageInfo = new PageInfo<Resource>(pageNo, pageSize);
@ -276,15 +276,19 @@ public class UdfFuncService extends BaseService{
}
/**
* query data resource by type
* query udf list
*
* @param loginUser login user
* @param type resource type
* @return resource list
* @param type udf type
* @return udf func list
*/
public Map<String, Object> queryResourceList(User loginUser, Integer type) {
Map<String, Object> result = new HashMap<>();
List<UdfFunc> udfFuncList = udfFuncMapper.getUdfFuncByType(loginUser.getId(), type);
public Map<String, Object> queryUdfFuncList(User loginUser, Integer type) {
Map<String, Object> result = new HashMap<>(5);
int userId = loginUser.getId();
if (isAdmin(loginUser)) {
userId = 0;
}
List<UdfFunc> udfFuncList = udfFuncMapper.getUdfFuncByType(userId, type);
result.put(Constants.DATA_LIST, udfFuncList);
putMsg(result, Status.SUCCESS);

135
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UsersService.java
Unescape View File

@ -31,21 +31,49 @@ import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.common.enums.ResourceType;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.dolphinscheduler.dao.entity.*;
import org.apache.dolphinscheduler.dao.mapper.*;
import org.apache.dolphinscheduler.common.utils.CollectionUtils;
import org.apache.dolphinscheduler.common.utils.EncryptionUtils;
import org.apache.dolphinscheduler.common.utils.HadoopUtils;
import org.apache.dolphinscheduler.common.utils.PropertyUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.dao.entity.AlertGroup;
import org.apache.dolphinscheduler.dao.entity.DatasourceUser;
import org.apache.dolphinscheduler.dao.entity.ProjectUser;
import org.apache.dolphinscheduler.dao.entity.Resource;
import org.apache.dolphinscheduler.dao.entity.ResourcesUser;
import org.apache.dolphinscheduler.dao.entity.Tenant;
import org.apache.dolphinscheduler.dao.entity.UDFUser;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.AlertGroupMapper;
import org.apache.dolphinscheduler.dao.mapper.DataSourceUserMapper;
import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper;
import org.apache.dolphinscheduler.dao.mapper.ProjectUserMapper;
import org.apache.dolphinscheduler.dao.mapper.ResourceMapper;
import org.apache.dolphinscheduler.dao.mapper.ResourceUserMapper;
import org.apache.dolphinscheduler.dao.mapper.TenantMapper;
import org.apache.dolphinscheduler.dao.mapper.UDFUserMapper;
import org.apache.dolphinscheduler.dao.mapper.UserMapper;
import org.apache.dolphinscheduler.dao.utils.ResourceProcessDefinitionUtils;
import java.io.IOException;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.io.IOException;
import java.text.MessageFormat;
import java.util.*;
import java.util.stream.Collectors;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
/**
* user service
@ -174,6 +202,37 @@ public class UsersService extends BaseService {
return user;
}
/***
* create User for ldap login
*/
@Transactional(rollbackFor = Exception.class)
public User createUser(UserType userType, String userId, String email) {
User user = new User();
Date now = new Date();
user.setUserName(userId);
user.setEmail(email);
// create general users, administrator users are currently built-in
user.setUserType(userType);
user.setCreateTime(now);
user.setUpdateTime(now);
user.setQueue("");
// save user
userMapper.insert(user);
return user;
}
/**
* get user by user name
*
* @param userName user name
* @return exist user or null
*/
public User getUserByUserName(String userName) {
return userMapper.queryByUserNameAccurately(userName);
}
/**
* query user by id
*
@ -208,7 +267,6 @@ public class UsersService extends BaseService {
/**
* get user id by user name
*
* @param name user name
* @return if name empty 0, user not exists -1, user exist user id
*/
@ -269,7 +327,7 @@ public class UsersService extends BaseService {
* @return update result code
* @throws Exception exception
*/
public Map<String, Object> updateUser(int userId,
public Map<String, Object> updateUser(User loginUser, int userId,
String userName,
String userPassword,
String email,
@ -280,13 +338,14 @@ public class UsersService extends BaseService {
Map<String, Object> result = new HashMap<>();
result.put(Constants.STATUS, false);
if (check(result, !hasPerm(loginUser, userId), Status.USER_NO_OPERATION_PERM)) {
return result;
}
User user = userMapper.selectById(userId);
if (user == null) {
putMsg(result, Status.USER_NOT_EXIST, userId);
return result;
}
if (StringUtils.isNotEmpty(userName)) {
if (!CheckUtils.checkUserName(userName)) {
@ -587,7 +646,7 @@ public class UsersService extends BaseService {
*/
@Transactional(rollbackFor = RuntimeException.class)
public Map<String, Object> grantUDFFunction(User loginUser, int userId, String udfIds) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
//only admin can operate
if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) {
@ -627,14 +686,14 @@ public class UsersService extends BaseService {
/**
* grant datasource
*
* @param loginUser login user
* @param userId user id
* @param loginUser login user
* @param userId user id
* @param datasourceIds data source id array
* @return grant result code
*/
@Transactional(rollbackFor = RuntimeException.class)
public Map<String, Object> grantDataSource(User loginUser, int userId, String datasourceIds) {
Map<String, Object> result = new HashMap<>();
Map<String, Object> result = new HashMap<>(5);
result.put(Constants.STATUS, false);
//only admin can operate
@ -687,6 +746,18 @@ public class UsersService extends BaseService {
user = loginUser;
} else {
user = userMapper.queryDetailsById(loginUser.getId());
List<AlertGroup> alertGroups = alertGroupMapper.queryByUserId(loginUser.getId());
StringBuilder sb = new StringBuilder();
if (alertGroups != null && alertGroups.size() > 0) {
for (int i = 0; i < alertGroups.size() - 1; i++) {
sb.append(alertGroups.get(i).getGroupName() + ",");
}
sb.append(alertGroups.get(alertGroups.size() - 1));
user.setAlertGroup(sb.toString());
}
}
result.put(Constants.DATA_LIST, user);
@ -815,24 +886,6 @@ public class UsersService extends BaseService {
return result;
}
/**
* check
*
* @param result result
* @param bool bool
* @param userNoOperationPerm status
* @return check result
*/
private boolean check(Map<String, Object> result, boolean bool, Status userNoOperationPerm) {
//only admin can operate
if (bool) {
result.put(Constants.STATUS, userNoOperationPerm);
result.put(Constants.MSG, userNoOperationPerm.getMsg());
return true;
}
return false;
}
/**
* @param tenantId tenant id
* @return true if tenant exists, otherwise return false
@ -842,10 +895,6 @@ public class UsersService extends BaseService {
}
/**
* @param userName
* @param password
* @param email
* @param phone
* @return if check failed return the field, otherwise return null
*/
private String checkUserParams(String userName, String password, String email, String phone) {
@ -872,8 +921,8 @@ public class UsersService extends BaseService {
* copy resource files
*
* @param resourceComponent resource component
* @param srcBasePath src base path
* @param dstBasePath dst base path
* @param srcBasePath src base path
* @param dstBasePath dst base path
* @throws IOException io exception
*/
private void copyResourceFiles(ResourceComponent resourceComponent, String srcBasePath, String dstBasePath) throws IOException {
@ -908,10 +957,10 @@ public class UsersService extends BaseService {
/**
* register user, default state is 0, default tenant_id is 1, no phone, no queue
*
* @param userName user name
* @param userPassword user password
* @param userName user name
* @param userPassword user password
* @param repeatPassword repeat password
* @param email email
* @param email email
* @return register result code
* @throws Exception exception
*/
@ -941,7 +990,7 @@ public class UsersService extends BaseService {
* activate user, only system admin have permission, change user state code 0 to 1
*
* @param loginUser login user
* @param userName user name
* @param userName user name
* @return create result code
*/
public Map<String, Object> activateUser(User loginUser, String userName) {

76
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/WorkerGroupService.java
Unescape View File

@ -14,8 +14,11 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.service;
import static org.apache.dolphinscheduler.common.Constants.DEFAULT_WORKER_GROUP;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.common.Constants;
@ -26,11 +29,17 @@ import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.entity.WorkerGroup;
import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapper;
import org.apache.dolphinscheduler.service.zk.ZookeeperCachedOperator;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.*;
import java.util.stream.Collectors;
/**
* work group service
@ -38,14 +47,11 @@ import java.util.stream.Collectors;
@Service
public class WorkerGroupService extends BaseService {
@Autowired
ProcessInstanceMapper processInstanceMapper;
private static final String NO_NODE_EXCEPTION_REGEX = "KeeperException$NoNodeException";
@Autowired
protected ZookeeperCachedOperator zookeeperCachedOperator;
@Autowired
ProcessInstanceMapper processInstanceMapper;
/**
* query worker group paging
@ -56,7 +62,7 @@ public class WorkerGroupService extends BaseService {
* @param pageSize page size
* @return worker group list page
*/
public Map<String,Object> queryAllGroupPaging(User loginUser, Integer pageNo, Integer pageSize, String searchVal) {
public Map<String, Object> queryAllGroupPaging(User loginUser, Integer pageNo, Integer pageSize, String searchVal) {
// list from index
Integer fromIndex = (pageNo - 1) * pageSize;
@ -72,20 +78,20 @@ public class WorkerGroupService extends BaseService {
List<WorkerGroup> resultDataList = new ArrayList<>();
if (CollectionUtils.isNotEmpty(workerGroups)){
if (CollectionUtils.isNotEmpty(workerGroups)) {
List<WorkerGroup> searchValDataList = new ArrayList<>();
if (StringUtils.isNotEmpty(searchVal)){
for (WorkerGroup workerGroup : workerGroups){
if (workerGroup.getName().contains(searchVal)){
if (StringUtils.isNotEmpty(searchVal)) {
for (WorkerGroup workerGroup : workerGroups) {
if (workerGroup.getName().contains(searchVal)) {
searchValDataList.add(workerGroup);
}
}
}else {
} else {
searchValDataList = workerGroups;
}
if (searchValDataList.size() < pageSize){
if (searchValDataList.size() < pageSize) {
toIndex = (pageNo - 1) * pageSize + searchValDataList.size();
}
resultDataList = searchValDataList.subList(fromIndex, toIndex);
@ -100,14 +106,12 @@ public class WorkerGroupService extends BaseService {
return result;
}
/**
* query all worker group
*
* @return all worker group list
*/
public Map<String,Object> queryAllGroup() {
public Map<String, Object> queryAllGroup() {
Map<String, Object> result = new HashMap<>();
List<WorkerGroup> workerGroups = getWorkerGroups(false);
@ -120,30 +124,46 @@ public class WorkerGroupService extends BaseService {
return result;
}
/**
* get worker groups
* get worker groups
*
* @param isPaging whether paging
* @return WorkerGroup list
*/
private List<WorkerGroup> getWorkerGroups(boolean isPaging) {
String workerPath = zookeeperCachedOperator.getZookeeperConfig().getDsRoot()+"/nodes" +"/worker";
List<String> workerGroupList = zookeeperCachedOperator.getChildrenKeys(workerPath);
String workerPath = zookeeperCachedOperator.getZookeeperConfig().getDsRoot() + Constants.ZOOKEEPER_DOLPHINSCHEDULER_WORKERS;
List<WorkerGroup> workerGroups = new ArrayList<>();
List<String> workerGroupList;
try {
workerGroupList = zookeeperCachedOperator.getChildrenKeys(workerPath);
} catch (Exception e) {
if (e.getMessage().contains(NO_NODE_EXCEPTION_REGEX)) {
if (isPaging) {
return workerGroups;
} else {
//ignore noNodeException return Default
WorkerGroup wg = new WorkerGroup();
wg.setName(DEFAULT_WORKER_GROUP);
workerGroups.add(wg);
return workerGroups;
}
} else {
throw e;
}
}
// available workerGroup list
List<String> availableWorkerGroupList = new ArrayList<>();
List<WorkerGroup> workerGroups = new ArrayList<>();
for (String workerGroup : workerGroupList){
String workerGroupPath= workerPath + "/" + workerGroup;
for (String workerGroup : workerGroupList) {
String workerGroupPath = workerPath + "/" + workerGroup;
List<String> childrenNodes = zookeeperCachedOperator.getChildrenKeys(workerGroupPath);
if (CollectionUtils.isNotEmpty(childrenNodes)){
if (CollectionUtils.isNotEmpty(childrenNodes)) {
availableWorkerGroupList.add(workerGroup);
WorkerGroup wg = new WorkerGroup();
wg.setName(workerGroup);
if (isPaging){
if (isPaging) {
wg.setIpList(childrenNodes);
String registeredIpValue = zookeeperCachedOperator.get(workerGroupPath + "/" + childrenNodes.get(0));
wg.setCreateTime(DateUtils.stringToDate(registeredIpValue.split(",")[6]));

24
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/AccessTokenServiceImpl.java
Unescape View File

@ -86,9 +86,14 @@ public class AccessTokenServiceImpl extends BaseService implements AccessTokenSe
* @param token token string
* @return create result code
*/
public Map<String, Object> createToken(int userId, String expireTime, String token) {
public Map<String, Object> createToken(User loginUser, int userId, String expireTime, String token) {
Map<String, Object> result = new HashMap<>(5);
if (!hasPerm(loginUser,userId)){
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
if (userId <= 0) {
throw new IllegalArgumentException("User id should not less than or equals to 0.");
}
@ -118,8 +123,12 @@ public class AccessTokenServiceImpl extends BaseService implements AccessTokenSe
* @param expireTime token expire time
* @return token string
*/
public Map<String, Object> generateToken(int userId, String expireTime) {
public Map<String, Object> generateToken(User loginUser, int userId, String expireTime) {
Map<String, Object> result = new HashMap<>(5);
if (!hasPerm(loginUser,userId)){
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
String token = EncryptionUtils.getMd5(userId + expireTime + System.currentTimeMillis());
result.put(Constants.DATA_LIST, token);
putMsg(result, Status.SUCCESS);
@ -144,8 +153,8 @@ public class AccessTokenServiceImpl extends BaseService implements AccessTokenSe
return result;
}
if (loginUser.getId() != accessToken.getUserId() &&
loginUser.getUserType() != UserType.ADMIN_USER) {
if (!hasPerm(loginUser,accessToken.getUserId())){
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
@ -164,9 +173,12 @@ public class AccessTokenServiceImpl extends BaseService implements AccessTokenSe
* @param token token string
* @return update result code
*/
public Map<String, Object> updateToken(int id, int userId, String expireTime, String token) {
public Map<String, Object> updateToken(User loginUser, int id, int userId, String expireTime, String token) {
Map<String, Object> result = new HashMap<>(5);
if (!hasPerm(loginUser,userId)){
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
AccessToken accessToken = accessTokenMapper.selectById(id);
if (accessToken == null) {
logger.error("access token not exist, access token id {}", id);

169
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/DataAnalysisServiceImpl.java
Unescape View File

@ -16,7 +16,6 @@
*/
package org.apache.dolphinscheduler.api.service.impl;
import org.apache.dolphinscheduler.api.dto.CommandStateCount;
import org.apache.dolphinscheduler.api.dto.DefineUserDto;
import org.apache.dolphinscheduler.api.dto.TaskCountDto;
@ -45,14 +44,14 @@ import org.apache.dolphinscheduler.service.process.ProcessService;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Date;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
@ -62,8 +61,6 @@ import org.springframework.stereotype.Service;
@Service
public class DataAnalysisServiceImpl extends BaseService implements DataAnalysisService {
private static final Logger logger = LoggerFactory.getLogger(DataAnalysisServiceImpl.class);
@Autowired
private ProjectMapper projectMapper;
@ -88,10 +85,6 @@ public class DataAnalysisServiceImpl extends BaseService implements DataAnalysis
@Autowired
private ProcessService processService;
private static final String COMMAND_STATE = "commandState";
private static final String ERROR_COMMAND_STATE = "errorCommandState";
/**
* statistical task instance status data
*
@ -137,16 +130,17 @@ public class DataAnalysisServiceImpl extends BaseService implements DataAnalysis
return result;
}
Date start;
Date end;
try {
Date start = null;
Date end = null;
if (StringUtils.isNotEmpty(startDate) && StringUtils.isNotEmpty(endDate)) {
start = DateUtils.getScheduleDate(startDate);
end = DateUtils.getScheduleDate(endDate);
} catch (Exception e) {
logger.error(e.getMessage(), e);
putErrorRequestParamsMsg(result);
return result;
if (Objects.isNull(start) || Objects.isNull(end)) {
putErrorRequestParamsMsg(result);
return result;
}
}
Integer[] projectIdArray = getProjectIdsArrays(loginUser, projectId);
List<ExecuteStatusCount> processInstanceStateCounts =
instanceStateCounter.apply(start, end, projectIdArray);
@ -204,79 +198,39 @@ public class DataAnalysisServiceImpl extends BaseService implements DataAnalysis
* statistics based on task status execution, failure, completion, wait, total
*/
Date start = null;
if (StringUtils.isNotEmpty(startDate)) {
start = DateUtils.getScheduleDate(startDate);
if (Objects.isNull(start)) {
putErrorRequestParamsMsg(result);
return result;
}
}
Date end = null;
if (startDate != null && endDate != null) {
try {
start = DateUtils.getScheduleDate(startDate);
end = DateUtils.getScheduleDate(endDate);
} catch (Exception e) {
logger.error(e.getMessage(), e);
if (StringUtils.isNotEmpty(endDate)) {
end = DateUtils.getScheduleDate(endDate);
if (Objects.isNull(end)) {
putErrorRequestParamsMsg(result);
return result;
}
}
Integer[] projectIdArray = getProjectIdsArrays(loginUser, projectId);
// count command state
List<CommandCount> commandStateCounts =
commandMapper.countCommandState(
loginUser.getId(),
start,
end,
projectIdArray);
// count normal command state
Map<CommandType, Integer> normalCountCommandCounts = commandMapper.countCommandState(loginUser.getId(), start, end, projectIdArray)
.stream()
.collect(Collectors.toMap(CommandCount::getCommandType, CommandCount::getCount));
// count error command state
List<CommandCount> errorCommandStateCounts =
errorCommandMapper.countCommandState(
start, end, projectIdArray);
// enumMap
Map<CommandType, Map<String, Integer>> dataMap = new EnumMap<>(CommandType.class);
Map<String, Integer> commonCommand = new HashMap<>();
commonCommand.put(COMMAND_STATE, 0);
commonCommand.put(ERROR_COMMAND_STATE, 0);
// init data map
/**
* START_PROCESS, START_CURRENT_TASK_PROCESS, RECOVER_TOLERANCE_FAULT_PROCESS, RECOVER_SUSPENDED_PROCESS,
START_FAILURE_TASK_PROCESS,COMPLEMENT_DATA,SCHEDULER, REPEAT_RUNNING,PAUSE,STOP,RECOVER_WAITTING_THREAD;
*/
dataMap.put(CommandType.START_PROCESS, commonCommand);
dataMap.put(CommandType.START_CURRENT_TASK_PROCESS, commonCommand);
dataMap.put(CommandType.RECOVER_TOLERANCE_FAULT_PROCESS, commonCommand);
dataMap.put(CommandType.RECOVER_SUSPENDED_PROCESS, commonCommand);
dataMap.put(CommandType.START_FAILURE_TASK_PROCESS, commonCommand);
dataMap.put(CommandType.COMPLEMENT_DATA, commonCommand);
dataMap.put(CommandType.SCHEDULER, commonCommand);
dataMap.put(CommandType.REPEAT_RUNNING, commonCommand);
dataMap.put(CommandType.PAUSE, commonCommand);
dataMap.put(CommandType.STOP, commonCommand);
dataMap.put(CommandType.RECOVER_WAITTING_THREAD, commonCommand);
// put command state
for (CommandCount executeStatusCount : commandStateCounts) {
Map<String, Integer> commandStateCountsMap = new HashMap<>(dataMap.get(executeStatusCount.getCommandType()));
commandStateCountsMap.put(COMMAND_STATE, executeStatusCount.getCount());
dataMap.put(executeStatusCount.getCommandType(), commandStateCountsMap);
}
Map<CommandType, Integer> errorCommandCounts = errorCommandMapper.countCommandState(start, end, projectIdArray)
.stream()
.collect(Collectors.toMap(CommandCount::getCommandType, CommandCount::getCount));
// put error command state
for (CommandCount errorExecutionStatus : errorCommandStateCounts) {
Map<String, Integer> errorCommandStateCountsMap = new HashMap<>(dataMap.get(errorExecutionStatus.getCommandType()));
errorCommandStateCountsMap.put(ERROR_COMMAND_STATE, errorExecutionStatus.getCount());
dataMap.put(errorExecutionStatus.getCommandType(), errorCommandStateCountsMap);
}
List<CommandStateCount> list = new ArrayList<>();
for (Map.Entry<CommandType, Map<String, Integer>> next : dataMap.entrySet()) {
CommandStateCount commandStateCount = new CommandStateCount(next.getValue().get(ERROR_COMMAND_STATE),
next.getValue().get(COMMAND_STATE), next.getKey());
list.add(commandStateCount);
}
List<CommandStateCount> list = Arrays.stream(CommandType.values())
.map(commandType -> new CommandStateCount(
errorCommandCounts.getOrDefault(commandType, 0),
normalCountCommandCounts.getOrDefault(commandType, 0),
commandType)
).collect(Collectors.toList());
result.put(Constants.DATA_LIST, list);
putMsg(result, Status.SUCCESS);
@ -311,59 +265,10 @@ public class DataAnalysisServiceImpl extends BaseService implements DataAnalysis
return result;
}
// TODO tasksQueueList and tasksKillList is never updated.
List<String> tasksQueueList = new ArrayList<>();
List<String> tasksKillList = new ArrayList<>();
//TODO need to add detail data info
Map<String, Integer> dataMap = new HashMap<>();
if (loginUser.getUserType() == UserType.ADMIN_USER) {
dataMap.put("taskQueue", tasksQueueList.size());
dataMap.put("taskKill", tasksKillList.size());
result.put(Constants.DATA_LIST, dataMap);
putMsg(result, Status.SUCCESS);
return result;
}
int[] tasksQueueIds = new int[tasksQueueList.size()];
int[] tasksKillIds = new int[tasksKillList.size()];
int i = 0;
for (String taskQueueStr : tasksQueueList) {
if (StringUtils.isNotEmpty(taskQueueStr)) {
String[] splits = taskQueueStr.split("_");
if (splits.length >= 4) {
tasksQueueIds[i++] = Integer.parseInt(splits[3]);
}
}
}
i = 0;
for (String taskKillStr : tasksKillList) {
if (StringUtils.isNotEmpty(taskKillStr)) {
String[] splits = taskKillStr.split("-");
if (splits.length == 2) {
tasksKillIds[i++] = Integer.parseInt(splits[1]);
}
}
}
Integer taskQueueCount = 0;
Integer taskKillCount = 0;
Integer[] projectIds = getProjectIdsArrays(loginUser, projectId);
if (tasksQueueIds.length != 0) {
taskQueueCount = taskInstanceMapper.countTask(
projectIds,
tasksQueueIds);
}
if (tasksKillIds.length != 0) {
taskKillCount = taskInstanceMapper.countTask(projectIds, tasksKillIds);
}
dataMap.put("taskQueue", taskQueueCount);
dataMap.put("taskKill", taskKillCount);
dataMap.put("taskQueue", 0);
dataMap.put("taskKill", 0);
result.put(Constants.DATA_LIST, dataMap);
putMsg(result, Status.SUCCESS);
return result;

1
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/LoggerServiceImpl.java
Unescape View File

@ -130,7 +130,6 @@ public class LoggerServiceImpl implements LoggerService {
logClient.getLogBytes(host, Constants.RPC_PORT, taskInstance.getLogPath()));
}
/**
* get host
*

172
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
Unescape View File

@ -17,7 +17,7 @@
package org.apache.dolphinscheduler.api.service.impl;
import static org.apache.dolphinscheduler.common.Constants.CMDPARAM_SUB_PROCESS_DEFINE_ID;
import static org.apache.dolphinscheduler.common.Constants.CMD_PARAM_SUB_PROCESS_DEFINE_ID;
import org.apache.dolphinscheduler.api.dto.ProcessMeta;
import org.apache.dolphinscheduler.api.dto.treeview.Instance;
@ -49,6 +49,7 @@ import org.apache.dolphinscheduler.common.model.TaskNodeRelation;
import org.apache.dolphinscheduler.common.process.ProcessDag;
import org.apache.dolphinscheduler.common.process.Property;
import org.apache.dolphinscheduler.common.process.ResourceInfo;
import org.apache.dolphinscheduler.common.task.AbstractParameters;
import org.apache.dolphinscheduler.common.thread.Stopper;
import org.apache.dolphinscheduler.common.utils.CollectionUtils;
import org.apache.dolphinscheduler.common.utils.DateUtils;
@ -77,7 +78,6 @@ import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Date;
import java.util.HashMap;
import java.util.HashSet;
@ -85,7 +85,6 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.stream.Collectors;
@ -159,6 +158,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param connects connects for nodes
* @return create result code
*/
@Override
public Map<String, Object> createProcessDefinition(User loginUser,
String projectName,
String name,
@ -222,7 +222,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
// return processDefinition object with ID
result.put(Constants.DATA_LIST, processDefineMapper.selectById(processDefine.getId()));
putMsg(result, Status.SUCCESS);
result.put("processDefinitionId", processDefine.getId());
result.put(PROCESSDEFINITIONID, processDefine.getId());
return result;
}
@ -233,16 +233,35 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @return resource ids
*/
private String getResourceIds(ProcessData processData) {
return Optional.ofNullable(processData.getTasks())
.orElse(Collections.emptyList())
.stream()
.map(taskNode -> TaskParametersUtils.getParameters(taskNode.getType(), taskNode.getParams()))
.filter(Objects::nonNull)
.flatMap(parameters -> parameters.getResourceFilesList().stream())
.map(ResourceInfo::getId)
.distinct()
.map(Objects::toString)
.collect(Collectors.joining(","));
List<TaskNode> tasks = processData.getTasks();
Set<Integer> resourceIds = new HashSet<>();
StringBuilder sb = new StringBuilder();
if (CollectionUtils.isEmpty(tasks)) {
return sb.toString();
}
for (TaskNode taskNode : tasks) {
String taskParameter = taskNode.getParams();
AbstractParameters params = TaskParametersUtils.getParameters(taskNode.getType(), taskParameter);
if (params == null) {
continue;
}
if (CollectionUtils.isNotEmpty(params.getResourceFilesList())) {
Set<Integer> tempSet = params.getResourceFilesList().
stream()
.filter(t -> t.getId() != 0)
.map(ResourceInfo::getId)
.collect(Collectors.toSet());
resourceIds.addAll(tempSet);
}
}
for (int i : resourceIds) {
if (sb.length() > 0) {
sb.append(",");
}
sb.append(i);
}
return sb.toString();
}
/**
@ -252,6 +271,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param projectName project name
* @return definition list
*/
@Override
public Map<String, Object> queryProcessDefinitionList(User loginUser, String projectName) {
HashMap<String, Object> result = new HashMap<>();
@ -281,6 +301,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param userId user id
* @return process definition page
*/
@Override
public Map<String, Object> queryProcessDefinitionListPaging(User loginUser, String projectName, String searchVal, Integer pageNo, Integer pageSize, Integer userId) {
Map<String, Object> result = new HashMap<>();
@ -313,6 +334,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param processId process definition id
* @return process definition detail
*/
@Override
public Map<String, Object> queryProcessDefinitionById(User loginUser, String projectName, Integer processId) {
Map<String, Object> result = new HashMap<>();
@ -347,10 +369,16 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param connects connects for nodes
* @return update result code
*/
public Map<String, Object> updateProcessDefinition(User loginUser, String projectName, int id, String name,
String processDefinitionJson, String desc,
String locations, String connects) {
Map<String, Object> result = new HashMap<>();
@Override
public Map<String, Object> updateProcessDefinition(User loginUser,
String projectName,
int id,
String name,
String processDefinitionJson,
String desc,
String locations,
String connects) {
Map<String, Object> result = new HashMap<>(5);
Project project = projectMapper.queryByName(projectName);
Map<String, Object> checkResult = projectService.checkProjectAndAuth(loginUser, project, projectName);
@ -365,16 +393,24 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
return checkProcessJson;
}
ProcessDefinition processDefine = processService.findProcessDefineById(id);
// check process definition exists
if (processDefine == null) {
// check process definition exists
putMsg(result, Status.PROCESS_DEFINE_NOT_EXIST, id);
return result;
} else if (processDefine.getReleaseState() == ReleaseState.ONLINE) {
}
if (processDefine.getReleaseState() == ReleaseState.ONLINE) {
// online can not permit edit
putMsg(result, Status.PROCESS_DEFINE_NOT_ALLOWED_EDIT, processDefine.getName());
return result;
} else {
putMsg(result, Status.SUCCESS);
}
if (!name.equals(processDefine.getName())) {
// check whether the new process define name exist
ProcessDefinition definition = processDefineMapper.verifyByDefineName(project.getId(), name);
if (definition != null) {
putMsg(result, Status.VERIFY_PROCESS_DEFINITION_NAME_UNIQUE_ERROR, name);
return result;
}
}
Date now = new Date();
@ -423,6 +459,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param name name
* @return true if process definition name not exists, otherwise false
*/
@Override
public Map<String, Object> verifyProcessDefinitionName(User loginUser, String projectName, String name) {
Map<String, Object> result = new HashMap<>();
@ -433,11 +470,11 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
if (resultEnum != Status.SUCCESS) {
return checkResult;
}
ProcessDefinition processDefinition = processDefineMapper.queryByDefineName(project.getId(), name);
ProcessDefinition processDefinition = processDefineMapper.verifyByDefineName(project.getId(), name);
if (processDefinition == null) {
putMsg(result, Status.SUCCESS);
} else {
putMsg(result, Status.PROCESS_INSTANCE_EXIST, name);
putMsg(result, Status.VERIFY_PROCESS_DEFINITION_NAME_UNIQUE_ERROR, name);
}
return result;
}
@ -450,6 +487,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param processDefinitionId process definition id
* @return delete result code
*/
@Override
@Transactional(rollbackFor = RuntimeException.class)
public Map<String, Object> deleteProcessDefinitionById(User loginUser, String projectName, Integer processDefinitionId) {
@ -481,9 +519,9 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
return result;
}
// check process instances is already running
List<ProcessInstance> processInstances = processInstanceService.queryByProcessDefineIdAndStatus(processDefinitionId, Constants.NOT_TERMINATED_STATES);
List<ProcessInstance> processInstances = processInstanceService.queryByProcessDefineIdAndStatus(processDefinitionId, Constants.NOT_TERMINATED_STATES);
if (CollectionUtils.isNotEmpty(processInstances)) {
putMsg(result, Status.DELETE_PROCESS_DEFINITION_BY_ID_FAIL,processInstances.size());
putMsg(result, Status.DELETE_PROCESS_DEFINITION_BY_ID_FAIL, processInstances.size());
return result;
}
@ -522,6 +560,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param releaseState release state
* @return release result code
*/
@Override
@Transactional(rollbackFor = RuntimeException.class)
public Map<String, Object> releaseProcessDefinition(User loginUser, String projectName, int id, int releaseState) {
HashMap<String, Object> result = new HashMap<>();
@ -589,6 +628,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
/**
* batch export process definition by ids
*/
@Override
public void batchExportProcessDefinitionByIds(User loginUser, String projectName, String processDefinitionIds, HttpServletResponse response) {
if (StringUtils.isEmpty(processDefinitionIds)) {
@ -665,6 +705,18 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
}
}
/**
* get export process metadata string
*
* @param processDefinitionId process definition id
* @param processDefinition process definition
* @return export process metadata string
*/
public String exportProcessMetaDataStr(Integer processDefinitionId, ProcessDefinition processDefinition) {
//create workflow json file
return JSONUtils.toJsonString(exportProcessMetaData(processDefinitionId, processDefinition));
}
/**
* get export process metadata string
*
@ -747,6 +799,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param currentProjectName current project name
* @return import process
*/
@Override
@Transactional(rollbackFor = RuntimeException.class)
public Map<String, Object> importProcessDefinition(User loginUser, MultipartFile file, String currentProjectName) {
Map<String, Object> result = new HashMap<>();
@ -835,14 +888,19 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
String processDefinitionName,
String importProcessParam) {
Map<String, Object> createProcessResult = null;
createProcessResult = createProcessDefinition(loginUser
, currentProjectName,
processDefinitionName + "_import_" + DateUtils.getCurrentTimeStamp(),
importProcessParam,
processMeta.getProcessDefinitionDescription(),
processMeta.getProcessDefinitionLocations(),
processMeta.getProcessDefinitionConnects());
putMsg(result, Status.SUCCESS);
try {
createProcessResult = createProcessDefinition(loginUser
, currentProjectName,
processDefinitionName + "_import_" + DateUtils.getCurrentTimeStamp(),
importProcessParam,
processMeta.getProcessDefinitionDescription(),
processMeta.getProcessDefinitionLocations(),
processMeta.getProcessDefinitionConnects());
putMsg(result, Status.SUCCESS);
} catch (Exception e) {
logger.error("import process meta json data: {}", e.getMessage(), e);
putMsg(result, Status.IMPORT_PROCESS_DEFINE_ERROR);
}
return createProcessResult;
}
@ -913,7 +971,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
}
//recursive sub-process parameter correction map key for old process id value for new process id
Map<Integer, Integer> subProcessIdMap = new HashMap<>();
Map<Integer, Integer> subProcessIdMap = new HashMap<>(20);
List<Object> subProcessList = StreamUtils.asStream(jsonArray.elements())
.filter(elem -> checkTaskHasSubProcess(JSONUtils.parseObject(elem.toString()).path("type").asText()))
@ -1074,6 +1132,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param processDefinitionJson process definition json
* @return check result code
*/
@Override
public Map<String, Object> checkProcessNodeList(ProcessData processData, String processDefinitionJson) {
Map<String, Object> result = new HashMap<>();
@ -1125,6 +1184,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param defineId define id
* @return task node list
*/
@Override
public Map<String, Object> getTaskNodeListByDefinitionId(Integer defineId) {
Map<String, Object> result = new HashMap<>();
@ -1161,6 +1221,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param defineIdList define id list
* @return task node list
*/
@Override
public Map<String, Object> getTaskNodeListByDefinitionIdList(String defineIdList) {
Map<String, Object> result = new HashMap<>();
@ -1198,6 +1259,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @param projectId project id
* @return process definitions in the project
*/
@Override
public Map<String, Object> queryProcessDefinitionAllByProjectId(Integer projectId) {
HashMap<String, Object> result = new HashMap<>();
@ -1217,6 +1279,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
* @return tree view json data
* @throws Exception exception
*/
@Override
public Map<String, Object> viewTree(Integer processId, Integer limit) throws Exception {
Map<String, Object> result = new HashMap<>();
@ -1301,7 +1364,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
String taskJson = taskInstance.getTaskJson();
taskNode = JSONUtils.parseObject(taskJson, TaskNode.class);
subProcessId = Integer.parseInt(JSONUtils.parseObject(
taskNode.getParams()).path(CMDPARAM_SUB_PROCESS_DEFINE_ID).asText());
taskNode.getParams()).path(CMD_PARAM_SUB_PROCESS_DEFINE_ID).asText());
}
treeViewDto.getInstances().add(new Instance(taskInstance.getId(), taskInstance.getName(), taskInstance.getTaskType(), taskInstance.getState().toString()
, taskInstance.getStartTime(), taskInstance.getEndTime(), taskInstance.getHost(), DateUtils.format2Readable(endTime.getTime() - startTime.getTime()), subProcessId));
@ -1323,7 +1386,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
}
runningNodeMap.remove(nodeName);
}
if (waitingRunningNodeMap.size() == 0) {
if (waitingRunningNodeMap == null || waitingRunningNodeMap.size() == 0) {
break;
} else {
runningNodeMap.putAll(waitingRunningNodeMap);
@ -1429,6 +1492,41 @@ public class ProcessDefinitionServiceImpl extends BaseService implements
}
}
/**
* copy process definition
*
* @param loginUser login user
* @param projectName project name
* @param processId process definition id
* @return copy result code
*/
public Map<String, Object> copyProcessDefinition(User loginUser, String projectName, Integer processId) {
Map<String, Object> result = new HashMap<>(5);
Project project = projectMapper.queryByName(projectName);
Map<String, Object> checkResult = projectService.checkProjectAndAuth(loginUser, project, projectName);
Status resultStatus = (Status) checkResult.get(Constants.STATUS);
if (resultStatus != Status.SUCCESS) {
return checkResult;
}
ProcessDefinition processDefinition = processDefineMapper.selectById(processId);
if (processDefinition == null) {
putMsg(result, Status.PROCESS_DEFINE_NOT_EXIST, processId);
return result;
} else {
return createProcessDefinition(
loginUser,
projectName,
processDefinition.getName() + "_copy_" + System.currentTimeMillis(),
processDefinition.getProcessDefinitionJson(),
processDefinition.getDescription(),
processDefinition.getLocations(),
processDefinition.getConnects());
}
}
/**
* batch copy process definition
*

22
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
Unescape View File

@ -375,6 +375,28 @@ public class ProjectServiceImpl extends BaseService implements ProjectService {
return result;
}
/**
* query authorized and user create project list by user
*
* @param loginUser login user
* @return
*/
public Map<String, Object> queryProjectCreatedAndAuthorizedByUser(User loginUser) {
Map<String, Object> result = new HashMap<>();
List<Project> projects = null;
if (loginUser.getUserType() == UserType.ADMIN_USER) {
projects = projectMapper.selectList(null);
} else {
projects = projectMapper.queryProjectCreatedAndAuthorizedByUserId(loginUser.getId());
}
result.put(Constants.DATA_LIST, projects);
putMsg(result, Status.SUCCESS);
return result;
}
/**
* check whether have read permission
*

32
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TenantServiceImpl.java
Unescape View File

@ -76,7 +76,6 @@ public class TenantServiceImpl extends BaseService implements TenantService {
*
* @param loginUser login user
* @param tenantCode tenant code
* @param tenantName tenant name
* @param queueId queue id
* @param desc description
* @return create result code
@ -85,7 +84,6 @@ public class TenantServiceImpl extends BaseService implements TenantService {
@Transactional(rollbackFor = Exception.class)
public Map<String, Object> createTenant(User loginUser,
String tenantCode,
String tenantName,
int queueId,
String desc) throws Exception {
@ -113,7 +111,6 @@ public class TenantServiceImpl extends BaseService implements TenantService {
return result;
}
tenant.setTenantCode(tenantCode);
tenant.setTenantName(tenantName);
tenant.setQueueId(queueId);
tenant.setDescription(desc);
tenant.setCreateTime(now);
@ -166,13 +163,12 @@ public class TenantServiceImpl extends BaseService implements TenantService {
* @param loginUser login user
* @param id tennat id
* @param tenantCode tennat code
* @param tenantName tennat name
* @param queueId queue id
* @param desc description
* @return update result code
* @throws Exception exception
*/
public Map<String, Object> updateTenant(User loginUser, int id, String tenantCode, String tenantName, int queueId,
public Map<String, Object> updateTenant(User loginUser, int id, String tenantCode, int queueId,
String desc) throws Exception {
Map<String, Object> result = new HashMap<>(5);
@ -215,10 +211,6 @@ public class TenantServiceImpl extends BaseService implements TenantService {
tenant.setTenantCode(tenantCode);
}
if (StringUtils.isNotEmpty(tenantName)) {
tenant.setTenantName(tenantName);
}
if (queueId != 0) {
tenant.setQueueId(queueId);
}
@ -291,6 +283,26 @@ public class TenantServiceImpl extends BaseService implements TenantService {
return processInstanceMapper.queryByTenantIdAndStatus(tenant.getId(), Constants.NOT_TERMINATED_STATES);
}
/**
* query tenant list
*
* @param tenantCode tenant code
* @return tenant list
*/
public Map<String, Object> queryTenantList(String tenantCode) {
Map<String, Object> result = new HashMap<>(5);
List<Tenant> resourceList = tenantMapper.queryByTenantCode(tenantCode);
if (CollectionUtils.isNotEmpty(resourceList)) {
result.put(Constants.DATA_LIST, resourceList);
putMsg(result, Status.SUCCESS);
} else {
putMsg(result, Status.TENANT_NOT_EXIST);
}
return result;
}
/**
* query tenant list
*
@ -317,7 +329,7 @@ public class TenantServiceImpl extends BaseService implements TenantService {
public Result verifyTenantCode(String tenantCode) {
Result result = new Result();
if (checkTenantExists(tenantCode)) {
putMsg(result, Status.TENANT_NAME_EXIST, tenantCode);
putMsg(result, Status.TENANT_CODE_EXIST, tenantCode);
} else {
putMsg(result, Status.SUCCESS);
}

20
dolphinscheduler-api/src/main/resources/application-api.properties
Unescape View File

@ -30,8 +30,12 @@ spring.jackson.time-zone=GMT+8
spring.servlet.multipart.max-file-size=1024MB
spring.servlet.multipart.max-request-size=1024MB
# enable response compression
server.compression.enabled=true
server.compression.mime-types=text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml
#post content
server.jetty.max-http-post-size=5000000
server.jetty.max-http-form-post-size=5000000
spring.messages.encoding=UTF-8
@ -41,6 +45,18 @@ spring.messages.basename=i18n/messages
# Authentication types (supported types: PASSWORD)
security.authentication.type=PASSWORD
#============================================================================
# LDAP Config
# mock ldap server from https://www.forumsys.com/tutorials/integration-how-to/ldap/online-ldap-test-server/
#============================================================================
# admin userId
#security.authentication.ldap.user.admin=read-only-admin
# ldap server config
#ldap.urls=ldap://ldap.forumsys.com:389/
#ldap.base.dn=dc=example,dc=com
#ldap.username=cn=read-only-admin,dc=example,dc=com
#ldap.password=password
#ldap.user.identity.attribute=uid
#ldap.user.email.attribute=mail

2
dolphinscheduler-api/src/main/resources/i18n/messages.properties
Unescape View File

@ -122,7 +122,6 @@ VERIFY_QUEUE_NOTES=verify queue
TENANT_TAG=tenant related operation
CREATE_TENANT_NOTES=create tenant
TENANT_CODE=tenant code
TENANT_NAME=tenant name
QUEUE_NAME=queue name
PASSWORD=password
DATA_SOURCE_OTHER=jdbc connection params, format:{"key1":"value1",...}
@ -259,6 +258,7 @@ QUERY_ALERT_GROUP_LIST_PAGING_NOTES=query alert group list paging
EXPORT_PROCESS_DEFINITION_BY_ID_NOTES=export process definition by id
BATCH_EXPORT_PROCESS_DEFINITION_BY_IDS_NOTES= batch export process definition by ids
QUERY_USER_CREATED_PROJECT_NOTES= query user created project
QUERY_AUTHORIZED_AND_USER_CREATED_PROJECT_NOTES= query authorized and user created project
COPY_PROCESS_DEFINITION_NOTES= copy process definition notes
MOVE_PROCESS_DEFINITION_NOTES= move process definition notes
TARGET_PROJECT_ID= target project id

2
dolphinscheduler-api/src/main/resources/i18n/messages_en_US.properties
Unescape View File

@ -122,7 +122,6 @@ VERIFY_QUEUE_NOTES=verify queue
TENANT_TAG=tenant related operation
CREATE_TENANT_NOTES=create tenant
TENANT_CODE=tenant code
TENANT_NAME=tenant name
QUEUE_NAME=queue name
PASSWORD=password
DATA_SOURCE_OTHER=jdbc connection params, format:{"key1":"value1",...}
@ -259,6 +258,7 @@ QUERY_ALERT_GROUP_LIST_PAGING_NOTES=query alert group list paging
EXPORT_PROCESS_DEFINITION_BY_ID_NOTES=export process definition by id
BATCH_EXPORT_PROCESS_DEFINITION_BY_IDS_NOTES= batch export process definition by ids
QUERY_USER_CREATED_PROJECT_NOTES= query user created project
QUERY_AUTHORIZED_AND_USER_CREATED_PROJECT_NOTES= query authorized and user created project
COPY_PROCESS_DEFINITION_NOTES= copy process definition notes
MOVE_PROCESS_DEFINITION_NOTES= move process definition notes
TARGET_PROJECT_ID= target project id

489
dolphinscheduler-api/src/main/resources/i18n/messages_zh_CN.properties
Unescape View File

@ -15,253 +15,248 @@
# limitations under the License.
#
QUERY_SCHEDULE_LIST_NOTES=\u67E5\u8BE2\u5B9A\u65F6\u5217\u8868
PROCESS_INSTANCE_EXECUTOR_TAG=\u6D41\u7A0B\u5B9E\u4F8B\u6267\u884C\u76F8\u5173\u64CD\u4F5C
RUN_PROCESS_INSTANCE_NOTES=\u8FD0\u884C\u6D41\u7A0B\u5B9E\u4F8B
START_NODE_LIST=\u5F00\u59CB\u8282\u70B9\u5217\u8868(\u8282\u70B9name)
TASK_DEPEND_TYPE=\u4EFB\u52A1\u4F9D\u8D56\u7C7B\u578B
COMMAND_TYPE=\u6307\u4EE4\u7C7B\u578B
RUN_MODE=\u8FD0\u884C\u6A21\u5F0F
TIMEOUT=\u8D85\u65F6\u65F6\u95F4
EXECUTE_ACTION_TO_PROCESS_INSTANCE_NOTES=\u6267\u884C\u6D41\u7A0B\u5B9E\u4F8B\u7684\u5404\u79CD\u64CD\u4F5C(\u6682\u505C\u3001\u505C\u6B62\u3001\u91CD\u8DD1\u3001\u6062\u590D\u7B49)
EXECUTE_TYPE=\u6267\u884C\u7C7B\u578B
START_CHECK_PROCESS_DEFINITION_NOTES=\u68C0\u67E5\u6D41\u7A0B\u5B9A\u4E49
DESC=\u5907\u6CE8(\u63CF\u8FF0)
GROUP_NAME=\u7EC4\u540D\u79F0
GROUP_TYPE=\u7EC4\u7C7B\u578B
QUERY_ALERT_GROUP_LIST_NOTES=\u544A\u8B66\u7EC4\u5217\u8868\
QUERY_SCHEDULE_LIST_NOTES=查询定时列表
PROCESS_INSTANCE_EXECUTOR_TAG=流程实例执行相关操作
RUN_PROCESS_INSTANCE_NOTES=运行流程实例
START_NODE_LIST=开始节点列表(节点name)
TASK_DEPEND_TYPE=任务依赖类型
COMMAND_TYPE=指令类型
RUN_MODE=运行模式
TIMEOUT=超时时间
EXECUTE_ACTION_TO_PROCESS_INSTANCE_NOTES=执行流程实例的各种操作(暂停、停止、重跑、恢复等)
EXECUTE_TYPE=执行类型
START_CHECK_PROCESS_DEFINITION_NOTES=检查流程定义
DESC=备注(描述)
GROUP_NAME=组名称
GROUP_TYPE=组类型
QUERY_ALERT_GROUP_LIST_NOTES=告警组列表\
UPDATE_ALERT_GROUP_NOTES=\u7F16\u8F91(\u66F4\u65B0)\u544A\u8B66\u7EC4
DELETE_ALERT_GROUP_BY_ID_NOTES=\u5220\u9664\u544A\u8B66\u7EC4\u901A\u8FC7ID
VERIFY_ALERT_GROUP_NAME_NOTES=\u68C0\u67E5\u544A\u8B66\u7EC4\u662F\u5426\u5B58\u5728
GRANT_ALERT_GROUP_NOTES=\u6388\u6743\u544A\u8B66\u7EC4
USER_IDS=\u7528\u6237ID\u5217\u8868
ALERT_GROUP_TAG=\u544A\u8B66\u7EC4\u76F8\u5173\u64CD\u4F5C
ALERT_PLUGIN_INSTANCE_TAG=\u544A\u8B66\u7EC4\u548C\u544A\u8B66\u63D2\u4EF6\u76F8\u5173\u64CD\u4F5C
UPDATE_ALERT_PLUGIN_INSTANCE_NOTES=\u66F4\u65B0\u544A\u8B66\u7EC4\u63D2\u4EF6\u5B9E\u4F8B
CREATE_ALERT_PLUGIN_INSTANCE_NOTES=\u521B\u5EFA\u544A\u8B66\u7EC4\u63D2\u4EF6\u5B9E\u4F8B
DELETE_ALERT_PLUGIN_INSTANCE_NOTES=\u5220\u9664\u544A\u8B66\u7EC4\u63D2\u4EF6\u5B9E\u4F8B
GET_ALERT_PLUGIN_INSTANCE_NOTES=\u83B7\u53D6\u544A\u8B66\u7EC4\u63D2\u4EF6\u5B9E\u4F8B
WORKER_GROUP_TAG=Worker\u5206\u7EC4\u7BA1\u7406
SAVE_WORKER_GROUP_NOTES=\u521B\u5EFAWorker\u5206\u7EC4\
UPDATE_ALERT_GROUP_NOTES=编辑(更新)告警组
DELETE_ALERT_GROUP_BY_ID_NOTES=删除告警组通过ID
VERIFY_ALERT_GROUP_NAME_NOTES=检查告警组是否存在
GRANT_ALERT_GROUP_NOTES=授权告警组
USER_IDS=用户ID列表
ALERT_GROUP_TAG=告警组相关操作
WORKER_GROUP_TAG=Worker分组管理
SAVE_WORKER_GROUP_NOTES=创建Worker分组\
WORKER_GROUP_NAME=Worker\u5206\u7EC4\u540D\u79F0
WORKER_IP_LIST=Worker ip\u5217\u8868\uFF0C\u6CE8\u610F\uFF1A\u591A\u4E2AIP\u5730\u5740\u4EE5\u9017\u53F7\u5206\u5272\
WORKER_GROUP_NAME=Worker分组名称
WORKER_IP_LIST=Worker ip列表，注意：多个IP地址以逗号分割\
QUERY_WORKER_GROUP_PAGING_NOTES=Worker\u5206\u7EC4\u7BA1\u7406
QUERY_WORKER_GROUP_LIST_NOTES=\u67E5\u8BE2worker group\u5206\u7EC4
DELETE_WORKER_GROUP_BY_ID_NOTES=\u5220\u9664worker group\u901A\u8FC7ID
DATA_ANALYSIS_TAG=\u4EFB\u52A1\u72B6\u6001\u5206\u6790\u76F8\u5173\u64CD\u4F5C
COUNT_TASK_STATE_NOTES=\u4EFB\u52A1\u72B6\u6001\u7EDF\u8BA1
COUNT_PROCESS_INSTANCE_NOTES=\u7EDF\u8BA1\u6D41\u7A0B\u5B9E\u4F8B\u72B6\u6001
COUNT_PROCESS_DEFINITION_BY_USER_NOTES=\u7EDF\u8BA1\u7528\u6237\u521B\u5EFA\u7684\u6D41\u7A0B\u5B9A\u4E49
COUNT_COMMAND_STATE_NOTES=\u7EDF\u8BA1\u547D\u4EE4\u72B6\u6001
COUNT_QUEUE_STATE_NOTES=\u7EDF\u8BA1\u961F\u5217\u91CC\u4EFB\u52A1\u72B6\u6001
ACCESS_TOKEN_TAG=access token\u76F8\u5173\u64CD\u4F5C\uFF0C\u9700\u8981\u5148\u767B\u5F55
MONITOR_TAG=\u76D1\u63A7\u76F8\u5173\u64CD\u4F5C
MASTER_LIST_NOTES=master\u670D\u52A1\u5217\u8868
WORKER_LIST_NOTES=worker\u670D\u52A1\u5217\u8868
QUERY_DATABASE_STATE_NOTES=\u67E5\u8BE2\u6570\u636E\u5E93\u72B6\u6001
QUERY_ZOOKEEPER_STATE_NOTES=\u67E5\u8BE2Zookeeper\u72B6\u6001
TASK_STATE=\u4EFB\u52A1\u5B9E\u4F8B\u72B6\u6001
SOURCE_TABLE=\u6E90\u8868
DEST_TABLE=\u76EE\u6807\u8868
TASK_DATE=\u4EFB\u52A1\u65F6\u95F4
QUERY_HISTORY_TASK_RECORD_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u5386\u53F2\u4EFB\u52A1\u8BB0\u5F55\u5217\u8868
DATA_SOURCE_TAG=\u6570\u636E\u6E90\u76F8\u5173\u64CD\u4F5C
CREATE_DATA_SOURCE_NOTES=\u521B\u5EFA\u6570\u636E\u6E90
DATA_SOURCE_NAME=\u6570\u636E\u6E90\u540D\u79F0
DATA_SOURCE_NOTE=\u6570\u636E\u6E90\u63CF\u8FF0
DB_TYPE=\u6570\u636E\u6E90\u7C7B\u578B
DATA_SOURCE_HOST=IP\u4E3B\u673A\u540D
DATA_SOURCE_PORT=\u6570\u636E\u6E90\u7AEF\u53E3
DATABASE_NAME=\u6570\u636E\u5E93\u540D
QUEUE_TAG=\u961F\u5217\u76F8\u5173\u64CD\u4F5C
QUERY_QUEUE_LIST_NOTES=\u67E5\u8BE2\u961F\u5217\u5217\u8868
QUERY_QUEUE_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u961F\u5217\u5217\u8868
CREATE_QUEUE_NOTES=\u521B\u5EFA\u961F\u5217
YARN_QUEUE_NAME=hadoop yarn\u961F\u5217\u540D
QUEUE_ID=\u961F\u5217ID
TENANT_DESC=\u79DF\u6237\u63CF\u8FF0
QUERY_TENANT_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u79DF\u6237\u5217\u8868
QUERY_TENANT_LIST_NOTES=\u67E5\u8BE2\u79DF\u6237\u5217\u8868
UPDATE_TENANT_NOTES=\u66F4\u65B0\u79DF\u6237
DELETE_TENANT_NOTES=\u5220\u9664\u79DF\u6237
RESOURCES_TAG=\u8D44\u6E90\u4E2D\u5FC3\u76F8\u5173\u64CD\u4F5C
CREATE_RESOURCE_NOTES=\u521B\u5EFA\u8D44\u6E90
RESOURCE_TYPE=\u8D44\u6E90\u6587\u4EF6\u7C7B\u578B
RESOURCE_NAME=\u8D44\u6E90\u6587\u4EF6\u540D\u79F0
RESOURCE_DESC=\u8D44\u6E90\u6587\u4EF6\u63CF\u8FF0
RESOURCE_FILE=\u8D44\u6E90\u6587\u4EF6
RESOURCE_ID=\u8D44\u6E90ID
QUERY_RESOURCE_LIST_NOTES=\u67E5\u8BE2\u8D44\u6E90\u5217\u8868
DELETE_RESOURCE_BY_ID_NOTES=\u5220\u9664\u8D44\u6E90\u901A\u8FC7ID
VIEW_RESOURCE_BY_ID_NOTES=\u6D4F\u89C8\u8D44\u6E90\u901A\u901A\u8FC7ID
ONLINE_CREATE_RESOURCE_NOTES=\u5728\u7EBF\u521B\u5EFA\u8D44\u6E90
SUFFIX=\u8D44\u6E90\u6587\u4EF6\u540E\u7F00
CONTENT=\u8D44\u6E90\u6587\u4EF6\u5185\u5BB9
UPDATE_RESOURCE_NOTES=\u5728\u7EBF\u66F4\u65B0\u8D44\u6E90\u6587\u4EF6
DOWNLOAD_RESOURCE_NOTES=\u4E0B\u8F7D\u8D44\u6E90\u6587\u4EF6
CREATE_UDF_FUNCTION_NOTES=\u521B\u5EFAUDF\u51FD\u6570
UDF_TYPE=UDF\u7C7B\u578B
FUNC_NAME=\u51FD\u6570\u540D\u79F0
CLASS_NAME=\u5305\u540D\u7C7B\u540D
ARG_TYPES=\u53C2\u6570
UDF_DESC=udf\u63CF\u8FF0\uFF0C\u4F7F\u7528\u8BF4\u660E
VIEW_UDF_FUNCTION_NOTES=\u67E5\u770Budf\u51FD\u6570
UPDATE_UDF_FUNCTION_NOTES=\u66F4\u65B0udf\u51FD\u6570
QUERY_UDF_FUNCTION_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2udf\u51FD\u6570\u5217\u8868
VERIFY_UDF_FUNCTION_NAME_NOTES=\u9A8C\u8BC1udf\u51FD\u6570\u540D
DELETE_UDF_FUNCTION_NOTES=\u5220\u9664UDF\u51FD\u6570
AUTHORIZED_FILE_NOTES=\u6388\u6743\u6587\u4EF6
UNAUTHORIZED_FILE_NOTES=\u53D6\u6D88\u6388\u6743\u6587\u4EF6
AUTHORIZED_UDF_FUNC_NOTES=\u6388\u6743udf\u51FD\u6570
UNAUTHORIZED_UDF_FUNC_NOTES=\u53D6\u6D88udf\u51FD\u6570\u6388\u6743
VERIFY_QUEUE_NOTES=\u9A8C\u8BC1\u961F\u5217
TENANT_TAG=\u79DF\u6237\u76F8\u5173\u64CD\u4F5C
CREATE_TENANT_NOTES=\u521B\u5EFA\u79DF\u6237
TENANT_CODE=\u79DF\u6237\u7F16\u7801
TENANT_NAME=\u79DF\u6237\u540D\u79F0
QUEUE_NAME=\u961F\u5217\u540D
PASSWORD=\u5BC6\u7801
DATA_SOURCE_OTHER=jdbc\u8FDE\u63A5\u53C2\u6570\uFF0C\u683C\u5F0F\u4E3A:{"key1":"value1",...}
PROJECT_TAG=\u9879\u76EE\u76F8\u5173\u64CD\u4F5C
CREATE_PROJECT_NOTES=\u521B\u5EFA\u9879\u76EE
PROJECT_DESC=\u9879\u76EE\u63CF\u8FF0
UPDATE_PROJECT_NOTES=\u66F4\u65B0\u9879\u76EE
PROJECT_ID=\u9879\u76EEID
QUERY_PROJECT_BY_ID_NOTES=\u901A\u8FC7\u9879\u76EEID\u67E5\u8BE2\u9879\u76EE\u4FE1\u606F
QUERY_PROJECT_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u9879\u76EE\u5217\u8868
QUERY_ALL_PROJECT_LIST_NOTES=\u67E5\u8BE2\u6240\u6709\u9879\u76EE
DELETE_PROJECT_BY_ID_NOTES=\u5220\u9664\u9879\u76EE\u901A\u8FC7ID
QUERY_UNAUTHORIZED_PROJECT_NOTES=\u67E5\u8BE2\u672A\u6388\u6743\u7684\u9879\u76EE
QUERY_AUTHORIZED_PROJECT_NOTES=\u67E5\u8BE2\u6388\u6743\u9879\u76EE
TASK_RECORD_TAG=\u4EFB\u52A1\u8BB0\u5F55\u76F8\u5173\u64CD\u4F5C
QUERY_TASK_RECORD_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u4EFB\u52A1\u8BB0\u5F55\u5217\u8868
CREATE_TOKEN_NOTES=\u521B\u5EFAtoken\uFF0C\u6CE8\u610F\u9700\u8981\u5148\u767B\u5F55
QUERY_ACCESS_TOKEN_LIST_NOTES=\u5206\u9875\u67E5\u8BE2access token\u5217\u8868
SCHEDULE=\u5B9A\u65F6
WARNING_TYPE=\u53D1\u9001\u7B56\u7565
WARNING_GROUP_ID=\u53D1\u9001\u7EC4ID
FAILURE_STRATEGY=\u5931\u8D25\u7B56\u7565
RECEIVERS=\u6536\u4EF6\u4EBA
RECEIVERS_CC=\u6536\u4EF6\u4EBA(\u6284\u9001)
WORKER_GROUP_ID=Worker Server\u5206\u7EC4ID
PROCESS_INSTANCE_PRIORITY=\u6D41\u7A0B\u5B9E\u4F8B\u4F18\u5148\u7EA7
UPDATE_SCHEDULE_NOTES=\u66F4\u65B0\u5B9A\u65F6
SCHEDULE_ID=\u5B9A\u65F6ID
ONLINE_SCHEDULE_NOTES=\u5B9A\u65F6\u4E0A\u7EBF
OFFLINE_SCHEDULE_NOTES=\u5B9A\u65F6\u4E0B\u7EBF
QUERY_SCHEDULE_NOTES=\u67E5\u8BE2\u5B9A\u65F6
QUERY_SCHEDULE_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u5B9A\u65F6
LOGIN_TAG=\u7528\u6237\u767B\u5F55\u76F8\u5173\u64CD\u4F5C
USER_NAME=\u7528\u6237\u540D
PROJECT_NAME=\u9879\u76EE\u540D\u79F0
CREATE_PROCESS_DEFINITION_NOTES=\u521B\u5EFA\u6D41\u7A0B\u5B9A\u4E49
PROCESS_DEFINITION_NAME=\u6D41\u7A0B\u5B9A\u4E49\u540D\u79F0
PROCESS_DEFINITION_JSON=\u6D41\u7A0B\u5B9A\u4E49\u8BE6\u7EC6\u4FE1\u606F(json\u683C\u5F0F)
PROCESS_DEFINITION_LOCATIONS=\u6D41\u7A0B\u5B9A\u4E49\u8282\u70B9\u5750\u6807\u4F4D\u7F6E\u4FE1\u606F(json\u683C\u5F0F)
PROCESS_INSTANCE_LOCATIONS=\u6D41\u7A0B\u5B9E\u4F8B\u8282\u70B9\u5750\u6807\u4F4D\u7F6E\u4FE1\u606F(json\u683C\u5F0F)
PROCESS_DEFINITION_CONNECTS=\u6D41\u7A0B\u5B9A\u4E49\u8282\u70B9\u56FE\u6807\u8FDE\u63A5\u4FE1\u606F(json\u683C\u5F0F)
PROCESS_INSTANCE_CONNECTS=\u6D41\u7A0B\u5B9E\u4F8B\u8282\u70B9\u56FE\u6807\u8FDE\u63A5\u4FE1\u606F(json\u683C\u5F0F)
PROCESS_DEFINITION_DESC=\u6D41\u7A0B\u5B9A\u4E49\u63CF\u8FF0\u4FE1\u606F
PROCESS_DEFINITION_TAG=\u6D41\u7A0B\u5B9A\u4E49\u76F8\u5173\u64CD\u4F5C
SIGNOUT_NOTES=\u9000\u51FA\u767B\u5F55
USER_PASSWORD=\u7528\u6237\u5BC6\u7801
UPDATE_PROCESS_INSTANCE_NOTES=\u66F4\u65B0\u6D41\u7A0B\u5B9E\u4F8B
QUERY_PROCESS_INSTANCE_LIST_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5B9E\u4F8B\u5217\u8868
VERIFY_PROCESS_DEFINITION_NAME_NOTES=\u9A8C\u8BC1\u6D41\u7A0B\u5B9A\u4E49\u540D\u5B57
LOGIN_NOTES=\u7528\u6237\u767B\u5F55
UPDATE_PROCESS_DEFINITION_NOTES=\u66F4\u65B0\u6D41\u7A0B\u5B9A\u4E49
PROCESS_DEFINITION_ID=\u6D41\u7A0B\u5B9A\u4E49ID
RELEASE_PROCESS_DEFINITION_NOTES=\u53D1\u5E03\u6D41\u7A0B\u5B9A\u4E49
QUERY_PROCESS_DEFINITION_BY_ID_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5B9A\u4E49\u901A\u8FC7\u6D41\u7A0B\u5B9A\u4E49ID
QUERY_PROCESS_DEFINITION_LIST_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5B9A\u4E49\u5217\u8868
QUERY_PROCESS_DEFINITION_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u6D41\u7A0B\u5B9A\u4E49\u5217\u8868
QUERY_ALL_DEFINITION_LIST_NOTES=\u67E5\u8BE2\u6240\u6709\u6D41\u7A0B\u5B9A\u4E49
PAGE_NO=\u9875\u7801\u53F7
PROCESS_INSTANCE_ID=\u6D41\u7A0B\u5B9E\u4F8BID
PROCESS_INSTANCE_IDS=\u6D41\u7A0B\u5B9E\u4F8BID\u96C6\u5408
PROCESS_INSTANCE_JSON=\u6D41\u7A0B\u5B9E\u4F8B\u4FE1\u606F(json\u683C\u5F0F)
SCHEDULE_TIME=\u5B9A\u65F6\u65F6\u95F4
SYNC_DEFINE=\u66F4\u65B0\u6D41\u7A0B\u5B9E\u4F8B\u7684\u4FE1\u606F\u662F\u5426\u540C\u6B65\u5230\u6D41\u7A0B\u5B9A\u4E49
RECOVERY_PROCESS_INSTANCE_FLAG=\u662F\u5426\u6062\u590D\u6D41\u7A0B\u5B9E\u4F8B
SEARCH_VAL=\u641C\u7D22\u503C
USER_ID=\u7528\u6237ID
PAGE_SIZE=\u9875\u5927\u5C0F
LIMIT=\u663E\u793A\u591A\u5C11\u6761
VIEW_TREE_NOTES=\u6811\u72B6\u56FE
GET_NODE_LIST_BY_DEFINITION_ID_NOTES=\u83B7\u5F97\u4EFB\u52A1\u8282\u70B9\u5217\u8868\u901A\u8FC7\u6D41\u7A0B\u5B9A\u4E49ID
PROCESS_DEFINITION_ID_LIST=\u6D41\u7A0B\u5B9A\u4E49id\u5217\u8868
QUERY_PROCESS_DEFINITION_All_BY_PROJECT_ID_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5B9A\u4E49\u901A\u8FC7\u9879\u76EEID
BATCH_DELETE_PROCESS_DEFINITION_BY_IDS_NOTES=\u6279\u91CF\u5220\u9664\u6D41\u7A0B\u5B9A\u4E49\u901A\u8FC7\u6D41\u7A0B\u5B9A\u4E49ID\u96C6\u5408
DELETE_PROCESS_DEFINITION_BY_ID_NOTES=\u5220\u9664\u6D41\u7A0B\u5B9A\u4E49\u901A\u8FC7\u6D41\u7A0B\u5B9A\u4E49ID
QUERY_PROCESS_INSTANCE_BY_ID_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5B9E\u4F8B\u901A\u8FC7\u6D41\u7A0B\u5B9E\u4F8BID
DELETE_PROCESS_INSTANCE_BY_ID_NOTES=\u5220\u9664\u6D41\u7A0B\u5B9E\u4F8B\u901A\u8FC7\u6D41\u7A0B\u5B9E\u4F8BID
TASK_ID=\u4EFB\u52A1\u5B9E\u4F8BID
SKIP_LINE_NUM=\u5FFD\u7565\u884C\u6570
QUERY_TASK_INSTANCE_LOG_NOTES=\u67E5\u8BE2\u4EFB\u52A1\u5B9E\u4F8B\u65E5\u5FD7
DOWNLOAD_TASK_INSTANCE_LOG_NOTES=\u4E0B\u8F7D\u4EFB\u52A1\u5B9E\u4F8B\u65E5\u5FD7
USERS_TAG=\u7528\u6237\u76F8\u5173\u64CD\u4F5C
SCHEDULER_TAG=\u5B9A\u65F6\u76F8\u5173\u64CD\u4F5C
CREATE_SCHEDULE_NOTES=\u521B\u5EFA\u5B9A\u65F6
CREATE_USER_NOTES=\u521B\u5EFA\u7528\u6237
TENANT_ID=\u79DF\u6237ID
QUEUE=\u4F7F\u7528\u7684\u961F\u5217
EMAIL=\u90AE\u7BB1
PHONE=\u624B\u673A\u53F7
QUERY_USER_LIST_NOTES=\u67E5\u8BE2\u7528\u6237\u5217\u8868
UPDATE_USER_NOTES=\u66F4\u65B0\u7528\u6237
DELETE_USER_BY_ID_NOTES=\u5220\u9664\u7528\u6237\u901A\u8FC7ID
GRANT_PROJECT_NOTES=\u6388\u6743\u9879\u76EE
PROJECT_IDS=\u9879\u76EEIDS(\u5B57\u7B26\u4E32\u683C\u5F0F\uFF0C\u591A\u4E2A\u9879\u76EE\u4EE5","\u5206\u5272)
GRANT_RESOURCE_NOTES=\u6388\u6743\u8D44\u6E90\u6587\u4EF6
RESOURCE_IDS=\u8D44\u6E90ID\u5217\u8868(\u5B57\u7B26\u4E32\u683C\u5F0F\uFF0C\u591A\u4E2A\u8D44\u6E90ID\u4EE5","\u5206\u5272)
GET_USER_INFO_NOTES=\u83B7\u53D6\u7528\u6237\u4FE1\u606F
LIST_USER_NOTES=\u7528\u6237\u5217\u8868
VERIFY_USER_NAME_NOTES=\u9A8C\u8BC1\u7528\u6237\u540D
UNAUTHORIZED_USER_NOTES=\u53D6\u6D88\u6388\u6743
ALERT_GROUP_ID=\u62A5\u8B66\u7EC4ID
AUTHORIZED_USER_NOTES=\u6388\u6743\u7528\u6237
GRANT_UDF_FUNC_NOTES=\u6388\u6743udf\u51FD\u6570
UDF_IDS=udf\u51FD\u6570id\u5217\u8868(\u5B57\u7B26\u4E32\u683C\u5F0F\uFF0C\u591A\u4E2Audf\u51FD\u6570ID\u4EE5","\u5206\u5272)
GRANT_DATASOURCE_NOTES=\u6388\u6743\u6570\u636E\u6E90
DATASOURCE_IDS=\u6570\u636E\u6E90ID\u5217\u8868(\u5B57\u7B26\u4E32\u683C\u5F0F\uFF0C\u591A\u4E2A\u6570\u636E\u6E90ID\u4EE5","\u5206\u5272)
QUERY_SUBPROCESS_INSTANCE_BY_TASK_ID_NOTES=\u67E5\u8BE2\u5B50\u6D41\u7A0B\u5B9E\u4F8B\u901A\u8FC7\u4EFB\u52A1\u5B9E\u4F8BID
QUERY_PARENT_PROCESS_INSTANCE_BY_SUB_PROCESS_INSTANCE_ID_NOTES=\u67E5\u8BE2\u7236\u6D41\u7A0B\u5B9E\u4F8B\u4FE1\u606F\u901A\u8FC7\u5B50\u6D41\u7A0B\u5B9E\u4F8BID
QUERY_PROCESS_INSTANCE_GLOBAL_VARIABLES_AND_LOCAL_VARIABLES_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5B9E\u4F8B\u5168\u5C40\u53D8\u91CF\u548C\u5C40\u90E8\u53D8\u91CF
VIEW_GANTT_NOTES=\u6D4F\u89C8Gantt\u56FE
SUB_PROCESS_INSTANCE_ID=\u5B50\u6D41\u7A0B\u662F\u54A7ID
TASK_NAME=\u4EFB\u52A1\u5B9E\u4F8B\u540D
TASK_INSTANCE_TAG=\u4EFB\u52A1\u5B9E\u4F8B\u76F8\u5173\u64CD\u4F5C
LOGGER_TAG=\u65E5\u5FD7\u76F8\u5173\u64CD\u4F5C
PROCESS_INSTANCE_TAG=\u6D41\u7A0B\u5B9E\u4F8B\u76F8\u5173\u64CD\u4F5C
EXECUTION_STATUS=\u5DE5\u4F5C\u6D41\u548C\u4EFB\u52A1\u8282\u70B9\u7684\u8FD0\u884C\u72B6\u6001
HOST=\u8FD0\u884C\u4EFB\u52A1\u7684\u4E3B\u673AIP\u5730\u5740
START_DATE=\u5F00\u59CB\u65F6\u95F4
END_DATE=\u7ED3\u675F\u65F6\u95F4
QUERY_TASK_LIST_BY_PROCESS_INSTANCE_ID_NOTES=\u901A\u8FC7\u6D41\u7A0B\u5B9E\u4F8BID\u67E5\u8BE2\u4EFB\u52A1\u5217\u8868
UPDATE_DATA_SOURCE_NOTES=\u66F4\u65B0\u6570\u636E\u6E90
DATA_SOURCE_ID=\u6570\u636E\u6E90ID
QUERY_DATA_SOURCE_NOTES=\u67E5\u8BE2\u6570\u636E\u6E90\u901A\u8FC7ID
QUERY_DATA_SOURCE_LIST_BY_TYPE_NOTES=\u67E5\u8BE2\u6570\u636E\u6E90\u5217\u8868\u901A\u8FC7\u6570\u636E\u6E90\u7C7B\u578B
QUERY_DATA_SOURCE_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u6570\u636E\u6E90\u5217\u8868
CONNECT_DATA_SOURCE_NOTES=\u8FDE\u63A5\u6570\u636E\u6E90
CONNECT_DATA_SOURCE_TEST_NOTES=\u8FDE\u63A5\u6570\u636E\u6E90\u6D4B\u8BD5
DELETE_DATA_SOURCE_NOTES=\u5220\u9664\u6570\u636E\u6E90
VERIFY_DATA_SOURCE_NOTES=\u9A8C\u8BC1\u6570\u636E\u6E90
UNAUTHORIZED_DATA_SOURCE_NOTES=\u672A\u6388\u6743\u7684\u6570\u636E\u6E90
AUTHORIZED_DATA_SOURCE_NOTES=\u6388\u6743\u7684\u6570\u636E\u6E90
DELETE_SCHEDULER_BY_ID_NOTES=\u6839\u636E\u5B9A\u65F6id\u5220\u9664\u5B9A\u65F6\u6570\u636E
QUERY_ALERT_GROUP_LIST_PAGING_NOTES=\u5206\u9875\u67E5\u8BE2\u544A\u8B66\u7EC4\u5217\u8868
EXPORT_PROCESS_DEFINITION_BY_ID_NOTES=\u901A\u8FC7\u5DE5\u4F5C\u6D41ID\u5BFC\u51FA\u5DE5\u4F5C\u6D41\u5B9A\u4E49
BATCH_EXPORT_PROCESS_DEFINITION_BY_IDS_NOTES=\u6279\u91CF\u5BFC\u51FA\u5DE5\u4F5C\u6D41\u5B9A\u4E49
QUERY_USER_CREATED_PROJECT_NOTES= \u67E5\u8BE2\u7528\u6237\u521B\u5EFA\u7684\u9879\u76EE
COPY_PROCESS_DEFINITION_NOTES= \u590D\u5236\u5DE5\u4F5C\u6D41\u5B9A\u4E49
MOVE_PROCESS_DEFINITION_NOTES= \u79FB\u52A8\u5DE5\u4F5C\u6D41\u5B9A\u4E49
TARGET_PROJECT_ID= \u76EE\u6807\u9879\u76EEID
IS_COPY = \u662F\u5426\u590D\u5236
DELETE_PROCESS_DEFINITION_VERSION_NOTES=\u5220\u9664\u6D41\u7A0B\u5386\u53F2\u7248\u672C
QUERY_PROCESS_DEFINITION_VERSIONS_NOTES=\u67E5\u8BE2\u6D41\u7A0B\u5386\u53F2\u7248\u672C\u4FE1\u606F
SWITCH_PROCESS_DEFINITION_VERSION_NOTES=\u5207\u6362\u6D41\u7A0B\u7248\u672C
VERSION=\u7248\u672C\u53F7
QUERY_WORKER_GROUP_PAGING_NOTES=Worker分组管理
QUERY_WORKER_GROUP_LIST_NOTES=查询worker group分组
DELETE_WORKER_GROUP_BY_ID_NOTES=删除worker group通过ID
DATA_ANALYSIS_TAG=任务状态分析相关操作
COUNT_TASK_STATE_NOTES=任务状态统计
COUNT_PROCESS_INSTANCE_NOTES=统计流程实例状态
COUNT_PROCESS_DEFINITION_BY_USER_NOTES=统计用户创建的流程定义
COUNT_COMMAND_STATE_NOTES=统计命令状态
COUNT_QUEUE_STATE_NOTES=统计队列里任务状态
ACCESS_TOKEN_TAG=access token相关操作，需要先登录
MONITOR_TAG=监控相关操作
MASTER_LIST_NOTES=master服务列表
WORKER_LIST_NOTES=worker服务列表
QUERY_DATABASE_STATE_NOTES=查询数据库状态
QUERY_ZOOKEEPER_STATE_NOTES=查询Zookeeper状态
TASK_STATE=任务实例状态
SOURCE_TABLE=源表
DEST_TABLE=目标表
TASK_DATE=任务时间
QUERY_HISTORY_TASK_RECORD_LIST_PAGING_NOTES=分页查询历史任务记录列表
DATA_SOURCE_TAG=数据源相关操作
CREATE_DATA_SOURCE_NOTES=创建数据源
DATA_SOURCE_NAME=数据源名称
DATA_SOURCE_NOTE=数据源描述
DB_TYPE=数据源类型
DATA_SOURCE_HOST=IP主机名
DATA_SOURCE_PORT=数据源端口
DATABASE_NAME=数据库名
QUEUE_TAG=队列相关操作
QUERY_QUEUE_LIST_NOTES=查询队列列表
QUERY_QUEUE_LIST_PAGING_NOTES=分页查询队列列表
CREATE_QUEUE_NOTES=创建队列
YARN_QUEUE_NAME=hadoop yarn队列名
QUEUE_ID=队列ID
TENANT_DESC=租户描述
QUERY_TENANT_LIST_PAGING_NOTES=分页查询租户列表
QUERY_TENANT_LIST_NOTES=查询租户列表
UPDATE_TENANT_NOTES=更新租户
DELETE_TENANT_NOTES=删除租户
RESOURCES_TAG=资源中心相关操作
CREATE_RESOURCE_NOTES=创建资源
RESOURCE_TYPE=资源文件类型
RESOURCE_NAME=资源文件名称
RESOURCE_DESC=资源文件描述
RESOURCE_FILE=资源文件
RESOURCE_ID=资源ID
QUERY_RESOURCE_LIST_NOTES=查询资源列表
DELETE_RESOURCE_BY_ID_NOTES=删除资源通过ID
VIEW_RESOURCE_BY_ID_NOTES=浏览资源通通过ID
ONLINE_CREATE_RESOURCE_NOTES=在线创建资源
SUFFIX=资源文件后缀
CONTENT=资源文件内容
UPDATE_RESOURCE_NOTES=在线更新资源文件
DOWNLOAD_RESOURCE_NOTES=下载资源文件
CREATE_UDF_FUNCTION_NOTES=创建UDF函数
UDF_TYPE=UDF类型
FUNC_NAME=函数名称
CLASS_NAME=包名类名
ARG_TYPES=参数
UDF_DESC=udf描述，使用说明
VIEW_UDF_FUNCTION_NOTES=查看udf函数
UPDATE_UDF_FUNCTION_NOTES=更新udf函数
QUERY_UDF_FUNCTION_LIST_PAGING_NOTES=分页查询udf函数列表
VERIFY_UDF_FUNCTION_NAME_NOTES=验证udf函数名
DELETE_UDF_FUNCTION_NOTES=删除UDF函数
AUTHORIZED_FILE_NOTES=授权文件
UNAUTHORIZED_FILE_NOTES=取消授权文件
AUTHORIZED_UDF_FUNC_NOTES=授权udf函数
UNAUTHORIZED_UDF_FUNC_NOTES=取消udf函数授权
VERIFY_QUEUE_NOTES=验证队列
TENANT_TAG=租户相关操作
CREATE_TENANT_NOTES=创建租户
TENANT_CODE=租户编码
QUEUE_NAME=队列名
PASSWORD=密码
DATA_SOURCE_OTHER=jdbc连接参数，格式为:{"key1":"value1",...}
PROJECT_TAG=项目相关操作
CREATE_PROJECT_NOTES=创建项目
PROJECT_DESC=项目描述
UPDATE_PROJECT_NOTES=更新项目
PROJECT_ID=项目ID
QUERY_PROJECT_BY_ID_NOTES=通过项目ID查询项目信息
QUERY_PROJECT_LIST_PAGING_NOTES=分页查询项目列表
QUERY_ALL_PROJECT_LIST_NOTES=查询所有项目
DELETE_PROJECT_BY_ID_NOTES=删除项目通过ID
QUERY_UNAUTHORIZED_PROJECT_NOTES=查询未授权的项目
QUERY_AUTHORIZED_PROJECT_NOTES=查询授权项目
TASK_RECORD_TAG=任务记录相关操作
QUERY_TASK_RECORD_LIST_PAGING_NOTES=分页查询任务记录列表
CREATE_TOKEN_NOTES=创建token，注意需要先登录
QUERY_ACCESS_TOKEN_LIST_NOTES=分页查询access token列表
SCHEDULE=定时
WARNING_TYPE=发送策略
WARNING_GROUP_ID=发送组ID
FAILURE_STRATEGY=失败策略
RECEIVERS=收件人
RECEIVERS_CC=收件人(抄送)
WORKER_GROUP_ID=Worker Server分组ID
PROCESS_INSTANCE_PRIORITY=流程实例优先级
UPDATE_SCHEDULE_NOTES=更新定时
SCHEDULE_ID=定时ID
ONLINE_SCHEDULE_NOTES=定时上线
OFFLINE_SCHEDULE_NOTES=定时下线
QUERY_SCHEDULE_NOTES=查询定时
QUERY_SCHEDULE_LIST_PAGING_NOTES=分页查询定时
LOGIN_TAG=用户登录相关操作
USER_NAME=用户名
PROJECT_NAME=项目名称
CREATE_PROCESS_DEFINITION_NOTES=创建流程定义
PROCESS_DEFINITION_NAME=流程定义名称
PROCESS_DEFINITION_JSON=流程定义详细信息(json格式)
PROCESS_DEFINITION_LOCATIONS=流程定义节点坐标位置信息(json格式)
PROCESS_INSTANCE_LOCATIONS=流程实例节点坐标位置信息(json格式)
PROCESS_DEFINITION_CONNECTS=流程定义节点图标连接信息(json格式)
PROCESS_INSTANCE_CONNECTS=流程实例节点图标连接信息(json格式)
PROCESS_DEFINITION_DESC=流程定义描述信息
PROCESS_DEFINITION_TAG=流程定义相关操作
SIGNOUT_NOTES=退出登录
USER_PASSWORD=用户密码
UPDATE_PROCESS_INSTANCE_NOTES=更新流程实例
QUERY_PROCESS_INSTANCE_LIST_NOTES=查询流程实例列表
VERIFY_PROCESS_DEFINITION_NAME_NOTES=验证流程定义名字
LOGIN_NOTES=用户登录
UPDATE_PROCESS_DEFINITION_NOTES=更新流程定义
PROCESS_DEFINITION_ID=流程定义ID
RELEASE_PROCESS_DEFINITION_NOTES=发布流程定义
QUERY_PROCESS_DEFINITION_BY_ID_NOTES=查询流程定义通过流程定义ID
QUERY_PROCESS_DEFINITION_LIST_NOTES=查询流程定义列表
QUERY_PROCESS_DEFINITION_LIST_PAGING_NOTES=分页查询流程定义列表
QUERY_ALL_DEFINITION_LIST_NOTES=查询所有流程定义
PAGE_NO=页码号
PROCESS_INSTANCE_ID=流程实例ID
PROCESS_INSTANCE_IDS=流程实例ID集合
PROCESS_INSTANCE_JSON=流程实例信息(json格式)
SCHEDULE_TIME=定时时间
SYNC_DEFINE=更新流程实例的信息是否同步到流程定义
RECOVERY_PROCESS_INSTANCE_FLAG=是否恢复流程实例
SEARCH_VAL=搜索值
USER_ID=用户ID
PAGE_SIZE=页大小
LIMIT=显示多少条
VIEW_TREE_NOTES=树状图
GET_NODE_LIST_BY_DEFINITION_ID_NOTES=获得任务节点列表通过流程定义ID
PROCESS_DEFINITION_ID_LIST=流程定义id列表
QUERY_PROCESS_DEFINITION_All_BY_PROJECT_ID_NOTES=查询流程定义通过项目ID
BATCH_DELETE_PROCESS_DEFINITION_BY_IDS_NOTES=批量删除流程定义通过流程定义ID集合
DELETE_PROCESS_DEFINITION_BY_ID_NOTES=删除流程定义通过流程定义ID
QUERY_PROCESS_INSTANCE_BY_ID_NOTES=查询流程实例通过流程实例ID
DELETE_PROCESS_INSTANCE_BY_ID_NOTES=删除流程实例通过流程实例ID
TASK_ID=任务实例ID
SKIP_LINE_NUM=忽略行数
QUERY_TASK_INSTANCE_LOG_NOTES=查询任务实例日志
DOWNLOAD_TASK_INSTANCE_LOG_NOTES=下载任务实例日志
USERS_TAG=用户相关操作
SCHEDULER_TAG=定时相关操作
CREATE_SCHEDULE_NOTES=创建定时
CREATE_USER_NOTES=创建用户
TENANT_ID=租户ID
QUEUE=使用的队列
EMAIL=邮箱
PHONE=手机号
QUERY_USER_LIST_NOTES=查询用户列表
UPDATE_USER_NOTES=更新用户
DELETE_USER_BY_ID_NOTES=删除用户通过ID
GRANT_PROJECT_NOTES=授权项目
PROJECT_IDS=项目IDS(字符串格式，多个项目以","分割)
GRANT_RESOURCE_NOTES=授权资源文件
RESOURCE_IDS=资源ID列表(字符串格式，多个资源ID以","分割)
GET_USER_INFO_NOTES=获取用户信息
LIST_USER_NOTES=用户列表
VERIFY_USER_NAME_NOTES=验证用户名
UNAUTHORIZED_USER_NOTES=取消授权
ALERT_GROUP_ID=报警组ID
AUTHORIZED_USER_NOTES=授权用户
GRANT_UDF_FUNC_NOTES=授权udf函数
UDF_IDS=udf函数id列表(字符串格式，多个udf函数ID以","分割)
GRANT_DATASOURCE_NOTES=授权数据源
DATASOURCE_IDS=数据源ID列表(字符串格式，多个数据源ID以","分割)
QUERY_SUBPROCESS_INSTANCE_BY_TASK_ID_NOTES=查询子流程实例通过任务实例ID
QUERY_PARENT_PROCESS_INSTANCE_BY_SUB_PROCESS_INSTANCE_ID_NOTES=查询父流程实例信息通过子流程实例ID
QUERY_PROCESS_INSTANCE_GLOBAL_VARIABLES_AND_LOCAL_VARIABLES_NOTES=查询流程实例全局变量和局部变量
VIEW_GANTT_NOTES=浏览Gantt图
SUB_PROCESS_INSTANCE_ID=子流程是咧ID
TASK_NAME=任务实例名
TASK_INSTANCE_TAG=任务实例相关操作
LOGGER_TAG=日志相关操作
PROCESS_INSTANCE_TAG=流程实例相关操作
EXECUTION_STATUS=工作流和任务节点的运行状态
HOST=运行任务的主机IP地址
START_DATE=开始时间
END_DATE=结束时间
QUERY_TASK_LIST_BY_PROCESS_INSTANCE_ID_NOTES=通过流程实例ID查询任务列表
UPDATE_DATA_SOURCE_NOTES=更新数据源
DATA_SOURCE_ID=数据源ID
QUERY_DATA_SOURCE_NOTES=查询数据源通过ID
QUERY_DATA_SOURCE_LIST_BY_TYPE_NOTES=查询数据源列表通过数据源类型
QUERY_DATA_SOURCE_LIST_PAGING_NOTES=分页查询数据源列表
CONNECT_DATA_SOURCE_NOTES=连接数据源
CONNECT_DATA_SOURCE_TEST_NOTES=连接数据源测试
DELETE_DATA_SOURCE_NOTES=删除数据源
VERIFY_DATA_SOURCE_NOTES=验证数据源
UNAUTHORIZED_DATA_SOURCE_NOTES=未授权的数据源
AUTHORIZED_DATA_SOURCE_NOTES=授权的数据源
DELETE_SCHEDULER_BY_ID_NOTES=根据定时id删除定时数据
QUERY_ALERT_GROUP_LIST_PAGING_NOTES=分页查询告警组列表
EXPORT_PROCESS_DEFINITION_BY_ID_NOTES=通过工作流ID导出工作流定义
BATCH_EXPORT_PROCESS_DEFINITION_BY_IDS_NOTES=批量导出工作流定义
QUERY_USER_CREATED_PROJECT_NOTES= 查询用户创建的项目
QUERY_AUTHORIZED_AND_USER_CREATED_PROJECT_NOTES= 查询授权和用户创建的项目
COPY_PROCESS_DEFINITION_NOTES= 复制工作流定义
MOVE_PROCESS_DEFINITION_NOTES= 移动工作流定义
TARGET_PROJECT_ID= 目标项目ID
IS_COPY = 是否复制
DELETE_PROCESS_DEFINITION_VERSION_NOTES=删除流程历史版本
QUERY_PROCESS_DEFINITION_VERSIONS_NOTES=查询流程历史版本信息
SWITCH_PROCESS_DEFINITION_VERSION_NOTES=切换流程版本
VERSION=版本号

1
dolphinscheduler-api/src/main/resources/logback-api.xml
Unescape View File

@ -55,7 +55,6 @@
<root level="INFO">
<appender-ref ref="STDOUT"/>
<appender-ref ref="APILOGFILE"/>
</root>

4
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
Unescape View File

@ -117,14 +117,14 @@ public class ProcessDefinitionControllerTest {
public void testVerifyProcessDefinitionName() throws Exception {
Map<String, Object> result = new HashMap<>();
putMsg(result, Status.PROCESS_INSTANCE_EXIST);
putMsg(result, Status.VERIFY_PROCESS_DEFINITION_NAME_UNIQUE_ERROR);
String projectName = "test";
String name = "dag_test";
Mockito.when(processDefinitionService.verifyProcessDefinitionName(user, projectName, name)).thenReturn(result);
Result response = processDefinitionController.verifyProcessDefinitionName(user, projectName, name);
Assert.assertEquals(Status.PROCESS_INSTANCE_EXIST.getCode(), response.getCode().intValue());
Assert.assertEquals(Status.VERIFY_PROCESS_DEFINITION_NAME_UNIQUE_ERROR.getCode(), response.getCode().intValue());
}

72
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TaskInstanceControllerTest.java
Unescape View File

@ -14,52 +14,60 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.api.controller;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.when;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.service.TaskInstanceService;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.ExecutionStatus;
import org.apache.dolphinscheduler.dao.entity.TaskInstance;
import java.util.HashMap;
import java.util.Map;
import org.junit.Assert;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MvcResult;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import org.junit.runner.RunWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.MockitoJUnitRunner;
/**
* task instance controller test
*/
public class TaskInstanceControllerTest extends AbstractControllerTest{
private static Logger logger = LoggerFactory.getLogger(TaskInstanceControllerTest.class);
@RunWith(MockitoJUnitRunner.Silent.class)
public class TaskInstanceControllerTest {
@InjectMocks
private TaskInstanceController taskInstanceController;
@Mock
private TaskInstanceService taskInstanceService;
@Test
public void testQueryTaskListPaging() throws Exception {
public void testQueryTaskListPaging() {
MultiValueMap<String, String> paramsMap = new LinkedMultiValueMap<>();
//paramsMap.add("processInstanceId","1380");
paramsMap.add("searchVal","");
paramsMap.add("taskName","");
//paramsMap.add("stateType","");
paramsMap.add("startDate","2019-02-26 19:48:00");
paramsMap.add("endDate","2019-02-26 19:48:22");
paramsMap.add("pageNo","1");
paramsMap.add("pageSize","20");
Map<String,Object> result = new HashMap<>();
Integer pageNo = 1;
Integer pageSize = 20;
PageInfo pageInfo = new PageInfo<TaskInstance>(pageNo, pageSize);
result.put(Constants.DATA_LIST, pageInfo);
result.put(Constants.STATUS, Status.SUCCESS);
MvcResult mvcResult = mockMvc.perform(get("/projects/{projectName}/task-instance/list-paging","cxc_1113")
.header(SESSION_ID, sessionId)
.params(paramsMap))
.andExpect(status().isOk())
.andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
.andReturn();
when(taskInstanceService.queryTaskListPaging(any(), eq(""), eq(1), eq(""), eq(""), eq(""),any(), any(),
eq(""), Mockito.any(), eq("192.168.xx.xx"), any(), any())).thenReturn(result);
Result taskResult = taskInstanceController.queryTaskListPaging(null, "", 1, "", "",
"", "", ExecutionStatus.SUCCESS,"192.168.xx.xx", "2020-01-01 00:00:00", "2020-01-02 00:00:00",pageNo, pageSize);
Assert.assertEquals(Integer.valueOf(Status.SUCCESS.getCode()), taskResult.getCode());
Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class);
Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue());
logger.info(mvcResult.getResponse().getContentAsString());
}
}

8
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TenantControllerTest.java
Unescape View File

@ -43,7 +43,6 @@ public class TenantControllerTest extends AbstractControllerTest{
public void testCreateTenant() throws Exception {
MultiValueMap<String, String> paramsMap = new LinkedMultiValueMap<>();
paramsMap.add("tenantCode","tenantCode");
paramsMap.add("tenantName","tenantName");
paramsMap.add("queueId","1");
paramsMap.add("description","tenant description");
@ -84,7 +83,6 @@ public class TenantControllerTest extends AbstractControllerTest{
MultiValueMap<String, String> paramsMap = new LinkedMultiValueMap<>();
paramsMap.add("id","9");
paramsMap.add("tenantCode","cxc_te");
paramsMap.add("tenantName","tenant_update_2");
paramsMap.add("queueId","1");
paramsMap.add("description","tenant description");
@ -96,7 +94,7 @@ public class TenantControllerTest extends AbstractControllerTest{
.andReturn();
Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class);
Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue());
Assert.assertEquals(Status.TENANT_NOT_EXIST.getCode(),result.getCode().intValue());
logger.info(mvcResult.getResponse().getContentAsString());
}
@ -133,7 +131,7 @@ public class TenantControllerTest extends AbstractControllerTest{
.andReturn();
Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class);
Assert.assertEquals(Status.TENANT_NAME_EXIST.getCode(), result.getCode().intValue());
Assert.assertEquals(Status.TENANT_CODE_EXIST.getCode(), result.getCode().intValue());
logger.info(mvcResult.getResponse().getContentAsString());
}
@ -165,7 +163,7 @@ public class TenantControllerTest extends AbstractControllerTest{
.andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
.andReturn();
Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class);
Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue());
Assert.assertEquals(Status.TENANT_NOT_EXIST.getCode(),result.getCode().intValue());
logger.info(mvcResult.getResponse().getContentAsString());
}
}

Some files were not shown because too many files have changed in this diff Show More

Write Preview
Loading…
Cancel
Save