From 0c8d08cbaa0636ef93cbcb01c43a34f002bb7fdb Mon Sep 17 00:00:00 2001 From: Kirs Date: Fri, 22 Jan 2021 11:01:46 +0800 Subject: [PATCH] [Feature-#3049] Alert Plugin Design(merge alert_plugin_design to dev) (#4495) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui]Alert plugin design (#3734) * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui] Add alarm instance page * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601) * DS SPI * Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI * Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI * add TODO * delete * compile * spi commit * Plugin Alert * fix some bug * add todo * change web ui from alpacajs to form-create * remove module * add plugin schema * add license header * update alert and spi module version * update the alert plugin sub module version * comment the maven.local.repository param * move utils from spi to common module * add license header * add license header and delete some chinese comment * update spi packages * delete no use alert_xx.properties * update mysql.connector.version back to 5.1.34 * delete no use comment in pom.xml * update email stmp password * add license * add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file * format the code style * format new clase file with checkstyle * update plugin params to Builder model * move JSONUtils to SPI because plugin can not dependency common module * move JSONUtils to SPI because plugin can not dependency common module * delete collection dependency * replace PluginParamsTransfer to spi PluginParamsTransfer * update dolphinscheduler-maven-plugin to 1.0.0 * update license * update apache-rat-plugin add exclude '.iml' file * check license * ArtifactResolver only use in development and configPlugins is not empty * ArtifactResolver only use in development and configPlugins is not empty * ArtifactResolver only use in development and configPlugins is not empty * default datasource should be postgresql * add license files * add license files * postgresql port should be 5432 * postgresql test * mv show_type to spi add license header to AlertConstants * check style fix * copy check style file from branch dev * alert show_type set by plugin * alert show_type set by plugin * add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java * add Bean to TaskCallbackServiceTestConfig * add Bean to TaskCallbackServiceTestConfig * fix check style * check style fix * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * fix check style * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * check style fix * rollback test change * rollback test change * rollback dao pom change * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * add ut to pom.xml * add upgrade schema to global schema * fix ut failed * fix ut failed * fix ut failed * fix ut failed * add test EmailAlertPluginTest to pom.xml * fix ut failed * fix ut failed * fix check style * update license header to presto license header * presto license header not check * fix ut coverage * fix ut coverage * fix ut * fix ut * fix ut * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage Co-authored-by: break60 <790061044@qq.com> * Replace the tenant management page ui with element-ui (#3891) * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui] Add alarm instance page * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui]Alert plugin design (#3734) * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui] Add alarm instance page * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601) * DS SPI * Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI * Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI * add TODO * delete * compile * spi commit * Plugin Alert * fix some bug * add todo * change web ui from alpacajs to form-create * remove module * add plugin schema * add license header * update alert and spi module version * update the alert plugin sub module version * comment the maven.local.repository param * move utils from spi to common module * add license header * add license header and delete some chinese comment * update spi packages * delete no use alert_xx.properties * update mysql.connector.version back to 5.1.34 * delete no use comment in pom.xml * update email stmp password * add license * add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file * format the code style * format new clase file with checkstyle * update plugin params to Builder model * move JSONUtils to SPI because plugin can not dependency common module * move JSONUtils to SPI because plugin can not dependency common module * delete collection dependency * replace PluginParamsTransfer to spi PluginParamsTransfer * update dolphinscheduler-maven-plugin to 1.0.0 * update license * update apache-rat-plugin add exclude '.iml' file * check license * ArtifactResolver only use in development and configPlugins is not empty * ArtifactResolver only use in development and configPlugins is not empty * ArtifactResolver only use in development and configPlugins is not empty * default datasource should be postgresql * add license files * add license files * postgresql port should be 5432 * postgresql test * mv show_type to spi add license header to AlertConstants * check style fix * copy check style file from branch dev * alert show_type set by plugin * alert show_type set by plugin * add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java * add Bean to TaskCallbackServiceTestConfig * add Bean to TaskCallbackServiceTestConfig * fix check style * check style fix * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * fix check style * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * check style fix * rollback test change * rollback test change * rollback dao pom change * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * add ut to pom.xml * add upgrade schema to global schema * fix ut failed * fix ut failed * fix ut failed * fix ut failed * add test EmailAlertPluginTest to pom.xml * fix ut failed * fix ut failed * fix check style * update license header to presto license header * presto license header not check * fix ut coverage * fix ut coverage * fix ut * fix ut * fix ut * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage Co-authored-by: break60 <790061044@qq.com> * The tenant management module ui is replaced with element-ui * fix * fix * fix * fix * fix * modify * fix Co-authored-by: gaojun2048 <32193458+gaojun2048@users.noreply.github.com> * the alert module support service. * the alert module support service. * Delete blank lines. * add code checkstyle. * Solve the sonar bug. * Re-execute build. * add test. * update ut. * add test class. * update test. * [feature#3356] alert-spi support DingTalk&WeChat (#3869) * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui]Alert plugin design (#3734) * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [feature][ui] Add alarm instance page * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601) * DS SPI * Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI * Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI * add TODO * delete * compile * spi commit * Plugin Alert * fix some bug * add todo * change web ui from alpacajs to form-create * remove module * add plugin schema * add license header * update alert and spi module version * update the alert plugin sub module version * comment the maven.local.repository param * move utils from spi to common module * add license header * add license header and delete some chinese comment * update spi packages * delete no use alert_xx.properties * update mysql.connector.version back to 5.1.34 * delete no use comment in pom.xml * update email stmp password * add license * add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file * format the code style * format new clase file with checkstyle * update plugin params to Builder model * move JSONUtils to SPI because plugin can not dependency common module * move JSONUtils to SPI because plugin can not dependency common module * delete collection dependency * replace PluginParamsTransfer to spi PluginParamsTransfer * update dolphinscheduler-maven-plugin to 1.0.0 * update license * update apache-rat-plugin add exclude '.iml' file * check license * ArtifactResolver only use in development and configPlugins is not empty * ArtifactResolver only use in development and configPlugins is not empty * ArtifactResolver only use in development and configPlugins is not empty * default datasource should be postgresql * add license files * add license files * postgresql port should be 5432 * postgresql test * mv show_type to spi add license header to AlertConstants * check style fix * copy check style file from branch dev * alert show_type set by plugin * alert show_type set by plugin * add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java * add Bean to TaskCallbackServiceTestConfig * add Bean to TaskCallbackServiceTestConfig * fix check style * check style fix * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * fix check style * [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683) * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * check style fix * rollback test change * rollback test change * rollback dao pom change * [feature-3665][ui]Add element-ui (#3666) * [feature-3665][ui]Add element-ui * add license * Add form-create plug-in and alarm group management add sample demo * Modify node version * fix * fix * add ut to pom.xml * add upgrade schema to global schema * fix ut failed * fix ut failed * fix ut failed * fix ut failed * add test EmailAlertPluginTest to pom.xml * fix ut failed * fix ut failed * fix check style * update license header to presto license header * presto license header not check * fix ut coverage * fix ut coverage * fix ut * fix ut * fix ut * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage * fix ut coverage Co-authored-by: break60 <790061044@qq.com> * [feature#3356] alert-spi support DingTalk this closes # 3356 * add test * code style * we chat alert * support we chat alert * support we chat alert * support we chat alert,update ding talk alert * code style * add test * code style * clean old code * clean old code * code smell * code style * add test * simple config * code style * code style * code style * delete old file * fix log content error Co-authored-by: break60 <790061044@qq.com> Co-authored-by: gaojun2048 <32193458+gaojun2048@users.noreply.github.com> * [FEATURE#4029] alert spi support script alert this closes #4029 * code style * code smell * add test * code style * code smell * code style * fix name error * init sms plugin * modify http type * delete dolphinscheduler-alert-sms plugin * dolphinscheduler plugin need add dolphinscheduler-plugin in the pom.xml file * delete alert http plugin (#4152) delete alert http plugin (#4152) This part of the code was merged by mistake, so I need to delete it * [Feature-3749][Alert-SPI] SqlTask should send notifications by alert server api (#4080) * add sqltask send sync alert server. * update alert-sms license. * update AlertServer test. * remote EmailAlertPluginTest. * update sqltask. * update test class. * [Alert plugin design] add http alert plugin (#4165) * add http alert plugin * [Feature-3311] Add alert plugin instance function (#4038) * [Feature-#3252][build]plugins need put in plugins dir when install (#4259) Update maven build, put the alert plugins in the ${install_dir}/lib/plugin/alert/${plugin_module_name} directory (assembly build and rpm build). Note: When adding a new plug-in, it must be reflected in provisio/dolphinscheduler.xml. this closes #3252 * [Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages (#4311) * [Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages *Some plugins (such as alert plugin) need to provide UI interfaces to users. *We use from-creat to dynamically generate UI interfaces. Related parameters are mainly provided by pluginParams. *From-create can generate dynamic ui based on this parameter. this closes #4310 * add license head * rename * add ut * add license * add query plugin detail interface * fix error * [Feature-#3828][server] Delete recipients and CCs in the process defi… (#4303) * [Feature-#3828][server] Delete recipients and CCs in the process definition and select the associated alert group instead * The specific alarm instance associated with the alert group * The process definition only associates the alert group, and does not care about the specific alarm instance. this closes #3828 * remove test cc * remove test cc * fix test error * add sql upgrade script delete alert column(show_type、alert_type、re、cc) * code style * update test Note the statement to delete the column, and provide related instructions * delete AlertGroup field(alert_type) * fix ut error * fix ut error * fix ut error * code style * delete api-doc receiverCC * delete UserAlertGroupMapper * delete UserAlertGroup * delete UserAlertGroupApi * alert group add coulmn user_id * alter user_id to create_user_id * userinfo remove alert group info * Replace the front-end code of the plugin branch with the front-end code of the dev branch (#4353) * Replace the front-end code of the plugin branch with the front-end code of the dev branch * Merge dev to alert_plugin_design (#4351) * fixed queryProcessInstanceListPaging query slow (#3893) because queryProcessInstanceListPaging has contain longtext type field by instance.*,so queryProcessInstanceListPaging query slow , but longtext field not used. * fixed queryTaskInstanceListPaging query slow (#3892) because queryTaskInstanceListPaging has contain longtext type field by instance.*,so queryTaskInstanceListPaging query slow , but longtext field not used. * add verify tenant name cannot contain special characters. (#3865) * [Fix][Docker] fix startup.sh variables cannot reference in single quotes. (#3895) * [Fix][Docker] fix startup.sh variables cannot reference in single quotes.(#3894) Signed-off-by: YoungMan * Update Dockerfile update the comment and delete the maridb client Co-authored-by: dailidong * [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935) * [CodeClean][DAO]Remove redundant comments (#3939) * fix bug Delete invalid field: executorcores Modify verification prompt * fix bug Delete invalid field: executorcores Modify verification prompt * fix bug Delete invalid field: executorcores Modify verification prompt * dag add close button * reset last version * reset last version * dag add close buttion dag add close buttion * update CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * Update CreateWorkflowLocator.java modify submit workflow button * Update CreateWorkflowLocator.java * Update CreateWorkflowLocator.java modify CLICK_ADD_BUTTON * Update CreateWorkflowLocator.java delete print * Update CreateWorkflowLocator.java 1 * Update CreateWorkflowLocator.java 1 * Setting '-XX:+DisableExplicitGC ' causes netty memory leaks in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M' * Update dag.vue * Update dag.vue * Update dag.vue * Update CreateWorkflowLocator.java * Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks" This reverts commit 3a2cba7a * Setting '-XX:+DisableExplicitGC ' causes netty memory leaks in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M' * Update dolphinscheduler-daemon.sh * Remove redundant comments ProcessInstanceMapper# queryProcessInstanceListPaging * Remove redundant comments ProcessInstanceMapper# queryProcessInstanceListPaging Co-authored-by: dailidong Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> * fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master * When running a task, the resource file is lost, which results in an error * When running a task, the resource file is lost, which results in an error * When running a task, the resource file is lost, which results in an error * [Fix][api] Add queryProjectCreatedAndAuthorizedByUser function (#3658) * fix getLoginUserCreated bug * fix getLoginUserCreated bug 1 * fix checkstyle * modify function name * modify comment * fix unit test error * rollback sqoop task hive target file Co-authored-by: sunchaohe Co-authored-by: dailidong * When running a task, the resource file is lost, which results in an error * When running a task, the resource file is lost, which results in an error * [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty * solve too many files, close logClientService (#3971) * fix #3966 sub process doesnot send alert mail after process instance ending. (#3972) Co-authored-by: baoliang * [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975) * [Fix-#3618][server] resolve task executed finished but not release the file handle * [Fix-#3618][server] resolve task executed finished but not release the file handle * [Fix-#3958][api] files should not be created successfully in the directory of the authorized file * [FIX-3966] The timeout warning does not take effect in sub_process (#3982) * fix #3966 sub process doesnot send alert mail after process instance ending. * fix bug 3964: sub_process The timeout warning does not take effect add timeout warning for sub_process/dependent task. * fix code smell * fix code smell * fix code smell * update worker group inherit from parent Co-authored-by: baoliang * [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog(). (#3963) * Enhance user experience, add close button to file details page * [Test-3557][API] full cover DataAnalysisServiceImpl (#3605) * generate equals and hashCode method * make up for the state of oblivion * simplify countQueueState method * delete try catch, because this never throw exception * full cover DataAnalysisServiceImpl * improve test coverage * adjust code style * simplify countCommandState method * reduce duplication * simplify countTaskDtos method * Update DataAnalysisServiceImpl.java Co-authored-by: dailidong * fix import dolphinscheduler_mysql.sql insert admin user data * [FIX-3929] condition task would post wrong tasks when failover. (#3999) * fix #3966 sub process doesnot send alert mail after process instance ending. * fix bug 3964: sub_process The timeout warning does not take effect add timeout warning for sub_process/dependent task. * fix code smell * fix code smell * fix code smell * update worker group inherit from parent * remove stdout in logback configuration * fix bug #3929 condition task would post error when failover. * remove unused test * add comments * add skip node judge Co-authored-by: baoliang * [FIX-3929] because of no lock, start up failover would dispatch two same tasks. (#4004) * fix #3966 sub process doesnot send alert mail after process instance ending. * fix bug 3964: sub_process The timeout warning does not take effect add timeout warning for sub_process/dependent task. * fix code smell * fix code smell * fix code smell * update worker group inherit from parent * remove stdout in logback configuration * fix bug #3929 condition task would post error when failover. * remove unused test * add comments * add skip node judge * fix bug 3929: because of no lock, start up failover would dispatch two same tasks. Co-authored-by: baoliang * revert pom version to 1.3.3-release * [maven-release-plugin] prepare release 1.3.3 * [maven-release-plugin] prepare for next development iteration * the resources API document does not match the interface parameters * [release]revert pom version to 1.3.3-release * fix bug 4010: remove failed condition tasks from error-task-list. (#4011) Co-authored-by: baoliang * [maven-release-plugin] prepare release 1.3.3 * [maven-release-plugin] prepare for next development iteration * fix bug * reset code * Update udfs.vue * [Feature-3941][ui] Remove tenant name (#3949) * FeatureUI-3941: Removed Tenant * FeatureUI-3941: Removed tenantName in the suggested pages * FeatureUI-3941: Updated the suggested files * FeatureUI-3941: Updated the TenantController * FeatureUI-3941: Updated the TenantController and the pom.xml * FeatureUI-3941: Updated the TenantControllerTest * [Improvement][script] daemon script usage (#3968) * Improvement daemon script usage * don't remove status command don't remove status command Co-authored-by: dailidong * fix #3688, Optimization of table building statements (#3689) * merge from 1.3.3-release * merge from 1.3.3-release * [Improvement-4012][common/remote] Json util code integration,remove the remote module json util (#4013) * Json util code integration,remove the remote module json util. * update code checkstyle. * update code checkstyle. * add code checkstyle. * add test class. * update JSONUtils class. * update JSONUtils class. * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * refactor code style * refactor code style * refactor code style * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * refactor ut test * refactor ut test * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * fix #3900 kill multi yarn app in one job * refactor ut * merge from 1.3.3-release * refactor ut * refactor ut * refactor * refactor * refactor code style * refactor code style * refactor code style * refactor code style * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * add ProcessUtils UT * refactor code style * refactor code style * refactor code style * refactor code style * [Draft][Merge][133-dev]133 merge dev (#4031) * [Feture-3327][ui]Add the function of re-uploading files in the resource center * [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394) * Before creating a workflow, clear the canvas * [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306) * [Feture-3327][ui]Add the function of re-uploading files in the resource center Co-authored-by: wuchunfu <319355703@qq.com> * [Improvement-3327][api]support re-upload the resource file (#3395) * [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396) * [Fix-3390][api]Running hive sql task need find the hdfs path correctly * [Fix-3390][api]Running hive sql task need find the hdfs path correctly * update soft version * hive UDF function to modify the background color * fix * fix bug: Fix master task dependency check bug * cancel spark task version check (#3406) Co-authored-by: Eights-Li * [Bug][ui]Fix front-end bug #3413 * [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410) * Optimize dag * Update actions.js (#3401) * [Fix-3256][ui] Fix admin user info update error (#3425) (#3428) * [PROPOSAL-3139] Datasource selection changes from radio to select * [PROPOSAL-3139] Datasource selection changes from radio to select * [BUG FIX] issues #3256 * [BUG FIX] issues #3256 * [BUG FIX] issues #3256 * [Fix-3256][ui] Fix admin user info update error * [Fix-3256][ui] Fix admin user info update error * [Fix-3256][ui] Fix admin user info update error * [Fix-3256][ui] Fix admin user info update error * reset createUser.vue * [Fix-3256][ui] Fix admin user info update error Co-authored-by: dailidong Co-authored-by: wuchunfu <319355703@qq.com> Co-authored-by: dailidong * [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure * dag connection add check * fix * [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454) * Remove node deep monitoring * If worker group id is null,don't need to set the value of the worker group (#3460) * [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x * [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group * [ui]Code optimization * fix * fix * [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js * [optimization][ui]Prevent the shell script input box from being empty * [Fix-3462][api]If login user is admin,need list all udfs (#3465) * [Fix-3462][api]If login user is admin,need list all udfs * [Fix-3462][api]add the test on the method of QueryUdfFuncList * [Fix-3462][api]fix the code smell * [Fix-3462][api]fix the code smell * [Fix-3462][api]fix the code smell * [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482) * [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data * [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory * fix bug: Fix master task dependency check bug (#3473) Co-authored-by: lenboo * [maven-release-plugin] prepare release 1.3.2 * [maven-release-plugin] prepare for next development iteration * fix ci_e2e fail (#3497) * [Fix-3469][api]Should filter the resource by the different program type (#3498) * [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource * [Fix-3469][api]Should list python file and jar file * [Fix-3469][api]Should filter the resource by the different program type * [Fix-3469][api]fix the code smell * test release 1.3.2 version rollback * test release 1.3.2 version rollback * test release 1.3.2 version rollback (#3499) * [Feature] JVM parameter optimization , related issue #3370 * [Feature] JVM parameter optimization , related issue #3370 * test release 1.3.2 version rollback * test release 1.3.2 version rollback Co-authored-by: qiaozhanwei * [maven-release-plugin] prepare release 1.3.2 * [maven-release-plugin] prepare for next development iteration * [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types * fix * Revert "fix ci_e2e fail (#3497)" This reverts commit e367f90bb73c9682739308a0a98887a1c0f407ef. * test * test release 1.3.2 version rollback * test release 1.3.2 version rollback (#3503) * [Feature] JVM parameter optimization , related issue #3370 * [Feature] JVM parameter optimization , related issue #3370 * test release 1.3.2 version rollback * test release 1.3.2 version rollback * test * test release 1.3.2 version rollback Co-authored-by: qiaozhanwei * [maven-release-plugin] prepare release 1.3.2 * [maven-release-plugin] prepare for next development iteration * test release 1.3.2 version rollback (#3504) * [Feature] JVM parameter optimization , related issue #3370 * [Feature] JVM parameter optimization , related issue #3370 * test release 1.3.2 version rollback * test release 1.3.2 version rollback * test * test release 1.3.2 version rollback Co-authored-by: qiaozhanwei * [maven-release-plugin] prepare release 1.3.2 * [maven-release-plugin] prepare for next development iteration * fix ds muti-level directory in zk, which lead to fail to assign work * add login user check some actions in api * [Hotfix][ci] Fix e2e ci docker image build error * modify tag 1.3.0 to HEAD * modify tag 1.3.0 to HEAD (#3525) * [Feature] JVM parameter optimization , related issue #3370 * [Feature] JVM parameter optimization , related issue #3370 * test release 1.3.2 version rollback * test release 1.3.2 version rollback * test * test release 1.3.2 version rollback * modify tag 1.3.0 to HEAD Co-authored-by: qiaozhanwei * remove OGNL part of the mybaits notice (#3526) * [maven-release-plugin] prepare release 1.3.2 * [maven-release-plugin] prepare for next development iteration * release 1.3.2 version rollback (#3527) * [Feature] JVM parameter optimization , related issue #3370 * [Feature] JVM parameter optimization , related issue #3370 * test release 1.3.2 version rollback * test release 1.3.2 version rollback * test * test release 1.3.2 version rollback * modify tag 1.3.0 to HEAD Co-authored-by: qiaozhanwei * [ui]Script input box to modify the delay loading time * fix * fix * fix * fix * modify general user can't create token * [ui]It is forbidden to select non-existent resources and modify the tree display data format * modify general user can't create token (#3533) * [Feature] JVM parameter optimization , related issue #3370 * [Feature] JVM parameter optimization , related issue #3370 * test release 1.3.2 version rollback * test release 1.3.2 version rollback * test * test release 1.3.2 version rollback * modify tag 1.3.0 to HEAD * modify general user can't create token Co-authored-by: qiaozhanwei * if task is null , set task type is null instead of "null" * [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537) * [Fix-3536][api]If user didn't have tenant,create resource will NPE * [Fix-3536][api]If user didn't have tenant,create resource directory will NPE * modify general user can't create,delete,update token (#3538) Co-authored-by: qiaozhanwei * [ui]Resource delete OK button to increase loading, change the number of homepage display cursor * fix * [Fix-3616][Server] when worker akc/response master exception , async retry (#3748) * [fixbug][ui]Repair the master and worker management instrument display * [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741) * [fixbug][ui]Remove non-existent or deleted resources disabled * [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select * [fix-3553][ui]Repair click workflow connection, select the entire path * fix * fix * [Fix-3238][docker]Fix that can not create folder in docker with standalone mode * [Fix-3616][Server] when worker akc/response master exception , async retry (#3776) * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry Co-authored-by: qiaozhanwei * The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected. * [Improvement-3720][ui] js mailbox verification fix * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784) * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected. * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect Co-authored-by: qiaozhanwei Co-authored-by: zhuangchong Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> * [Fix-3124][docker]Fix that can not build a docker image on windows (#3765) * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786) * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected. * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect Co-authored-by: qiaozhanwei Co-authored-by: zhuangchong Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> * [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787) * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * [Fix-3616][Server] when worker akc/response master exception , async retry * The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected. * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect * [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect * [BugFixed] issue #3258 (#3265) * 'ExecutionStatus' * '3258' * Update WorkerGroupServiceTest.java * Delete UserState.java * Delete ResourceSyncService.java * Delete core-site.xml * Delete hdfs-site.xml Co-authored-by: dailidong Co-authored-by: qiaozhanwei Co-authored-by: qiaozhanwei Co-authored-by: zhuangchong Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> Co-authored-by: dailidong * [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage * [Bug-3713][HadoopUtils] catfile method Stream not closed (#3715) * fix bug Delete invalid field: executorcores Modify verification prompt * fix bug Delete invalid field: executorcores Modify verification prompt * fix bug Delete invalid field: executorcores Modify verification prompt * dag add close button * reset last version * reset last version * dag add close buttion dag add close buttion * update CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * Update CreateWorkflowLocator.java modify submit workflow button * Update CreateWorkflowLocator.java * Update CreateWorkflowLocator.java modify CLICK_ADD_BUTTON * Update CreateWorkflowLocator.java delete print * Update CreateWorkflowLocator.java 1 * Update CreateWorkflowLocator.java 1 * Setting '-XX:+DisableExplicitGC ' causes netty memory leaks in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M' * Update dag.vue * Update dag.vue * Update dag.vue * Update CreateWorkflowLocator.java * Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks" This reverts commit 3a2cba7a * Setting '-XX:+DisableExplicitGC ' causes netty memory leaks in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M' * Update dolphinscheduler-daemon.sh * catfile method Stream not closed * catfile method Stream not closed Co-authored-by: dailidong Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> * [Fix-#3713][common]Fix that catfile method Stream not closed * [Fix-#3713][common]Fix that catfile method Stream not closed (#3810) * [Bug-3713][HadoopUtils] catfile method Stream not closed (#3715) * fix bug Delete invalid field: executorcores Modify verification prompt * fix bug Delete invalid field: executorcores Modify verification prompt * fix bug Delete invalid field: executorcores Modify verification prompt * dag add close button * reset last version * reset last version * dag add close buttion dag add close buttion * update CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * updae CLICK_SAVE_WORKFLOW_BUTTON xpath * Update CreateWorkflowLocator.java modify submit workflow button * Update CreateWorkflowLocator.java * Update CreateWorkflowLocator.java modify CLICK_ADD_BUTTON * Update CreateWorkflowLocator.java delete print * Update CreateWorkflowLocator.java 1 * Update CreateWorkflowLocator.java 1 * Setting '-XX:+DisableExplicitGC ' causes netty memory leaks in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M' * Update dag.vue * Update dag.vue * Update dag.vue * Update CreateWorkflowLocator.java * Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks" This reverts commit 3a2cba7a * Setting '-XX:+DisableExplicitGC ' causes netty memory leaks in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M' * Update dolphinscheduler-daemon.sh * catfile method Stream not closed * catfile method Stream not closed Co-authored-by: dailidong Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> * [Fix-#3713][common]Fix that catfile method Stream not closed Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com> Co-authored-by: dailidong Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> * [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading * [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132) * Bugfix: Fix the new tenant already exists prompt * Feature: Add test cases * Update TenantServiceTest.java Co-authored-by: dailidong Co-authored-by: qiaozhanwei * Set up JDK 11 for SonarCloud in github action. (#3052) * Set up JDK 11 for SonarCloud in github action. * Fix javadoc error with JDK 11. * Prevent Javadoc from stopping if it finds any html errors. * [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state * add verify tenant name cannot contain special characters. * [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers * fix * fix * fix * [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862) * [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file. * [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file. * [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking * [fixbug-3887][ui]Fix missing English translation of re-upload files * add process define name verify. (#3879) * Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters." * revert pr 3872 * [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830) * fix bug(#3617): after subtask fault tolerance, 2 task instances are generated. * delete unused code * update code smell * refactor sub work command process * add process service ut * add license header * fix some code smell * chang ut java8 to java11 * update sonar to java11 * copy ut config from dev * remove checkstyle * revert to 1.3.3 * change proess service test to executor service * add process service test * add process service test * revert * revert * add comments * change dev to 1.3.3-release * revert Co-authored-by: baoliang * [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907) * [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error (#3899) * fix bug : error message * fix code smell * fix code smell * [FIX_#3789][remote]cherry pick from dev to support netty heart beat * [FIX_#3789][remote]cherry pick from dev to support netty heart beat * [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913) * [FIX_#3789][remote]cherry pick from dev to support netty heart beat * [FIX_#3789][remote]cherry pick from dev to support netty heart beat Co-authored-by: Kirs * Repair check box cannot be canceled * [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly * [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly * [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918) * [FIX_#3789][remote]cherry pick from dev to support netty heart beat * [FIX_#3789][remote]cherry pick from dev to support netty heart beat * [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly * [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly Co-authored-by: Kirs * [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un * Workflow definition name re-modified and added check * [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly. * update code. * [#3931][ui]Field name optimization for spark, flink, and mr * change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934) * [maven-release-plugin] prepare release 1.3.3 * [maven-release-plugin] prepare for next development iteration * [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935) * fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master * [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty * solve too many files, close logClientService (#3971) * fix #3966 sub process doesnot send alert mail after process instance ending. (#3972) Co-authored-by: baoliang * [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975) * [Fix-#3618][server] resolve task executed finished but not release the file handle * [Fix-#3618][server] resolve task executed finished but not release the file handle * [Fix-#3958][api] files should not be created successfully in the directory of the authorized file * [FIX-3966] The timeout warning does not take effect in sub_process (#3982) * fix #3966 sub process doesnot send alert mail after process instance ending. * fix bug 3964: sub_process The timeout warning does not take effect add timeout warning for sub_process/dependent task. * fix code smell * fix code smell * fix code smell * update worker group inherit from parent Co-authored-by: baoliang * fix import dolphinscheduler_mysql.sql insert admin user data * [FIX-3929] condition task would post wrong tasks when failover. (#3999) * fix #3966 sub process doesnot send alert mail after process instance ending. * fix bug 3964: sub_process The timeout warning does not take effect add timeout warning for sub_process/dependent task. * fix code smell * fix code smell * fix code smell * update worker group inherit from parent * remove stdout in logback configuration * fix bug #3929 condition task would post error when failover. * remove unused test * add comments * add skip node judge Co-authored-by: baoliang * [FIX-3929] because of no lock, start up failover would dispatch two same tasks. (#4004) * fix #3966 sub process doesnot send alert mail after process instance ending. * fix bug 3964: sub_process The timeout warning does not take effect add timeout warning for sub_process/dependent task. * fix code smell * fix code smell * fix code smell * update worker group inherit from parent * remove stdout in logback configuration * fix bug #3929 condition task would post error when failover. * remove unused test * add comments * add skip node judge * fix bug 3929: because of no lock, start up failover would dispatch two same tasks. Co-authored-by: baoliang * revert pom version to 1.3.3-release * [maven-release-plugin] prepare release 1.3.3 * [maven-release-plugin] prepare for next development iteration * [release]revert pom version to 1.3.3-release * fix bug 4010: remove failed condition tasks from error-task-list. (#4011) Co-authored-by: baoliang * [maven-release-plugin] prepare release 1.3.3 * [maven-release-plugin] prepare for next development iteration * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * refactor code style * refactor code style * refactor code style * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * refactor ut test * refactor ut test * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * refactor ut * merge from 1.3.3-release * refactor ut * refactor ut * refactor * refactor * refactor code style * refactor code style * refactor code style * refactor code style * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * merge from 1.3.3-release * refactor code style Co-authored-by: break60 <790061044@qq.com> Co-authored-by: wuchunfu <319355703@qq.com> Co-authored-by: lgcareer <18610854716@163.com> Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> Co-authored-by: lenboo Co-authored-by: qiaozhanwei Co-authored-by: Yelli Co-authored-by: Eights-Li Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> Co-authored-by: dailidong Co-authored-by: qiaozhanwei Co-authored-by: XiaotaoYi Co-authored-by: Yichao Yang <1048262223@qq.com> Co-authored-by: zhuangchong Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com> Co-authored-by: muzhongjiang Co-authored-by: Jave-Chen Co-authored-by: zhuangchong Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com> Co-authored-by: Kirs Co-authored-by: lgcareer Co-authored-by: wulingqi * Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057) This reverts commit ad2d9f99d01f3baef8641e0a7e741c08a39f1f24. * [Fix][API] Condition task null pointer exception (#4056) * add ProcessUtils UT * modify ProcessUtils & ProcessUtilsTest * Enhance user experience, add close button, return to the previous page (#4006) Co-authored-by: zhanglong * [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067) * [Fix][api] Fix build parameter error of sqlserver when create. (#4015) * [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog(). * [Fix][api] Fix build parameter error of sqlserver when create. * [Feature-3985][Datax] Datax supports setting up running memory (#3986) * Datax supports setting up running memory * Datax supports setting up running memory * Datax supports setting up running memory * When running a task, the resource file is lost, which results in an error * add unit test * add unit test * add unit test * add test unit * add test unit * add test unit * fix code smell * add test unit * add test unit * [Improvement-3767][api] Task instance supports query by process instance name (#3825) * Task instance supports query by process instance name. * add test code checkstyle. * add test param. * resolve the sonar duplication check. * solve logger single-line string exceeds 200 characters. * resolve the sonar check. * Resolve code conflicts. Co-authored-by: zhuangchong * deal with magic value * [Feature-4050][server] Spark task support udv inject (#4061) * #4050 spark task support udv inject * modify spark task UT * modify sparkTaskExecutionCtx * add exp for spark task get main jar method * deal with magic value * [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481) * JSONUtils.toMap call improvement. * [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error * 解决单独执行子节点空指针的问题 * using OSUtils.execCmd when kill yarn app * 解决单独执行子节点空指针的问题(增加checkstyle) * 解决单独执行子节点空指针的问题(增加checkstyle) * [FIX-#4083][server]fix taskInstance state change error Concurrent processing of ack message and result message causes the execution sequence to be wrong # this close # 4083 * code style * fix replaceNRTtoUnderline NullPointerException #4098 (#4100) * fix replaceNRTtoUnderline NullPointerException * add unit Test * add taskResponseTest * add taskResponseTest * code smell * Time is too small and the task is not finished * Time is too small and the task is not finished * [FIX-4034][server] fix sqoop import fail (#4036) * fix #4043, sqoop import query fail * fix #4043, sqoop task hard code & code style * add license for SqoopConstants * add private constructor for SqoopConstants * fixed sqoop mysql pwd have special character * fix checkstyle * fix sqoop task log * remove unused constants * Time is too small and the task is not finished * Time is too small and the task is not finished * test * remove assert * test * test * fix task instance status judgment error * fix: security page disappear delay problem when force refresh under GENERAL_USER * improvement: resovle download url with resolveURL to prevent change of apiPrefix * fix sqoop task jdbc string contains special char (#4105) * [Bug][Common] read file garbled (#3479) * fix bug : Random code problem Co-authored-by: zhanglong * [Improvement-3933][db operation] Improve the performance of sql query (#3940) * optimize select * case * emove redundancy * bug fixed * Update en_US.js * Update startup.sh * optimize * optimize code * optimize * bug fixed * add ut * bug fixed * bug fixed * bug fixed * bug fixed * Delete WorkFlowLineageMapper.xml * Delete createTenement.vue * recove wrongly deleted file * Update WorkFlowLineageMapper.xml * Update createTenement.vue * [Feature][API]enable response resources gzip compression (#4121) * enable response compression * add server.compression.mime-types with default value explicitly * [Improvement][API] ignore noNodeException when get worker groups (#4120) * ignore noNodeException when get worker groups * add ut * [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118) * FIX SPELL * FIX SPELL AND Optimizing code conventions * add ut cannot construct process instance, return null; * add ut testExportProcessMetaData * add ut testExportProcessMetaData * add ut testImportProcessSchedule * add ut MasterExecThreadTest * add ut MasterExecThreadTest * add ut testSubProcessViewTree * add ut testComplementWithStartNodeList * add ut testRecurseFindSubProcessId * add ut testRecurseFindSubProcessId * add ut testRecurseFindSubProcessId * [FIX#4033] $[] conflicts with mysql keywords (#4111) * [FIX#4033] $[] conflicts with mysql keywords We currently only use this symbol for dates, so I filtered out the number type. this close #4033 * test * fix error * split sqoop import hive database and table (#4141) * upgrade quartz version to 2.3.0 * add HikariCP-java6,c3p0,mchange-commons-java license * upgrade jackson version to 2.9.10 * remove c3p0,mchange-commons-java license * update Check code style * update code style * Update pom.xml * Update pom.xml * Update LICENSE not need to add HikariCP-java6 * Delete LICENSE-HikariCP-java6.txt * Update known-dependencies.txt * Update TaskInstanceMapper.xml optimize page of [TaskInstance] load data slow * [FIX-PR-4097][server-master]task ack miss (#4189) When the message of successful execution arrives earlier than the message of ack, the message of ack will be discarded, resulting in some information missing * fix bug #4125 (#4127) * fix bug #4125 * code style! * code style. Co-authored-by: chengp * [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201) * fix 4190: When the amount of json data is large, process list page display slowly * fix 4190: When the amount of json data is large, process list page display slowly * fix 4190: When the amount of json data is large, process list page display slowly Co-authored-by: baoliang * [Fix-3457][flink] fix flink args build problem (#4166) * [Fix][Flink] fix flink args build problem * [Fix][Flink] fix FlinkArgsUtilsTest * [Improvement][UI] hide version and cluster input when deployMode is local * [common]del windows file( not support windows ) (#4204) * del support win * del support win * [Feature-4138][Master] dispatch workgroup error add sleep time (#4139) * When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second * When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second * fix code smell & code style * fix code smell & code style Co-authored-by: zhanglong * [FIX-#4172][server-worker] kill task NPE (#4182) * [FIX-#4172][server-worker] kill task NPE The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear Modification method: write into the cache when the task is received, and mark it as preData If the task is killed before the Process is generated, delete the cache directly at this time It will be judged before the process is generated. If the task has been killed, it will not be executed. After the new process is created, write it into the cache, and judge again, if kill, then kill the process. this closes #4172 * Delete the commented out code Add spring beans * code smell * add test * add test * fix error * test * test * revert * fix error * [Feature-3878]Replace the page with element-ui (#4065) * [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui * Change node version * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * Replace the page with element-ui * Replace user management with elment-ui * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * Repair the last Sunday of each month * support auto eslint for .js, .vue file on save * update .eslintrc.yml and license check exclude .eslintignore * fix eslint: fix syntax by npm run lint automatically * fix eslint: fatal syntax errors * fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '==' * fix eslint: Unexpected side effect in 'cacheParams' computed property * fix eslint: assigned a value but never used * fix eslint: component has been registered but not used * fix eslint: unexpected mutation of prop * fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219) * deleted invalid code as assigned in issue 4215 (#4221) deleted the code at following two places: org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo * [FIX-#3177]Task time parameter parsing error (#4224) * [FIX-#3177]Task time parameter parsing error rerun schedule time error this closes #3177 * [FIX-#3177]Task time parameter parsing error rerun schedule time error this closes #3177 * fix sql error (#4227) * [FIX-3177]Task time parameter parsing error (#4228) * [FIX-3177]Task time parameter parsing error when system.datetime !=null $[datetime] = system.datetime else $[datetime] = current time * remove unused import * fix date_convert null * fix time cover * fix time cover * reformat * add ut * [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162) * fix(ui): re-login problem in new tab * refresh page in new tab automatically * support to reload router view in lightweight * optimize visibility code * [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225) Add no-var rule for eslint, and fix related files Add npm run lint:fix command, which distinguished from the default command npm run lint The behavior of command npm run lint and command npm run lint:fix should be distinguished The first command is used to lint only, and the second one is used to lint and fix problems automatically * [FIX][UI ] security user state tenantCode User Type Display abnormal (#4255) * fix user state error * fix userType error * fix tenantCode error * [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250) * [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. * [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. * [Fix-4222][Master]Remove useless import * [Fix-4222][Master]Reformat code style * [Fix-4222][Master]Reformat code style * [Fix-4222][Master]Reformat code style * [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest * [Fix-4222][Master]Fix code smell * [Fix-4222][Master]Reformat code style * [Fix-4222][Master]Fix code smell Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> * update bug template (#4260) * [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108) * when the tenant does not exist, the task execution should throw an exception * remote method createWorkDirAndUserIfAbsent * set the task status failed when the tenant code does not exist. * add taskLog. * update check os user exists * update TaskExecuteThreadTest test method. * solving sonar fail. * remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262) * remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters() * remove unused imports in AbstractTask.java * reformat * [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272) * fix version funcation delete. (#4265) * [improvement][config] Update datasource.properties，add mysql meta data template (#4266) * Update datasource.properties [新增]元数据mysql连接模版 * Update datasource.properties Co-authored-by: dailidong * fix taskInstance submitTime is empty. (#4274) * [Improvement-3878]Tenant list delete user name (#4278) * [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269) * [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html * Fix checkstyle error * Update swagger-models in known-dependencies.txt * [Improvement-3878][ui]Fix the list style (#4280) * [Improvement-3878]Tenant list delete user name * [Improvement-3878][ui]Fix the list style * [FIX] [UI] fix create project cancel button invalid (#4282) * fix create project cancel button invalid * [Improvement] Refactor code to support distributed tracing (#4270) * Refactor code to support tracing * Extension network protocol, support context and version * Extension master asynchronous queue support context * Extract scan task method from MasterSchedulerService for tracing * fix * fix * add test case * fix * fix Co-authored-by: hailin0 * [Improvement] Add Flink job name (#4285) * [Improvement] Add Flink job name * fix typo * [Improvement][ui]List vacancy optimization and icon icon repair (#4286) * [Improvement][ui] List vacancy optimization and icon icon repair * [FEATURE-736] integrate ldap authentication (#3743) * [FEATURE-736] integrate ldap authentication * add ldap authentication type * refactor authentication with ldap and password * add createUser for ldap in user service * remove duplicate password authenticator * [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290) * fix flink name not display in process definition editor * fix flink name not escape problem * simplify escape method * fix database is the mysql DB keyword. (#4295) * fix the token management list does not display the user name. (#4302) * Dev imp server process utils (#4263) * Adds the comment on the WINDOWSATTERN Co-authored-by: Kirs Co-authored-by: 0002939 * fix version close function does not take effect. (#4307) * fix user management authorization operation exceptions do not prompt exception information. (#4292) * [Fix][UI] Fix ui style problem and refactor form style (#4329) * rename from-model to form-model * [UI] remove duplicated css style code * [UI] refactor css style of form model * fix form model overflow problem in ie * Align each item one by one in locale * fix el-dialog width is too wide * fix locale * [Improvement][UI] Improve script box and dialog css style (#4331) * [Fix-4335][UI] Fix IE 9-11 not supported * [FIx-4338][UI] Fix invalid date problem in IE * [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341) Co-authored-by: hailin0 * [Fix][common] only two yarns can be selected(#4314) (#4344) * Handling conflicts manually * code style * ut skip ui * fix ut error * fix ut error * delete SchedulerController Unused field * delete SchedulerController Unused field * delete SchedulerController Unused field * ci skip ui * ignore AlertPluginManagerTest.java and DolphinPluginLoaderTest.java * fix ut error * fix ut error * test * code style * replace ui * replace ui * replace ui * replace ui * fix license * fix license Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com> Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com> Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com> Co-authored-by: dailidong Co-authored-by: lgcareer <18610854716@163.com> Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com> Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> Co-authored-by: baoliang Co-authored-by: BoYiZhang Co-authored-by: zixi0825 <649790970@qq.com> Co-authored-by: sunchaohe Co-authored-by: break60 <790061044@qq.com> Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com> Co-authored-by: t1mon <178317391@qq.com> Co-authored-by: zhanglong Co-authored-by: 孙继峰 Co-authored-by: wulingqi Co-authored-by: lgcareer Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com> Co-authored-by: huzekang <1040080742@qq.com> Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> Co-authored-by: Eights-LI Co-authored-by: wuchunfu <319355703@qq.com> Co-authored-by: qiaozhanwei Co-authored-by: Yelli Co-authored-by: Eights-Li Co-authored-by: qiaozhanwei Co-authored-by: XiaotaoYi Co-authored-by: Yichao Yang <1048262223@qq.com> Co-authored-by: zhuangchong Co-authored-by: muzhongjiang Co-authored-by: Jave-Chen Co-authored-by: zhuangchong Co-authored-by: hailin0 Co-authored-by: yangquan Co-authored-by: liliang1991 Co-authored-by: liang.li.c Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com> Co-authored-by: chengshiwen Co-authored-by: Yarlung Co-authored-by: karlsun Co-authored-by: wangxj3 <857234426@qq.com> Co-authored-by: chengp <5058557@qq.com> Co-authored-by: chengp Co-authored-by: Saksham Gupta Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com> Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com> Co-authored-by: hailin0 Co-authored-by: zh0122 Co-authored-by: geosmart Co-authored-by: 李丛阳 Co-authored-by: 0002939 Co-authored-by: kamisamak <1057372918@qq.com> * fix alert-spi error todo sql script * add page query plugin instance add plugin instance check name * add page query plugin instance add plugin instance check name * add props field * ding talk some filed is not required * fix bug * fix error * revert * add name field provide to ui * fix send error * add ui * delete useless constant definitions code style * code style * fix license head error * fix ut error * fix ut error * del unused jar * import alert-plugin * upgrade jboss-logging * upgrade jboss-logging * fix ut error * fix Vulnerabilities * fix Vulnerabilities * fix Vulnerabilities * code smell * fix Vulnerabilities * add public constructors * fix Vulnerabilities * resolve conflicts Co-authored-by: break60 <790061044@qq.com> Co-authored-by: gaojun2048 <32193458+gaojun2048@users.noreply.github.com> Co-authored-by: zhuangchong Co-authored-by: dailidong Co-authored-by: samz406 Co-authored-by: gaojun2048 <540957506@qq.com> Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com> Co-authored-by: Yichao Yang <1048262223@qq.com> Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com> Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com> Co-authored-by: lgcareer <18610854716@163.com> Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com> Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> Co-authored-by: baoliang Co-authored-by: BoYiZhang Co-authored-by: zixi0825 <649790970@qq.com> Co-authored-by: sunchaohe Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com> Co-authored-by: t1mon <178317391@qq.com> Co-authored-by: zhanglong Co-authored-by: 孙继峰 Co-authored-by: wulingqi Co-authored-by: lgcareer Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com> Co-authored-by: huzekang <1040080742@qq.com> Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> Co-authored-by: Eights-LI Co-authored-by: wuchunfu <319355703@qq.com> Co-authored-by: qiaozhanwei Co-authored-by: Yelli Co-authored-by: Eights-Li Co-authored-by: qiaozhanwei Co-authored-by: XiaotaoYi Co-authored-by: zhuangchong Co-authored-by: muzhongjiang Co-authored-by: Jave-Chen Co-authored-by: hailin0 Co-authored-by: yangquan Co-authored-by: liliang1991 Co-authored-by: liang.li.c Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com> Co-authored-by: chengshiwen Co-authored-by: Yarlung Co-authored-by: karlsun Co-authored-by: wangxj3 <857234426@qq.com> Co-authored-by: chengp <5058557@qq.com> Co-authored-by: chengp Co-authored-by: Saksham Gupta Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com> Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com> Co-authored-by: hailin0 Co-authored-by: zh0122 Co-authored-by: geosmart Co-authored-by: 李丛阳 Co-authored-by: 0002939 Co-authored-by: kamisamak <1057372918@qq.com> --- LICENSE | 4 + .../dolphinscheduler/requirements.yaml~HEAD | 25 + .../dolphinscheduler/requirements.yaml~dev | 25 + .../dolphinscheduler-alert-dingtalk/pom.xml | 82 ++ .../alert/dingtalk/DingTalkAlertChannel.java | 41 + .../dingtalk/DingTalkAlertChannelFactory.java | 89 ++ .../alert/dingtalk/DingTalkAlertPlugin.java | 34 + .../dingtalk/DingTalkParamsConstants.java | 57 ++ .../plugin/alert/dingtalk/DingTalkSender.java | 204 +++++ .../DingTalkAlertChannelFactoryTest.java | 48 ++ .../alert/dingtalk/DingTalkSenderTest.java | 57 ++ .../dolphinscheduler-alert-email/pom.xml | 128 +++ .../plugin/alert/email/EmailAlertChannel.java | 69 ++ .../alert/email/EmailAlertChannelFactory.java | 137 ++++ .../plugin/alert/email/EmailAlertPlugin.java | 33 + .../plugin/alert/email/EmailConstants.java | 75 ++ .../plugin/alert/email}/ExcelUtils.java | 24 +- .../alert/email/MailParamsConstants.java | 65 ++ .../plugin/alert/email/MailSender.java | 429 ++++++++++ .../alert/email}/template/AlertTemplate.java | 21 +- .../email/template}/DefaultHTMLTemplate.java | 53 +- .../email/EmailAlertChannelFactoryTest.java | 75 ++ .../alert/email/EmailAlertChannelTest.java | 161 ++++ .../plugin/alert/email}/ExcelUtilsTest.java | 10 +- .../plugin/alert/email/MailUtilsTest.java | 136 ++++ .../template/DefaultHTMLTemplateTest.java | 105 +++ .../dolphinscheduler-alert-http/pom.xml | 76 ++ .../plugin/alert/http/HttpAlertChannel.java | 41 + .../alert/http/HttpAlertChannelFactory.java | 78 ++ .../plugin/alert/http/HttpAlertConstants.java | 35 + .../plugin/alert/http/HttpAlertPlugin.java | 34 + .../plugin/alert/http/HttpSender.java | 166 ++++ .../http/HttpAlertChannelFactoryTest.java | 46 +- .../alert/http/HttpAlertChannelTest.java | 104 +++ .../alert/http/HttpAlertPluginTest.java | 38 + .../plugin/alert/http/HttpSenderTest.java | 46 ++ .../dolphinscheduler-alert-script/pom.xml | 79 ++ .../plugin/alert/script/OSUtils.java | 32 + .../plugin/alert/script/ProcessUtils.java | 62 ++ .../alert/script/ScriptAlertChannel.java | 40 + .../script/ScriptAlertChannelFactory.java | 71 ++ .../alert/script/ScriptAlertPlugin.java | 35 + .../alert/script/ScriptParamsConstants.java | 40 + .../plugin/alert/script/ScriptSender.java | 74 ++ .../plugin/alert/script/ScriptType.java | 49 +- .../plugin/alert/script/StreamGobbler.java | 53 +- .../plugin/alert/script/ProcessUtilsTest.java | 37 + .../script/ScriptAlertChannelFactoryTest.java | 48 ++ .../plugin/alert/script/ScriptSenderTest.java | 58 ++ .../src/test/script/shell/example.sh | 25 + .../src/test/script/shell/scriptTest.sh | 13 +- .../dolphinscheduler-alert-wechat/pom.xml | 78 ++ .../alert/wechat/WeChatAlertChannel.java | 41 + .../wechat/WeChatAlertChannelFactory.java | 94 +++ .../alert/wechat/WeChatAlertConstants.java | 34 + .../wechat/WeChatAlertParamsConstants.java | 56 ++ .../alert/wechat/WeChatAlertPlugin.java | 34 + .../plugin/alert/wechat/WeChatSender.java | 335 ++++++++ .../wechat/WeChatAlertChannelFactoryTest.java | 48 ++ .../plugin/alert/wechat/WeChatSenderTest.java | 89 ++ dolphinscheduler-alert-plugin/pom.xml | 41 + dolphinscheduler-alert/pom.xml | 79 +- .../dolphinscheduler/alert/AlertServer.java | 126 ++- .../alert/manager/EmailManager.java | 110 +-- .../manager/EnterpriseWeChatManager.java | 59 -- .../alert/manager/MsgManager.java | 10 +- .../plugin/AbstractDolphinPluginManager.java | 29 + .../alert/plugin/AlertPluginManager.java | 99 +++ .../plugin/DolphinPluginClassLoader.java | 139 ++++ .../alert/plugin/DolphinPluginDiscovery.java | 139 ++++ .../alert/plugin/DolphinPluginLoader.java | 194 +++++ .../plugin/DolphinPluginManagerConfig.java | 121 +++ .../alert/plugin/EmailAlertPlugin.java | 147 ---- .../processor/AlertRequestProcessor.java | 67 ++ .../alert/runner/AlertSender.java | 174 ++-- .../alert/utils/Constants.java | 172 +--- .../alert/utils/DingTalkUtils.java | 141 ---- .../alert/utils/EnterpriseWeChatUtils.java | 286 ------- .../alert/utils/MailUtils.java | 375 --------- .../alert/utils/PropertyUtils.java | 2 +- .../src/main/resources/alert.properties | 49 +- .../alert/AlertServerTest.java | 91 +++ .../alert/plugin/AlertPluginManagerTest.java | 65 ++ .../alert/plugin/DolphinPluginLoaderTest.java | 60 ++ .../alert/plugin/EmailAlertPluginTest.java | 253 ++++-- .../processor/AlertRequestProcessorTest.java | 61 ++ .../alert/runner/AlertSenderTest.java | 181 +++++ .../template/AlertTemplateFactoryTest.java | 65 -- .../impl/DefaultHTMLTemplateTest.java | 96 --- .../alert/utils/DingTalkUtilsTest.java | 120 --- .../utils/EnterpriseWeChatUtilsTest.java | 283 ------- .../alert/utils/FuncUtilsTest.java | 10 +- .../alert/utils/MailUtilsTest.java | 190 ----- .../api/controller/AlertGroupController.java | 113 ++- .../AlertPluginInstanceController.java | 240 ++++++ .../api/controller/ExecutorController.java | 113 ++- .../api/controller/SchedulerController.java | 132 +-- .../api/controller/UiPluginController.java | 93 +++ .../dolphinscheduler/api/enums/Status.java | 17 +- .../api/service/AlertGroupService.java | 84 +- .../service/AlertPluginInstanceService.java | 89 ++ .../api/service/ExecutorService.java | 47 +- .../api/service/ProcessInstanceService.java | 3 +- .../api/service/SchedulerService.java | 97 ++- .../api/service/UiPluginService.java | 14 +- .../api/service/UsersService.java | 10 +- .../impl/AlertPluginInstanceServiceImpl.java | 208 +++++ .../impl/ProcessDefinitionServiceImpl.java | 7 +- .../ProcessDefinitionVersionServiceImpl.java | 23 +- .../api/service/impl/UiPluginServiceImpl.java | 85 ++ .../api/vo/AlertPluginInstanceVO.java | 117 +++ .../main/resources/i18n/messages.properties | 5 + .../resources/i18n/messages_en_US.properties | 5 + .../controller/ExecutorControllerTest.java | 100 +-- .../TaskInstanceControllerTest.java | 1 + .../security/SecurityConfigLDAPTest.java~HEAD | 45 ++ .../security/SecurityConfigLDAPTest.java~dev | 45 ++ .../api/service/AlertGroupServiceTest.java | 129 ++- .../api/service/ExecutorService2Test.java | 147 ++-- .../ProcessDefinitionVersionServiceTest.java | 23 +- .../service/ProcessInstanceServiceTest.java | 2 - .../api/service/UiPluginServiceTest.java | 86 ++ .../service/UserAlertGroupServiceTest.java | 53 -- dolphinscheduler-common/pom.xml | 5 - .../dolphinscheduler/common/Constants.java | 8 +- .../common/enums/PluginType.java | 70 ++ .../common/plugin/FilePluginManager.java | 107 --- .../common/plugin/PluginClassLoader.java | 154 ---- .../common/task/sql/SqlParameters.java | 67 +- .../common/plugin/FilePluginManagerTest.java | 72 -- .../common/plugin/PluginClassLoaderTest.java | 61 -- .../apache/dolphinscheduler/dao/AlertDao.java | 102 +-- .../dolphinscheduler/dao/PluginDao.java | 96 +++ .../dolphinscheduler/dao/entity/Alert.java | 121 +-- .../dao/entity/AlertGroup.java | 71 +- ...ertGroup.java => AlertPluginInstance.java} | 104 ++- .../dao/entity/PluginDefine.java | 125 +++ .../dao/entity/ProcessDefinition.java | 93 +-- .../dao/entity/ProcessDefinitionVersion.java | 88 +- .../dao/entity/ProcessInstance.java | 36 +- .../dao/mapper/AlertGroupMapper.java | 13 +- ...er.java => AlertPluginInstanceMapper.java} | 32 +- .../dao/mapper/PluginDefineMapper.java | 61 ++ .../src/main/resources/datasource.properties | 8 +- .../dao/mapper/AlertGroupMapper.xml | 19 +- .../dao/mapper/AlertMapper.xml | 5 +- ...pper.xml => AlertPluginInstanceMapper.xml} | 44 +- .../dao/mapper/PluginDefineMapper.xml | 45 ++ .../dao/mapper/ProcessDefinitionMapper.xml | 13 +- .../mapper/ProcessDefinitionVersionMapper.xml | 5 +- .../dao/mapper/ProjectMapper.xml | 1 + .../dolphinscheduler/dao/AlertDaoTest.java | 21 +- .../dao/mapper/AlertGroupMapperTest.java | 189 ++--- .../dao/mapper/AlertMapperTest.java | 67 +- .../mapper/AlertPluginInstanceMapperTest.java | 112 +++ .../dao/mapper/PluginDefineTest.java | 77 ++ .../mapper/ProcessDefinitionMapperTest.java | 4 - .../ProcessDefinitionVersionMapperTest.java | 4 - .../dao/mapper/UserAlertGroupMapperTest.java | 196 ----- .../dao/mapper/UserMapperTest.java | 122 +-- dolphinscheduler-dist/pom.xml | 2 +- dolphinscheduler-dist/release-docs/LICENSE | 12 +- dolphinscheduler-dist/release-docs/NOTICE | 23 - .../licenses/LICENSE-aether-api.txt | 86 ++ .../LICENSE-animal-sniffer-annotations.txt | 21 + .../licenses/LICENSE-checker-compat-qual.txt | 22 + .../licenses/LICENSE-j2objc-annotations.txt | 201 +++++ .../licenses/LICENSE-javax.mail.txt | 759 ------------------ .../licenses/LICENSE-resolver.txt | 202 +++++ .../LICENSE-@form-create-element-ui | 21 + .../src/main/provisio/dolphinscheduler.xml | 67 ++ .../plugin/utils/PropertyUtils.java | 190 ----- .../plugin/model/AlertDataTest.java | 80 -- .../plugin/utils/PropertyUtilsTest.java | 83 -- .../remote/command/CommandType.java | 12 +- .../alert/AlertSendRequestCommand.java | 80 ++ .../alert/AlertSendResponseCommand.java | 75 ++ .../alert/AlertSendResponseResult.java | 52 ++ .../remote/utils/JsonSerializer.java | 96 +++ .../alert/AlertSendRequestCommandTest.java | 42 + .../alert/AlertSendResponseCommandTest.java | 48 ++ dolphinscheduler-server/pom.xml | 325 ++++---- .../server/log/TaskLogAppender.java | 8 +- .../runner/MasterBaseTaskExecThread.java | 64 +- .../master/runner/MasterTaskExecThread.java | 59 +- .../server/utils/AlertManager.java | 14 +- .../server/worker/WorkerServer.java | 23 +- .../server/worker/config/WorkerConfig.java | 19 +- .../processor/TaskExecuteProcessor.java | 18 +- .../worker/runner/TaskExecuteThread.java | 11 +- .../server/worker/task/TaskManager.java | 6 +- .../server/worker/task/sql/SqlTask.java | 103 +-- .../src/main/resources/worker.properties | 3 + .../server/registry/DependencyConfig.java | 84 +- .../TaskCallbackServiceTestConfig.java | 71 +- .../worker/runner/TaskExecuteThreadTest.java | 17 +- .../server/worker/task/TaskManagerTest.java | 27 +- .../server/worker/task/sql/SqlTaskTest.java | 111 +++ .../service/alert/AlertClientService.java | 120 +++ .../service/process/ProcessService.java | 5 +- .../service/alert/AlertClientServiceTest.java | 152 ++++ .../pom.xml | 36 +- .../spi/DolphinSchedulerPlugin.java | 38 + .../spi/alert/AlertChannel.java | 17 +- .../spi/alert/AlertChannelFactory.java | 52 ++ .../spi/alert/AlertConstants.java | 24 + .../spi/alert}/AlertData.java | 56 +- .../dolphinscheduler/spi/alert/AlertInfo.java | 37 +- .../spi/alert/AlertResult.java | 30 +- .../dolphinscheduler/spi/alert}/ShowType.java | 8 +- .../classloader/ThreadContextClassLoader.java | 27 +- .../spi/params/InputParam.java | 94 +++ .../spi/params/PasswordParam.java | 87 ++ .../spi/params/PluginParamsTransfer.java | 51 ++ .../spi/params/RadioParam.java | 106 +++ .../spi/params/base/DataType.java | 27 +- .../spi/params/base/FormType.java | 38 + .../spi/params/base/ParamsOptions.java | 72 ++ .../spi/params/base/ParamsProps.java | 50 ++ .../spi/params/base/PluginParams.java | 170 ++++ .../spi/params/base/PropsType.java | 37 + .../spi/params/base/TriggerType.java | 36 + .../spi/params/base/Validate.java | 137 ++++ .../dolphinscheduler/spi/utils/JSONUtils.java | 156 ++++ .../spi/utils/StringUtils.java | 41 + .../spi/params/PluginParamsTransferTest.java | 201 +++++ install.sh | 2 + pom.xml | 124 ++- sql/dolphinscheduler-postgre.sql | 95 ++- sql/dolphinscheduler_mysql.sql | 91 ++- .../mysql/dolphinscheduler_ddl.sql | 69 ++ .../mysql/dolphinscheduler_dml.sql | 18 + .../postgresql/dolphinscheduler_ddl.sql | 65 ++ .../postgresql/dolphinscheduler_dml.sql | 16 + style/checkstyle.xml | 8 +- style/intellij-java-code-style.xml | 2 - tools/dependencies/known-dependencies.txt | 13 +- 237 files changed, 12419 insertions(+), 6087 deletions(-) create mode 100644 docker/kubernetes/dolphinscheduler/requirements.yaml~HEAD create mode 100644 docker/kubernetes/dolphinscheduler/requirements.yaml~dev create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/pom.xml create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannel.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactory.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertPlugin.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkParamsConstants.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSender.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactoryTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSenderTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannel.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactory.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertPlugin.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailConstants.java rename {dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils => dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email}/ExcelUtils.java (87%) create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailParamsConstants.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailSender.java rename {dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert => dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email}/template/AlertTemplate.java (72%) rename {dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/impl => dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template}/DefaultHTMLTemplate.java (66%) create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactoryTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelTest.java rename {dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils => dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email}/ExcelUtilsTest.java (89%) create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/MailUtilsTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplateTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/pom.xml create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannel.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactory.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertConstants.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPlugin.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSender.java rename dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertInfoTest.java => dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactoryTest.java (53%) create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPluginTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSenderTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/pom.xml create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/OSUtils.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtils.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannel.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactory.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertPlugin.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptParamsConstants.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSender.java rename dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertInfo.java => dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptType.java (51%) rename dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/DingTalkManager.java => dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/StreamGobbler.java (50%) create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtilsTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactoryTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSenderTest.java create mode 100755 dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/example.sh rename dolphinscheduler-plugin-api/src/test/resources/plugin.properties => dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/scriptTest.sh (91%) mode change 100644 => 100755 create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannel.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactory.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertConstants.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertParamsConstants.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertPlugin.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSender.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactoryTest.java create mode 100644 dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSenderTest.java create mode 100644 dolphinscheduler-alert-plugin/pom.xml delete mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EnterpriseWeChatManager.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AbstractDolphinPluginManager.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManager.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginClassLoader.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginDiscovery.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoader.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginManagerConfig.java delete mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPlugin.java create mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessor.java delete mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtils.java delete mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtils.java delete mode 100644 dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/MailUtils.java create mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/AlertServerTest.java create mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManagerTest.java create mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoaderTest.java create mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessorTest.java create mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/runner/AlertSenderTest.java delete mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactoryTest.java delete mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplateTest.java delete mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtilsTest.java delete mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtilsTest.java delete mode 100644 dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/MailUtilsTest.java create mode 100644 dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertPluginInstanceController.java create mode 100644 dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UiPluginController.java create mode 100644 dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertPluginInstanceService.java rename dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginManager.java => dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UiPluginService.java (74%) create mode 100644 dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/AlertPluginInstanceServiceImpl.java create mode 100644 dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UiPluginServiceImpl.java create mode 100644 dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo/AlertPluginInstanceVO.java create mode 100644 dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~HEAD create mode 100644 dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~dev create mode 100644 dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UiPluginServiceTest.java delete mode 100644 dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UserAlertGroupServiceTest.java create mode 100644 dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/PluginType.java delete mode 100644 dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/FilePluginManager.java delete mode 100644 dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoader.java delete mode 100644 dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/FilePluginManagerTest.java delete mode 100644 dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoaderTest.java create mode 100644 dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/PluginDao.java rename dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/{UserAlertGroup.java => AlertPluginInstance.java} (51%) create mode 100644 dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/PluginDefine.java rename dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/{UserAlertGroupMapper.java => AlertPluginInstanceMapper.java} (62%) create mode 100644 dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.java rename dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/{UserAlertGroupMapper.xml => AlertPluginInstanceMapper.xml} (52%) create mode 100644 dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.xml create mode 100644 dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapperTest.java create mode 100644 dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineTest.java delete mode 100644 dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapperTest.java create mode 100644 dolphinscheduler-dist/release-docs/licenses/LICENSE-aether-api.txt create mode 100644 dolphinscheduler-dist/release-docs/licenses/LICENSE-animal-sniffer-annotations.txt create mode 100644 dolphinscheduler-dist/release-docs/licenses/LICENSE-checker-compat-qual.txt create mode 100644 dolphinscheduler-dist/release-docs/licenses/LICENSE-j2objc-annotations.txt delete mode 100644 dolphinscheduler-dist/release-docs/licenses/LICENSE-javax.mail.txt create mode 100644 dolphinscheduler-dist/release-docs/licenses/LICENSE-resolver.txt create mode 100644 dolphinscheduler-dist/release-docs/licenses/ui-licenses/LICENSE-@form-create-element-ui create mode 100644 dolphinscheduler-dist/src/main/provisio/dolphinscheduler.xml delete mode 100644 dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtils.java delete mode 100644 dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertDataTest.java delete mode 100644 dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtilsTest.java create mode 100644 dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommand.java create mode 100644 dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommand.java create mode 100644 dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseResult.java create mode 100644 dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/JsonSerializer.java create mode 100644 dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommandTest.java create mode 100644 dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommandTest.java mode change 100644 => 100755 dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/log/TaskLogAppender.java create mode 100644 dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTaskTest.java create mode 100644 dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/alert/AlertClientService.java create mode 100644 dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/alert/AlertClientServiceTest.java rename {dolphinscheduler-plugin-api => dolphinscheduler-spi}/pom.xml (64%) create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/DolphinSchedulerPlugin.java rename dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/spi/AlertPluginProvider.java => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannel.java (74%) create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannelFactory.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertConstants.java rename {dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert}/AlertData.java (61%) rename dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/api/AlertPlugin.java => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertInfo.java (60%) rename dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/PluginName.java => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertResult.java (63%) rename {dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert}/ShowType.java (89%) rename dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UserAlertGroupService.java => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/classloader/ThreadContextClassLoader.java (58%) create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/InputParam.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PasswordParam.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransfer.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/RadioParam.java rename dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactory.java => dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/DataType.java (64%) create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/FormType.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsOptions.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsProps.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PluginParams.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PropsType.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/TriggerType.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/Validate.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/JSONUtils.java create mode 100644 dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/StringUtils.java create mode 100644 dolphinscheduler-spi/src/test/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransferTest.java create mode 100644 sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql create mode 100644 sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_dml.sql create mode 100644 sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_ddl.sql create mode 100644 sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_dml.sql diff --git a/LICENSE b/LICENSE index 9a0c6aa66a..5d7c9f4cf7 100644 --- a/LICENSE +++ b/LICENSE @@ -216,3 +216,7 @@ The text of each license is the standard Apache 2.0 license. ScriptRunner from https://github.com/mybatis/mybatis-3 Apache 2.0 mvnw files from https://github.com/takari/maven-wrapper Apache 2.0 PropertyPlaceholderHelper from https://github.com/spring-projects/spring-framework Apache 2.0 + DolphinPluginClassLoader from https://github.com/prestosql/presto Apache 2.0 + DolphinPluginDiscovery from https://github.com/prestosql/presto Apache 2.0 + DolphinPluginLoader from https://github.com/prestosql/presto Apache 2.0 + diff --git a/docker/kubernetes/dolphinscheduler/requirements.yaml~HEAD b/docker/kubernetes/dolphinscheduler/requirements.yaml~HEAD new file mode 100644 index 0000000000..e219975995 --- /dev/null +++ b/docker/kubernetes/dolphinscheduler/requirements.yaml~HEAD @@ -0,0 +1,25 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +dependencies: +- name: postgresql + version: 8.x.x + repository: https://charts.bitnami.com/bitnami + condition: postgresql.enabled +- name: zookeeper + version: 5.x.x + repository: https://charts.bitnami.com/bitnami + condition: redis.enabled \ No newline at end of file diff --git a/docker/kubernetes/dolphinscheduler/requirements.yaml~dev b/docker/kubernetes/dolphinscheduler/requirements.yaml~dev new file mode 100644 index 0000000000..e219975995 --- /dev/null +++ b/docker/kubernetes/dolphinscheduler/requirements.yaml~dev @@ -0,0 +1,25 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +dependencies: +- name: postgresql + version: 8.x.x + repository: https://charts.bitnami.com/bitnami + condition: postgresql.enabled +- name: zookeeper + version: 5.x.x + repository: https://charts.bitnami.com/bitnami + condition: redis.enabled \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/pom.xml b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/pom.xml new file mode 100644 index 0000000000..930b92f855 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/pom.xml @@ -0,0 +1,82 @@ + + + + + dolphinscheduler-alert-plugin + org.apache.dolphinscheduler + 1.3.4-SNAPSHOT + + 4.0.0 + + org.apache.dolphinscheduler + dolphinscheduler-alert-dingtalk + dolphinscheduler-plugin + + + + + org.apache.dolphinscheduler + dolphinscheduler-spi + provided + + + org.apache.httpcomponents + httpclient + + + com.google.guava + guava + + + + ch.qos.logback + logback-classic + + + + org.slf4j + slf4j-api + + + + com.fasterxml.jackson.core + jackson-annotations + provided + + + + junit + junit + test + + + + org.mockito + mockito-core + jar + test + + + + + dolphinscheduler-alert-dingtalk-${project.version} + + + \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannel.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannel.java new file mode 100644 index 0000000000..7adfacce39 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannel.java @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; + +import java.util.Map; + +/** + * DingTalkAlertChannel + */ +public class DingTalkAlertChannel implements AlertChannel { + + @Override + public AlertResult process(AlertInfo alertInfo) { + + AlertData alertData = alertInfo.getAlertData(); + String alertParams = alertInfo.getAlertParams(); + Map paramsMap = PluginParamsTransfer.getPluginParamsMap(alertParams); + return new DingTalkSender(paramsMap).sendDingTalkMsg(alertData.getTitle(), alertData.getContent()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactory.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactory.java new file mode 100644 index 0000000000..0a6851a4ac --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactory.java @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.PasswordParam; +import org.apache.dolphinscheduler.spi.params.RadioParam; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.Arrays; +import java.util.List; + +/** + * DingTalkAlertChannelFactory + */ +public class DingTalkAlertChannelFactory implements AlertChannelFactory { + @Override + public String getName() { + return "DingTalk"; + } + + @Override + public List getParams() { + InputParam webHookParam = InputParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_WEB_HOOK, DingTalkParamsConstants.DING_TALK_WEB_HOOK) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + InputParam keywordParam = InputParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_KEYWORD, DingTalkParamsConstants.DING_TALK_KEYWORD) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + RadioParam isEnableProxy = + RadioParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_PROXY_ENABLE, DingTalkParamsConstants.NAME_DING_TALK_PROXY_ENABLE) + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(true) + .addValidate(Validate.newBuilder() + .setRequired(false) + .build()) + .build(); + InputParam proxyParam = + InputParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_PROXY, DingTalkParamsConstants.DING_TALK_PROXY) + .addValidate(Validate.newBuilder() + .setRequired(false).build()) + .build(); + + InputParam portParam = InputParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_PORT, DingTalkParamsConstants.DING_TALK_PORT) + .addValidate(Validate.newBuilder() + .setRequired(false).build()) + .build(); + + InputParam userParam = + InputParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_USER, DingTalkParamsConstants.DING_TALK_USER) + .addValidate(Validate.newBuilder() + .setRequired(false).build()) + .build(); + PasswordParam passwordParam = PasswordParam.newBuilder(DingTalkParamsConstants.NAME_DING_TALK_PASSWORD, DingTalkParamsConstants.DING_TALK_PASSWORD) + .setPlaceholder("if enable use authentication, you need input password") + .build(); + + return Arrays.asList(webHookParam, keywordParam, isEnableProxy, proxyParam, portParam, userParam, passwordParam); + } + + @Override + public AlertChannel create() { + return new DingTalkAlertChannel(); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertPlugin.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertPlugin.java new file mode 100644 index 0000000000..2d15f499c9 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertPlugin.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +import com.google.common.collect.ImmutableList; + +/** + * DingTalkAlertPlugin + */ +public class DingTalkAlertPlugin implements DolphinSchedulerPlugin { + + @Override + public Iterable getAlertChannelFactorys() { + return ImmutableList.of(new DingTalkAlertChannelFactory()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkParamsConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkParamsConstants.java new file mode 100644 index 0000000000..fb9f5f9f5d --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkParamsConstants.java @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +/** + * DingTalkParamsConstants + */ +public class DingTalkParamsConstants { + + public DingTalkParamsConstants() { + throw new IllegalStateException("Utility class"); + } + + static final String DING_TALK_WEB_HOOK = "dingtalk.webhook"; + + static final String NAME_DING_TALK_WEB_HOOK = "dingTalkWebHook"; + + static final String DING_TALK_KEYWORD = "dingtalk.keyword"; + + static final String NAME_DING_TALK_KEYWORD = "dingTalkKeyword"; + + public static final String DING_TALK_PROXY_ENABLE = "dingtalk.isEnableProxy"; + + static final String NAME_DING_TALK_PROXY_ENABLE = "dingTalkIsEnableProxy"; + + static final String DING_TALK_PROXY = "dingtalk.proxy"; + + static final String NAME_DING_TALK_PROXY = "dingTalkProxy"; + + static final String DING_TALK_PORT = "dingtalk.port"; + + static final String NAME_DING_TALK_PORT = "dingTalkPort"; + + static final String DING_TALK_USER = "dingtalk.user"; + + static final String NAME_DING_TALK_USER = "dingTalkUser"; + + static final String DING_TALK_PASSWORD = "dingtalk.password"; + + static final String NAME_DING_TALK_PASSWORD = "dingTalkPassword"; + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSender.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSender.java new file mode 100644 index 0000000000..4500f3e28e --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/main/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSender.java @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import org.apache.commons.codec.binary.StringUtils; +import org.apache.http.HttpEntity; +import org.apache.http.HttpHost; +import org.apache.http.auth.AuthScope; +import org.apache.http.auth.UsernamePasswordCredentials; +import org.apache.http.client.CredentialsProvider; +import org.apache.http.client.config.RequestConfig; +import org.apache.http.client.methods.CloseableHttpResponse; +import org.apache.http.client.methods.HttpPost; +import org.apache.http.entity.StringEntity; +import org.apache.http.impl.client.BasicCredentialsProvider; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.http.util.EntityUtils; + +import java.io.IOException; +import java.util.HashMap; +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Ding Talk Sender + */ +public class DingTalkSender { + private static final Logger logger = LoggerFactory.getLogger(DingTalkSender.class); + + private String url; + + private String keyword; + + private Boolean enableProxy; + + private String proxy; + + private Integer port; + + private String user; + + private String password; + + DingTalkSender(Map config) { + url = config.get(DingTalkParamsConstants.NAME_DING_TALK_WEB_HOOK); + keyword = config.get(DingTalkParamsConstants.NAME_DING_TALK_KEYWORD); + enableProxy = Boolean.valueOf(config.get(DingTalkParamsConstants.NAME_DING_TALK_PROXY_ENABLE)); + if (Boolean.TRUE.equals(enableProxy)) { + port = Integer.parseInt(config.get(DingTalkParamsConstants.NAME_DING_TALK_PORT)); + proxy = config.get(DingTalkParamsConstants.NAME_DING_TALK_PROXY); + user = config.get(DingTalkParamsConstants.DING_TALK_USER); + password = config.get(DingTalkParamsConstants.NAME_DING_TALK_PASSWORD); + } + + } + + public AlertResult sendDingTalkMsg(String msg, String charset) { + AlertResult alertResult; + try { + String resp = sendMsg(msg, charset); + return checkSendDingTalkSendMsgResult(resp); + } catch (Exception e) { + logger.info("send ding talk alert msg exception : {}", e.getMessage()); + alertResult = new AlertResult(); + alertResult.setStatus("false"); + alertResult.setMessage("send ding talk alert fail."); + } + return alertResult; + } + + private String sendMsg(String msg, String charset) throws IOException { + + String msgToJson = textToJsonString(msg + "#" + keyword); + HttpPost httpPost = constructHttpPost(url, msgToJson, charset); + + CloseableHttpClient httpClient; + if (Boolean.TRUE.equals(enableProxy)) { + httpClient = getProxyClient(proxy, port, user, password); + RequestConfig rcf = getProxyConfig(proxy, port); + httpPost.setConfig(rcf); + } else { + httpClient = getDefaultClient(); + } + + try { + CloseableHttpResponse response = httpClient.execute(httpPost); + String resp; + try { + HttpEntity entity = response.getEntity(); + resp = EntityUtils.toString(entity, charset); + EntityUtils.consume(entity); + } finally { + response.close(); + } + logger.info("Ding Talk send [ %s ], resp:{%s}", msg, resp); + return resp; + } finally { + httpClient.close(); + } + } + + private static HttpPost constructHttpPost(String url, String msg, String charset) { + HttpPost post = new HttpPost(url); + StringEntity entity = new StringEntity(msg, charset); + post.setEntity(entity); + post.addHeader("Content-Type", "application/json; charset=utf-8"); + return post; + } + + private static CloseableHttpClient getProxyClient(String proxy, int port, String user, String password) { + HttpHost httpProxy = new HttpHost(proxy, port); + CredentialsProvider provider = new BasicCredentialsProvider(); + provider.setCredentials(new AuthScope(httpProxy), new UsernamePasswordCredentials(user, password)); + return HttpClients.custom().setDefaultCredentialsProvider(provider).build(); + } + + private static CloseableHttpClient getDefaultClient() { + return HttpClients.createDefault(); + } + + private static RequestConfig getProxyConfig(String proxy, int port) { + HttpHost httpProxy = new HttpHost(proxy, port); + return RequestConfig.custom().setProxy(httpProxy).build(); + } + + private static String textToJsonString(String text) { + Map items = new HashMap<>(); + items.put("msgtype", "text"); + Map textContent = new HashMap<>(); + byte[] byt = StringUtils.getBytesUtf8(text); + String txt = StringUtils.newStringUtf8(byt); + textContent.put("content", txt); + items.put("text", textContent); + return JSONUtils.toJsonString(items); + } + + public static class DingTalkSendMsgResponse { + private Integer errcode; + private String errmsg; + + public Integer getErrcode() { + return errcode; + } + + public void setErrcode(Integer errcode) { + this.errcode = errcode; + } + + public String getErrmsg() { + return errmsg; + } + + public void setErrmsg(String errmsg) { + this.errmsg = errmsg; + } + } + + private static AlertResult checkSendDingTalkSendMsgResult(String result) { + AlertResult alertResult = new AlertResult(); + alertResult.setStatus("false"); + + if (null == result) { + alertResult.setMessage("send ding talk msg error"); + logger.info("send ding talk msg error,ding talk server resp is null"); + return alertResult; + } + DingTalkSendMsgResponse sendMsgResponse = JSONUtils.parseObject(result, DingTalkSendMsgResponse.class); + if (null == sendMsgResponse) { + alertResult.setMessage("send ding talk msg fail"); + logger.info("send ding talk msg error,resp error"); + return alertResult; + } + if (sendMsgResponse.errcode == 0) { + alertResult.setStatus("true"); + alertResult.setMessage("send ding talk msg success"); + return alertResult; + } + alertResult.setMessage(String.format("alert send ding talk msg error : %s", sendMsgResponse.getErrmsg())); + logger.info("alert send ding talk msg error : {}", sendMsgResponse.getErrmsg()); + return alertResult; + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactoryTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactoryTest.java new file mode 100644 index 0000000000..2a26daad63 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkAlertChannelFactoryTest.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +/** + * DingTalkAlertChannelFactoryTest + */ +public class DingTalkAlertChannelFactoryTest { + + @Test + public void testGetParams() { + DingTalkAlertChannelFactory dingTalkAlertChannelFactory = new DingTalkAlertChannelFactory(); + List params = dingTalkAlertChannelFactory.getParams(); + JSONUtils.toJsonString(params); + Assert.assertEquals(7, params.size()); + } + + @Test + public void testCreate() { + DingTalkAlertChannelFactory dingTalkAlertChannelFactory = new DingTalkAlertChannelFactory(); + AlertChannel alertChannel = dingTalkAlertChannelFactory.create(); + Assert.assertNotNull(alertChannel); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSenderTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSenderTest.java new file mode 100644 index 0000000000..4512402dc4 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-dingtalk/src/test/java/org/apache/dolphinscheduler/plugin/alert/dingtalk/DingTalkSenderTest.java @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.dingtalk; + +import org.apache.dolphinscheduler.spi.alert.AlertResult; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +/** + * DingTalkSenderTest + */ +public class DingTalkSenderTest { + + private static Map dingTalkConfig = new HashMap<>(); + + @Before + public void initDingTalkConfig() { + + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_KEYWORD, "keyWord"); + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_WEB_HOOK, "url"); + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_PROXY_ENABLE, "false"); + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_PASSWORD, "password"); + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_PORT, "9988"); + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_USER, "user1,user2"); + } + + @Test + public void testSend() { + DingTalkSender dingTalkSender = new DingTalkSender(dingTalkConfig); + dingTalkSender.sendDingTalkMsg("keyWord+Welcome", "UTF-8"); + dingTalkConfig.put(DingTalkParamsConstants.NAME_DING_TALK_PROXY_ENABLE, "true"); + dingTalkSender = new DingTalkSender(dingTalkConfig); + AlertResult alertResult = dingTalkSender.sendDingTalkMsg("keyWord+Welcome", "UTF-8"); + Assert.assertEquals("false",alertResult.getStatus()); + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml new file mode 100644 index 0000000000..f4524b2046 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml @@ -0,0 +1,128 @@ + + + + + dolphinscheduler-alert-plugin + org.apache.dolphinscheduler + 1.3.4-SNAPSHOT + + 4.0.0 + + org.apache.dolphinscheduler + dolphinscheduler-alert-email + + dolphinscheduler-plugin + + + + org.apache.dolphinscheduler + dolphinscheduler-spi + provided + + + + com.fasterxml.jackson.core + jackson-annotations + provided + + + com.fasterxml.jackson.core + jackson-databind + provided + + + org.apache.commons + commons-collections4 + + + + org.apache.poi + poi + + + + com.google.guava + guava + + + + ch.qos.logback + logback-classic + + + + org.slf4j + slf4j-api + + + + org.apache.commons + commons-email + + + + com.fasterxml.jackson.core + jackson-annotations + provided + + + + com.fasterxml.jackson.core + jackson-databind + provided + + + + junit + junit + test + + + + org.mockito + mockito-core + jar + test + + + + org.powermock + powermock-module-junit4 + test + + + + org.powermock + powermock-api-mockito2 + test + + + org.mockito + mockito-core + + + + + + + dolphinscheduler-alert-email-${project.version} + + + \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannel.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannel.java new file mode 100644 index 0000000000..c793af5710 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannel.java @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; + +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * email alert channel . use email to seed the alertInfo + */ +public class EmailAlertChannel implements AlertChannel { + private static final Logger logger = LoggerFactory.getLogger(EmailAlertChannel.class); + + @Override + public AlertResult process(AlertInfo info) { + + AlertData alert = info.getAlertData(); + String alertParams = info.getAlertParams(); + Map paramsMap = PluginParamsTransfer.getPluginParamsMap(alertParams); + MailSender mailSender = new MailSender(paramsMap); + AlertResult alertResult = mailSender.sendMails(alert.getTitle(), alert.getContent()); + + //send flag + boolean flag = false; + + if (alertResult == null) { + alertResult = new AlertResult(); + alertResult.setStatus("false"); + alertResult.setMessage("alert send error."); + logger.info("alert send error : {}", alertResult.getMessage()); + return alertResult; + } + + flag = Boolean.parseBoolean(String.valueOf(alertResult.getStatus())); + + if (flag) { + logger.info("alert send success"); + alertResult.setMessage("email send success."); + } else { + alertResult.setMessage("alert send error."); + logger.info("alert send error : {}", alertResult.getMessage()); + } + + return alertResult; + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactory.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactory.java new file mode 100644 index 0000000000..f8aa17fca3 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactory.java @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.PasswordParam; +import org.apache.dolphinscheduler.spi.params.RadioParam; +import org.apache.dolphinscheduler.spi.params.base.DataType; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.ArrayList; +import java.util.List; + +/** + * email alert factory + */ +public class EmailAlertChannelFactory implements AlertChannelFactory { + @Override + public String getName() { + return "Email"; + } + + @Override + public List getParams() { + + List paramsList = new ArrayList<>(); + InputParam receivesParam = InputParam.newBuilder(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS, MailParamsConstants.PLUGIN_DEFAULT_EMAIL_RECEIVERS) + .setPlaceholder("please input receives") + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam receiveCcsParam = InputParam.newBuilder(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS, MailParamsConstants.PLUGIN_DEFAULT_EMAIL_RECEIVERCCS) + .build(); + + InputParam mailSmtpHost = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_HOST, MailParamsConstants.MAIL_SMTP_HOST) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam mailSmtpPort = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_PORT, MailParamsConstants.MAIL_SMTP_PORT) + .setValue(25) + .addValidate(Validate.newBuilder() + .setRequired(true) + .setType(DataType.NUMBER.getDataType()) + .build()) + .build(); + + InputParam mailSender = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SENDER, MailParamsConstants.MAIL_SENDER) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + RadioParam enableSmtpAuth = RadioParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_AUTH, MailParamsConstants.MAIL_SMTP_AUTH) + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(true) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam mailUser = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_USER, MailParamsConstants.MAIL_USER) + .setPlaceholder("if enable use authentication, you need input user") + .build(); + + PasswordParam mailPassword = PasswordParam.newBuilder(MailParamsConstants.NAME_MAIL_PASSWD, MailParamsConstants.MAIL_PASSWD) + .setPlaceholder("if enable use authentication, you need input password") + .build(); + + RadioParam enableTls = RadioParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_STARTTLS_ENABLE, MailParamsConstants.MAIL_SMTP_STARTTLS_ENABLE) + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(false) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + RadioParam enableSsl = RadioParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_SSL_ENABLE, MailParamsConstants.MAIL_SMTP_SSL_ENABLE) + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(false) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam sslTrust = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_SSL_TRUST, MailParamsConstants.MAIL_SMTP_SSL_TRUST) + .setValue("*") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + RadioParam showType = RadioParam.newBuilder(AlertConstants.SHOW_TYPE, AlertConstants.SHOW_TYPE) + .addParamsOptions(new ParamsOptions(ShowType.TABLE.getDescp(), ShowType.TABLE.getDescp(), false)) + .addParamsOptions(new ParamsOptions(ShowType.TEXT.getDescp(), ShowType.TEXT.getDescp(), false)) + .addParamsOptions(new ParamsOptions(ShowType.ATTACHMENT.getDescp(), ShowType.ATTACHMENT.getDescp(), false)) + .addParamsOptions(new ParamsOptions(ShowType.TABLEATTACHMENT.getDescp(), ShowType.TABLEATTACHMENT.getDescp(), false)) + .setValue(ShowType.TABLE.getDescp()) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + paramsList.add(receivesParam); + paramsList.add(receiveCcsParam); + paramsList.add(mailSmtpHost); + paramsList.add(mailSmtpPort); + paramsList.add(mailSender); + paramsList.add(enableSmtpAuth); + paramsList.add(mailUser); + paramsList.add(mailPassword); + paramsList.add(enableTls); + paramsList.add(enableSsl); + paramsList.add(sslTrust); + paramsList.add(showType); + + return paramsList; + } + + @Override + public AlertChannel create() { + return new EmailAlertChannel(); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertPlugin.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertPlugin.java new file mode 100644 index 0000000000..175b518189 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertPlugin.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +import com.google.common.collect.ImmutableList; + +/** + * email alert plugin + */ +public class EmailAlertPlugin implements DolphinSchedulerPlugin { + @Override + public Iterable getAlertChannelFactorys() { + return ImmutableList.of(new EmailAlertChannelFactory()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailConstants.java new file mode 100644 index 0000000000..d0e85ffb03 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/EmailConstants.java @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +public class EmailConstants { + + + public static final String XLS_FILE_PATH = "xls.file.path"; + + public static final String MAIL_TRANSPORT_PROTOCOL = "mail.transport.protocol"; + + public static final String DEFAULT_SMTP_PORT = "25"; + + public static final String TEXT_HTML_CHARSET_UTF_8 = "text/html;charset=utf-8"; + + public static final int NUMBER_1000 = 1000; + + public static final String TR = ""; + + public static final String TD = ""; + + public static final String TD_END = ""; + + public static final String TR_END = ""; + + public static final String TITLE = "title"; + + public static final String CONTENT = "content"; + + public static final String TH = ""; + + public static final String TH_END = ""; + + public static final String MARKDOWN_QUOTE = ">"; + + public static final String MARKDOWN_ENTER = "\n"; + + public static final String HTML_HEADER_PREFIX = new StringBuilder("") + .append("") + .append("") + .append("dolphinscheduler") + .append("") + .append("") + .append("") + .append("") + .append(" ") + .toString(); + + public static final String TABLE_BODY_HTML_TAIL = "
"; + + public static final String UTF_8 = "UTF-8"; + + public static final String EXCEL_SUFFIX_XLS = ".xls"; + + public static final String SINGLE_SLASH = "/"; +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/ExcelUtils.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/ExcelUtils.java similarity index 87% rename from dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/ExcelUtils.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/ExcelUtils.java index 76ce66ac2f..1579f9548e 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/ExcelUtils.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/ExcelUtils.java @@ -15,11 +15,11 @@ * limitations under the License. */ -package org.apache.dolphinscheduler.alert.utils; +package org.apache.dolphinscheduler.plugin.alert.email; -import org.apache.dolphinscheduler.common.utils.CollectionUtils; -import org.apache.dolphinscheduler.common.utils.JSONUtils; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; +import org.apache.commons.collections4.CollectionUtils; import org.apache.poi.hssf.usermodel.HSSFCell; import org.apache.poi.hssf.usermodel.HSSFRow; import org.apache.poi.hssf.usermodel.HSSFSheet; @@ -31,6 +31,7 @@ import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.util.ArrayList; +import java.util.Iterator; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; @@ -43,12 +44,12 @@ import org.slf4j.LoggerFactory; */ public class ExcelUtils { - private static final Logger logger = LoggerFactory.getLogger(ExcelUtils.class); - - private ExcelUtils() { - throw new IllegalStateException(ExcelUtils.class.getName()); + public ExcelUtils() { + throw new IllegalStateException("Utility class"); } + private static final Logger logger = LoggerFactory.getLogger(ExcelUtils.class); + /** * generate excel file * @@ -71,7 +72,9 @@ public class ExcelUtils { List headerList = new ArrayList<>(); - for (Map.Entry en : headerMap.entrySet()) { + Iterator> iter = headerMap.entrySet().iterator(); + while (iter.hasNext()) { + Map.Entry en = iter.next(); headerList.add(en.getKey()); } @@ -122,11 +125,12 @@ public class ExcelUtils { } //setting file output - fos = new FileOutputStream(xlsFilePath + Constants.SINGLE_SLASH + title + Constants.EXCEL_SUFFIX_XLS); + fos = new FileOutputStream(xlsFilePath + EmailConstants.SINGLE_SLASH + title + EmailConstants.EXCEL_SUFFIX_XLS); wb.write(fos); } catch (Exception e) { + logger.error("generate excel error", e); throw new RuntimeException("generate excel error", e); } finally { if (wb != null) { @@ -146,4 +150,4 @@ public class ExcelUtils { } } -} +} \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailParamsConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailParamsConstants.java new file mode 100644 index 0000000000..3eaa420301 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailParamsConstants.java @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +/** + * mail plugin params json use + */ +public class MailParamsConstants { + + public MailParamsConstants() { + throw new IllegalStateException("Utility class"); + } + + public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERS = "$t('receivers')"; + public static final String NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS = "receivers"; + + public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERCCS = "$t('receiverCcs')"; + public static final String NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS = "receiverCcs"; + + public static final String MAIL_PROTOCOL = "transport.protocol"; + public static final String NAME_MAIL_PROTOCOL = "protocol"; + + public static final String MAIL_SMTP_HOST = "smtp.host"; + public static final String NAME_MAIL_SMTP_HOST = "serverHost"; + + public static final String MAIL_SMTP_PORT = "smtp.port"; + public static final String NAME_MAIL_SMTP_PORT = "serverPort"; + + public static final String MAIL_SENDER = "sender"; + public static final String NAME_MAIL_SENDER = "sender"; + + public static final String MAIL_SMTP_AUTH = "smtp.auth"; + public static final String NAME_MAIL_SMTP_AUTH = "enableSmtpAuth"; + + public static final String MAIL_USER = "user"; + public static final String NAME_MAIL_USER = "user"; + + public static final String MAIL_PASSWD = "passwd"; + public static final String NAME_MAIL_PASSWD = "passwd"; + + public static final String MAIL_SMTP_STARTTLS_ENABLE = "smtp.starttls.enable"; + public static final String NAME_MAIL_SMTP_STARTTLS_ENABLE = "starttlsEnable"; + + public static final String MAIL_SMTP_SSL_ENABLE = "smtp.ssl.enable"; + public static final String NAME_MAIL_SMTP_SSL_ENABLE = "sslEnable"; + + public static final String MAIL_SMTP_SSL_TRUST = "smtp.ssl.trust"; + public static final String NAME_MAIL_SMTP_SSL_TRUST = "smtpSslTrust"; + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailSender.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailSender.java new file mode 100644 index 0000000000..03765b248d --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/MailSender.java @@ -0,0 +1,429 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import static java.util.Objects.requireNonNull; + +import org.apache.dolphinscheduler.plugin.alert.email.template.AlertTemplate; +import org.apache.dolphinscheduler.plugin.alert.email.template.DefaultHTMLTemplate; +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.utils.StringUtils; + +import org.apache.commons.collections4.CollectionUtils; +import org.apache.commons.mail.EmailException; +import org.apache.commons.mail.HtmlEmail; + +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import javax.mail.Authenticator; +import javax.mail.Message; +import javax.mail.MessagingException; +import javax.mail.PasswordAuthentication; +import javax.mail.Session; +import javax.mail.Transport; +import javax.mail.internet.InternetAddress; +import javax.mail.internet.MimeBodyPart; +import javax.mail.internet.MimeMessage; +import javax.mail.internet.MimeMultipart; +import javax.mail.internet.MimeUtility; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.sun.mail.smtp.SMTPProvider; + +/** + * mail utils + */ +public class MailSender { + + public static final Logger logger = LoggerFactory.getLogger(MailSender.class); + + private List receivers; + private List receiverCcs; + private String mailProtocol = "SMTP"; + private String mailSmtpHost; + private String mailSmtpPort; + private String mailSender; + private String enableSmtpAuth; + private String mailUser; + private String mailPasswd; + private String mailUseStartTLS; + private String mailUseSSL; + private String xlsFilePath; + private String sslTrust; + private String showType; + private AlertTemplate alertTemplate; + + public MailSender(Map config) { + + String receiversConfig = config.get(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS); + if (receiversConfig == null || "".equals(receiversConfig)) { + throw new RuntimeException(MailParamsConstants.PLUGIN_DEFAULT_EMAIL_RECEIVERS + "must not be null"); + } + + receivers = Arrays.asList(receiversConfig.split(",")); + + String receiverCcsConfig = config.get(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS); + + receiverCcs = new ArrayList<>(); + if (receiverCcsConfig != null && !"".equals(receiverCcsConfig)) { + receiverCcs = Arrays.asList(receiverCcsConfig.split(",")); + } + + mailSmtpHost = config.get(MailParamsConstants.NAME_MAIL_SMTP_HOST); + requireNonNull(mailSmtpHost, MailParamsConstants.MAIL_SMTP_HOST + " must not null"); + + mailSmtpPort = config.get(MailParamsConstants.NAME_MAIL_SMTP_PORT); + requireNonNull(mailSmtpPort, MailParamsConstants.MAIL_SMTP_PORT + " must not null"); + + mailSender = config.get(MailParamsConstants.NAME_MAIL_SENDER); + requireNonNull(mailSender, MailParamsConstants.MAIL_SENDER + " must not null"); + + enableSmtpAuth = config.get(MailParamsConstants.NAME_MAIL_SMTP_AUTH); + + mailUser = config.get(MailParamsConstants.NAME_MAIL_USER); + requireNonNull(mailUser, MailParamsConstants.MAIL_USER + " must not null"); + + mailPasswd = config.get(MailParamsConstants.NAME_MAIL_PASSWD); + requireNonNull(mailPasswd, MailParamsConstants.MAIL_PASSWD + " must not null"); + + mailUseStartTLS = config.get(MailParamsConstants.NAME_MAIL_SMTP_STARTTLS_ENABLE); + requireNonNull(mailUseStartTLS, MailParamsConstants.MAIL_SMTP_STARTTLS_ENABLE + " must not null"); + + mailUseSSL = config.get(MailParamsConstants.NAME_MAIL_SMTP_SSL_ENABLE); + requireNonNull(mailUseSSL, MailParamsConstants.MAIL_SMTP_SSL_ENABLE + " must not null"); + + sslTrust = config.get(MailParamsConstants.NAME_MAIL_SMTP_SSL_TRUST); + requireNonNull(sslTrust, MailParamsConstants.MAIL_SMTP_SSL_TRUST + " must not null"); + + showType = config.get(AlertConstants.SHOW_TYPE); + requireNonNull(showType, AlertConstants.SHOW_TYPE + " must not null"); + + xlsFilePath = config.get(EmailConstants.XLS_FILE_PATH); + if (StringUtils.isBlank(xlsFilePath)) { + xlsFilePath = "/tmp/xls"; + } + + alertTemplate = new DefaultHTMLTemplate(); + } + + /** + * send mail to receivers + * + * @param title title + * @param content content + * @return + */ + public AlertResult sendMails(String title, String content) { + return sendMails(this.receivers, this.receiverCcs, title, content); + } + + /** + * send mail to receivers + * + * @param title email title + * @param content email content + * @return + */ + public AlertResult sendMailsToReceiverOnly(String title, String content) { + return sendMails(this.receivers, null, title, content); + } + + /** + * send mail + * + * @param receivers receivers + * @param receiverCcs receiverCcs + * @param title title + * @param content content + * @return + */ + public AlertResult sendMails(List receivers, List receiverCcs, String title, String content) { + AlertResult alertResult = new AlertResult(); + alertResult.setStatus("false"); + + // if there is no receivers && no receiversCc, no need to process + if (CollectionUtils.isEmpty(receivers) && CollectionUtils.isEmpty(receiverCcs)) { + return alertResult; + } + + receivers.removeIf(StringUtils::isEmpty); + + if (showType.equals(ShowType.TABLE.getDescp()) || showType.equals(ShowType.TEXT.getDescp())) { + // send email + HtmlEmail email = new HtmlEmail(); + + try { + Session session = getSession(); + email.setMailSession(session); + email.setFrom(mailSender); + email.setCharset(EmailConstants.UTF_8); + if (CollectionUtils.isNotEmpty(receivers)) { + // receivers mail + for (String receiver : receivers) { + email.addTo(receiver); + } + } + + if (CollectionUtils.isNotEmpty(receiverCcs)) { + //cc + for (String receiverCc : receiverCcs) { + email.addCc(receiverCc); + } + } + // sender mail + return getStringObjectMap(title, content, alertResult, email); + } catch (Exception e) { + handleException(alertResult, e); + } + } else if (showType.equals(ShowType.ATTACHMENT.getDescp()) || showType.equals(ShowType.TABLEATTACHMENT.getDescp())) { + try { + + String partContent = (showType.equals(ShowType.ATTACHMENT.getDescp()) ? "Please see the attachment " + title + EmailConstants.EXCEL_SUFFIX_XLS : htmlTable(content, false)); + + attachment(title, content, partContent); + + alertResult.setStatus("true"); + return alertResult; + } catch (Exception e) { + handleException(alertResult, e); + return alertResult; + } + } + return alertResult; + + } + + /** + * html table content + * + * @param content the content + * @param showAll if show the whole content + * @return the html table form + */ + private String htmlTable(String content, boolean showAll) { + return alertTemplate.getMessageFromTemplate(content, ShowType.TABLE, showAll); + } + + /** + * html table content + * + * @param content the content + * @return the html table form + */ + private String htmlTable(String content) { + return htmlTable(content, true); + } + + /** + * html text content + * + * @param content the content + * @return text in html form + */ + private String htmlText(String content) { + return alertTemplate.getMessageFromTemplate(content, ShowType.TEXT); + } + + /** + * send mail as Excel attachment + * + * @param title + * @param content + * @param partContent + * @throws Exception + */ + private void attachment(String title, String content, String partContent) throws Exception { + MimeMessage msg = getMimeMessage(); + + attachContent(title, content, partContent, msg); + } + + /** + * get MimeMessage + * + * @return + * @throws MessagingException + */ + private MimeMessage getMimeMessage() throws MessagingException { + + // 1. The first step in creating mail: creating session + Session session = getSession(); + // Setting debug mode, can be turned off + session.setDebug(false); + + // 2. creating mail: Creating a MimeMessage + MimeMessage msg = new MimeMessage(session); + // 3. set sender + msg.setFrom(new InternetAddress(mailSender)); + // 4. set receivers + for (String receiver : receivers) { + msg.addRecipients(Message.RecipientType.TO, InternetAddress.parse(receiver)); + } + return msg; + } + + /** + * get session + * + * @return the new Session + */ + private Session getSession() { + Properties props = new Properties(); + props.setProperty(MailParamsConstants.MAIL_SMTP_HOST, mailSmtpHost); + props.setProperty(MailParamsConstants.MAIL_SMTP_PORT, mailSmtpPort); + props.setProperty(MailParamsConstants.MAIL_SMTP_AUTH, enableSmtpAuth); + props.setProperty(EmailConstants.MAIL_TRANSPORT_PROTOCOL, mailProtocol); + props.setProperty(MailParamsConstants.MAIL_SMTP_STARTTLS_ENABLE, mailUseStartTLS); + props.setProperty(MailParamsConstants.MAIL_SMTP_SSL_ENABLE, mailUseSSL); + props.setProperty(MailParamsConstants.MAIL_SMTP_SSL_TRUST, sslTrust); + + Authenticator auth = new Authenticator() { + @Override + protected PasswordAuthentication getPasswordAuthentication() { + // mail username and password + return new PasswordAuthentication(mailUser, mailPasswd); + } + }; + + Session session = Session.getInstance(props, auth); + session.addProvider(new SMTPProvider()); + return session; + } + + /** + * attach content + * + * @param title + * @param content + * @param partContent + * @param msg + * @throws MessagingException + * @throws IOException + */ + private void attachContent(String title, String content, String partContent, MimeMessage msg) throws MessagingException, IOException { + /** + * set receiverCc + */ + if (CollectionUtils.isNotEmpty(receiverCcs)) { + for (String receiverCc : receiverCcs) { + msg.addRecipients(Message.RecipientType.CC, InternetAddress.parse(receiverCc)); + } + } + + // set subject + msg.setSubject(title); + MimeMultipart partList = new MimeMultipart(); + // set signature + MimeBodyPart part1 = new MimeBodyPart(); + part1.setContent(partContent, EmailConstants.TEXT_HTML_CHARSET_UTF_8); + // set attach file + MimeBodyPart part2 = new MimeBodyPart(); + File file = new File(xlsFilePath + EmailConstants.SINGLE_SLASH + title + EmailConstants.EXCEL_SUFFIX_XLS); + if (!file.getParentFile().exists()) { + file.getParentFile().mkdirs(); + } + // make excel file + + ExcelUtils.genExcelFile(content, title, xlsFilePath); + + part2.attachFile(file); + part2.setFileName(MimeUtility.encodeText(title + EmailConstants.EXCEL_SUFFIX_XLS, EmailConstants.UTF_8, "B")); + // add components to collection + partList.addBodyPart(part1); + partList.addBodyPart(part2); + msg.setContent(partList); + // 5. send Transport + Transport.send(msg); + // 6. delete saved file + deleteFile(file); + } + + /** + * the string object map + * + * @param title + * @param content + * @param alertResult + * @param email + * @return + * @throws EmailException + */ + private AlertResult getStringObjectMap(String title, String content, AlertResult alertResult, HtmlEmail email) throws EmailException { + + /** + * the subject of the message to be sent + */ + email.setSubject(title); + /** + * to send information, you can use HTML tags in mail content because of the use of HtmlEmail + */ + if (showType.equals(ShowType.TABLE.getDescp())) { + email.setMsg(htmlTable(content)); + } else if (showType.equals(ShowType.TEXT.getDescp())) { + email.setMsg(htmlText(content)); + } + + // send + email.setDebug(true); + Thread.currentThread().setContextClassLoader(getClass().getClassLoader()); + email.send(); + + alertResult.setStatus("true"); + + return alertResult; + } + + /** + * file delete + * + * @param file the file to delete + */ + public void deleteFile(File file) { + if (file.exists()) { + if (file.delete()) { + logger.info("delete success: {}", file.getAbsolutePath() + file.getName()); + } else { + logger.info("delete fail: {}", file.getAbsolutePath() + file.getName()); + } + } else { + logger.info("file not exists: {}", file.getAbsolutePath() + file.getName()); + } + } + + /** + * handle exception + * + * @param alertResult + * @param e + */ + private void handleException(AlertResult alertResult, Exception e) { + logger.error("Send email to {} failed", receivers, e); + alertResult.setMessage("Send email to {" + String.join(",", receivers) + "} failed，" + e.toString()); + } + +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplate.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template/AlertTemplate.java similarity index 72% rename from dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplate.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template/AlertTemplate.java index 81b5e65f27..dec993d4d0 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplate.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template/AlertTemplate.java @@ -14,9 +14,10 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.alert.template; -import org.apache.dolphinscheduler.common.enums.ShowType; +package org.apache.dolphinscheduler.plugin.alert.email.template; + +import org.apache.dolphinscheduler.spi.alert.ShowType; /** * alert message template @@ -25,20 +26,22 @@ public interface AlertTemplate { /** * get a message from a specified alert template - * @param content alert message content - * @param showType show type - * @param showAll whether to show all + * + * @param content alert message content + * @param showType show type + * @param showAll whether to show all * @return a message from a specified alert template */ - String getMessageFromTemplate(String content, ShowType showType,boolean showAll); + String getMessageFromTemplate(String content, ShowType showType, boolean showAll); /** * default showAll is true - * @param content alert message content + * + * @param content alert message content * @param showType show type * @return a message from a specified alert template */ - default String getMessageFromTemplate(String content,ShowType showType){ - return getMessageFromTemplate(content,showType,true); + default String getMessageFromTemplate(String content, ShowType showType) { + return getMessageFromTemplate(content, showType, true); } } diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplate.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplate.java similarity index 66% rename from dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplate.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplate.java index f590849660..06decd6ffc 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplate.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/main/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplate.java @@ -15,21 +15,19 @@ * limitations under the License. */ -package org.apache.dolphinscheduler.alert.template.impl; +package org.apache.dolphinscheduler.plugin.alert.email.template; -import static org.apache.dolphinscheduler.common.utils.Preconditions.checkNotNull; +import static java.util.Objects.requireNonNull; -import org.apache.dolphinscheduler.alert.template.AlertTemplate; -import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.apache.dolphinscheduler.common.utils.JSONUtils; -import org.apache.dolphinscheduler.common.utils.StringUtils; +import org.apache.dolphinscheduler.plugin.alert.email.EmailConstants; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; +import org.apache.dolphinscheduler.spi.utils.StringUtils; import java.util.Iterator; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; -import java.util.Map.Entry; import java.util.Set; import org.slf4j.Logger; @@ -52,7 +50,7 @@ public class DefaultHTMLTemplate implements AlertTemplate { case TABLE: return getTableTypeMessage(content, showAll); case TEXT: - return getTextTypeMessage(content); + return getTextTypeMessage(content, showAll); default: throw new IllegalArgumentException(String.format("not support showType: %s in DefaultHTMLTemplate", showType)); } @@ -70,8 +68,8 @@ public class DefaultHTMLTemplate implements AlertTemplate { if (StringUtils.isNotEmpty(content)) { List mapItemsList = JSONUtils.toList(content, LinkedHashMap.class); - if (!showAll && mapItemsList.size() > Constants.NUMBER_1000) { - mapItemsList = mapItemsList.subList(0, Constants.NUMBER_1000); + if (!showAll && mapItemsList.size() > EmailConstants.NUMBER_1000) { + mapItemsList = mapItemsList.subList(0, EmailConstants.NUMBER_1000); } StringBuilder contents = new StringBuilder(200); @@ -81,21 +79,21 @@ public class DefaultHTMLTemplate implements AlertTemplate { String title = ""; for (LinkedHashMap mapItems : mapItemsList) { - Set> entries = mapItems.entrySet(); + Set> entries = mapItems.entrySet(); - Iterator> iterator = entries.iterator(); + Iterator> iterator = entries.iterator(); - StringBuilder t = new StringBuilder(Constants.TR); - StringBuilder cs = new StringBuilder(Constants.TR); + StringBuilder t = new StringBuilder(EmailConstants.TR); + StringBuilder cs = new StringBuilder(EmailConstants.TR); while (iterator.hasNext()) { Map.Entry entry = iterator.next(); - t.append(Constants.TH).append(entry.getKey()).append(Constants.TH_END); - cs.append(Constants.TD).append(String.valueOf(entry.getValue())).append(Constants.TD_END); + t.append(EmailConstants.TH).append(entry.getKey()).append(EmailConstants.TH_END); + cs.append(EmailConstants.TD).append(String.valueOf(entry.getValue())).append(EmailConstants.TD_END); } - t.append(Constants.TR_END); - cs.append(Constants.TR_END); + t.append(EmailConstants.TR_END); + cs.append(EmailConstants.TR_END); if (flag) { title = t.toString(); } @@ -113,17 +111,18 @@ public class DefaultHTMLTemplate implements AlertTemplate { * get alert message which type is TEXT * * @param content message content + * @param showAll weather to show all * @return alert message */ - private String getTextTypeMessage(String content) { + private String getTextTypeMessage(String content, boolean showAll) { if (StringUtils.isNotEmpty(content)) { ArrayNode list = JSONUtils.parseArray(content); StringBuilder contents = new StringBuilder(100); for (JsonNode jsonNode : list) { - contents.append(Constants.TR); - contents.append(Constants.TD).append(jsonNode.toString()).append(Constants.TD_END); - contents.append(Constants.TR_END); + contents.append(EmailConstants.TR); + contents.append(EmailConstants.TD).append(jsonNode.toString()).append(EmailConstants.TD_END); + contents.append(EmailConstants.TR_END); } return getMessageFromHtmlTemplate(null, contents.toString()); @@ -136,16 +135,16 @@ public class DefaultHTMLTemplate implements AlertTemplate { /** * get alert message from a html template * - * @param title message title + * @param title message title * @param content message content * @return alert message which use html template */ private String getMessageFromHtmlTemplate(String title, String content) { - checkNotNull(content); - String htmlTableThead = StringUtils.isEmpty(title) ? "" : String.format("%s%n", title); + requireNonNull(content, "content must not null"); + String htmlTableThead = StringUtils.isEmpty(title) ? "" : String.format("%s\n", title); - return Constants.HTML_HEADER_PREFIX + htmlTableThead + content + Constants.TABLE_BODY_HTML_TAIL; + return EmailConstants.HTML_HEADER_PREFIX + htmlTableThead + content + EmailConstants.TABLE_BODY_HTML_TAIL; } } diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactoryTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactoryTest.java new file mode 100644 index 0000000000..977cd8fefc --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelFactoryTest.java @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.List; + +import org.junit.After; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +/** + * EmailAlertChannelFactory Tester. + * + * @version 1.0 + * @since 
Aug 20, 2020
+ */ +public class EmailAlertChannelFactoryTest { + + @Before + public void before() throws Exception { + } + + @After + public void after() throws Exception { + } + + /** + * Method: getName() + */ + @Test + public void testGetName() throws Exception { + } + + /** + * Method: getParams() + */ + @Test + public void testGetParams() throws Exception { + EmailAlertChannelFactory emailAlertChannelFactory = new EmailAlertChannelFactory(); + List params = emailAlertChannelFactory.getParams(); + System.out.println(JSONUtils.toJsonString(params)); + Assert.assertEquals(12, params.size()); + } + + /** + * Method: create() + */ + @Test + public void testCreate() throws Exception { + EmailAlertChannelFactory emailAlertChannelFactory = new EmailAlertChannelFactory(); + AlertChannel alertChannel = emailAlertChannelFactory.create(); + Assert.assertNotNull(alertChannel); + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelTest.java new file mode 100644 index 0000000000..97a1013399 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/EmailAlertChannelTest.java @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.PasswordParam; +import org.apache.dolphinscheduler.spi.params.RadioParam; +import org.apache.dolphinscheduler.spi.params.base.DataType; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +/** + * EmailAlertChannel Tester. + */ +public class EmailAlertChannelTest { + + /** + * Method: process(AlertInfo info) + */ + @Test + public void testProcess() { + EmailAlertChannel emailAlertChannel = new EmailAlertChannel(); + AlertData alertData = new AlertData(); + LinkedHashMap map1 = new LinkedHashMap<>(); + map1.put("mysql service name", "mysql200"); + map1.put("mysql address", "192.168.xx.xx"); + map1.put("port", "3306"); + map1.put("no index of number", "80"); + map1.put("database client connections", "190"); + List> maps = new ArrayList<>(); + maps.add(0, map1); + String mapjson = JSONUtils.toJsonString(maps); + + alertData.setId(10) + .setContent(mapjson) + .setLog("10") + .setTitle("test"); + AlertInfo alertInfo = new AlertInfo(); + alertInfo.setAlertData(alertData); + alertInfo.setAlertParams(getEmailAlertParams()); + AlertResult alertResult = emailAlertChannel.process(alertInfo); + Assert.assertNotNull(alertResult); + Assert.assertEquals("false", alertResult.getStatus()); + } + + public String getEmailAlertParams() { + List paramsList = new ArrayList<>(); + InputParam receivesParam = InputParam.newBuilder("receivers", "receivers") + .setValue("540957506@qq.com") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam mailSmtpHost = InputParam.newBuilder("serverHost", "smtp.host") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("smtp.126.com") + .build(); + + InputParam mailSmtpPort = InputParam.newBuilder("serverPort", "smtp.port") + .addValidate(Validate.newBuilder() + .setRequired(true) + .setType(DataType.NUMBER.getDataType()) + .build()) + .setValue(25) + .build(); + + InputParam mailSender = InputParam.newBuilder("sender", "sender") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("dolphinscheduler@126.com") + .build(); + + RadioParam enableSmtpAuth = RadioParam.newBuilder("enableSmtpAuth", "smtp.auth") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue(false) + .build(); + + InputParam mailUser = InputParam.newBuilder("user", "user") + .setPlaceholder("if enable use authentication, you need input user") + .setValue("dolphinscheduler@126.com") + .build(); + + PasswordParam mailPassword = PasswordParam.newBuilder("passwd", "passwd") + .setPlaceholder("if enable use authentication, you need input password") + .setValue("escheduler123") + .build(); + + RadioParam enableTls = RadioParam.newBuilder("starttlsEnable", "starttls.enable") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue(true) + .build(); + + RadioParam enableSsl = RadioParam.newBuilder("sslEnable", "smtp.ssl.enable") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue(true) + .build(); + + InputParam sslTrust = InputParam.newBuilder("smtpSslTrust", "smtp.ssl.trust") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("smtp.126.com") + .build(); + + List emailShowTypeList = new ArrayList<>(); + emailShowTypeList.add(new ParamsOptions(ShowType.TABLE.getDescp(), ShowType.TABLE.getDescp(), false)); + emailShowTypeList.add(new ParamsOptions(ShowType.TEXT.getDescp(), ShowType.TEXT.getDescp(), false)); + emailShowTypeList.add(new ParamsOptions(ShowType.ATTACHMENT.getDescp(), ShowType.ATTACHMENT.getDescp(), false)); + emailShowTypeList.add(new ParamsOptions(ShowType.TABLEATTACHMENT.getDescp(), ShowType.TABLEATTACHMENT.getDescp(), false)); + RadioParam showType = RadioParam.newBuilder(AlertConstants.SHOW_TYPE, "showType") + .setParamsOptionsList(emailShowTypeList) + .setValue(ShowType.TABLE.getDescp()) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + paramsList.add(receivesParam); + paramsList.add(mailSmtpHost); + paramsList.add(mailSmtpPort); + paramsList.add(mailSender); + paramsList.add(enableSmtpAuth); + paramsList.add(mailUser); + paramsList.add(mailPassword); + paramsList.add(enableTls); + paramsList.add(enableSsl); + paramsList.add(sslTrust); + paramsList.add(showType); + + return JSONUtils.toJsonString(paramsList); + } +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/ExcelUtilsTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/ExcelUtilsTest.java similarity index 89% rename from dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/ExcelUtilsTest.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/ExcelUtilsTest.java index c4833252b2..5c8b195176 100644 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/ExcelUtilsTest.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/ExcelUtilsTest.java @@ -15,7 +15,7 @@ * limitations under the License. */ -package org.apache.dolphinscheduler.alert.utils; +package org.apache.dolphinscheduler.plugin.alert.email; import static org.junit.Assert.assertTrue; @@ -63,7 +63,7 @@ public class ExcelUtilsTest { //Define dest file path String xlsFilePath = rootPath + System.getProperty("file.separator"); - logger.info("XLS_FILE_PATH: " + xlsFilePath); + logger.info("xlsFilePath: " + xlsFilePath); //Define correctContent String correctContent = "[{\"name\":\"ds name\",\"value\":\"ds value\"}]"; @@ -78,7 +78,7 @@ public class ExcelUtilsTest { ExcelUtils.genExcelFile(correctContent, title, xlsFilePath); //Test file exists - File xlsFile = new File(xlsFilePath + Constants.SINGLE_SLASH + title + Constants.EXCEL_SUFFIX_XLS); + File xlsFile = new File(xlsFilePath + EmailConstants.SINGLE_SLASH + title + EmailConstants.EXCEL_SUFFIX_XLS); assertTrue(xlsFile.exists()); //Expected RuntimeException @@ -98,7 +98,7 @@ public class ExcelUtilsTest { @Test public void testGenExcelFileByCheckDir() { ExcelUtils.genExcelFile("[{\"a\": \"a\"},{\"a\": \"a\"}]", "t", "/tmp/xls"); - File file = new File("/tmp/xls" + Constants.SINGLE_SLASH + "t" + Constants.EXCEL_SUFFIX_XLS); + File file = new File("/tmp/xls" + EmailConstants.SINGLE_SLASH + "t" + EmailConstants.EXCEL_SUFFIX_XLS); file.delete(); } -} +} \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/MailUtilsTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/MailUtilsTest.java new file mode 100644 index 0000000000..e19c819dae --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/MailUtilsTest.java @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email; + +import org.apache.dolphinscheduler.plugin.alert.email.template.AlertTemplate; +import org.apache.dolphinscheduler.plugin.alert.email.template.DefaultHTMLTemplate; +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +import org.junit.BeforeClass; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + */ +public class MailUtilsTest { + private static final Logger logger = LoggerFactory.getLogger(MailUtilsTest.class); + + private static Map emailConfig = new HashMap<>(); + + private static AlertTemplate alertTemplate; + + static MailSender mailSender; + + @BeforeClass + public static void initEmailConfig() { + emailConfig.put(MailParamsConstants.NAME_MAIL_PROTOCOL, "smtp"); + emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_HOST, "xxx.xxx.com"); + emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_PORT, "25"); + emailConfig.put(MailParamsConstants.NAME_MAIL_SENDER, "xxx1.xxx.com"); + emailConfig.put(MailParamsConstants.NAME_MAIL_USER, "xxx2.xxx.com"); + emailConfig.put(MailParamsConstants.NAME_MAIL_PASSWD, "111111"); + emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_STARTTLS_ENABLE, "true"); + emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_SSL_ENABLE, "false"); + emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_SSL_TRUST, "false"); + emailConfig.put(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS, "347801120@qq.com"); + emailConfig.put(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS, "347801120@qq.com"); + emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.TEXT.getDescp()); + alertTemplate = new DefaultHTMLTemplate(); + mailSender = new MailSender(emailConfig); + } + + @Test + public void testSendMails() { + + String content = "[\"id:69\"," + + "\"name:UserBehavior-0--1193959466\"," + + "\"Job name: Start workflow\"," + + "\"State: SUCCESS\"," + + "\"Recovery:NO\"," + + "\"Run time: 1\"," + + "\"Start time: 2018-08-06 10:31:34.0\"," + + "\"End time: 2018-08-06 10:31:49.0\"," + + "\"Host: 192.168.xx.xx\"," + + "\"Notify group :4\"]"; + + mailSender.sendMails( + "Mysql Exception", + content); + } + + public String list2String() { + + LinkedHashMap map1 = new LinkedHashMap<>(); + map1.put("mysql service name", "mysql200"); + map1.put("mysql address", "192.168.xx.xx"); + map1.put("port", "3306"); + map1.put("no index of number", "80"); + map1.put("database client connections", "190"); + + LinkedHashMap map2 = new LinkedHashMap<>(); + map2.put("mysql service name", "mysql210"); + map2.put("mysql address", "192.168.xx.xx"); + map2.put("port", "3306"); + map2.put("no index of number", "10"); + map2.put("database client connections", "90"); + + List> maps = new ArrayList<>(); + maps.add(0, map1); + maps.add(1, map2); + String mapjson = JSONUtils.toJsonString(maps); + logger.info(mapjson); + + return mapjson; + + } + + @Test + public void testSendTableMail() { + String title = "Mysql Exception"; + String content = list2String(); + emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.TABLE.getDescp()); + mailSender = new MailSender(emailConfig); + mailSender.sendMails(title, content); + } + + @Test + public void testAttachmentFile() throws Exception { + String content = list2String(); + emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.ATTACHMENT.getDescp()); + mailSender = new MailSender(emailConfig); + mailSender.sendMails("gaojing", content); + } + + @Test + public void testTableAttachmentFile() throws Exception { + String content = list2String(); + emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.TABLEATTACHMENT.getDescp()); + mailSender = new MailSender(emailConfig); + mailSender.sendMails("gaojing", content); + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplateTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplateTest.java new file mode 100644 index 0000000000..3d941962d9 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/src/test/java/org/apache/dolphinscheduler/plugin/alert/email/template/DefaultHTMLTemplateTest.java @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.email.template; + +import static org.junit.Assert.assertEquals; + +import org.apache.dolphinscheduler.plugin.alert.email.EmailConstants; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; + +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * test class for DefaultHTMLTemplate + */ +public class DefaultHTMLTemplateTest { + + private static final Logger logger = LoggerFactory.getLogger(DefaultHTMLTemplateTest.class); + + /** + * only need test method GetMessageFromTemplate + */ + @Test + public void testGetMessageFromTemplate() { + + DefaultHTMLTemplate template = new DefaultHTMLTemplate(); + + String tableTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TABLE, true); + + assertEquals(tableTypeMessage, generateMockTableTypeResultByHand()); + + String textTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TEXT, true); + + assertEquals(textTypeMessage, generateMockTextTypeResultByHand()); + } + + /** + * generate some simulation data + */ + private String list2String() { + + LinkedHashMap map1 = new LinkedHashMap<>(); + map1.put("mysql service name", "mysql200"); + map1.put("mysql address", "192.168.xx.xx"); + map1.put("database client connections", "190"); + map1.put("port", "3306"); + map1.put("no index of number", "80"); + + LinkedHashMap map2 = new LinkedHashMap<>(); + map2.put("mysql service name", "mysql210"); + map2.put("mysql address", "192.168.xx.xx"); + map2.put("database client connections", "90"); + map2.put("port", "3306"); + map2.put("no index of number", "10"); + + List> maps = new ArrayList<>(); + maps.add(0, map1); + maps.add(1, map2); + String mapjson = JSONUtils.toJsonString(maps); + logger.info(mapjson); + + return mapjson; + } + + private String generateMockTableTypeResultByHand() { + + return EmailConstants.HTML_HEADER_PREFIX + + "" + + "mysql service namemysql addressdatabase client connectionsportno index of number" + + "\n" + + "mysql200192.168.xx.xx190330680" + + "mysql210192.168.xx.xx90330610" + + EmailConstants.TABLE_BODY_HTML_TAIL; + + } + + private String generateMockTextTypeResultByHand() { + + return EmailConstants.HTML_HEADER_PREFIX + + "{\"mysql service name\":\"mysql200\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"190\",\"port\":\"3306\",\"no index of number\":\"80\"}" + + "{\"mysql service name\":\"mysql210\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"90\",\"port\":\"3306\",\"no index of number\":\"10\"}" + + EmailConstants.TABLE_BODY_HTML_TAIL; + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/pom.xml b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/pom.xml new file mode 100644 index 0000000000..5223730e3e --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/pom.xml @@ -0,0 +1,76 @@ + + + + + dolphinscheduler-alert-plugin + org.apache.dolphinscheduler + 1.3.4-SNAPSHOT + + 4.0.0 + + dolphinscheduler-alert-http + dolphinscheduler-plugin + + + org.apache.dolphinscheduler + dolphinscheduler-spi + provided + + + + com.google.guava + guava + + + + ch.qos.logback + logback-classic + + + + org.apache.httpcomponents + httpclient + + + + com.fasterxml.jackson.core + jackson-databind + provided + + + + junit + junit + test + + + + org.mockito + mockito-core + jar + test + + + + + dolphinscheduler-alert-http-${project.version} + + + \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannel.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannel.java new file mode 100644 index 0000000000..27bc1903d8 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannel.java @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; + +import java.util.Map; + +/** + * http alert channel,use sms message to seed the alertInfo + */ +public class HttpAlertChannel implements AlertChannel { + @Override + public AlertResult process(AlertInfo alertInfo) { + + AlertData alertData = alertInfo.getAlertData(); + String alertParams = alertInfo.getAlertParams(); + Map paramsMap = PluginParamsTransfer.getPluginParamsMap(alertParams); + + return new HttpSender(paramsMap).send(alertData.getContent()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactory.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactory.java new file mode 100644 index 0000000000..6b8dd305fd --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactory.java @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.Arrays; +import java.util.List; + +/** + * http alert factory + */ +public class HttpAlertChannelFactory implements AlertChannelFactory { + @Override + public String getName() { + return "Http"; + } + + @Override + public List getParams() { + + InputParam url = InputParam.newBuilder(HttpAlertConstants.URL, HttpAlertConstants.URL) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam headerParams = InputParam.newBuilder(HttpAlertConstants.HEADER_PARAMS, HttpAlertConstants.HEADER_PARAMS) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam bodyParams = InputParam.newBuilder(HttpAlertConstants.BODY_PARAMS, HttpAlertConstants.BODY_PARAMS) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam contentField = InputParam.newBuilder(HttpAlertConstants.CONTENT_FIELD, HttpAlertConstants.CONTENT_FIELD) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam requestType = InputParam.newBuilder(HttpAlertConstants.REQUEST_TYPE, HttpAlertConstants.REQUEST_TYPE) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + return Arrays.asList(url, requestType, headerParams, bodyParams, contentField); + } + + @Override + public AlertChannel create() { + return new HttpAlertChannel(); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertConstants.java new file mode 100644 index 0000000000..965860d868 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertConstants.java @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +public class HttpAlertConstants { + + private HttpAlertConstants() { + } + + public static final String URL = "url"; + + public static final String HEADER_PARAMS = "headerParams"; + + public static final String BODY_PARAMS = "bodyParams"; + + public static final String CONTENT_FIELD = "contentField"; + + public static final String REQUEST_TYPE = "requestType"; + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPlugin.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPlugin.java new file mode 100644 index 0000000000..973f1617a6 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPlugin.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +import com.google.common.collect.ImmutableList; + +/** + * http alertPlugins + */ +public class HttpAlertPlugin implements DolphinSchedulerPlugin { + + @Override + public Iterable getAlertChannelFactorys() { + return ImmutableList.of(new HttpAlertChannelFactory()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSender.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSender.java new file mode 100644 index 0000000000..32d3cdb52f --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/main/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSender.java @@ -0,0 +1,166 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; +import org.apache.dolphinscheduler.spi.utils.StringUtils; + +import org.apache.http.HttpEntity; +import org.apache.http.client.methods.CloseableHttpResponse; +import org.apache.http.client.methods.HttpGet; +import org.apache.http.client.methods.HttpPost; +import org.apache.http.client.methods.HttpRequestBase; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClientBuilder; +import org.apache.http.util.EntityUtils; + +import java.util.HashMap; +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.fasterxml.jackson.databind.node.ObjectNode; + +/** + * http send message + */ +public class HttpSender { + + public static final Logger logger = LoggerFactory.getLogger(HttpSender.class); + + private String url; + + private final String headerParams; + + private final String bodyParams; + + private final String contentField; + + private final String requestType; + + private HttpRequestBase httpRequest; + + + private static final String URL_SPLICE_CHAR = "?"; + + /** + * request type post + */ + private static final String REQUEST_TYPE_POST = "POST"; + + /** + * request type get + */ + private static final String REQUEST_TYPE_GET = "GET"; + + private static final String DEFAULT_CHARSET = "utf-8"; + + public HttpSender(Map paramsMap) { + + url = paramsMap.get(HttpAlertConstants.URL); + headerParams = paramsMap.get(HttpAlertConstants.HEADER_PARAMS); + bodyParams = paramsMap.get(HttpAlertConstants.BODY_PARAMS); + contentField = paramsMap.get(HttpAlertConstants.CONTENT_FIELD); + requestType = paramsMap.get(HttpAlertConstants.REQUEST_TYPE); + } + + public AlertResult send(String msg) { + + AlertResult alertResult = new AlertResult(); + + createHttpRequest(msg); + + if (httpRequest == null) { + alertResult.setStatus("false"); + alertResult.setMessage("Request types are not supported"); + return alertResult; + } + + try { + CloseableHttpClient httpClient = HttpClientBuilder.create().build(); + CloseableHttpResponse response = httpClient.execute(httpRequest); + HttpEntity entity = response.getEntity(); + String resp = EntityUtils.toString(entity, DEFAULT_CHARSET); + alertResult.setStatus("true"); + alertResult.setMessage(resp); + } catch (Exception e) { + logger.error("send http alert msg exception : {}", e.getMessage()); + alertResult.setStatus("false"); + alertResult.setMessage("send http request alert fail."); + } + + return alertResult; + } + + private void createHttpRequest(String msg) { + + if (REQUEST_TYPE_POST.equals(requestType)) { + httpRequest = new HttpPost(url); + //POST request add param in request body + setMsgInRequestBody(msg); + } else if (REQUEST_TYPE_GET.equals(requestType)) { + //GET request add param in url + setMsgInUrl(msg); + httpRequest = new HttpGet(url); + } + setHeader(); + } + + /** + * add msg param in url + */ + private void setMsgInUrl(String msg) { + + if (StringUtils.isNotBlank(contentField)) { + String type = "&"; + //check splice char is & or ? + if (!url.contains(URL_SPLICE_CHAR)) { + type = URL_SPLICE_CHAR; + } + url = String.format("%s%s%s=%s", url, type, contentField, msg); + } + } + + /** + * set header params + */ + private void setHeader() { + + if (httpRequest == null) { + return; + } + + HashMap map = JSONUtils.parseObject(headerParams, HashMap.class); + for (Map.Entry entry : map.entrySet()) { + httpRequest.setHeader(entry.getKey(), String.valueOf(entry.getValue())); + } + } + + /** + * set body params + */ + private String setMsgInRequestBody(String msg) { + ObjectNode objectNode = JSONUtils.parseObject(bodyParams); + //set msg content field + objectNode.put(contentField, msg); + return objectNode.toString(); + } + +} diff --git a/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertInfoTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactoryTest.java similarity index 53% rename from dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertInfoTest.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactoryTest.java index 13eb595ac3..25181ebd26 100644 --- a/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertInfoTest.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelFactoryTest.java @@ -14,41 +14,41 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.plugin.model; -import org.junit.Before; -import org.junit.Test; +package org.apache.dolphinscheduler.plugin.alert.http; -import java.util.HashMap; -import java.util.Map; +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; -import static org.junit.Assert.*; +import java.util.List; -public class AlertInfoTest { +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; - private AlertInfo alertInfo; +/** + * HttpAlertChannelFactory UT + */ +public class HttpAlertChannelFactoryTest { + + private HttpAlertChannelFactory httpAlertChannelFactory; @Before - public void before() { - alertInfo = new AlertInfo(); + public void init() { + httpAlertChannelFactory = new HttpAlertChannelFactory(); } @Test - public void getAlertProps() { - Map map = new HashMap<>(); - alertInfo.setAlertProps(map); - assertNotNull(alertInfo.getAlertProps()); - } + public void getParamsTest() { - @Test - public void getProp() { - alertInfo.addProp("k", "v"); - assertEquals("v", alertInfo.getProp("k")); + List pluginParamsList = httpAlertChannelFactory.getParams(); + Assert.assertEquals(5, pluginParamsList.size()); } @Test - public void getAlertData() { - alertInfo.setAlertData(new AlertData()); - assertNotNull(alertInfo.getAlertData()); + public void createTest() { + AlertChannel alertChannel = httpAlertChannelFactory.create(); + Assert.assertNotNull(alertChannel); } -} \ No newline at end of file + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelTest.java new file mode 100644 index 0000000000..31a438b4fc --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertChannelTest.java @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +/** + * HttpAlertChannel UT + */ +public class HttpAlertChannelTest { + + @Test + public void processTest() { + + HttpAlertChannel alertChannel = new HttpAlertChannel(); + AlertInfo alertInfo = new AlertInfo(); + AlertData alertData = new AlertData(); + alertData.setContent("Fault tolerance warning"); + alertInfo.setAlertData(alertData); + AlertResult alertResult = alertChannel.process(alertInfo); + Assert.assertEquals("Request types are not supported", alertResult.getMessage()); + } + + @Test + public void processTest2() { + + HttpAlertChannel alertChannel = new HttpAlertChannel(); + AlertInfo alertInfo = new AlertInfo(); + AlertData alertData = new AlertData(); + alertData.setContent("Fault tolerance warning"); + alertInfo.setAlertData(alertData); + alertInfo.setAlertParams(getParams()); + AlertResult alertResult = alertChannel.process(alertInfo); + Assert.assertEquals("true", alertResult.getStatus()); + } + + /** + * create params + */ + private String getParams() { + + List paramsList = new ArrayList<>(); + InputParam urlParam = InputParam.newBuilder("url", "url") + .setValue("http://www.baidu.com") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam headerParams = InputParam.newBuilder("headerParams", "headerParams") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("{\"Content-Type\":\"application/json\"}") + .build(); + + InputParam bodyParams = InputParam.newBuilder("bodyParams", "bodyParams") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("{\"number\":\"13457654323\"}") + .build(); + + InputParam content = InputParam.newBuilder("contentField", "contentField") + .setValue("content") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam requestType = InputParam.newBuilder("requestType", "requestType") + .setValue("POST") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + paramsList.add(urlParam); + paramsList.add(headerParams); + paramsList.add(bodyParams); + paramsList.add(content); + paramsList.add(requestType); + + return JSONUtils.toJsonString(paramsList); + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPluginTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPluginTest.java new file mode 100644 index 0000000000..7dac686e88 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpAlertPluginTest.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +import org.junit.Assert; +import org.junit.Test; + +/** + * HttpAlertPlugin UT + */ +public class HttpAlertPluginTest { + + @Test + public void getAlertChannelFactorysTest() { + + HttpAlertPlugin httpAlertPlugin = new HttpAlertPlugin(); + Iterable alertChannelFactorys = httpAlertPlugin.getAlertChannelFactorys(); + Assert.assertNotNull(alertChannelFactorys); + + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSenderTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSenderTest.java new file mode 100644 index 0000000000..d59c4d47bb --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-http/src/test/java/org/apache/dolphinscheduler/plugin/alert/http/HttpSenderTest.java @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.http; + +import org.apache.dolphinscheduler.spi.alert.AlertResult; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Assert; +import org.junit.Test; + +/** + * HttpSender UT + */ +public class HttpSenderTest { + + @Test + public void sendTest() { + + Map paramsMap = new HashMap<>(); + paramsMap.put(HttpAlertConstants.URL, "http://www.baidu.com"); + paramsMap.put(HttpAlertConstants.REQUEST_TYPE, "POST"); + paramsMap.put(HttpAlertConstants.HEADER_PARAMS, "{\"Content-Type\":\"application/json\"}"); + paramsMap.put(HttpAlertConstants.BODY_PARAMS, "{\"number\":\"13457654323\"}"); + paramsMap.put(HttpAlertConstants.CONTENT_FIELD, "content"); + HttpSender httpSender = new HttpSender(paramsMap); + AlertResult alertResult = httpSender.send("Fault tolerance warning"); + Assert.assertEquals("true", alertResult.getStatus()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/pom.xml b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/pom.xml new file mode 100644 index 0000000000..8c35b3cbf6 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/pom.xml @@ -0,0 +1,79 @@ + + + + + dolphinscheduler-alert-plugin + org.apache.dolphinscheduler + 1.3.4-SNAPSHOT + + 4.0.0 + + org.apache.dolphinscheduler + dolphinscheduler-alert-script + dolphinscheduler-plugin + + + + + org.apache.dolphinscheduler + dolphinscheduler-spi + provided + + + + com.google.guava + guava + + + + ch.qos.logback + logback-classic + + + + org.slf4j + slf4j-api + + + + com.fasterxml.jackson.core + jackson-annotations + provided + + + + junit + junit + test + + + + org.mockito + mockito-core + jar + test + + + + + dolphinscheduler-alert-script-${project.version} + + + \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/OSUtils.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/OSUtils.java new file mode 100644 index 0000000000..acf3e310cc --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/OSUtils.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +/** + * OSUtils + */ +public class OSUtils { + + public OSUtils() { + throw new UnsupportedOperationException("Construct OSUtils"); + } + + static Boolean isWindows() { + return System.getProperty("os.name").startsWith("Windows"); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtils.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtils.java new file mode 100644 index 0000000000..d63a350051 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtils.java @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import java.io.IOException; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * ProcessUtils + */ +public class ProcessUtils { + + private static final Logger logger = LoggerFactory.getLogger(ProcessUtils.class); + + private ProcessUtils() { + throw new IllegalStateException("Utility class"); + } + + /** + * executeScript + * + * @param cmd cmd params + * @return exit code + */ + static Integer executeScript(String... cmd) { + + int exitCode = -1; + + ProcessBuilder processBuilder = new ProcessBuilder(cmd); + try { + Process process = processBuilder.start(); + StreamGobbler inputStreamGobbler = new StreamGobbler(process.getInputStream()); + StreamGobbler errorStreamGobbler = new StreamGobbler(process.getErrorStream()); + + inputStreamGobbler.start(); + errorStreamGobbler.start(); + return process.waitFor(); + } catch (IOException | InterruptedException e) { + logger.error("execute alert script error {}", e.getMessage()); + Thread.currentThread().interrupt(); + } + + return exitCode; + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannel.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannel.java new file mode 100644 index 0000000000..2a0021277d --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannel.java @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; + +import java.util.Map; + +/** + * ScriptAlertChannel + */ +public class ScriptAlertChannel implements AlertChannel { + + @Override + public AlertResult process(AlertInfo alertinfo) { + AlertData alertData = alertinfo.getAlertData(); + String alertParams = alertinfo.getAlertParams(); + Map paramsMap = PluginParamsTransfer.getPluginParamsMap(alertParams); + return new ScriptSender(paramsMap).sendScriptAlert(alertData.getTitle()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactory.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactory.java new file mode 100644 index 0000000000..70ab76381a --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactory.java @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.RadioParam; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.Arrays; +import java.util.List; + +/** + * ScriptAlertChannelFactory + */ +public class ScriptAlertChannelFactory implements AlertChannelFactory { + + @Override + public String getName() { + return "Script"; + } + + @Override + public List getParams() { + + InputParam scriptUserParam = InputParam.newBuilder(ScriptParamsConstants.NAME_SCRIPT_USER_PARAMS, ScriptParamsConstants.SCRIPT_USER_PARAMS) + .addValidate(Validate.newBuilder() + .setRequired(false) + .build()) + .setPlaceholder("please enter your custom parameters, which will be passed to you when calling your script") + .build(); + // need check file type and file exist + InputParam scriptPathParam = InputParam.newBuilder(ScriptParamsConstants.NAME_SCRIPT_PATH, ScriptParamsConstants.SCRIPT_PATH) + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .setPlaceholder("please upload the file to the disk directory of the alert server, and ensure that the path is absolute and has the corresponding access rights") + .build(); + + RadioParam scriptTypeParams = RadioParam.newBuilder(ScriptParamsConstants.NAME_SCRIPT_TYPE, ScriptParamsConstants.SCRIPT_TYPE) + .addParamsOptions(new ParamsOptions(ScriptType.SHELL.getDescp(), ScriptType.SHELL.getCode(), false)) + .setValue(ScriptType.SHELL.getCode()) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + return Arrays.asList(scriptUserParam, scriptPathParam, scriptTypeParams); + } + + @Override + public AlertChannel create() { + return new ScriptAlertChannel(); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertPlugin.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertPlugin.java new file mode 100644 index 0000000000..f2ba0e87d0 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertPlugin.java @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +import com.google.common.collect.ImmutableList; + +/** + * ScriptAlertPlugin + */ +public class ScriptAlertPlugin implements DolphinSchedulerPlugin { + + @Override + public Iterable getAlertChannelFactorys() { + return ImmutableList.of(new ScriptAlertChannelFactory()); + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptParamsConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptParamsConstants.java new file mode 100644 index 0000000000..98e4eee788 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptParamsConstants.java @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +/** + * ScriptParamsConstants + */ +public class ScriptParamsConstants { + + public ScriptParamsConstants() { + throw new IllegalStateException("Utility class"); + } + + static final String SCRIPT_TYPE = "type"; + + static final String NAME_SCRIPT_TYPE = "type"; + + static final String SCRIPT_PATH = "path"; + + static final String NAME_SCRIPT_PATH = "path"; + + static final String SCRIPT_USER_PARAMS = "user.params"; + + static final String NAME_SCRIPT_USER_PARAMS = "userParams"; +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSender.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSender.java new file mode 100644 index 0000000000..377c318b8e --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSender.java @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.apache.dolphinscheduler.spi.alert.AlertResult; + +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * ScriptSender + */ +public class ScriptSender { + + private static final Logger logger = LoggerFactory.getLogger(ScriptSender.class); + + private String scriptPath; + + private Integer scriptType; + + private String userParams; + + ScriptSender(Map config) { + scriptPath = config.get(ScriptParamsConstants.NAME_SCRIPT_PATH); + scriptType = Integer.parseInt(config.get(ScriptParamsConstants.NAME_SCRIPT_TYPE)); + userParams = config.get(ScriptParamsConstants.NAME_SCRIPT_USER_PARAMS); + } + + AlertResult sendScriptAlert(String msg) { + AlertResult alertResult = new AlertResult(); + if (ScriptType.of(scriptType).equals(ScriptType.SHELL)) { + return executeShellScript(msg); + } + return alertResult; + } + + private AlertResult executeShellScript(String msg) { + AlertResult alertResult = new AlertResult(); + alertResult.setStatus("false"); + if (Boolean.TRUE.equals(OSUtils.isWindows())) { + alertResult.setMessage("shell script not support windows os"); + return alertResult; + } + String[] cmd = {"/bin/sh", "-c", scriptPath + " " + msg + " " + userParams}; + int exitCode = ProcessUtils.executeScript(cmd); + + if (exitCode == 0) { + alertResult.setStatus("true"); + alertResult.setMessage("send script alert msg success"); + return alertResult; + } + alertResult.setMessage("send script alert msg error,exitCode is " + exitCode); + logger.info("send script alert msg error,exitCode is {}", exitCode); + return alertResult; + } + +} diff --git a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertInfo.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptType.java similarity index 51% rename from dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertInfo.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptType.java index 1d71ed7d8a..59b17d0a58 100644 --- a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertInfo.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptType.java @@ -14,48 +14,49 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.plugin.model; + +package org.apache.dolphinscheduler.plugin.alert.script; import java.util.HashMap; import java.util.Map; /** - * AlertInfo + * ScriptType */ -public class AlertInfo { +public enum ScriptType { - private Map alertProps; - private AlertData alertData; + SHELL(0, "SHELL"), + ; - public AlertInfo() { - this.alertProps = new HashMap<>(); + ScriptType(int code, String descp) { + this.code = code; + this.descp = descp; } - public Map getAlertProps() { - return alertProps; - } + private final int code; + private final String descp; - public AlertInfo setAlertProps(Map alertProps) { - this.alertProps = alertProps; - return this; + public int getCode() { + return code; } - public AlertInfo addProp(String key, Object value) { - this.alertProps.put(key, value); - return this; + public String getDescp() { + return descp; } - public Object getProp(String key) { - return this.alertProps.get(key); - } + private static final Map SCRIPT_TYPE_MAP = new HashMap<>(); - public AlertData getAlertData() { - return alertData; + static { + for (ScriptType scriptType : ScriptType.values()) { + SCRIPT_TYPE_MAP.put(scriptType.code, scriptType); + } } - public AlertInfo setAlertData(AlertData alertData) { - this.alertData = alertData; - return this; + public static ScriptType of(Integer code) { + if (SCRIPT_TYPE_MAP.containsKey(code)) { + return SCRIPT_TYPE_MAP.get(code); + } + throw new IllegalArgumentException("invalid code : " + code); } } diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/DingTalkManager.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/StreamGobbler.java similarity index 50% rename from dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/DingTalkManager.java rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/StreamGobbler.java index 871ad958c4..2ec25280b4 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/DingTalkManager.java +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/main/java/org/apache/dolphinscheduler/plugin/alert/script/StreamGobbler.java @@ -15,40 +15,47 @@ * limitations under the License. */ -package org.apache.dolphinscheduler.alert.manager; - -import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.alert.utils.DingTalkUtils; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; +package org.apache.dolphinscheduler.plugin.alert.script; +import java.io.BufferedReader; import java.io.IOException; -import java.util.HashMap; -import java.util.Map; +import java.io.InputStream; +import java.io.InputStreamReader; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** - * Ding Talk Manager + * StreamGobbler */ -public class DingTalkManager { - private static final Logger logger = LoggerFactory.getLogger(DingTalkManager.class); +public class StreamGobbler extends Thread { + + private static final Logger logger = LoggerFactory.getLogger(StreamGobbler.class); + + private InputStream inputStream; + + StreamGobbler(InputStream inputStream) { + this.inputStream = inputStream; + } + + @Override + public void run() { + InputStreamReader inputStreamReader = new InputStreamReader(inputStream); + BufferedReader inputBufferReader = new BufferedReader(inputStreamReader); - public Map send(AlertInfo alert) { - Map retMap = new HashMap<>(); - retMap.put(Constants.STATUS, false); - logger.info("send message {}", alert.getAlertData().getTitle()); try { - String msg = buildMessage(alert); - DingTalkUtils.sendDingTalkMsg(msg, Constants.UTF_8); + String line; + StringBuilder output = new StringBuilder(); + while ((line = inputBufferReader.readLine()) != null) { + output.append(line); + output.append(System.getProperty("line.separator")); + } + if (output.length() > 0) { + logger.info("out put msg is{}",output.toString()); + } } catch (IOException e) { - logger.error(e.getMessage(), e); + logger.error("I/O error occurs {}", e.getMessage()); } - retMap.put(Constants.STATUS, true); - return retMap; } - private String buildMessage(AlertInfo alert) { - return alert.getAlertData().getContent(); - } -} +} \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtilsTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtilsTest.java new file mode 100644 index 0000000000..1bf98d2019 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ProcessUtilsTest.java @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.junit.Test; + +/** + * ProcessUtilsTest + */ +public class ProcessUtilsTest { + + private static final String rootPath = System.getProperty("user.dir"); + + private static final String shellFilPath = rootPath + "/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/example.sh"; + + private String[] cmd = {"/bin/sh", "-c", shellFilPath + " " + "testMsg" + " " + "userParams"}; + + @Test + public void testExecuteScript() { + ProcessUtils.executeScript(cmd); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactoryTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactoryTest.java new file mode 100644 index 0000000000..72f2197315 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptAlertChannelFactoryTest.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +/** + * ScriptAlertChannelFactoryTest + */ +public class ScriptAlertChannelFactoryTest { + + @Test + public void testGetParams() { + ScriptAlertChannelFactory scriptAlertChannelFactory = new ScriptAlertChannelFactory(); + List params = scriptAlertChannelFactory.getParams(); + JSONUtils.toJsonString(params); + Assert.assertEquals(3, params.size()); + } + + @Test + public void testCreate() { + ScriptAlertChannelFactory scriptAlertChannelFactory = new ScriptAlertChannelFactory(); + AlertChannel alertChannel = scriptAlertChannelFactory.create(); + Assert.assertNotNull(alertChannel); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSenderTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSenderTest.java new file mode 100644 index 0000000000..1cd74cfaba --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/java/org/apache/dolphinscheduler/plugin/alert/script/ScriptSenderTest.java @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.script; + +import org.apache.dolphinscheduler.spi.alert.AlertResult; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +/** + * ScriptSenderTest + */ +public class ScriptSenderTest { + + private static Map scriptConfig = new HashMap<>(); + + private static final String rootPath = System.getProperty("user.dir"); + + private static final String shellFilPath = rootPath + "/src/test/script/shell/scriptTest.sh"; + + @Before + public void initScriptConfig() { + + scriptConfig.put(ScriptParamsConstants.NAME_SCRIPT_TYPE, String.valueOf(ScriptType.SHELL.getCode())); + scriptConfig.put(ScriptParamsConstants.NAME_SCRIPT_USER_PARAMS, "userParams"); + scriptConfig.put(ScriptParamsConstants.NAME_SCRIPT_PATH, shellFilPath); + } + + @Test + public void testScriptSenderTest() { + ScriptSender scriptSender = new ScriptSender(scriptConfig); + AlertResult alertResult; + alertResult = scriptSender.sendScriptAlert("success"); + Assert.assertEquals("true", alertResult.getStatus()); + alertResult = scriptSender.sendScriptAlert("errorMsg"); + Assert.assertEquals("false", alertResult.getStatus()); + } + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/example.sh b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/example.sh new file mode 100755 index 0000000000..708dcd004b --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/example.sh @@ -0,0 +1,25 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + + +msg=$1 +content=$2 + +# Write your specific logic here + +# Set the exit code according to your execution result, and alert needs to use it to judge the status of this alarm result +exit 0 diff --git a/dolphinscheduler-plugin-api/src/test/resources/plugin.properties b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/scriptTest.sh old mode 100644 new mode 100755 similarity index 91% rename from dolphinscheduler-plugin-api/src/test/resources/plugin.properties rename to dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/scriptTest.sh index d2ea3831be..02eba48a81 --- a/dolphinscheduler-plugin-api/src/test/resources/plugin.properties +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-script/src/test/script/shell/scriptTest.sh @@ -15,8 +15,11 @@ # limitations under the License. # -test.string=teststring -test.false=false -test.true=true -cba=3.1 -test.long=100 \ No newline at end of file +msg=$1 +content=$2 + +if [ $msg = errorMsg ] + then + exit 12 +fi +exit 0 \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml new file mode 100644 index 0000000000..123cebc5c5 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml @@ -0,0 +1,78 @@ + + + + + dolphinscheduler-alert-plugin + org.apache.dolphinscheduler + 1.3.4-SNAPSHOT + + 4.0.0 + + org.apache.dolphinscheduler + dolphinscheduler-alert-wechat + dolphinscheduler-plugin + + + + org.apache.dolphinscheduler + dolphinscheduler-spi + provided + + + + com.google.guava + guava + + + + ch.qos.logback + logback-classic + + + + org.apache.httpcomponents + httpclient + + + + com.fasterxml.jackson.core + jackson-databind + provided + + + + junit + junit + test + + + + org.mockito + mockito-core + jar + test + + + + + + dolphinscheduler-alert-wechat-${project.version} + + \ No newline at end of file diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannel.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannel.java new file mode 100644 index 0000000000..4cdd4d375c --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannel.java @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; + +import java.util.Map; + +/** + * WeChatAlertChannel + */ +public class WeChatAlertChannel implements AlertChannel { + + @Override + public AlertResult process(AlertInfo info) { + AlertData alertData = info.getAlertData(); + String alertParams = info.getAlertParams(); + Map paramsMap = PluginParamsTransfer.getPluginParamsMap(alertParams); + return new WeChatSender(paramsMap).sendEnterpriseWeChat(alertData.getTitle(), alertData.getContent()); + + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactory.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactory.java new file mode 100644 index 0000000000..636c571322 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactory.java @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.RadioParam; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.Arrays; +import java.util.List; + +/** + * WeChatAlertChannelFactory + */ +public class WeChatAlertChannelFactory implements AlertChannelFactory { + + @Override + public String getName() { + return "WeChat"; + } + + @Override + public List getParams() { + InputParam corpIdParam = InputParam.newBuilder(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_CORP_ID, WeChatAlertParamsConstants.ENTERPRISE_WE_CHAT_CORP_ID) + .setPlaceholder("please input corp id ") + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam secretParam = InputParam.newBuilder(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_SECRET, WeChatAlertParamsConstants.ENTERPRISE_WE_CHAT_SECRET) + .setPlaceholder("please input secret ") + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam usersParam = InputParam.newBuilder(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_USERS, WeChatAlertParamsConstants.ENTERPRISE_WE_CHAT_USERS) + .setPlaceholder("please input users ") + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam userSendMsgParam = InputParam.newBuilder(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_USER_SEND_MSG, WeChatAlertParamsConstants.ENTERPRISE_WE_CHAT_USER_SEND_MSG) + .setPlaceholder("please input corp id ") + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + InputParam agentIdParam = InputParam.newBuilder(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_AGENT_ID, WeChatAlertParamsConstants.ENTERPRISE_WE_CHAT_AGENT_ID) + .setPlaceholder("please input agent id ") + .addValidate(Validate.newBuilder() + .setRequired(true) + .build()) + .build(); + + RadioParam showType = RadioParam.newBuilder(AlertConstants.SHOW_TYPE, AlertConstants.SHOW_TYPE) + .addParamsOptions(new ParamsOptions(ShowType.TABLE.getDescp(), ShowType.TABLE.getDescp(), false)) + .addParamsOptions(new ParamsOptions(ShowType.TEXT.getDescp(), ShowType.TEXT.getDescp(), false)) + .setValue(ShowType.TABLE.getDescp()) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + return Arrays.asList(corpIdParam, secretParam, usersParam, userSendMsgParam, agentIdParam, showType); + } + + @Override + public AlertChannel create() { + return new WeChatAlertChannel(); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertConstants.java new file mode 100644 index 0000000000..904060c3e0 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertConstants.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +/** + * WeChatAlertConstants + */ +public class WeChatAlertConstants { + + static final String MARKDOWN_QUOTE = ">"; + + static final String MARKDOWN_ENTER = "\n"; + + static final String CHARSET = "UTF-8"; + + static final String WE_CHAT_PUSH_URL = "https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token={token}"; + + static final String WE_CHAT_TOKEN_URL = "https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid={corpId}&corpsecret={secret}"; +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertParamsConstants.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertParamsConstants.java new file mode 100644 index 0000000000..138bd83b3a --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertParamsConstants.java @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +/** + * WeChatAlertParamsConstants + */ +public class WeChatAlertParamsConstants { + + + static final String ENTERPRISE_WE_CHAT_CORP_ID = "corp.id"; + + static final String NAME_ENTERPRISE_WE_CHAT_CORP_ID = "corpId"; + + + static final String ENTERPRISE_WE_CHAT_SECRET = "secret"; + + static final String NAME_ENTERPRISE_WE_CHAT_SECRET = "secret"; + + static final String ENTERPRISE_WE_CHAT_TEAM_SEND_MSG = "team.send.msg"; + + static final String NAME_ENTERPRISE_WE_CHAT_TEAM_SEND_MSG = "teamSendMsg"; + + + static final String ENTERPRISE_WE_CHAT_USER_SEND_MSG = "user.send.msg"; + + static final String NAME_ENTERPRISE_WE_CHAT_USER_SEND_MSG = "userSendMsg"; + + + static final String ENTERPRISE_WE_CHAT_AGENT_ID = "agent.id"; + + static final String NAME_ENTERPRISE_WE_CHAT_AGENT_ID = "agentId"; + + + static final String ENTERPRISE_WE_CHAT_USERS = "users"; + + + static final String NAME_ENTERPRISE_WE_CHAT_USERS = "users"; + + +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertPlugin.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertPlugin.java new file mode 100644 index 0000000000..56c3c01f20 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertPlugin.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +import com.google.common.collect.ImmutableList; + +/** + * WeChatAlertPlugin + */ +public class WeChatAlertPlugin implements DolphinSchedulerPlugin { + + @Override + public Iterable getAlertChannelFactorys() { + return ImmutableList.of(new WeChatAlertChannelFactory()); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSender.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSender.java new file mode 100644 index 0000000000..f4fd090744 --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/main/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSender.java @@ -0,0 +1,335 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +import static java.util.Objects.requireNonNull; + +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; +import org.apache.dolphinscheduler.spi.utils.StringUtils; + +import org.apache.http.HttpEntity; +import org.apache.http.client.methods.CloseableHttpResponse; +import org.apache.http.client.methods.HttpGet; +import org.apache.http.client.methods.HttpPost; +import org.apache.http.entity.StringEntity; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.http.util.EntityUtils; + +import java.io.IOException; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Set; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * WeChatSender + */ +public class WeChatSender { + + private static Logger logger = LoggerFactory.getLogger(WeChatSender.class); + + private String weChatAgentId; + + private String weChatUsers; + + private String weChatTeamSendMsg; + + private String weChatUserSendMsg; + + private String weChatTokenUrlReplace; + + private String weChatToken; + + private String showType; + + + private static final String agentIdRegExp = "{agentId}"; + private static final String msgRegExp = "{msg}"; + private static final String userRegExp = "{toUser}"; + private static final String corpIdRegex = "{corpId}"; + private static final String secretRegex = "{secret}"; + private static final String toPartyRegex = "{toParty}"; + private static final String toUserRegex = "{toUser}"; + private static final String tokenRegex = "{token}"; + + WeChatSender(Map config) { + weChatAgentId = config.get(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_AGENT_ID); + weChatUsers = config.get(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_USERS); + String weChatCorpId = config.get(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_CORP_ID); + String weChatSecret = config.get(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_SECRET); + String weChatTokenUrl = WeChatAlertConstants.WE_CHAT_TOKEN_URL; + weChatTeamSendMsg = config.get(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_TEAM_SEND_MSG); + weChatUserSendMsg = config.get(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_USER_SEND_MSG); + showType = config.get(AlertConstants.SHOW_TYPE); + requireNonNull(showType, AlertConstants.SHOW_TYPE + " must not null"); + weChatTokenUrlReplace = weChatTokenUrl + .replace(corpIdRegex, weChatCorpId) + .replace(secretRegex, weChatSecret); + weChatToken = getToken(); + } + + /** + * make user multi user message + * + * @param toUser the toUser + * @param agentId the agentId + * @param msg the msg + * @return Enterprise WeChat send message + */ + private String makeUserSendMsg(Collection toUser, String agentId, String msg) { + String listUser = mkString(toUser); + return weChatUserSendMsg.replace(userRegExp, listUser) + .replace(agentIdRegExp, agentId) + .replace(msgRegExp, msg); + } + + /** + * send Enterprise WeChat + * + * @return Enterprise WeChat resp, demo: {"errcode":0,"errmsg":"ok","invaliduser":""} + * @throws Exception the Exception + */ + public AlertResult sendEnterpriseWeChat(String title, String content) { + AlertResult alertResult; + List userList = Arrays.asList(weChatUsers.split(",")); + String data = markdownByAlert(title, content); + String msg = makeUserSendMsg(userList, weChatAgentId, data); + if (null == weChatToken) { + alertResult = new AlertResult(); + alertResult.setMessage("send we chat alert fail,get weChat token error"); + alertResult.setStatus("false"); + return alertResult; + } + String enterpriseWeChatPushUrlReplace = WeChatAlertConstants.WE_CHAT_PUSH_URL.replace(tokenRegex, weChatToken); + + try { + return checkWeChatSendMsgResult(post(enterpriseWeChatPushUrlReplace, msg)); + } catch (Exception e) { + logger.info("send we chat alert msg exception : {}", e.getMessage()); + alertResult = new AlertResult(); + alertResult.setMessage("send we chat alert fail"); + alertResult.setStatus("false"); + } + return alertResult; + } + + private static String post(String url, String data) throws IOException { + try (CloseableHttpClient httpClient = HttpClients.createDefault()) { + HttpPost httpPost = new HttpPost(url); + httpPost.setEntity(new StringEntity(data, WeChatAlertConstants.CHARSET)); + CloseableHttpResponse response = httpClient.execute(httpPost); + String resp; + try { + HttpEntity entity = response.getEntity(); + resp = EntityUtils.toString(entity, WeChatAlertConstants.CHARSET); + EntityUtils.consume(entity); + } finally { + response.close(); + } + logger.info("Enterprise WeChat send [{}], param:{}, resp:{}", + url, data, resp); + return resp; + } + } + + /** + * convert table to markdown style + * + * @param title the title + * @param content the content + * @return markdown table content + */ + private static String markdownTable(String title, String content) { + List mapItemsList = JSONUtils.toList(content, LinkedHashMap.class); + if (null == mapItemsList || mapItemsList.isEmpty()) { + logger.error("itemsList is null"); + throw new RuntimeException("itemsList is null"); + } + StringBuilder contents = new StringBuilder(200); + for (LinkedHashMap mapItems : mapItemsList) { + Set> entries = mapItems.entrySet(); + Iterator> iterator = entries.iterator(); + StringBuilder t = new StringBuilder(String.format("`%s`%s", title, WeChatAlertConstants.MARKDOWN_ENTER)); + + while (iterator.hasNext()) { + + Map.Entry entry = iterator.next(); + t.append(WeChatAlertConstants.MARKDOWN_QUOTE); + t.append(entry.getKey()).append(":").append(entry.getValue()); + t.append(WeChatAlertConstants.MARKDOWN_ENTER); + } + contents.append(t); + } + + return contents.toString(); + } + + /** + * convert text to markdown style + * + * @param title the title + * @param content the content + * @return markdown text + */ + private static String markdownText(String title, String content) { + if (StringUtils.isNotEmpty(content)) { + List mapItemsList = JSONUtils.toList(content, LinkedHashMap.class); + if (null == mapItemsList || mapItemsList.isEmpty()) { + logger.error("itemsList is null"); + throw new RuntimeException("itemsList is null"); + } + + StringBuilder contents = new StringBuilder(100); + contents.append(String.format("`%s`%n", title)); + for (LinkedHashMap mapItems : mapItemsList) { + + Set> entries = mapItems.entrySet(); + for (Entry entry : entries) { + contents.append(WeChatAlertConstants.MARKDOWN_QUOTE); + contents.append(entry.getKey()).append(":").append(entry.getValue()); + contents.append(WeChatAlertConstants.MARKDOWN_ENTER); + } + + } + return contents.toString(); + } + return null; + } + + /** + * Determine the mardown style based on the show type of the alert + * + * @return the markdown alert table/text + */ + private String markdownByAlert(String title, String content) { + String result = ""; + if (showType.equals(ShowType.TABLE.getDescp())) { + result = markdownTable(title, content); + } else if (showType.equals(ShowType.TEXT.getDescp())) { + result = markdownText(title, content); + } + return result; + + } + + private String getToken() { + try { + return get(weChatTokenUrlReplace); + } catch (IOException e) { + logger.info("we chat alert get token error{}", e.getMessage()); + } + return null; + } + + private static String get(String url) throws IOException { + String resp; + + try (CloseableHttpClient httpClient = HttpClients.createDefault()) { + HttpGet httpGet = new HttpGet(url); + try (CloseableHttpResponse response = httpClient.execute(httpGet)) { + HttpEntity entity = response.getEntity(); + resp = EntityUtils.toString(entity, WeChatAlertConstants.CHARSET); + EntityUtils.consume(entity); + } + + HashMap map = JSONUtils.parseObject(resp, HashMap.class); + if (map != null && null != map.get("access_token")) { + return map.get("access_token").toString(); + } else { + return null; + } + } + } + + private static String mkString(Iterable list) { + + if (null == list || StringUtils.isEmpty("|")) { + return null; + } + + StringBuilder sb = new StringBuilder(); + boolean first = true; + for (String item : list) { + if (first) { + first = false; + } else { + sb.append("|"); + } + sb.append(item); + } + return sb.toString(); + } + + public static class WeChatSendMsgResponse { + private Integer errcode; + private String errmsg; + + public Integer getErrcode() { + return errcode; + } + + public void setErrcode(Integer errcode) { + this.errcode = errcode; + } + + public String getErrmsg() { + return errmsg; + } + + public void setErrmsg(String errmsg) { + this.errmsg = errmsg; + } + } + + private static AlertResult checkWeChatSendMsgResult(String result) { + AlertResult alertResult = new AlertResult(); + alertResult.setStatus("false"); + + if (null == result) { + alertResult.setMessage("we chat send fail"); + logger.info("send we chat msg error,resp is null"); + return alertResult; + } + WeChatSendMsgResponse sendMsgResponse = JSONUtils.parseObject(result, WeChatSendMsgResponse.class); + if (null == sendMsgResponse) { + alertResult.setMessage("we chat send fail"); + logger.info("send we chat msg error,resp error"); + return alertResult; + } + if (sendMsgResponse.errcode == 0) { + alertResult.setStatus("true"); + alertResult.setMessage("we chat alert send success"); + return alertResult; + } + alertResult.setStatus("false"); + alertResult.setMessage(sendMsgResponse.getErrmsg()); + return alertResult; + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactoryTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactoryTest.java new file mode 100644 index 0000000000..cc62d5aabf --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatAlertChannelFactoryTest.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +/** + * WeChatAlertChannelFactoryTest + */ +public class WeChatAlertChannelFactoryTest { + + @Test + public void testGetParams() { + WeChatAlertChannelFactory weChatAlertChannelFactory = new WeChatAlertChannelFactory(); + List params = weChatAlertChannelFactory.getParams(); + JSONUtils.toJsonString(params); + Assert.assertEquals(6, params.size()); + } + + @Test + public void testCreate() { + WeChatAlertChannelFactory dingTalkAlertChannelFactory = new WeChatAlertChannelFactory(); + AlertChannel alertChannel = dingTalkAlertChannelFactory.create(); + Assert.assertNotNull(alertChannel); + } +} diff --git a/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSenderTest.java b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSenderTest.java new file mode 100644 index 0000000000..61010ebe3a --- /dev/null +++ b/dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/src/test/java/org/apache/dolphinscheduler/plugin/alert/wechat/WeChatSenderTest.java @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.plugin.alert.wechat; + +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.AlertResult; +import org.apache.dolphinscheduler.spi.alert.ShowType; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +/** + * WeChatSenderTest + */ +public class WeChatSenderTest { + + private static Map weChatConfig = new HashMap<>(); + + private String content = "[{\"id\":\"69\"," + + + "\"name\":\"UserBehavior-0--1193959466\"," + + + "\"Job name\":\"Start workflow\"," + + + "\"State\":\"SUCCESS\"," + + + "\"Recovery\":\"NO\"," + + + "\"Run time\":\"1\"," + + + "\"Start time\": \"2018-08-06 10:31:34.0\"," + + + "\"End time\": \"2018-08-06 10:31:49.0\"," + + + "\"Host\": \"192.168.xx.xx\"," + + + "\"Notify group\" :\"4\"}]"; + + @Before + public void initDingTalkConfig() { + // Just for this test, I will delete these configurations before this PR is merged + weChatConfig.put(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_AGENT_ID, "100000"); + weChatConfig.put(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_CORP_ID, "NAME_ENTERPRISE_WE_CHAT_CORP_ID"); + weChatConfig.put(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_SECRET, "NAME_ENTERPRISE_WE_CHAT_SECRET"); + weChatConfig.put(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_USER_SEND_MSG, "{\"touser\":\"{toUser}\",\"agentid\":{agentId}" + + + ",\"msgtype\":\"markdown\",\"markdown\":{\"content\":\"{msg}\"}}" + ); + weChatConfig.put(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_USERS, "Kris"); + weChatConfig.put(WeChatAlertParamsConstants.NAME_ENTERPRISE_WE_CHAT_TEAM_SEND_MSG, "msg"); + weChatConfig.put(AlertConstants.SHOW_TYPE, ShowType.TABLE.getDescp()); + } + + @Test + public void testSendWeChatTableMsg() { + WeChatSender weChatSender = new WeChatSender(weChatConfig); + + AlertResult alertResult = weChatSender.sendEnterpriseWeChat("test", content); + Assert.assertEquals("false", alertResult.getStatus()); + } + + @Test + public void testSendWeChatTextMsg() { + weChatConfig.put(AlertConstants.SHOW_TYPE, ShowType.TEXT.getDescp()); + WeChatSender weChatSender = new WeChatSender(weChatConfig); + AlertResult alertResult = weChatSender.sendEnterpriseWeChat("test", content); + Assert.assertEquals("false", alertResult.getStatus()); + } + +} diff --git a/dolphinscheduler-alert-plugin/pom.xml b/dolphinscheduler-alert-plugin/pom.xml new file mode 100644 index 0000000000..ede5051049 --- /dev/null +++ b/dolphinscheduler-alert-plugin/pom.xml @@ -0,0 +1,41 @@ + + + + + dolphinscheduler + org.apache.dolphinscheduler + 1.3.4-SNAPSHOT + + 4.0.0 + + org.apache.dolphinscheduler + dolphinscheduler-alert-plugin + pom + + + dolphinscheduler-alert-email + dolphinscheduler-alert-wechat + dolphinscheduler-alert-dingtalk + dolphinscheduler-alert-script + dolphinscheduler-alert-http + + + + \ No newline at end of file diff --git a/dolphinscheduler-alert/pom.xml b/dolphinscheduler-alert/pom.xml index a44d101ffe..cea763ccbd 100644 --- a/dolphinscheduler-alert/pom.xml +++ b/dolphinscheduler-alert/pom.xml @@ -32,38 +32,21 @@ - junit - junit - test + org.apache.dolphinscheduler + dolphinscheduler-spi - org.mockito - mockito-core - jar - test + org.apache.dolphinscheduler + dolphinscheduler-remote - - org.powermock - powermock-module-junit4 - test + org.apache.dolphinscheduler + dolphinscheduler-common - - org.powermock - powermock-api-mockito2 + junit + junit test - - - org.mockito - mockito-core - - - - - - org.apache.commons - commons-email @@ -108,6 +91,52 @@ + + org.sonatype.aether + aether-api + provided + + + + io.airlift.resolver + resolver + provided + + + + org.ow2.asm + asm + provided + + + + org.powermock + powermock-module-junit4 + test + + + + org.mockito + mockito-core + test + + + + mysql + mysql-connector-java + + + + org.powermock + powermock-api-mockito2 + test + + + org.mockito + mockito-core + + + diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/AlertServer.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/AlertServer.java index bf791ac9f0..54afc93442 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/AlertServer.java +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/AlertServer.java @@ -17,21 +17,32 @@ package org.apache.dolphinscheduler.alert; -import org.apache.dolphinscheduler.alert.plugin.EmailAlertPlugin; +import static org.apache.dolphinscheduler.common.Constants.ALERT_RPC_PORT; + +import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager; +import org.apache.dolphinscheduler.alert.plugin.DolphinPluginLoader; +import org.apache.dolphinscheduler.alert.plugin.DolphinPluginManagerConfig; +import org.apache.dolphinscheduler.alert.processor.AlertRequestProcessor; import org.apache.dolphinscheduler.alert.runner.AlertSender; import org.apache.dolphinscheduler.alert.utils.Constants; import org.apache.dolphinscheduler.alert.utils.PropertyUtils; -import org.apache.dolphinscheduler.common.plugin.FilePluginManager; import org.apache.dolphinscheduler.common.thread.Stopper; import org.apache.dolphinscheduler.dao.AlertDao; import org.apache.dolphinscheduler.dao.DaoFactory; +import org.apache.dolphinscheduler.dao.PluginDao; import org.apache.dolphinscheduler.dao.entity.Alert; +import org.apache.dolphinscheduler.remote.NettyRemotingServer; +import org.apache.dolphinscheduler.remote.command.CommandType; +import org.apache.dolphinscheduler.remote.config.NettyServerConfig; +import org.apache.dolphinscheduler.spi.utils.StringUtils; import java.util.List; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import com.google.common.collect.ImmutableList; + /** * alert of start */ @@ -42,34 +53,75 @@ public class AlertServer { */ private AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); + private PluginDao pluginDao = DaoFactory.getDaoInstance(PluginDao.class); + + private AlertSender alertSender; + private static AlertServer instance; - private FilePluginManager alertPluginManager; + private AlertPluginManager alertPluginManager; + + private DolphinPluginManagerConfig alertPluginManagerConfig; + + public static final String ALERT_PLUGIN_BINDING = "alert.plugin.binding"; + + public static final String ALERT_PLUGIN_DIR = "alert.plugin.dir"; + + public static final String MAVEN_LOCAL_REPOSITORY = "maven.local.repository"; + + /** + * netty server + */ + private NettyRemotingServer server; + + private static class AlertServerHolder { + private static final AlertServer INSTANCE = new AlertServer(); + } - private static final String[] whitePrefixes = new String[]{"org.apache.dolphinscheduler.plugin.utils."}; + public static final AlertServer getInstance() { + return AlertServerHolder.INSTANCE; + + } - private static final String[] excludePrefixes = new String[]{ - "org.apache.dolphinscheduler.plugin.", - "ch.qos.logback.", - "org.slf4j." - }; + private AlertServer() { - public AlertServer() { - alertPluginManager = - new FilePluginManager(PropertyUtils.getString(Constants.PLUGIN_DIR), whitePrefixes, excludePrefixes); - // add default alert plugins - alertPluginManager.addPlugin(new EmailAlertPlugin()); } - public static synchronized AlertServer getInstance() { - if (null == instance) { - instance = new AlertServer(); + private void initPlugin() { + alertPluginManager = new AlertPluginManager(); + alertPluginManagerConfig = new DolphinPluginManagerConfig(); + alertPluginManagerConfig.setPlugins(PropertyUtils.getString(ALERT_PLUGIN_BINDING)); + if (StringUtils.isNotBlank(PropertyUtils.getString(ALERT_PLUGIN_DIR))) { + alertPluginManagerConfig.setInstalledPluginsDir(PropertyUtils.getString(ALERT_PLUGIN_DIR, Constants.ALERT_PLUGIN_PATH).trim()); + } + + if (StringUtils.isNotBlank(PropertyUtils.getString(MAVEN_LOCAL_REPOSITORY))) { + alertPluginManagerConfig.setMavenLocalRepository(PropertyUtils.getString(MAVEN_LOCAL_REPOSITORY).trim()); + } + + DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager)); + try { + alertPluginLoader.loadPlugins(); + } catch (Exception e) { + throw new RuntimeException("load Alert Plugin Failed !", e); } - return instance; } - public void start() { - logger.info("alert server ready start "); + /** + * init netty remoting server + */ + private void initRemoteServer() { + NettyServerConfig serverConfig = new NettyServerConfig(); + serverConfig.setListenPort(ALERT_RPC_PORT); + this.server = new NettyRemotingServer(serverConfig); + this.server.registerProcessor(CommandType.ALERT_SEND_REQUEST, new AlertRequestProcessor(alertDao, alertPluginManager, pluginDao)); + this.server.start(); + } + + /** + * Cyclic alert info sending alert + */ + private void runSender() { while (Stopper.isRunning()) { try { Thread.sleep(Constants.ALERT_SCAN_INTERVAL); @@ -77,15 +129,43 @@ public class AlertServer { logger.error(e.getMessage(), e); Thread.currentThread().interrupt(); } - List alerts = alertDao.listWaitExecutionAlert(); - AlertSender alertSender = new AlertSender(alerts, alertDao, alertPluginManager); - alertSender.run(); + if (alertPluginManager == null || alertPluginManager.getAlertChannelMap().size() == 0) { + logger.warn("No Alert Plugin . Can not send alert info. "); + } else { + List alerts = alertDao.listWaitExecutionAlert(); + alertSender = new AlertSender(alerts, alertDao, alertPluginManager, pluginDao); + alertSender.run(); + } } } + /** + * start + */ + public void start() { + initPlugin(); + initRemoteServer(); + logger.info("alert server ready start "); + runSender(); + } + + /** + * stop + */ + public void stop() { + this.server.close(); + logger.info("alert server shut down"); + } + public static void main(String[] args) { AlertServer alertServer = AlertServer.getInstance(); alertServer.start(); + Runtime.getRuntime().addShutdownHook(new Thread() { + @Override + public void run() { + alertServer.stop(); + } + }); } } diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EmailManager.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EmailManager.java index 22f4b7a27f..874b866759 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EmailManager.java +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EmailManager.java @@ -1,55 +1,55 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.alert.manager; - -import org.apache.dolphinscheduler.alert.utils.MailUtils; - -import java.util.List; -import java.util.Map; - -/** - * email send manager - */ -public class EmailManager { - /** - * email send - * @param receiversList the receiver list - * @param receiversCcList the cc List - * @param title the title - * @param content the content - * @param showType the showType - * @return the send result - */ - public Map send(List receiversList,List receiversCcList,String title,String content,String showType){ - - return MailUtils.sendMails(receiversList, receiversCcList, title, content, showType); - } - - /** - * msg send - * @param receiversList the receiver list - * @param title the title - * @param content the content - * @param showType the showType - * @return the send result - */ - public Map send(List receiversList,String title,String content,String showType){ - - return MailUtils.sendMails(receiversList,title, content, showType); - } - -} +///* +// * Licensed to the Apache Software Foundation (ASF) under one or more +// * contributor license agreements. See the NOTICE file distributed with +// * this work for additional information regarding copyright ownership. +// * The ASF licenses this file to You under the Apache License, Version 2.0 +// * (the "License"); you may not use this file except in compliance with +// * the License. You may obtain a copy of the License at +// * +// * http://www.apache.org/licenses/LICENSE-2.0 +// * +// * Unless required by applicable law or agreed to in writing, software +// * distributed under the License is distributed on an "AS IS" BASIS, +// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// * See the License for the specific language governing permissions and +// * limitations under the License. +// */ +//package org.apache.dolphinscheduler.alert.manager; +// +//import org.apache.dolphinscheduler.alert.utils.MailUtils; +// +//import java.util.List; +//import java.util.Map; +// +///** +// * email send manager +// */ +//public class EmailManager { +// /** +// * email send +// * @param receiversList the receiver list +// * @param receiversCcList the cc List +// * @param title the title +// * @param content the content +// * @param showType the showType +// * @return the send result +// */ +// public Map send(List receiversList,List receiversCcList,String title,String content,String showType){ +// +// return MailUtils.sendMails(receiversList, receiversCcList, title, content, showType); +// } +// +// /** +// * msg send +// * @param receiversList the receiver list +// * @param title the title +// * @param content the content +// * @param showType the showType +// * @return the send result +// */ +// public Map send(List receiversList,String title,String content,String showType){ +// +// return MailUtils.sendMails(receiversList,title, content, showType); +// } +// +//} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EnterpriseWeChatManager.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EnterpriseWeChatManager.java deleted file mode 100644 index 43649d6758..0000000000 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/EnterpriseWeChatManager.java +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.alert.manager; - -import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.alert.utils.EnterpriseWeChatUtils; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -/** - * Enterprise WeChat Manager - */ -public class EnterpriseWeChatManager { - private static final Logger logger = LoggerFactory.getLogger(EnterpriseWeChatManager.class); - /** - * Enterprise We Chat send - * @param alertInfo the alert info - * @param token the token - * @return the send result - */ - public Map send(AlertInfo alertInfo, String token){ - Map retMap = new HashMap<>(); - retMap.put(Constants.STATUS, false); - String agentId = EnterpriseWeChatUtils.ENTERPRISE_WE_CHAT_AGENT_ID; - String users = EnterpriseWeChatUtils.ENTERPRISE_WE_CHAT_USERS; - List userList = Arrays.asList(users.split(",")); - logger.info("send message {}", alertInfo.getAlertData().getTitle()); - String msg = EnterpriseWeChatUtils.makeUserSendMsg(userList, agentId,EnterpriseWeChatUtils.markdownByAlert(alertInfo.getAlertData())); - try { - EnterpriseWeChatUtils.sendEnterpriseWeChat(Constants.UTF_8, msg, token); - } catch (IOException e) { - logger.error(e.getMessage(),e); - } - retMap.put(Constants.STATUS, true); - return retMap; - } - -} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/MsgManager.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/MsgManager.java index 359492699d..e7fb161162 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/MsgManager.java +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/manager/MsgManager.java @@ -14,23 +14,27 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.alert.manager; import org.apache.dolphinscheduler.dao.entity.Alert; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * SMS send manager */ -public class MsgManager { +public class MsgManager { private static final Logger logger = LoggerFactory.getLogger(MsgManager.class); + /** * SMS send + * * @param alert the alert */ - public void send(Alert alert){ - logger.info("send message {}",alert); + public void send(Alert alert) { + logger.info("send message {}", alert); } } diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AbstractDolphinPluginManager.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AbstractDolphinPluginManager.java new file mode 100644 index 0000000000..9fd847d0e8 --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AbstractDolphinPluginManager.java @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import org.apache.dolphinscheduler.dao.DaoFactory; +import org.apache.dolphinscheduler.dao.PluginDao; +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; + +public abstract class AbstractDolphinPluginManager { + + protected PluginDao pluginDao = DaoFactory.getDaoInstance(PluginDao.class); + + public abstract void installPlugin(DolphinSchedulerPlugin dolphinSchedulerPlugin); +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManager.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManager.java new file mode 100644 index 0000000000..a660087c89 --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManager.java @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import static java.lang.String.format; +import static java.util.Objects.requireNonNull; + +import static com.google.common.base.Preconditions.checkState; + +import org.apache.dolphinscheduler.common.enums.PluginType; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; +import org.apache.dolphinscheduler.spi.classloader.ThreadContextClassLoader; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; + +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * load the configured alert plugin and manager them + */ +public class AlertPluginManager extends AbstractDolphinPluginManager { + private static final Logger logger = LoggerFactory.getLogger(AlertPluginManager.class); + + private final Map alertChannelFactoryMap = new ConcurrentHashMap<>(); + private final Map alertChannelMap = new ConcurrentHashMap<>(); + + public void addAlertChannelFactory(AlertChannelFactory alertChannelFactory) { + requireNonNull(alertChannelFactory, "alertChannelFactory is null"); + + if (alertChannelFactoryMap.putIfAbsent(alertChannelFactory.getName(), alertChannelFactory) != null) { + throw new IllegalArgumentException(format("Alert Plugin '{}' is already registered", alertChannelFactory.getName())); + } + + try { + loadAlertChannel(alertChannelFactory.getName()); + } catch (Exception e) { + throw new IllegalArgumentException(format("Alert Plugin '{}' is can not load .", alertChannelFactory.getName())); + } + } + + protected void loadAlertChannel(String name) { + requireNonNull(name, "name is null"); + + AlertChannelFactory alertChannelFactory = alertChannelFactoryMap.get(name); + checkState(alertChannelFactory != null, "Alert Plugin {} is not registered", name); + + try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(alertChannelFactory.getClass().getClassLoader())) { + AlertChannel alertChannel = alertChannelFactory.create(); + this.alertChannelMap.put(name, alertChannel); + } + + logger.info("-- Loaded Alert Plugin {} --", name); + } + + public Map getAlertChannelFactoryMap() { + return alertChannelFactoryMap; + } + + public Map getAlertChannelMap() { + return alertChannelMap; + } + + @Override + public void installPlugin(DolphinSchedulerPlugin dolphinSchedulerPlugin) { + for (AlertChannelFactory alertChannelFactory : dolphinSchedulerPlugin.getAlertChannelFactorys()) { + logger.info("Registering Alert Plugin '{}'", alertChannelFactory.getName()); + this.addAlertChannelFactory(alertChannelFactory); + List params = alertChannelFactory.getParams(); + String nameEn = alertChannelFactory.getName(); + String paramsJson = PluginParamsTransfer.transferParamsToJson(params); + + PluginDefine pluginDefine = new PluginDefine(nameEn, PluginType.ALERT.getDesc(), paramsJson); + pluginDao.addOrUpdatePluginDefine(pluginDefine); + } + } +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginClassLoader.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginClassLoader.java new file mode 100644 index 0000000000..81dc4f3489 --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginClassLoader.java @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import static java.util.Objects.requireNonNull; + +import java.io.IOException; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.net.URL; +import java.net.URLClassLoader; +import java.util.Enumeration; +import java.util.List; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Iterables; + +class DolphinPluginClassLoader + extends URLClassLoader { + private static final ClassLoader PLATFORM_CLASS_LOADER = findPlatformClassLoader(); + + private final ClassLoader spiClassLoader; + private final List spiPackages; + private final List spiResources; + + public DolphinPluginClassLoader( + List urls, + ClassLoader spiClassLoader, + Iterable spiPackages) { + this(urls, + spiClassLoader, + spiPackages, + Iterables.transform(spiPackages, DolphinPluginClassLoader::classNameToResource)); + } + + private DolphinPluginClassLoader( + List urls, + ClassLoader spiClassLoader, + Iterable spiPackages, + Iterable spiResources) { + // plugins should not have access to the system (application) class loader + super(urls.toArray(new URL[urls.size()]), PLATFORM_CLASS_LOADER); + this.spiClassLoader = requireNonNull(spiClassLoader, "spiClassLoader is null"); + this.spiPackages = ImmutableList.copyOf(spiPackages); + this.spiResources = ImmutableList.copyOf(spiResources); + } + + @Override + protected Class loadClass(String name, boolean resolve) + throws ClassNotFoundException { + // grab the magic lock + synchronized (getClassLoadingLock(name)) { + // Check if class is in the loaded classes cache + Class cachedClass = findLoadedClass(name); + if (cachedClass != null) { + return resolveClass(cachedClass, resolve); + } + + // If this is an SPI class, only check SPI class loader + if (isSpiClass(name)) { + return resolveClass(spiClassLoader.loadClass(name), resolve); + } + + // Look for class locally + return super.loadClass(name, resolve); + } + } + + private Class resolveClass(Class clazz, boolean resolve) { + if (resolve) { + resolveClass(clazz); + } + return clazz; + } + + @Override + public URL getResource(String name) { + // If this is an SPI resource, only check SPI class loader + if (isSpiResource(name)) { + return spiClassLoader.getResource(name); + } + + // Look for resource locally + return super.getResource(name); + } + + @Override + public Enumeration getResources(String name) + throws IOException { + // If this is an SPI resource, use SPI resources + if (isSpiClass(name)) { + return spiClassLoader.getResources(name); + } + + // Use local resources + return super.getResources(name); + } + + private boolean isSpiClass(String name) { + return spiPackages.stream().anyMatch(name::startsWith); + } + + private boolean isSpiResource(String name) { + return spiResources.stream().anyMatch(name::startsWith); + } + + private static String classNameToResource(String className) { + return className.replace('.', '/'); + } + + @SuppressWarnings("JavaReflectionMemberAccess") + private static ClassLoader findPlatformClassLoader() { + try { + // use platform class loader on Java 9 + Method method = ClassLoader.class.getMethod("getPlatformClassLoader"); + return (ClassLoader) method.invoke(null); + } catch (NoSuchMethodException ignored) { + // use null class loader on Java 8 + return null; + } catch (IllegalAccessException | InvocationTargetException e) { + throw new AssertionError(e); + } + } +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginDiscovery.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginDiscovery.java new file mode 100644 index 0000000000..7e5cb544ff --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginDiscovery.java @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import static java.nio.charset.StandardCharsets.UTF_8; +import static java.nio.file.Files.createDirectories; +import static java.nio.file.Files.walkFileTree; + +import static com.google.common.io.ByteStreams.toByteArray; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStreamWriter; +import java.io.UncheckedIOException; +import java.io.Writer; +import java.nio.file.FileVisitResult; +import java.nio.file.Path; +import java.nio.file.SimpleFileVisitor; +import java.nio.file.attribute.BasicFileAttributes; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; + +import org.objectweb.asm.ClassReader; +import org.sonatype.aether.artifact.Artifact; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; + +/** + * The role of this class is to load the plugin class during development + */ +final class DolphinPluginDiscovery { + private static final String JAVA_CLASS_FILE_SUFFIX = ".class"; + private static final String PLUGIN_SERVICES_FILE = "META-INF/services/" + DolphinSchedulerPlugin.class.getName(); + + private DolphinPluginDiscovery() { + } + + public static Set discoverPluginsFromArtifact(Artifact artifact, ClassLoader classLoader) + throws IOException { + if (!artifact.getExtension().equals("dolphinscheduler-plugin")) { + throw new RuntimeException("Unexpected extension for main artifact: " + artifact); + } + + File file = artifact.getFile(); + if (!file.getPath().endsWith("/target/classes")) { + throw new RuntimeException("Unexpected file for main artifact: " + file); + } + if (!file.isDirectory()) { + throw new RuntimeException("Main artifact file is not a directory: " + file); + } + + if (new File(file, PLUGIN_SERVICES_FILE).exists()) { + return ImmutableSet.of(); + } + + return listClasses(file.toPath()).stream() + .filter(name -> classInterfaces(name, classLoader).contains(DolphinSchedulerPlugin.class.getName())) + .collect(Collectors.toSet()); + } + + public static void writePluginServices(Iterable plugins, File root) + throws IOException { + Path path = root.toPath().resolve(PLUGIN_SERVICES_FILE); + createDirectories(path.getParent()); + try (Writer out = new OutputStreamWriter(new FileOutputStream(path.toFile()), UTF_8)) { + for (String plugin : plugins) { + out.write(plugin + "\n"); + } + } + } + + private static List listClasses(Path base) + throws IOException { + ImmutableList.Builder list = ImmutableList.builder(); + walkFileTree(base, new SimpleFileVisitor() { + @Override + public FileVisitResult visitFile(Path file, BasicFileAttributes attributes) { + if (file.getFileName().toString().endsWith(JAVA_CLASS_FILE_SUFFIX)) { + String name = file.subpath(base.getNameCount(), file.getNameCount()).toString(); + list.add(javaName(name.substring(0, name.length() - JAVA_CLASS_FILE_SUFFIX.length()))); + } + return FileVisitResult.CONTINUE; + } + }); + return list.build(); + } + + private static List classInterfaces(String name, ClassLoader classLoader) { + ImmutableList.Builder list = ImmutableList.builder(); + ClassReader reader = readClass(name, classLoader); + for (String binaryName : reader.getInterfaces()) { + list.add(javaName(binaryName)); + } + if (reader.getSuperName() != null) { + list.addAll(classInterfaces(javaName(reader.getSuperName()), classLoader)); + } + return list.build(); + } + + private static ClassReader readClass(String name, ClassLoader classLoader) { + try (InputStream in = classLoader.getResourceAsStream(binaryName(name) + JAVA_CLASS_FILE_SUFFIX)) { + if (in == null) { + throw new RuntimeException("Failed to read class: " + name); + } + return new ClassReader(toByteArray(in)); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + } + + private static String binaryName(String javaName) { + return javaName.replace('.', '/'); + } + + private static String javaName(String binaryName) { + return binaryName.replace('/', '.'); + } +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoader.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoader.java new file mode 100644 index 0000000000..7dffe1def8 --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoader.java @@ -0,0 +1,194 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import static java.lang.String.format; +import static java.util.Objects.requireNonNull; + +import static com.google.common.base.Preconditions.checkState; + +import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin; +import org.apache.dolphinscheduler.spi.classloader.ThreadContextClassLoader; + +import java.io.File; +import java.io.IOException; +import java.net.URL; +import java.net.URLClassLoader; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; +import java.util.ServiceLoader; +import java.util.Set; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.sonatype.aether.artifact.Artifact; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Ordering; + +import io.airlift.resolver.ArtifactResolver; + +/** + * Plugin Loader + * Load Plugin from pom when development and run server in IDE + * Load Plugin from the plugin directory when running on the server + */ +public class DolphinPluginLoader { + private static final Logger logger = LoggerFactory.getLogger(DolphinPluginLoader.class); + + /** + * All third-party jar packages used in the classes which in spi package need to be add + */ + private static final ImmutableList DOLPHIN_SPI_PACKAGES = ImmutableList.builder() + .add("org.apache.dolphinscheduler.spi.") + .add("com.fasterxml.jackson.") + .build(); + + private final File installedPluginsDir; + private final List configPlugins; + private ArtifactResolver resolver = null; + private final List dolphinPluginManagerList; + + public DolphinPluginLoader(DolphinPluginManagerConfig config, List dolphinPluginManagerList) { + installedPluginsDir = config.getInstalledPluginsDir(); + if (config.getPlugins() == null) { + this.configPlugins = ImmutableList.of(); + } else { + this.configPlugins = ImmutableList.copyOf(config.getPlugins()); + } + + this.dolphinPluginManagerList = requireNonNull(dolphinPluginManagerList, "dolphinPluginManagerList is null"); + if (configPlugins != null && configPlugins.size() > 0) { + this.resolver = new ArtifactResolver(config.getMavenLocalRepository(), config.getMavenRemoteRepository()); + } + } + + public void loadPlugins() + throws Exception { + for (File file : listPluginDirs(installedPluginsDir)) { + if (file.isDirectory()) { + loadPlugin(file.getAbsolutePath()); + } + } + + for (String plugin : configPlugins) { + loadPlugin(plugin); + } + } + + private void loadPlugin(String plugin) + throws Exception { + logger.info("-- Loading Alert plugin {} --", plugin); + URLClassLoader pluginClassLoader = buildPluginClassLoader(plugin); + try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(pluginClassLoader)) { + loadPlugin(pluginClassLoader); + } + logger.info("-- Finished loading Alert plugin {} --", plugin); + } + + private void loadPlugin(URLClassLoader pluginClassLoader) { + ServiceLoader serviceLoader = ServiceLoader.load(DolphinSchedulerPlugin.class, pluginClassLoader); + List plugins = ImmutableList.copyOf(serviceLoader); + checkState(!plugins.isEmpty(), "No service providers the plugin {}", DolphinSchedulerPlugin.class.getName()); + for (DolphinSchedulerPlugin plugin : plugins) { + logger.info("Installing {}", plugin.getClass().getName()); + for (AbstractDolphinPluginManager dolphinPluginManager : dolphinPluginManagerList) { + dolphinPluginManager.installPlugin(plugin); + } + } + } + + private URLClassLoader buildPluginClassLoader(String plugin) + throws Exception { + File file = new File(plugin); + + if (!file.isDirectory() && (file.getName().equals("pom.xml") || file.getName().endsWith(".pom"))) { + return buildPluginClassLoaderFromPom(file); + } + if (file.isDirectory()) { + return buildPluginClassLoaderFromDirectory(file); + } else { + throw new IllegalArgumentException(format("plugin must be a pom file or directory {} .", plugin)); + } + } + + private URLClassLoader buildPluginClassLoaderFromPom(File pomFile) + throws Exception { + List artifacts = resolver.resolvePom(pomFile); + URLClassLoader classLoader = createClassLoader(artifacts, pomFile.getPath()); + + Artifact artifact = artifacts.get(0); + Set plugins = DolphinPluginDiscovery.discoverPluginsFromArtifact(artifact, classLoader); + if (!plugins.isEmpty()) { + DolphinPluginDiscovery.writePluginServices(plugins, artifact.getFile()); + } + + return classLoader; + } + + private URLClassLoader buildPluginClassLoaderFromDirectory(File dir) + throws Exception { + logger.info("Classpath for {}:", dir.getName()); + List urls = new ArrayList<>(); + for (File file : listPluginDirs(dir)) { + logger.info(" {}", file); + urls.add(file.toURI().toURL()); + } + return createClassLoader(urls); + } + + private URLClassLoader createClassLoader(List artifacts, String name) + throws IOException { + logger.info("Classpath for {}:", name); + List urls = new ArrayList<>(); + for (Artifact artifact : sortArtifacts(artifacts)) { + if (artifact.getFile() == null) { + throw new RuntimeException("Could not resolve artifact: " + artifact); + } + File file = artifact.getFile().getCanonicalFile(); + logger.info(" {}", file); + urls.add(file.toURI().toURL()); + } + return createClassLoader(urls); + } + + private URLClassLoader createClassLoader(List urls) { + ClassLoader parent = getClass().getClassLoader(); + return new DolphinPluginClassLoader(urls, parent, DOLPHIN_SPI_PACKAGES); + } + + private static List listPluginDirs(File installedPluginsDir) { + if (installedPluginsDir != null && installedPluginsDir.isDirectory()) { + File[] files = installedPluginsDir.listFiles(); + if (files != null) { + Arrays.sort(files); + return ImmutableList.copyOf(files); + } + } + return ImmutableList.of(); + } + + private static List sortArtifacts(List artifacts) { + List list = new ArrayList<>(artifacts); + Collections.sort(list, Ordering.natural().nullsLast().onResultOf(Artifact::getFile)); + return list; + } + +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginManagerConfig.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginManagerConfig.java new file mode 100644 index 0000000000..195484db46 --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginManagerConfig.java @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import static java.lang.String.format; +import static java.util.Objects.requireNonNull; + +import java.io.File; +import java.util.List; + +import com.google.common.base.Splitter; +import com.google.common.collect.ImmutableList; + +/** + * Dolphin Scheduler Plugin Manager Config + */ +public class DolphinPluginManagerConfig { + + /** + * The dir of the Alert Plugin in. + * When AlertServer is running on the server, it will load the Alert Plugin from this directory. + */ + private File installedPluginsDir; + + /** + * The plugin should be load. + * The installedPluginsDir is empty when we development and run server in IDEA. Then we can config which plugin should be load by param name alert.plugin.binding in the alert.properties file + */ + private List plugins; + + /** + * Development, When AlertServer is running on IDE, AlertPluginLoad can load Alert Plugin from local Repository. + */ + private String mavenLocalRepository = System.getProperty("user.home") + "/.m2/repository"; + private List mavenRemoteRepository = ImmutableList.of("http://repo1.maven.org/maven2/"); + + public File getInstalledPluginsDir() { + return installedPluginsDir; + } + + /** + * @param pluginDir + */ + public DolphinPluginManagerConfig setInstalledPluginsDir(String pluginDir) { + requireNonNull(pluginDir, "pluginDir can not be null"); + File pluginDirFile = new File(pluginDir); + if (!pluginDirFile.exists()) { + throw new IllegalArgumentException(format("plugin dir not exists ! {}", pluginDirFile.getPath())); + } + this.installedPluginsDir = pluginDirFile; + return this; + } + + public List getPlugins() { + return plugins; + } + + public DolphinPluginManagerConfig setPlugins(List plugins) { + this.plugins = plugins; + return this; + } + + /** + * When development and run server in IDE, this method can set plugins in alert.properties . + * Then when you start AlertServer in IDE, the plugin can be load. + * eg: + * file: alert.properties + * alert.plugin=\ + * ../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml, \ + * ../dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml + * + * @param plugins + * @return + */ + public DolphinPluginManagerConfig setPlugins(String plugins) { + if (plugins == null) { + this.plugins = null; + } else { + this.plugins = ImmutableList.copyOf(Splitter.on(',').omitEmptyStrings().trimResults().split(plugins)); + } + return this; + } + + public String getMavenLocalRepository() { + return mavenLocalRepository; + } + + public DolphinPluginManagerConfig setMavenLocalRepository(String mavenLocalRepository) { + this.mavenLocalRepository = mavenLocalRepository; + return this; + } + + public List getMavenRemoteRepository() { + return mavenRemoteRepository; + } + + public DolphinPluginManagerConfig setMavenRemoteRepository(List mavenRemoteRepository) { + this.mavenRemoteRepository = mavenRemoteRepository; + return this; + } + + public DolphinPluginManagerConfig setMavenRemoteRepository(String mavenRemoteRepository) { + this.mavenRemoteRepository = ImmutableList.copyOf(Splitter.on(',').omitEmptyStrings().trimResults().split(mavenRemoteRepository)); + return this; + } +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPlugin.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPlugin.java deleted file mode 100644 index 5bbc21930f..0000000000 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPlugin.java +++ /dev/null @@ -1,147 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.dolphinscheduler.alert.plugin; - -import org.apache.dolphinscheduler.alert.manager.DingTalkManager; -import org.apache.dolphinscheduler.alert.manager.EmailManager; -import org.apache.dolphinscheduler.alert.manager.EnterpriseWeChatManager; -import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.alert.utils.DingTalkUtils; -import org.apache.dolphinscheduler.alert.utils.EnterpriseWeChatUtils; -import org.apache.dolphinscheduler.common.utils.CollectionUtils; -import org.apache.dolphinscheduler.common.utils.StringUtils; -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; -import org.apache.dolphinscheduler.plugin.model.AlertData; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; -import org.apache.dolphinscheduler.plugin.model.PluginName; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * EmailAlertPlugin - *

- * This plugin is a default plugin, and mix up email and enterprise wechat, because adapt with former alert behavior - */ -public class EmailAlertPlugin implements AlertPlugin { - - private static final Logger logger = LoggerFactory.getLogger(EmailAlertPlugin.class); - - private PluginName pluginName; - - private static final EmailManager emailManager = new EmailManager(); - private static final EnterpriseWeChatManager weChatManager = new EnterpriseWeChatManager(); - private static final DingTalkManager dingTalkManager = new DingTalkManager(); - - public EmailAlertPlugin() { - this.pluginName = new PluginName(); - this.pluginName.setEnglish(Constants.PLUGIN_DEFAULT_EMAIL_EN); - this.pluginName.setChinese(Constants.PLUGIN_DEFAULT_EMAIL_CH); - } - - @Override - public String getId() { - return Constants.PLUGIN_DEFAULT_EMAIL_ID; - } - - @Override - public PluginName getName() { - return pluginName; - } - - @Override - @SuppressWarnings("unchecked") - public Map process(AlertInfo info) { - Map retMaps = new HashMap<>(); - - AlertData alert = info.getAlertData(); - - List receiversList = (List) info.getProp(Constants.PLUGIN_DEFAULT_EMAIL_RECEIVERS); - - // receiving group list - // custom receiver - String receivers = alert.getReceivers(); - if (StringUtils.isNotEmpty(receivers)) { - String[] splits = receivers.split(","); - receiversList.addAll(Arrays.asList(splits)); - } - - List receiversCcList = new ArrayList<>(); - // Custom Copier - String receiversCc = alert.getReceiversCc(); - if (StringUtils.isNotEmpty(receiversCc)) { - String[] splits = receiversCc.split(","); - receiversCcList.addAll(Arrays.asList(splits)); - } - - if (CollectionUtils.isEmpty(receiversList) && CollectionUtils.isEmpty(receiversCcList)) { - logger.warn("alert send error : At least one receiver address required"); - retMaps.put(Constants.STATUS, "false"); - retMaps.put(Constants.MESSAGE, "execution failure,At least one receiver address required."); - return retMaps; - } - - retMaps = emailManager.send(receiversList, receiversCcList, alert.getTitle(), alert.getContent(), - alert.getShowType()); - - //send flag - boolean flag = false; - - if (retMaps == null) { - retMaps = new HashMap<>(); - retMaps.put(Constants.MESSAGE, "alert send error."); - retMaps.put(Constants.STATUS, "false"); - logger.info("alert send error : {}", retMaps.get(Constants.MESSAGE)); - return retMaps; - } - - flag = Boolean.parseBoolean(String.valueOf(retMaps.get(Constants.STATUS))); - - if (flag) { - logger.info("alert send success"); - retMaps.put(Constants.MESSAGE, "email send success."); - if (EnterpriseWeChatUtils.isEnable()) { - logger.info("Enterprise WeChat is enable!"); - try { - String token = EnterpriseWeChatUtils.getToken(); - weChatManager.send(info, token); - } catch (Exception e) { - logger.error(e.getMessage(), e); - } - } - - if (DingTalkUtils.IS_ENABLE_DING_TALK) { - logger.info("Ding Talk is enable."); - dingTalkManager.send(info); - } - - } else { - retMaps.put(Constants.MESSAGE, "alert send error."); - logger.info("alert send error : {}", retMaps.get(Constants.MESSAGE)); - } - - return retMaps; - } - -} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessor.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessor.java new file mode 100644 index 0000000000..5e8a8f89d6 --- /dev/null +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessor.java @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.processor; + +import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager; +import org.apache.dolphinscheduler.alert.runner.AlertSender; +import org.apache.dolphinscheduler.common.utils.Preconditions; +import org.apache.dolphinscheduler.dao.AlertDao; +import org.apache.dolphinscheduler.dao.PluginDao; +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.CommandType; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendRequestCommand; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseCommand; +import org.apache.dolphinscheduler.remote.processor.NettyRequestProcessor; +import org.apache.dolphinscheduler.remote.utils.JsonSerializer; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.netty.channel.Channel; + +/** + * alert request processor + */ +public class AlertRequestProcessor implements NettyRequestProcessor { + + private final Logger logger = LoggerFactory.getLogger(AlertRequestProcessor.class); + private AlertDao alertDao; + private PluginDao pluginDao; + private AlertPluginManager alertPluginManager; + + public AlertRequestProcessor(AlertDao alertDao, AlertPluginManager alertPluginManager, PluginDao pluginDao) { + this.alertDao = alertDao; + this.pluginDao = pluginDao; + this.alertPluginManager = alertPluginManager; + } + + @Override + public void process(Channel channel, Command command) { + Preconditions.checkArgument(CommandType.ALERT_SEND_REQUEST == command.getType(), + String.format("invalid command type : %s", command.getType())); + + AlertSendRequestCommand alertSendRequestCommand = JsonSerializer.deserialize( + command.getBody(), AlertSendRequestCommand.class); + logger.info("received command : {}", alertSendRequestCommand); + + AlertSender alertSender = new AlertSender(alertDao, alertPluginManager, pluginDao); + AlertSendResponseCommand alertSendResponseCommand = alertSender.syncHandler(alertSendRequestCommand.getGroupId(), alertSendRequestCommand.getTitle(), alertSendRequestCommand.getContent()); + channel.writeAndFlush(alertSendResponseCommand.convert2Command(command.getOpaque())); + + } +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/runner/AlertSender.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/runner/AlertSender.java index 1bae9c7724..d635574543 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/runner/AlertSender.java +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/runner/AlertSender.java @@ -14,23 +14,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.alert.runner; -import org.apache.dolphinscheduler.alert.utils.Constants; +import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager; import org.apache.dolphinscheduler.common.enums.AlertStatus; -import org.apache.dolphinscheduler.common.plugin.PluginManager; +import org.apache.dolphinscheduler.common.utils.CollectionUtils; import org.apache.dolphinscheduler.dao.AlertDao; +import org.apache.dolphinscheduler.dao.PluginDao; import org.apache.dolphinscheduler.dao.entity.Alert; -import org.apache.dolphinscheduler.dao.entity.User; -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; -import org.apache.dolphinscheduler.plugin.model.AlertData; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseCommand; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseResult; +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertData; +import org.apache.dolphinscheduler.spi.alert.AlertInfo; +import org.apache.dolphinscheduler.spi.alert.AlertResult; import java.util.ArrayList; import java.util.List; -import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * alert sender @@ -41,60 +46,135 @@ public class AlertSender { private List alertList; private AlertDao alertDao; - private PluginManager pluginManager; + private PluginDao pluginDao; + private AlertPluginManager alertPluginManager; - public AlertSender() { + public AlertSender(AlertPluginManager alertPluginManager) { + this.alertPluginManager = alertPluginManager; } - public AlertSender(List alertList, AlertDao alertDao, PluginManager pluginManager) { + public AlertSender(AlertDao alertDao, AlertPluginManager alertPluginManager, PluginDao pluginDao) { + super(); + this.alertDao = alertDao; + this.pluginDao = pluginDao; + this.alertPluginManager = alertPluginManager; + } + + public AlertSender(List alertList, AlertDao alertDao, AlertPluginManager alertPluginManager, PluginDao pluginDao) { super(); this.alertList = alertList; this.alertDao = alertDao; - this.pluginManager = pluginManager; + this.pluginDao = pluginDao; + this.alertPluginManager = alertPluginManager; } public void run() { - List users; - Map retMaps = null; for (Alert alert : alertList) { - users = alertDao.listUserByAlertgroupId(alert.getAlertGroupId()); - - // receiving group list - List receiversList = new ArrayList<>(); - for (User user : users) { - receiversList.add(user.getEmail()); + //get alert group from alert + int alertGroupId = alert.getAlertGroupId(); + List alertInstanceList = alertDao.listInstanceByAlertGroupId(alertGroupId); + if (CollectionUtils.isEmpty(alertInstanceList)) { + logger.error("send alert msg fail,no bind plugin instance."); + return; } - AlertData alertData = new AlertData(); alertData.setId(alert.getId()) - .setAlertGroupId(alert.getAlertGroupId()) - .setContent(alert.getContent()) - .setLog(alert.getLog()) - .setReceivers(alert.getReceivers()) - .setReceiversCc(alert.getReceiversCc()) - .setShowType(alert.getShowType().getDescp()) - .setTitle(alert.getTitle()); - - AlertInfo alertInfo = new AlertInfo(); - alertInfo.setAlertData(alertData); - - alertInfo.addProp("receivers", receiversList); - - AlertPlugin emailPlugin = pluginManager.findOne(Constants.PLUGIN_DEFAULT_EMAIL_ID); - retMaps = emailPlugin.process(alertInfo); - - if (retMaps == null) { - alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, "alert send error", alert.getId()); - logger.error("alert send error : return value is null"); - } else if (!Boolean.parseBoolean(String.valueOf(retMaps.get(Constants.STATUS)))) { - alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, String.valueOf(retMaps.get(Constants.MESSAGE)), alert.getId()); - logger.error("alert send error : {}", retMaps.get(Constants.MESSAGE)); - } else { - alertDao.updateAlert(AlertStatus.EXECUTION_SUCCESS, (String) retMaps.get(Constants.MESSAGE), alert.getId()); - logger.info("alert send success"); + .setContent(alert.getContent()) + .setLog(alert.getLog()) + .setTitle(alert.getTitle()); + + for (AlertPluginInstance instance : alertInstanceList) { + + AlertResult alertResult = this.alertResultHandler(instance, alertData); + AlertStatus alertStatus = Boolean.parseBoolean(String.valueOf(alertResult.getStatus())) ? AlertStatus.EXECUTION_SUCCESS : AlertStatus.EXECUTION_FAILURE; + alertDao.updateAlert(alertStatus, alertResult.getMessage(), alert.getId()); + } } } + /** + * sync send alert handler + * + * @param alertGroupId alertGroupId + * @param title title + * @param content content + * @return AlertSendResponseCommand + */ + public AlertSendResponseCommand syncHandler(int alertGroupId, String title, String content) { + + List alertInstanceList = alertDao.listInstanceByAlertGroupId(alertGroupId); + AlertData alertData = new AlertData(); + alertData.setContent(title) + .setTitle(content); + + boolean sendResponseStatus = true; + List sendResponseResults = new ArrayList<>(); + + if (CollectionUtils.isEmpty(alertInstanceList)) { + sendResponseStatus = false; + AlertSendResponseResult alertSendResponseResult = new AlertSendResponseResult(); + String message = String.format("Alert GroupId %s send error : not found alert instance", alertGroupId); + alertSendResponseResult.setStatus(sendResponseStatus); + alertSendResponseResult.setMessage(message); + sendResponseResults.add(alertSendResponseResult); + logger.error("Alert GroupId {} send error : not found alert instance", alertGroupId); + return new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + } + + for (AlertPluginInstance instance : alertInstanceList) { + AlertResult alertResult = this.alertResultHandler(instance, alertData); + AlertSendResponseResult alertSendResponseResult = new AlertSendResponseResult( + Boolean.parseBoolean(String.valueOf(alertResult.getStatus())), alertResult.getMessage()); + sendResponseStatus = sendResponseStatus && alertSendResponseResult.getStatus(); + sendResponseResults.add(alertSendResponseResult); + } + + return new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + } + + /** + * alert result handler + * + * @param instance instance + * @param alertData alertData + * @return AlertResult + */ + private AlertResult alertResultHandler(AlertPluginInstance instance, AlertData alertData) { + String pluginName = pluginDao.getPluginDefineById(instance.getPluginDefineId()).getPluginName(); + AlertChannel alertChannel = alertPluginManager.getAlertChannelMap().get(pluginName); + AlertResult alertResultExtend = new AlertResult(); + String pluginInstanceName = instance.getInstanceName(); + if (alertChannel == null) { + String message = String.format("Alert Plugin %s send error : return value is null", pluginInstanceName); + alertResultExtend.setStatus(String.valueOf(false)); + alertResultExtend.setMessage(message); + logger.error("Alert Plugin {} send error : not found plugin {}", pluginInstanceName, pluginName); + return alertResultExtend; + } + + AlertInfo alertInfo = new AlertInfo(); + alertInfo.setAlertData(alertData); + alertInfo.setAlertParams(instance.getPluginInstanceParams()); + AlertResult alertResult = alertChannel.process(alertInfo); + + if (alertResult == null) { + String message = String.format("Alert Plugin %s send error : return alertResult value is null", pluginInstanceName); + alertResultExtend.setStatus(String.valueOf(false)); + alertResultExtend.setMessage(message); + logger.info("Alert Plugin {} send error : return alertResult value is null", pluginInstanceName); + } else if (!Boolean.parseBoolean(String.valueOf(alertResult.getStatus()))) { + alertResultExtend.setStatus(String.valueOf(false)); + alertResultExtend.setMessage(alertResult.getMessage()); + logger.info("Alert Plugin {} send error : {}", pluginInstanceName, alertResult.getMessage()); + } else { + String message = String.format("Alert Plugin %s send success", pluginInstanceName); + alertResultExtend.setStatus(String.valueOf(true)); + alertResultExtend.setMessage(message); + logger.info("Alert Plugin {} send success", pluginInstanceName); + } + return alertResultExtend; + } + } diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/Constants.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/Constants.java index c0f916a809..8041fa565f 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/Constants.java +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/Constants.java @@ -30,177 +30,9 @@ public class Constants { */ public static final String ALERT_PROPERTIES_PATH = "/alert.properties"; - public static final String DATA_SOURCE_PROPERTIES_PATH = "/dao/data_source.properties"; - - public static final String SINGLE_SLASH = "/"; - - /** - * UTF-8 - */ - public static final String UTF_8 = "UTF-8"; - - public static final String STATUS = "status"; - - public static final String MESSAGE = "message"; - - public static final String MAIL_PROTOCOL = "mail.protocol"; - - public static final String MAIL_SERVER_HOST = "mail.server.host"; - - public static final String MAIL_SERVER_PORT = "mail.server.port"; - - public static final String MAIL_SENDER = "mail.sender"; - - public static final String MAIL_USER = "mail.user"; - - public static final String MAIL_PASSWD = "mail.passwd"; - - public static final String XLS_FILE_PATH = "xls.file.path"; - - public static final String MAIL_HOST = "mail.smtp.host"; - - public static final String MAIL_PORT = "mail.smtp.port"; - - public static final String MAIL_SMTP_AUTH = "mail.smtp.auth"; - - public static final String MAIL_TRANSPORT_PROTOCOL = "mail.transport.protocol"; - - public static final String MAIL_SMTP_STARTTLS_ENABLE = "mail.smtp.starttls.enable"; - - public static final String MAIL_SMTP_SSL_ENABLE = "mail.smtp.ssl.enable"; - - public static final String MAIL_SMTP_SSL_TRUST = "mail.smtp.ssl.trust"; - - public static final String TEXT_HTML_CHARSET_UTF_8 = "text/html;charset=utf-8"; - - public static final String STRING_TRUE = "true"; - - public static final String EXCEL_SUFFIX_XLS = ".xls"; - - public static final int NUMBER_1000 = 1000; - - public static final String SPRING_DATASOURCE_DRIVER_CLASS_NAME = "spring.datasource.driver-class-name"; - - public static final String SPRING_DATASOURCE_URL = "spring.datasource.url"; - - public static final String SPRING_DATASOURCE_USERNAME = "spring.datasource.username"; - - public static final String SPRING_DATASOURCE_PASSWORD = "spring.datasource.password"; - - public static final String SPRING_DATASOURCE_VALIDATION_QUERY_TIMEOUT = "spring.datasource.validationQueryTimeout"; - - public static final String SPRING_DATASOURCE_INITIAL_SIZE = "spring.datasource.initialSize"; - - public static final String SPRING_DATASOURCE_MIN_IDLE = "spring.datasource.minIdle"; - - public static final String SPRING_DATASOURCE_MAX_ACTIVE = "spring.datasource.maxActive"; - - public static final String SPRING_DATASOURCE_MAX_WAIT = "spring.datasource.maxWait"; - - public static final String SPRING_DATASOURCE_TIME_BETWEEN_EVICTION_RUNS_MILLIS = "spring.datasource.timeBetweenEvictionRunsMillis"; - - public static final String SPRING_DATASOURCE_MIN_EVICTABLE_IDLE_TIME_MILLIS = "spring.datasource.minEvictableIdleTimeMillis"; - - public static final String SPRING_DATASOURCE_VALIDATION_QUERY = "spring.datasource.validationQuery"; - - public static final String SPRING_DATASOURCE_TEST_WHILE_IDLE = "spring.datasource.testWhileIdle"; - - public static final String SPRING_DATASOURCE_TEST_ON_BORROW = "spring.datasource.testOnBorrow"; - - public static final String SPRING_DATASOURCE_TEST_ON_RETURN = "spring.datasource.testOnReturn"; - - public static final String SPRING_DATASOURCE_POOL_PREPARED_STATEMENTS = "spring.datasource.poolPreparedStatements"; - - public static final String SPRING_DATASOURCE_DEFAULT_AUTO_COMMIT = "spring.datasource.defaultAutoCommit"; - - public static final String SPRING_DATASOURCE_KEEP_ALIVE = "spring.datasource.keepAlive"; - - public static final String SPRING_DATASOURCE_MAX_POOL_PREPARED_STATEMENT_PER_CONNECTION_SIZE = "spring.datasource.maxPoolPreparedStatementPerConnectionSize"; - - public static final String DEVELOPMENT = "development"; - - public static final String TR = ""; - - public static final String TD = ""; - - public static final String TD_END = ""; - - public static final String TR_END = ""; - - public static final String TITLE = "title"; - - public static final String CONTENT = "content"; - - public static final String TH = ""; - - public static final String TH_END = ""; + /** default alert plugin dir **/ + public static final String ALERT_PLUGIN_PATH = "./lib/plugin/alert"; public static final int ALERT_SCAN_INTERVAL = 5000; - public static final String MARKDOWN_QUOTE = ">"; - - public static final String MARKDOWN_ENTER = "\n"; - - public static final String ENTERPRISE_WECHAT_ENABLE = "enterprise.wechat.enable"; - - public static final String ENTERPRISE_WECHAT_CORP_ID = "enterprise.wechat.corp.id"; - - public static final String ENTERPRISE_WECHAT_SECRET = "enterprise.wechat.secret"; - - public static final String ENTERPRISE_WECHAT_TOKEN_URL = "enterprise.wechat.token.url"; - - public static final String ENTERPRISE_WECHAT_PUSH_URL = "enterprise.wechat.push.url"; - - public static final String ENTERPRISE_WECHAT_TEAM_SEND_MSG = "enterprise.wechat.team.send.msg"; - - public static final String ENTERPRISE_WECHAT_USER_SEND_MSG = "enterprise.wechat.user.send.msg"; - - public static final String ENTERPRISE_WECHAT_AGENT_ID = "enterprise.wechat.agent.id"; - - public static final String ENTERPRISE_WECHAT_USERS = "enterprise.wechat.users"; - - - public static final String DINGTALK_WEBHOOK = "dingtalk.webhook"; - - public static final String DINGTALK_KEYWORD = "dingtalk.keyword"; - - public static final String DINGTALK_PROXY_ENABLE = "dingtalk.isEnableProxy"; - - public static final String DINGTALK_PROXY = "dingtalk.proxy"; - - public static final String DINGTALK_PORT = "dingtalk.port"; - - public static final String DINGTALK_USER = "dingtalk.user"; - - public static final String DINGTALK_PASSWORD = "dingtalk.password"; - - public static final String DINGTALK_ENABLE = "dingtalk.isEnable"; - - public static final String HTML_HEADER_PREFIX = "" - + "dolphinscheduler" - + "" - + "" - + "/head> "; - - public static final String TABLE_BODY_HTML_TAIL = "
"; - - /** - * plugin config - */ - public static final String PLUGIN_DIR = "plugin.dir"; - - public static final String PLUGIN_DEFAULT_EMAIL_ID = "email"; - - public static final String PLUGIN_DEFAULT_EMAIL_CH = "邮件"; - - public static final String PLUGIN_DEFAULT_EMAIL_EN = "email"; - - public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERS = "receivers"; - - public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERCCS = "receiverCcs"; - - public static final String RETMAP_MSG = "msg"; - } diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtils.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtils.java deleted file mode 100644 index 50a62e46fd..0000000000 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtils.java +++ /dev/null @@ -1,141 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.dolphinscheduler.alert.utils; - -import org.apache.dolphinscheduler.common.utils.*; - -import org.apache.commons.codec.binary.StringUtils; -import org.apache.http.HttpEntity; -import org.apache.http.HttpHost; -import org.apache.http.auth.AuthScope; -import org.apache.http.auth.UsernamePasswordCredentials; -import org.apache.http.client.CredentialsProvider; -import org.apache.http.client.config.RequestConfig; -import org.apache.http.client.methods.CloseableHttpResponse; -import org.apache.http.client.methods.HttpPost; -import org.apache.http.entity.StringEntity; -import org.apache.http.impl.client.BasicCredentialsProvider; -import org.apache.http.impl.client.CloseableHttpClient; -import org.apache.http.impl.client.HttpClients; -import org.apache.http.util.EntityUtils; - -import java.io.IOException; -import java.util.HashMap; -import java.util.Map; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * DingTalkUtils utils - * support send msg to ding talk by robot message push function. - * support PROXY setting - */ -public class DingTalkUtils { - public static final Logger logger = LoggerFactory.getLogger(DingTalkUtils.class); - - public static final boolean IS_ENABLE_DING_TALK = PropertyUtils.getBoolean(Constants.DINGTALK_ENABLE); - private static final String DING_TASK_URL = PropertyUtils.getString(Constants.DINGTALK_WEBHOOK); - private static final String KEYWORD = PropertyUtils.getString(Constants.DINGTALK_KEYWORD); - private static final Boolean IS_ENABLE_PROXY = PropertyUtils.getBoolean(Constants.DINGTALK_PROXY_ENABLE); - private static final String PROXY = PropertyUtils.getString(Constants.DINGTALK_PROXY); - private static final String USER = PropertyUtils.getString(Constants.DINGTALK_USER); - private static final String PASSWD = PropertyUtils.getString(Constants.DINGTALK_PASSWORD); - private static final Integer PORT = PropertyUtils.getInt(Constants.DINGTALK_PORT); - - private DingTalkUtils() { - throw new IllegalStateException(DingTalkUtils.class.getName()); - } - - /** - * send message interface - * only support text message format now. - * - * @param msg message context to send - * @param charset charset type - * @return result of sending msg - * @throws IOException the IOException - */ - public static String sendDingTalkMsg(String msg, String charset) throws IOException { - String msgToJson = textToJsonString(msg + "#" + KEYWORD); - HttpPost httpPost = constructHttpPost(msgToJson, charset); - - CloseableHttpClient httpClient; - if (Boolean.TRUE.equals(IS_ENABLE_PROXY)) { - httpClient = getProxyClient(); - RequestConfig rcf = getProxyConfig(); - httpPost.setConfig(rcf); - } else { - httpClient = getDefaultClient(); - } - - try { - CloseableHttpResponse response = httpClient.execute(httpPost); - String resp; - try { - HttpEntity entity = response.getEntity(); - resp = EntityUtils.toString(entity, charset); - EntityUtils.consume(entity); - } finally { - response.close(); - } - logger.info("Ding Talk send [{}], resp:{%s}", msg); - return resp; - } finally { - httpClient.close(); - } - } - - public static HttpPost constructHttpPost(String msg, String charset) { - HttpPost post = new HttpPost(DING_TASK_URL); - StringEntity entity = new StringEntity(msg, charset); - post.setEntity(entity); - post.addHeader("Content-Type", "application/json; charset=utf-8"); - return post; - } - - public static CloseableHttpClient getProxyClient() { - HttpHost httpProxy = new HttpHost(PROXY, PORT); - CredentialsProvider provider = new BasicCredentialsProvider(); - provider.setCredentials(new AuthScope(httpProxy), new UsernamePasswordCredentials(USER, PASSWD)); - return HttpClients.custom().setDefaultCredentialsProvider(provider).build(); - } - - public static CloseableHttpClient getDefaultClient() { - return HttpClients.createDefault(); - } - - public static RequestConfig getProxyConfig() { - HttpHost httpProxy = new HttpHost(PROXY, PORT); - return RequestConfig.custom().setProxy(httpProxy).build(); - } - - public static String textToJsonString(String text) { - Map items = new HashMap<>(); - items.put("msgtype", "text"); - Map textContent = new HashMap<>(); - byte[] byt = StringUtils.getBytesUtf8(text); - String txt = StringUtils.newStringUtf8(byt); - textContent.put("content", txt); - items.put("text", textContent); - - return JSONUtils.toJsonString(items); - - } - -} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtils.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtils.java deleted file mode 100644 index aeb6671791..0000000000 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtils.java +++ /dev/null @@ -1,286 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.dolphinscheduler.alert.utils; - -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.apache.dolphinscheduler.common.utils.JSONUtils; -import org.apache.dolphinscheduler.common.utils.StringUtils; -import org.apache.dolphinscheduler.plugin.model.AlertData; - -import org.apache.http.HttpEntity; -import org.apache.http.client.methods.CloseableHttpResponse; -import org.apache.http.client.methods.HttpGet; -import org.apache.http.client.methods.HttpPost; -import org.apache.http.entity.StringEntity; -import org.apache.http.impl.client.CloseableHttpClient; -import org.apache.http.impl.client.HttpClients; -import org.apache.http.util.EntityUtils; - -import java.io.IOException; -import java.util.Collection; -import java.util.Iterator; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.Set; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * Enterprise WeChat utils - */ -public class EnterpriseWeChatUtils { - - public static final Logger logger = LoggerFactory.getLogger(EnterpriseWeChatUtils.class); - public static final String ENTERPRISE_WE_CHAT_AGENT_ID = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_AGENT_ID); - public static final String ENTERPRISE_WE_CHAT_USERS = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_USERS); - private static final String ENTERPRISE_WE_CHAT_CORP_ID = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_CORP_ID); - private static final String ENTERPRISE_WE_CHAT_SECRET = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_SECRET); - private static final String ENTERPRISE_WE_CHAT_TOKEN_URL = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_TOKEN_URL); - private static final String ENTERPRISE_WE_CHAT_TOKEN_URL_REPLACE = ENTERPRISE_WE_CHAT_TOKEN_URL == null ? null : ENTERPRISE_WE_CHAT_TOKEN_URL - .replaceAll("\\{corpId}", ENTERPRISE_WE_CHAT_CORP_ID) - .replaceAll("\\{secret}", ENTERPRISE_WE_CHAT_SECRET); - private static final String ENTERPRISE_WE_CHAT_PUSH_URL = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_PUSH_URL); - private static final String ENTERPRISE_WE_CHAT_TEAM_SEND_MSG = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_TEAM_SEND_MSG); - private static final String ENTERPRISE_WE_CHAT_USER_SEND_MSG = PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_USER_SEND_MSG); - - private static final String AGENT_ID_REG_EXP = "\\{agentId}"; - private static final String MSG_REG_EXP = "\\{msg}"; - private static final String USER_REG_EXP = "\\{toUser}"; - - private EnterpriseWeChatUtils() { - throw new IllegalStateException(EnterpriseWeChatUtils.class.getName()); - } - - /** - * get Enterprise WeChat is enable - * - * @return isEnable - */ - public static boolean isEnable() { - Boolean isEnable = null; - try { - isEnable = PropertyUtils.getBoolean(Constants.ENTERPRISE_WECHAT_ENABLE); - } catch (Exception e) { - logger.error(e.getMessage(), e); - } - if (isEnable == null) { - return false; - } - return isEnable; - } - - /** - * get Enterprise WeChat token info - * - * @return token string info - * @throws IOException the IOException - */ - public static String getToken() throws IOException { - String resp; - - CloseableHttpClient httpClient = HttpClients.createDefault(); - try { - HttpGet httpGet = new HttpGet(ENTERPRISE_WE_CHAT_TOKEN_URL_REPLACE); - CloseableHttpResponse response = httpClient.execute(httpGet); - try { - HttpEntity entity = response.getEntity(); - resp = EntityUtils.toString(entity, Constants.UTF_8); - EntityUtils.consume(entity); - } finally { - response.close(); - } - - Map map = JSONUtils.toMap(resp); - return map == null ? null : map.get("access_token"); - } finally { - httpClient.close(); - } - } - - /** - * make team single Enterprise WeChat message - * - * @param toParty the toParty - * @param agentId the agentId - * @param msg the msg - * @return Enterprise WeChat send message - */ - public static String makeTeamSendMsg(String toParty, String agentId, String msg) { - return ENTERPRISE_WE_CHAT_TEAM_SEND_MSG.replaceAll("\\{toParty}", toParty) - .replaceAll(AGENT_ID_REG_EXP, agentId) - .replaceAll(MSG_REG_EXP, msg); - } - - /** - * make team multi Enterprise WeChat message - * - * @param toParty the toParty - * @param agentId the agentId - * @param msg the msg - * @return Enterprise WeChat send message - */ - public static String makeTeamSendMsg(Collection toParty, String agentId, String msg) { - String listParty = FuncUtils.mkString(toParty, "|"); - return ENTERPRISE_WE_CHAT_TEAM_SEND_MSG.replaceAll("\\{toParty}", listParty) - .replaceAll(AGENT_ID_REG_EXP, agentId) - .replaceAll(MSG_REG_EXP, msg); - } - - /** - * make team single user message - * - * @param toUser the toUser - * @param agentId the agentId - * @param msg the msg - * @return Enterprise WeChat send message - */ - public static String makeUserSendMsg(String toUser, String agentId, String msg) { - return ENTERPRISE_WE_CHAT_USER_SEND_MSG.replaceAll(USER_REG_EXP, toUser) - .replaceAll(AGENT_ID_REG_EXP, agentId) - .replaceAll(MSG_REG_EXP, msg); - } - - /** - * make team multi user message - * - * @param toUser the toUser - * @param agentId the agentId - * @param msg the msg - * @return Enterprise WeChat send message - */ - public static String makeUserSendMsg(Collection toUser, String agentId, String msg) { - String listUser = FuncUtils.mkString(toUser, "|"); - return ENTERPRISE_WE_CHAT_USER_SEND_MSG.replaceAll(USER_REG_EXP, listUser) - .replaceAll(AGENT_ID_REG_EXP, agentId) - .replaceAll(MSG_REG_EXP, msg); - } - - /** - * send Enterprise WeChat - * - * @param charset the charset - * @param data the data - * @param token the token - * @return Enterprise WeChat resp, demo: {"errcode":0,"errmsg":"ok","invaliduser":""} - * @throws IOException the IOException - */ - public static String sendEnterpriseWeChat(String charset, String data, String token) throws IOException { - String enterpriseWeChatPushUrlReplace = ENTERPRISE_WE_CHAT_PUSH_URL.replaceAll("\\{token}", token); - - CloseableHttpClient httpClient = HttpClients.createDefault(); - try { - HttpPost httpPost = new HttpPost(enterpriseWeChatPushUrlReplace); - httpPost.setEntity(new StringEntity(data, charset)); - CloseableHttpResponse response = httpClient.execute(httpPost); - String resp; - try { - HttpEntity entity = response.getEntity(); - resp = EntityUtils.toString(entity, charset); - EntityUtils.consume(entity); - } finally { - response.close(); - } - logger.info("Enterprise WeChat send [{}], param:{}, resp:{}", - ENTERPRISE_WE_CHAT_PUSH_URL, data, resp); - return resp; - } finally { - httpClient.close(); - } - } - - /** - * convert table to markdown style - * - * @param title the title - * @param content the content - * @return markdown table content - */ - public static String markdownTable(String title, String content) { - List mapItemsList = JSONUtils.toList(content, LinkedHashMap.class); - StringBuilder contents = new StringBuilder(200); - - if (null != mapItemsList) { - for (LinkedHashMap mapItems : mapItemsList) { - Set> entries = mapItems.entrySet(); - Iterator> iterator = entries.iterator(); - StringBuilder t = new StringBuilder(String.format("`%s`%s", title, Constants.MARKDOWN_ENTER)); - - while (iterator.hasNext()) { - - Map.Entry entry = iterator.next(); - t.append(Constants.MARKDOWN_QUOTE); - t.append(entry.getKey()).append(":").append(entry.getValue()); - t.append(Constants.MARKDOWN_ENTER); - } - contents.append(t); - } - } - return contents.toString(); - } - - /** - * convert text to markdown style - * - * @param title the title - * @param content the content - * @return markdown text - */ - public static String markdownText(String title, String content) { - if (StringUtils.isNotEmpty(content)) { - List mapItemsList = JSONUtils.toList(content, LinkedHashMap.class); - if (null != mapItemsList) { - StringBuilder contents = new StringBuilder(100); - contents.append(String.format("`%s`%n", title)); - for (LinkedHashMap mapItems : mapItemsList) { - - Set> entries = mapItems.entrySet(); - Iterator> iterator = entries.iterator(); - while (iterator.hasNext()) { - Map.Entry entry = iterator.next(); - contents.append(Constants.MARKDOWN_QUOTE); - contents.append(entry.getKey()).append(":").append(entry.getValue()); - contents.append(Constants.MARKDOWN_ENTER); - } - - } - return contents.toString(); - } - - } - return null; - } - - /** - * Determine the mardown style based on the show type of the alert - * - * @return the markdown alert table/text - */ - public static String markdownByAlert(AlertData alert) { - String result = ""; - if (alert.getShowType().equals(ShowType.TABLE.getDescp())) { - result = markdownTable(alert.getTitle(), alert.getContent()); - } else if (alert.getShowType().equals(ShowType.TEXT.getDescp())) { - result = markdownText(alert.getTitle(), alert.getContent()); - } - return result; - - } - -} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/MailUtils.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/MailUtils.java deleted file mode 100644 index f57481bd0c..0000000000 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/MailUtils.java +++ /dev/null @@ -1,375 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.dolphinscheduler.alert.utils; - -import org.apache.dolphinscheduler.alert.template.AlertTemplate; -import org.apache.dolphinscheduler.alert.template.AlertTemplateFactory; -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.apache.dolphinscheduler.common.utils.CollectionUtils; -import org.apache.dolphinscheduler.common.utils.StringUtils; - -import org.apache.commons.mail.EmailException; -import org.apache.commons.mail.HtmlEmail; - -import java.io.File; -import java.io.IOException; -import java.util.Collection; -import java.util.HashMap; -import java.util.Map; -import java.util.Properties; - -import javax.mail.Authenticator; -import javax.mail.Message; -import javax.mail.MessagingException; -import javax.mail.PasswordAuthentication; -import javax.mail.Session; -import javax.mail.Transport; -import javax.mail.internet.InternetAddress; -import javax.mail.internet.MimeBodyPart; -import javax.mail.internet.MimeMessage; -import javax.mail.internet.MimeMultipart; -import javax.mail.internet.MimeUtility; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * mail utils - */ -public class MailUtils { - - public static final Logger logger = LoggerFactory.getLogger(MailUtils.class); - - public static final String MAIL_PROTOCOL = PropertyUtils.getString(Constants.MAIL_PROTOCOL); - - public static final String MAIL_SERVER_HOST = PropertyUtils.getString(Constants.MAIL_SERVER_HOST); - - public static final Integer MAIL_SERVER_PORT = PropertyUtils.getInt(Constants.MAIL_SERVER_PORT); - - public static final String MAIL_SENDER = PropertyUtils.getString(Constants.MAIL_SENDER); - - public static final String MAIL_USER = PropertyUtils.getString(Constants.MAIL_USER); - - public static final String MAIL_PASSWD = PropertyUtils.getString(Constants.MAIL_PASSWD); - - public static final Boolean MAIL_USE_START_TLS = PropertyUtils.getBoolean(Constants.MAIL_SMTP_STARTTLS_ENABLE); - - public static final Boolean MAIL_USE_SSL = PropertyUtils.getBoolean(Constants.MAIL_SMTP_SSL_ENABLE); - - public static final String XLS_FILE_PATH = PropertyUtils.getString(Constants.XLS_FILE_PATH, "/tmp/xls"); - - public static final String STARTTLS_ENABLE = PropertyUtils.getString(Constants.MAIL_SMTP_STARTTLS_ENABLE); - - public static final Boolean SSL_ENABLE = PropertyUtils.getBoolean(Constants.MAIL_SMTP_SSL_ENABLE); - - public static final String SSL_TRUST = PropertyUtils.getString(Constants.MAIL_SMTP_SSL_TRUST); - - public static final AlertTemplate alertTemplate = AlertTemplateFactory.getMessageTemplate(); - - //Solve the problem of messy Chinese name in excel attachment - static { - System.setProperty("mail.mime.splitlongparameters", "false"); - } - - private MailUtils() { - throw new IllegalStateException(MailUtils.class.getName()); - } - - /** - * send mail to receivers - * - * @param receivers the receiver list - * @param title the title - * @param content the content - * @param showType the show type - * @return the result map - */ - public static Map sendMails(Collection receivers, String title, String content, String showType) { - return sendMails(receivers, null, title, content, showType); - } - - /** - * send mail - * - * @param receivers the receiver list - * @param receiversCc cc list - * @param title the title - * @param content the content - * @param showType the show type - * @return the send result - */ - public static Map sendMails(Collection receivers, Collection receiversCc, String title, String content, String showType) { - Map retMap = new HashMap<>(); - retMap.put(Constants.STATUS, false); - - // if there is no receivers && no receiversCc, no need to process - if (CollectionUtils.isEmpty(receivers) && CollectionUtils.isEmpty(receiversCc)) { - return retMap; - } - - receivers.removeIf(StringUtils::isEmpty); - - if (showType.equals(ShowType.TABLE.getDescp()) || showType.equals(ShowType.TEXT.getDescp())) { - // send email - HtmlEmail email = new HtmlEmail(); - - try { - Session session = getSession(); - email.setMailSession(session); - email.setFrom(MAIL_SENDER); - email.setCharset(Constants.UTF_8); - if (CollectionUtils.isNotEmpty(receivers)) { - // receivers mail - for (String receiver : receivers) { - email.addTo(receiver); - } - } - - if (CollectionUtils.isNotEmpty(receiversCc)) { - //cc - for (String receiverCc : receiversCc) { - email.addCc(receiverCc); - } - } - // sender mail - return getStringObjectMap(title, content, showType, retMap, email); - } catch (Exception e) { - handleException(receivers, retMap, e); - } - } else if (showType.equals(ShowType.ATTACHMENT.getDescp()) || showType.equals(ShowType.TABLEATTACHMENT.getDescp())) { - try { - - String partContent = (showType.equals(ShowType.ATTACHMENT.getDescp()) ? "Please see the attachment " + title + Constants.EXCEL_SUFFIX_XLS : htmlTable(content, false)); - - attachment(receivers, receiversCc, title, content, partContent); - - retMap.put(Constants.STATUS, true); - return retMap; - } catch (Exception e) { - handleException(receivers, retMap, e); - return retMap; - } - } - return retMap; - - } - - /** - * html table content - * - * @param content the content - * @param showAll if show the whole content - * @return the html table form - */ - private static String htmlTable(String content, boolean showAll) { - return alertTemplate.getMessageFromTemplate(content, ShowType.TABLE, showAll); - } - - /** - * html table content - * - * @param content the content - * @return the html table form - */ - private static String htmlTable(String content) { - return htmlTable(content, true); - } - - /** - * html text content - * - * @param content the content - * @return text in html form - */ - private static String htmlText(String content) { - return alertTemplate.getMessageFromTemplate(content, ShowType.TEXT); - } - - /** - * send mail as Excel attachment - * - * @param receivers the receiver list - * @param title the title - */ - private static void attachment(Collection receivers, Collection receiversCc, String title, String content, String partContent) throws Exception { - MimeMessage msg = getMimeMessage(receivers); - - attachContent(receiversCc, title, content, partContent, msg); - } - - /** - * get MimeMessage - * - * @param receivers receivers - * @return the MimeMessage - */ - private static MimeMessage getMimeMessage(Collection receivers) throws MessagingException { - - // 1. The first step in creating mail: creating session - Session session = getSession(); - // Setting debug mode, can be turned off - session.setDebug(false); - - // 2. creating mail: Creating a MimeMessage - MimeMessage msg = new MimeMessage(session); - // 3. set sender - msg.setFrom(new InternetAddress(MAIL_SENDER)); - // 4. set receivers - for (String receiver : receivers) { - msg.addRecipients(Message.RecipientType.TO, InternetAddress.parse(receiver)); - } - return msg; - } - - /** - * get session - * - * @return the new Session - */ - private static Session getSession() { - Properties props = new Properties(); - props.setProperty(Constants.MAIL_HOST, MAIL_SERVER_HOST); - props.setProperty(Constants.MAIL_PORT, String.valueOf(MAIL_SERVER_PORT)); - props.setProperty(Constants.MAIL_SMTP_AUTH, Constants.STRING_TRUE); - props.setProperty(Constants.MAIL_TRANSPORT_PROTOCOL, MAIL_PROTOCOL); - props.setProperty(Constants.MAIL_SMTP_STARTTLS_ENABLE, STARTTLS_ENABLE); - if (Boolean.TRUE.equals(SSL_ENABLE)) { - props.setProperty(Constants.MAIL_SMTP_SSL_ENABLE, "true"); - props.setProperty(Constants.MAIL_SMTP_SSL_TRUST, SSL_TRUST); - } - - Authenticator auth = new Authenticator() { - @Override - protected PasswordAuthentication getPasswordAuthentication() { - // mail username and password - return new PasswordAuthentication(MAIL_USER, MAIL_PASSWD); - } - }; - - return Session.getInstance(props, auth); - } - - /** - * attach content - * - * @param receiversCc the cc list - * @param title the title - * @param content the content - * @param partContent the partContent - * @param msg the message - */ - private static void attachContent(Collection receiversCc, String title, String content, String partContent, MimeMessage msg) throws MessagingException, IOException { - /* - * set receiverCc - */ - if (CollectionUtils.isNotEmpty(receiversCc)) { - for (String receiverCc : receiversCc) { - msg.addRecipients(Message.RecipientType.CC, InternetAddress.parse(receiverCc)); - } - } - - // set subject - msg.setSubject(title); - MimeMultipart partList = new MimeMultipart(); - // set signature - MimeBodyPart part1 = new MimeBodyPart(); - part1.setContent(partContent, Constants.TEXT_HTML_CHARSET_UTF_8); - // set attach file - MimeBodyPart part2 = new MimeBodyPart(); - File file = new File(XLS_FILE_PATH + Constants.SINGLE_SLASH + title + Constants.EXCEL_SUFFIX_XLS); - if (!file.getParentFile().exists()) { - file.getParentFile().mkdirs(); - } - // make excel file - - ExcelUtils.genExcelFile(content, title, XLS_FILE_PATH); - - part2.attachFile(file); - part2.setFileName(MimeUtility.encodeText(title + Constants.EXCEL_SUFFIX_XLS, Constants.UTF_8, "B")); - // add components to collection - partList.addBodyPart(part1); - partList.addBodyPart(part2); - msg.setContent(partList); - // 5. send Transport - Transport.send(msg); - // 6. delete saved file - deleteFile(file); - } - - /** - * the string object map - * - * @param title the title - * @param content the content - * @param showType the showType - * @param retMap the result map - * @param email the email - * @return the result map - */ - private static Map getStringObjectMap(String title, String content, String showType, Map retMap, HtmlEmail email) throws EmailException { - - /* - * the subject of the message to be sent - */ - email.setSubject(title); - /* - * to send information, you can use HTML tags in mail content because of the use of HtmlEmail - */ - if (showType.equals(ShowType.TABLE.getDescp())) { - email.setMsg(htmlTable(content)); - } else if (showType.equals(ShowType.TEXT.getDescp())) { - email.setMsg(htmlText(content)); - } - - // send - email.send(); - - retMap.put(Constants.STATUS, true); - - return retMap; - } - - /** - * file delete - * - * @param file the file to delete - */ - public static void deleteFile(File file) { - if (file.exists()) { - if (file.delete()) { - logger.info("delete success: {}", file.getAbsolutePath() + file.getName()); - } else { - logger.info("delete fail: {}", file.getAbsolutePath() + file.getName()); - } - } else { - logger.info("file not exists: {}", file.getAbsolutePath() + file.getName()); - } - } - - /** - * handle exception - * - * @param receivers the receiver list - * @param retMap the result map - * @param e the exception - */ - private static void handleException(Collection receivers, Map retMap, Exception e) { - logger.error("Send email to {} failed", receivers, e); - retMap.put(Constants.MESSAGE, "Send email to {" + String.join(",", receivers) + "} failed，" + e.toString()); - } -} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/PropertyUtils.java b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/PropertyUtils.java index 0eb241366a..8da9abe293 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/PropertyUtils.java +++ b/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/PropertyUtils.java @@ -83,7 +83,7 @@ public class PropertyUtils { /** * get property value * - * @param key property name + * @param key property name * @param defaultVal default value * @return property value */ diff --git a/dolphinscheduler-alert/src/main/resources/alert.properties b/dolphinscheduler-alert/src/main/resources/alert.properties index 6eb701841c..c153b9b83a 100644 --- a/dolphinscheduler-alert/src/main/resources/alert.properties +++ b/dolphinscheduler-alert/src/main/resources/alert.properties @@ -15,46 +15,19 @@ # limitations under the License. # -#alert type is EMAIL/SMS -alert.type=EMAIL +#This configuration file configures the configuration parameters related to the AlertServer. +#These parameters are only related to the AlertServer, and it has nothing to do with the specific Alert Plugin. +#eg : max retry num. +#eg : Alert Server Listener port -# mail server configuration -mail.protocol=SMTP -mail.server.host=xxx.xxx.com -mail.server.port=25 -mail.sender=xxx@xxx.com -mail.user=xxx@xxx.com -mail.passwd=111111 -# TLS -mail.smtp.starttls.enable=true -# SSL -mail.smtp.ssl.enable=false -mail.smtp.ssl.trust=xxx.xxx.com +#alert.plugin.dir config the Alert Plugin dir . AlertServer while find and load the Alert Plugin Jar from this dir when deploy and start AlertServer on the server . +#eg : +alert.plugin.dir=/opt/soft/spi/lib/plugin/alert -#xls file path,need create if not exist -#xls.file.path=/tmp/xls +#maven.local.repository=/Users/gaojun/Documents/jianguoyun/localRepository -# Enterprise WeChat configuration -enterprise.wechat.enable=false +#alert.plugin.binding config the Alert Plugin need be load when development and run in IDE +#alert.plugin.binding=\ +# ./dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml -#enterprise.wechat.corp.id=xxxxxxx -#enterprise.wechat.secret=xxxxxxx -#enterprise.wechat.agent.id=xxxxxxx -#enterprise.wechat.users=xxxxxxx -#enterprise.wechat.token.url=https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid={corpId}&corpsecret={secret} -#enterprise.wechat.push.url=https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token={token} -#enterprise.wechat.team.send.msg={\"toparty\":\"{toParty}\",\"agentid\":\"{agentId}\",\"msgtype\":\"text\",\"text\":{\"content\":\"{msg}\"},\"safe\":\"0\"} -#enterprise.wechat.user.send.msg={\"touser\":\"{toUser}\",\"agentid\":\"{agentId}\",\"msgtype\":\"markdown\",\"markdown\":{\"content\":\"{msg}\"}} - -plugin.dir=/Users/xx/your/path/to/plugin/dir - -#ding talk configuration -dingtalk.isEnable=flase -dingtalk.webhook=https://oapi.dingtalk.com/robot/send?access_token=xxxxx -dingtalk.keyword= -dingtalk.proxy= -dingtalk.port=80 -dingtalk.user= -dingtalk.password= -dingtalk.isEnableProxy=false diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/AlertServerTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/AlertServerTest.java new file mode 100644 index 0000000000..a8ead79be9 --- /dev/null +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/AlertServerTest.java @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert; + +import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager; +import org.apache.dolphinscheduler.alert.plugin.DolphinPluginLoader; +import org.apache.dolphinscheduler.alert.plugin.DolphinPluginManagerConfig; +import org.apache.dolphinscheduler.alert.runner.AlertSender; +import org.apache.dolphinscheduler.alert.utils.Constants; +import org.apache.dolphinscheduler.dao.AlertDao; +import org.apache.dolphinscheduler.dao.DaoFactory; +import org.apache.dolphinscheduler.dao.PluginDao; +import org.apache.dolphinscheduler.remote.NettyRemotingServer; +import org.apache.dolphinscheduler.spi.alert.AlertChannel; + +import java.util.concurrent.ConcurrentHashMap; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.powermock.api.mockito.PowerMockito; +import org.powermock.core.classloader.annotations.PrepareForTest; +import org.powermock.modules.junit4.PowerMockRunner; + +@RunWith(PowerMockRunner.class) +@PrepareForTest({AlertServer.class,DaoFactory.class}) +public class AlertServerTest { + + @Before + public void before() { + + } + + @Test + public void testMain() throws Exception { + AlertDao alertDao = PowerMockito.mock(AlertDao.class); + PowerMockito.mockStatic(DaoFactory.class); + PowerMockito.when(DaoFactory.getDaoInstance(AlertDao.class)).thenReturn(alertDao); + + PluginDao pluginDao = PowerMockito.mock(PluginDao.class); + PowerMockito.when(DaoFactory.getDaoInstance(PluginDao.class)).thenReturn(pluginDao); + + AlertChannel alertChannelMock = PowerMockito.mock(AlertChannel.class); + + AlertPluginManager alertPluginManager = PowerMockito.mock(AlertPluginManager.class); + PowerMockito.whenNew(AlertPluginManager.class).withNoArguments().thenReturn(alertPluginManager); + ConcurrentHashMap alertChannelMap = new ConcurrentHashMap<>(); + alertChannelMap.put("pluginName",alertChannelMock); + PowerMockito.when(alertPluginManager.getAlertChannelMap()).thenReturn(alertChannelMap); + + DolphinPluginManagerConfig alertPluginManagerConfig = PowerMockito.mock(DolphinPluginManagerConfig.class); + PowerMockito.whenNew(DolphinPluginManagerConfig.class).withNoArguments().thenReturn(alertPluginManagerConfig); + + NettyRemotingServer nettyRemotingServer = PowerMockito.mock(NettyRemotingServer.class); + PowerMockito.whenNew(NettyRemotingServer.class).withAnyArguments().thenReturn(nettyRemotingServer); + AlertSender alertSender = PowerMockito.mock(AlertSender.class); + PowerMockito.whenNew(AlertSender.class).withAnyArguments().thenReturn(alertSender); + + DolphinPluginLoader dolphinPluginLoader = PowerMockito.mock(DolphinPluginLoader.class); + PowerMockito.whenNew(DolphinPluginLoader.class).withAnyArguments().thenReturn(dolphinPluginLoader); + + AlertServer alertServer = AlertServer.getInstance(); + Assert.assertNotNull(alertServer); + + new Thread(() -> { + alertServer.start(); }) + .start(); + + Thread.sleep(5 * Constants.ALERT_SCAN_INTERVAL); + + alertServer.stop(); + + } + +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManagerTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManagerTest.java new file mode 100644 index 0000000000..5ed25cc004 --- /dev/null +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/AlertPluginManagerTest.java @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import org.apache.dolphinscheduler.alert.AlertServer; +import org.apache.dolphinscheduler.alert.utils.Constants; +import org.apache.dolphinscheduler.alert.utils.PropertyUtils; +import org.apache.dolphinscheduler.spi.utils.StringUtils; + +import org.junit.Assert; +import org.junit.Ignore; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.google.common.collect.ImmutableList; + +/** + * AlertPluginManager Tester. + */ +@Ignore +public class AlertPluginManagerTest { + + private static final Logger logger = LoggerFactory.getLogger(AlertPluginManagerTest.class); + + @Test + public void testLoadPlugins() throws Exception { + logger.info("begin test AlertPluginManagerTest"); + AlertPluginManager alertPluginManager = new AlertPluginManager(); + DolphinPluginManagerConfig alertPluginManagerConfig = new DolphinPluginManagerConfig(); + String path = DolphinPluginLoader.class.getClassLoader().getResource("").getPath(); + alertPluginManagerConfig.setPlugins(path + "../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml"); + if (StringUtils.isNotBlank(PropertyUtils.getString(AlertServer.ALERT_PLUGIN_DIR))) { + alertPluginManagerConfig.setInstalledPluginsDir(org.apache.dolphinscheduler.alert.utils.PropertyUtils.getString(AlertServer.ALERT_PLUGIN_DIR, Constants.ALERT_PLUGIN_PATH).trim()); + } + + if (StringUtils.isNotBlank(PropertyUtils.getString(AlertServer.MAVEN_LOCAL_REPOSITORY))) { + alertPluginManagerConfig.setMavenLocalRepository(PropertyUtils.getString(AlertServer.MAVEN_LOCAL_REPOSITORY).trim()); + } + + DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager)); + try { + alertPluginLoader.loadPlugins(); + } catch (Exception e) { + throw new RuntimeException("load Alert Plugin Failed !", e); + } + + Assert.assertNotNull(alertPluginManager.getAlertChannelFactoryMap().get("email alert")); + } +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoaderTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoaderTest.java new file mode 100644 index 0000000000..5c792db451 --- /dev/null +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/DolphinPluginLoaderTest.java @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.plugin; + +import org.junit.After; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Ignore; +import org.junit.Test; + +import com.google.common.collect.ImmutableList; + +/** + * DolphinPluginLoader Tester. + */ +@Ignore +public class DolphinPluginLoaderTest { + + @Before + public void before() throws Exception { + } + + @After + public void after() throws Exception { + } + + /** + * Method: loadPlugins() + */ + @Test + public void testLoadPlugins() throws Exception { + AlertPluginManager alertPluginManager = new AlertPluginManager(); + DolphinPluginManagerConfig alertPluginManagerConfig = new DolphinPluginManagerConfig(); + String path = DolphinPluginLoader.class.getClassLoader().getResource("").getPath(); + alertPluginManagerConfig.setPlugins(path + "../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml"); + DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager)); + try { + alertPluginLoader.loadPlugins(); + } catch (Exception e) { + throw new RuntimeException("load Alert Plugin Failed !", e); + } + + Assert.assertNotNull(alertPluginManager.getAlertChannelFactoryMap().get("email alert")); + } +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPluginTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPluginTest.java index cdbcf276d3..3983545140 100644 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPluginTest.java +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/plugin/EmailAlertPluginTest.java @@ -14,67 +14,226 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.alert.plugin; +import org.apache.dolphinscheduler.alert.AlertServer; +import org.apache.dolphinscheduler.alert.runner.AlertSender; import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; -import org.apache.dolphinscheduler.plugin.model.AlertData; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; -import org.apache.dolphinscheduler.plugin.model.PluginName; -import org.junit.Before; -import org.junit.Test; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import org.apache.dolphinscheduler.alert.utils.PropertyUtils; +import org.apache.dolphinscheduler.common.enums.AlertStatus; +import org.apache.dolphinscheduler.common.utils.JSONUtils; +import org.apache.dolphinscheduler.dao.AlertDao; +import org.apache.dolphinscheduler.dao.DaoFactory; +import org.apache.dolphinscheduler.dao.PluginDao; +import org.apache.dolphinscheduler.dao.entity.Alert; +import org.apache.dolphinscheduler.dao.entity.AlertGroup; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.spi.alert.AlertConstants; +import org.apache.dolphinscheduler.spi.alert.ShowType; +import org.apache.dolphinscheduler.spi.params.InputParam; +import org.apache.dolphinscheduler.spi.params.PasswordParam; +import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer; +import org.apache.dolphinscheduler.spi.params.RadioParam; +import org.apache.dolphinscheduler.spi.params.base.DataType; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; +import org.apache.dolphinscheduler.spi.utils.StringUtils; import java.util.ArrayList; +import java.util.Date; +import java.util.LinkedHashMap; import java.util.List; -import java.util.Map; -import static org.junit.Assert.*; +import org.junit.Assert; +import org.junit.Ignore; +import org.junit.Test; +import com.google.common.collect.ImmutableList; + +/** + * test load and use alert plugin + */ public class EmailAlertPluginTest { - private static final Logger logger = LoggerFactory.getLogger(EmailAlertPluginTest.class); + AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); + PluginDao pluginDao = DaoFactory.getDaoInstance(PluginDao.class); - private AlertPlugin plugin; + @Test + @Ignore + public void testRunSend() throws Exception { - @Before - public void before() { - plugin = new EmailAlertPlugin(); - } + //create alert group + AlertGroup alertGroup = new AlertGroup(); + alertGroup.setDescription("test alert group 1"); + alertGroup.setGroupName("testalertg1"); + alertDao.getAlertGroupMapper().insert(alertGroup); - @Test - public void getId() { - String id = plugin.getId(); - assertEquals(Constants.PLUGIN_DEFAULT_EMAIL_ID, id); - } + //add alert + Alert alert1 = new Alert(); + alert1.setTitle("test alert"); + LinkedHashMap map1 = new LinkedHashMap<>(); + map1.put("mysql service name", "mysql200"); + map1.put("mysql address", "192.168.xx.xx"); + map1.put("port", "3306"); + map1.put(AlertConstants.SHOW_TYPE, ShowType.TEXT.getDescp()); + map1.put("no index of number", "80"); + map1.put("database client connections", "190"); + + LinkedHashMap map2 = new LinkedHashMap<>(); + map2.put("mysql service name", "mysql210"); + map2.put("mysql address", "192.168.xx.xx"); + map2.put("port", "3306"); + map2.put("no index of number", "10"); + map1.put(AlertConstants.SHOW_TYPE, ShowType.TABLE.getDescp()); + map2.put("database client connections", "90"); + + List> maps = new ArrayList<>(); + maps.add(0, map1); + maps.add(1, map2); + String mapjson = JSONUtils.toJsonString(maps); + alert1.setContent(mapjson); + alert1.setLog("log log"); + alert1.setAlertGroupId(alertGroup.getId()); + alertDao.addAlert(alert1); + + List alertList = new ArrayList<>(); + alertList.add(alert1); + + //load email alert plugin + AlertPluginManager alertPluginManager = new AlertPluginManager(); + DolphinPluginManagerConfig alertPluginManagerConfig = new DolphinPluginManagerConfig(); + String path = DolphinPluginLoader.class.getClassLoader().getResource("").getPath(); + alertPluginManagerConfig.setPlugins(path + "../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml"); + if (StringUtils.isNotBlank(PropertyUtils.getString(AlertServer.ALERT_PLUGIN_DIR))) { + alertPluginManagerConfig.setInstalledPluginsDir(PropertyUtils.getString(AlertServer.ALERT_PLUGIN_DIR, Constants.ALERT_PLUGIN_PATH).trim()); + } + + if (StringUtils.isNotBlank(PropertyUtils.getString(AlertServer.MAVEN_LOCAL_REPOSITORY))) { + alertPluginManagerConfig.setMavenLocalRepository(PropertyUtils.getString(AlertServer.MAVEN_LOCAL_REPOSITORY).trim()); + } + + DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager)); + try { + alertPluginLoader.loadPlugins(); + } catch (Exception e) { + throw new RuntimeException("load Alert Plugin Failed !", e); + } + + //create email alert plugin instance + AlertPluginInstance alertPluginInstance = new AlertPluginInstance(); + alertPluginInstance.setCreateTime(new Date()); + alertPluginInstance.setInstanceName("test email alert"); + + List pluginDefineList = pluginDao.getPluginDefineMapper().queryByNameAndType("email alert", "alert"); + if (pluginDefineList == null || pluginDefineList.size() == 0) { + throw new RuntimeException("no alert plugin be load"); + } + PluginDefine pluginDefine = pluginDefineList.get(0); + alertPluginInstance.setPluginDefineId(pluginDefine.getId()); + alertPluginInstance.setPluginInstanceParams(getEmailAlertParams()); + alertDao.getAlertPluginInstanceMapper().insert(alertPluginInstance); + + AlertSender alertSender = new AlertSender(alertList, alertDao, alertPluginManager, pluginDao); + alertSender.run(); + + Alert alertResult = alertDao.getAlertMapper().selectById(alert1.getId()); + Assert.assertNotNull(alertResult); + Assert.assertEquals(alertResult.getAlertStatus(), AlertStatus.EXECUTION_FAILURE); + + alertDao.getAlertGroupMapper().deleteById(alertGroup.getId()); + alertDao.getAlertPluginInstanceMapper().deleteById(alertPluginInstance.getId()); + alertDao.getAlertMapper().deleteById(alert1.getId()); - @Test - public void getName() { - PluginName pluginName = plugin.getName(); - assertEquals(Constants.PLUGIN_DEFAULT_EMAIL_CH, pluginName.getChinese()); - assertEquals(Constants.PLUGIN_DEFAULT_EMAIL_EN, pluginName.getEnglish()); } - @Test - public void process() { - AlertInfo alertInfo = new AlertInfo(); - AlertData alertData = new AlertData(); - alertData.setId(1) - .setAlertGroupId(1) - .setContent("[\"alarm time：2018-02-05\", \"service name：MYSQL_ALTER\", \"alarm name：MYSQL_ALTER_DUMP\", " + - "\"get the alarm exception.！，interface error，exception information：timed out\", \"request address：http://blog.csdn.net/dreamInTheWorld/article/details/78539286\"]") - .setLog("test log") - .setReceivers("xx@xx.com") - .setReceiversCc("xx@xx.com") - .setShowType(ShowType.TEXT.getDescp()) - .setTitle("test title"); - - alertInfo.setAlertData(alertData); - List list = new ArrayList(){{ add("xx@xx.com"); }}; - alertInfo.addProp("receivers", list); -// Map ret = plugin.process(alertInfo); -// assertFalse(Boolean.parseBoolean(String.valueOf(ret.get(Constants.STATUS)))); + public String getEmailAlertParams() { + + List paramsList = new ArrayList<>(); + InputParam receivesParam = InputParam.newBuilder("receivers", "receivers") + .setValue("540957506@qq.com") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam mailSmtpHost = InputParam.newBuilder("mailServerHost", "mail.smtp.host") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("smtp.exmail.qq.com") + .build(); + + InputParam mailSmtpPort = InputParam.newBuilder("mailServerPort", "mail.smtp.port") + .addValidate(Validate.newBuilder() + .setRequired(true) + .setType(DataType.NUMBER.getDataType()) + .build()) + .setValue(25) + .build(); + + InputParam mailSender = InputParam.newBuilder("mailSender", "mail.sender") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("easyscheduler@analysys.com.cn") + .build(); + + RadioParam enableSmtpAuth = RadioParam.newBuilder("enableSmtpAuth", "mail.smtp.auth") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue(true) + .build(); + + InputParam mailUser = InputParam.newBuilder("mailUser", "mail.user") + .setPlaceholder("if enable use authentication, you need input user") + .setValue("easyscheduler@analysys.com.cn") + .build(); + + PasswordParam mailPassword = PasswordParam.newBuilder("mailPasswd", "mail.passwd") + .setPlaceholder("if enable use authentication, you need input password") + .setValue("xxxxxxx") + .build(); + + RadioParam enableTls = RadioParam.newBuilder("starttlsEnable", "mail.smtp.starttls.enable") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue(true) + .build(); + + RadioParam enableSsl = RadioParam.newBuilder("sslEnable", "mail.smtp.ssl.enable") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue(false) + .build(); + + InputParam sslTrust = InputParam.newBuilder("mailSmtpSslTrust", "mail.smtp.ssl.trust") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .setValue("smtp.exmail.qq.com") + .build(); + + List emailShowTypeList = new ArrayList<>(); + emailShowTypeList.add(new ParamsOptions(ShowType.TABLE.getDescp(), ShowType.TABLE.getDescp(), false)); + emailShowTypeList.add(new ParamsOptions(ShowType.TEXT.getDescp(), ShowType.TEXT.getDescp(), false)); + emailShowTypeList.add(new ParamsOptions(ShowType.ATTACHMENT.getDescp(), ShowType.ATTACHMENT.getDescp(), false)); + emailShowTypeList.add(new ParamsOptions(ShowType.TABLEATTACHMENT.getDescp(), ShowType.TABLEATTACHMENT.getDescp(), false)); + RadioParam showType = RadioParam.newBuilder(AlertConstants.SHOW_TYPE, "showType") + .setParamsOptionsList(emailShowTypeList) + .setValue(ShowType.TABLE.getDescp()) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + paramsList.add(receivesParam); + paramsList.add(mailSmtpHost); + paramsList.add(mailSmtpPort); + paramsList.add(mailSender); + paramsList.add(enableSmtpAuth); + paramsList.add(mailUser); + paramsList.add(mailPassword); + paramsList.add(enableTls); + paramsList.add(enableSsl); + paramsList.add(sslTrust); + paramsList.add(showType); + + return PluginParamsTransfer.transferParamsToJson(paramsList); } -} \ No newline at end of file +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessorTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessorTest.java new file mode 100644 index 0000000000..0126eb3dae --- /dev/null +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/processor/AlertRequestProcessorTest.java @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.processor; + +import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager; +import org.apache.dolphinscheduler.dao.AlertDao; +import org.apache.dolphinscheduler.dao.PluginDao; +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.CommandType; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendRequestCommand; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.powermock.api.mockito.PowerMockito; + +import io.netty.channel.Channel; + +/** + * alert request processor test + */ +public class AlertRequestProcessorTest { + + private AlertDao alertDao; + private PluginDao pluginDao; + private AlertPluginManager alertPluginManager; + + private AlertRequestProcessor alertRequestProcessor; + + @Before + public void before() { + alertDao = PowerMockito.mock(AlertDao.class); + pluginDao = PowerMockito.mock(PluginDao.class); + alertPluginManager = PowerMockito.mock(AlertPluginManager.class); + alertRequestProcessor = new AlertRequestProcessor(alertDao,alertPluginManager,pluginDao); + } + + @Test + public void testProcess() { + Channel channel = PowerMockito.mock(Channel.class); + AlertSendRequestCommand alertSendRequestCommand = new AlertSendRequestCommand(1,"title","content"); + Command reqCommand = alertSendRequestCommand.convert2Command(); + Assert.assertEquals(CommandType.ALERT_SEND_REQUEST,reqCommand.getType()); + alertRequestProcessor.process(channel,reqCommand); + } +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/runner/AlertSenderTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/runner/AlertSenderTest.java new file mode 100644 index 0000000000..2664bdcd29 --- /dev/null +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/runner/AlertSenderTest.java @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.alert.runner; + +import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager; +import org.apache.dolphinscheduler.dao.AlertDao; +import org.apache.dolphinscheduler.dao.PluginDao; +import org.apache.dolphinscheduler.dao.entity.Alert; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseCommand; +import org.apache.dolphinscheduler.spi.alert.AlertChannel; +import org.apache.dolphinscheduler.spi.alert.AlertResult; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.mockito.Mockito; +import org.powermock.api.mockito.PowerMockito; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * alert sender test + */ +public class AlertSenderTest { + + private static final Logger logger = LoggerFactory.getLogger(AlertSenderTest.class); + + private AlertDao alertDao; + private PluginDao pluginDao; + private AlertPluginManager alertPluginManager; + + private AlertSender alertSender; + + @Before + public void before() { + alertDao = PowerMockito.mock(AlertDao.class); + pluginDao = PowerMockito.mock(PluginDao.class); + alertPluginManager = PowerMockito.mock(AlertPluginManager.class); + + } + + @Test + public void testSyncHandler() { + + int alertGroupId = 1; + String title = "alert mail test title"; + String content = "alert mail test content"; + alertSender = new AlertSender(alertDao,alertPluginManager,pluginDao); + + //1.alert instance does not exist + PowerMockito.when(alertDao.listInstanceByAlertGroupId(alertGroupId)).thenReturn(null); + + AlertSendResponseCommand alertSendResponseCommand = alertSender.syncHandler(alertGroupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //2.alert plugin does not exist + int pluginDefineId = 1; + String pluginInstanceParams = "alert-instance-mail-params"; + String pluginInstanceName = "alert-instance-mail"; + List alertInstanceList = new ArrayList<>(); + AlertPluginInstance alertPluginInstance = new AlertPluginInstance( + pluginDefineId,pluginInstanceParams,pluginInstanceName); + alertInstanceList.add(alertPluginInstance); + PowerMockito.when(alertDao.listInstanceByAlertGroupId(1)).thenReturn(alertInstanceList); + + String pluginName = "alert-plugin-mail"; + PluginDefine pluginDefine = new PluginDefine(pluginName,"1",null); + PowerMockito.when(pluginDao.getPluginDefineById(pluginDefineId)).thenReturn(pluginDefine); + + alertSendResponseCommand = alertSender.syncHandler(alertGroupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //3.alert result value is null + AlertChannel alertChannelMock = PowerMockito.mock(AlertChannel.class); + PowerMockito.when(alertChannelMock.process(Mockito.any())).thenReturn(null); + Map alertChannelMap = new ConcurrentHashMap<>(); + alertChannelMap.put(pluginName,alertChannelMock); + PowerMockito.when(alertPluginManager.getAlertChannelMap()).thenReturn(alertChannelMap); + + alertSendResponseCommand = alertSender.syncHandler(alertGroupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //4.abnormal information inside the alert plug-in code + AlertResult alertResult = new AlertResult(); + alertResult.setStatus(String.valueOf(false)); + alertResult.setMessage("Abnormal information inside the alert plug-in code"); + PowerMockito.when(alertChannelMock.process(Mockito.any())).thenReturn(alertResult); + alertChannelMap = new ConcurrentHashMap<>(); + alertChannelMap.put(pluginName,alertChannelMock); + PowerMockito.when(alertPluginManager.getAlertChannelMap()).thenReturn(alertChannelMap); + + alertSendResponseCommand = alertSender.syncHandler(alertGroupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //5.alert plugin send success + alertResult = new AlertResult(); + alertResult.setStatus(String.valueOf(true)); + alertResult.setMessage(String.format("Alert Plugin %s send success",pluginInstanceName)); + PowerMockito.when(alertChannelMock.process(Mockito.any())).thenReturn(alertResult); + alertChannelMap = new ConcurrentHashMap<>(); + alertChannelMap.put(pluginName,alertChannelMock); + PowerMockito.when(alertPluginManager.getAlertChannelMap()).thenReturn(alertChannelMap); + + alertSendResponseCommand = alertSender.syncHandler(alertGroupId, title, content); + Assert.assertTrue(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + } + + @Test + public void testRun() { + int alertGroupId = 1; + String title = "alert mail test title"; + String content = "alert mail test content"; + List alertList = new ArrayList<>(); + Alert alert = new Alert(); + alert.setAlertGroupId(alertGroupId); + alert.setTitle(title); + alert.setContent(content); + alertList.add(alert); + + alertSender = new AlertSender(alertList,alertDao,alertPluginManager,pluginDao); + + int pluginDefineId = 1; + String pluginInstanceParams = "alert-instance-mail-params"; + String pluginInstanceName = "alert-instance-mail"; + List alertInstanceList = new ArrayList<>(); + AlertPluginInstance alertPluginInstance = new AlertPluginInstance( + pluginDefineId,pluginInstanceParams,pluginInstanceName); + alertInstanceList.add(alertPluginInstance); + PowerMockito.when(alertDao.listInstanceByAlertGroupId(alertGroupId)).thenReturn(alertInstanceList); + + String pluginName = "alert-plugin-mail"; + PluginDefine pluginDefine = new PluginDefine(pluginName,"1",null); + PowerMockito.when(pluginDao.getPluginDefineById(pluginDefineId)).thenReturn(pluginDefine); + + AlertResult alertResult = new AlertResult(); + alertResult.setStatus(String.valueOf(true)); + alertResult.setMessage(String.format("Alert Plugin %s send success",pluginInstanceName)); + AlertChannel alertChannelMock = PowerMockito.mock(AlertChannel.class); + PowerMockito.when(alertChannelMock.process(Mockito.any())).thenReturn(alertResult); + ConcurrentHashMap alertChannelMap = new ConcurrentHashMap<>(); + alertChannelMap.put(pluginName,alertChannelMock); + PowerMockito.when(alertPluginManager.getAlertChannelMap()).thenReturn(alertChannelMap); + Assert.assertTrue(Boolean.parseBoolean(alertResult.getStatus())); + alertSender.run(); + + } + +} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactoryTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactoryTest.java deleted file mode 100644 index 32201e6011..0000000000 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactoryTest.java +++ /dev/null @@ -1,65 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.alert.template; - -import org.apache.dolphinscheduler.alert.template.impl.DefaultHTMLTemplate; -import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.alert.utils.PropertyUtils; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.mockito.Mockito; -import org.powermock.api.mockito.PowerMockito; -import org.powermock.core.classloader.annotations.PrepareForTest; -import org.powermock.modules.junit4.PowerMockRunner; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import static org.mockito.Mockito.*; -import static org.junit.Assert.*; - -/** - * test class for AlertTemplateFactory - */ -@RunWith(PowerMockRunner.class) -@PrepareForTest(PropertyUtils.class) -public class AlertTemplateFactoryTest { - - private static final Logger logger = LoggerFactory.getLogger(AlertTemplateFactoryTest.class); - - /** - * GetMessageTemplate method test - */ - @Test - public void testGetMessageTemplate(){ - - PowerMockito.mockStatic(PropertyUtils.class); - - AlertTemplate defaultTemplate = AlertTemplateFactory.getMessageTemplate(); - - assertTrue(defaultTemplate instanceof DefaultHTMLTemplate); - } - - /** - * GetMessageTemplate method throw Exception test - */ - @Test - public void testGetMessageTemplateException(){ - - AlertTemplate defaultTemplate = AlertTemplateFactory.getMessageTemplate(); - assertTrue(defaultTemplate instanceof DefaultHTMLTemplate); - } -} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplateTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplateTest.java deleted file mode 100644 index c88f69224d..0000000000 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/template/impl/DefaultHTMLTemplateTest.java +++ /dev/null @@ -1,96 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.alert.template.impl; - -import org.apache.dolphinscheduler.common.utils.*; -import org.apache.dolphinscheduler.alert.utils.Constants; -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.junit.Test; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; - -import static org.junit.Assert.*; - -/** - * test class for DefaultHTMLTemplate - */ -public class DefaultHTMLTemplateTest{ - - private static final Logger logger = LoggerFactory.getLogger(DefaultHTMLTemplateTest.class); - - /** - * only need test method GetMessageFromTemplate - */ - @Test - public void testGetMessageFromTemplate(){ - - DefaultHTMLTemplate template = new DefaultHTMLTemplate(); - - String tableTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TABLE,true); - - assertEquals(tableTypeMessage,generateMockTableTypeResultByHand()); - - String textTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TEXT,true); - - assertEquals(textTypeMessage,generateMockTextTypeResultByHand()); - } - - /** - * generate some simulation data - */ - private String list2String(){ - - LinkedHashMap map1 = new LinkedHashMap<>(); - map1.put("mysql service name","mysql200"); - map1.put("mysql address","192.168.xx.xx"); - map1.put("database client connections","190"); - map1.put("port","3306"); - map1.put("no index of number","80"); - - LinkedHashMap map2 = new LinkedHashMap<>(); - map2.put("mysql service name","mysql210"); - map2.put("mysql address","192.168.xx.xx"); - map2.put("database client connections","90"); - map2.put("port","3306"); - map2.put("no index of number","10"); - - List> maps = new ArrayList<>(); - maps.add(0,map1); - maps.add(1,map2); - String mapjson = JSONUtils.toJsonString(maps); - logger.info(mapjson); - - return mapjson; - } - - private String generateMockTableTypeResultByHand(){ - - return Constants.HTML_HEADER_PREFIX + - "mysql service namemysql addressdatabase client connectionsportno index of number\n" + - "mysql200192.168.xx.xx190330680mysql210192.168.xx.xx90330610" + Constants.TABLE_BODY_HTML_TAIL; - - } - - private String generateMockTextTypeResultByHand(){ - - return Constants.HTML_HEADER_PREFIX + "{\"mysql service name\":\"mysql200\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"190\",\"port\":\"3306\",\"no index of number\":\"80\"}{\"mysql service name\":\"mysql210\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"90\",\"port\":\"3306\",\"no index of number\":\"10\"}" + Constants.TABLE_BODY_HTML_TAIL; - } -} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtilsTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtilsTest.java deleted file mode 100644 index ac62c17ea2..0000000000 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/DingTalkUtilsTest.java +++ /dev/null @@ -1,120 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.alert.utils; - -import org.apache.http.client.config.RequestConfig; -import org.apache.http.impl.client.CloseableHttpClient; -import org.junit.Assert; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.mockito.Mockito; -import org.powermock.api.mockito.PowerMockito; -import org.powermock.core.classloader.annotations.PowerMockIgnore; -import org.powermock.core.classloader.annotations.PrepareForTest; -import org.powermock.modules.junit4.PowerMockRunner; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; - -@PrepareForTest(PropertyUtils.class) -@RunWith(PowerMockRunner.class) -@PowerMockIgnore("javax.net.ssl.*") -public class DingTalkUtilsTest { - Logger logger = LoggerFactory.getLogger(DingTalkUtilsTest.class); - - private static final String mockUrl = "https://oapi.dingtalk.com/robot/send?access_token=test"; - private static final String mockKeyWords = "onway"; - private static final String msg = "ding talk test"; - - @Before - public void init(){ - PowerMockito.mockStatic(PropertyUtils.class); - Mockito.when(PropertyUtils.getString(Constants.DINGTALK_WEBHOOK)).thenReturn(mockUrl); - Mockito.when(PropertyUtils.getString(Constants.DINGTALK_KEYWORD)).thenReturn(mockKeyWords); - Mockito.when(PropertyUtils.getBoolean(Constants.DINGTALK_PROXY_ENABLE)).thenReturn(true); - Mockito.when(PropertyUtils.getString(Constants.DINGTALK_PROXY)).thenReturn("proxy.com.cn"); - Mockito.when(PropertyUtils.getString(Constants.DINGTALK_USER)).thenReturn("user"); - Mockito.when(PropertyUtils.getString(Constants.DINGTALK_PASSWORD)).thenReturn("pswd"); - Mockito.when(PropertyUtils.getInt(Constants.DINGTALK_PORT)).thenReturn(80); - } - -// @Test -// @Ignore -// public void testSendMsg() { -// try { -// String msgTosend = "msg to send"; -// logger.info(PropertyUtils.getString(Constants.DINGTALK_WEBHOOK)); -// String rsp = DingTalkUtils.sendDingTalkMsg(msgTosend, Constants.UTF_8); -// logger.info("send msg result:{}",rsp); -// String errmsg = JSONUtils.parseObject(rsp).getString("errmsg"); -// Assert.assertEquals("ok", errmsg); -// } catch (Exception e) { -// e.printStackTrace(); -// } -// } - - @Test - public void testCreateDefaultClient() { - CloseableHttpClient client = DingTalkUtils.getDefaultClient();; - try { - Assert.assertNotNull(client); - client.close(); - } catch (IOException ex) { - logger.info("close exception",ex.getMessage()); - new Throwable(); - } - } - @Test - public void testCreateProxyClient() { - CloseableHttpClient client = DingTalkUtils.getProxyClient(); - try { - Assert.assertNotNull(client); - client.close(); - } catch (IOException ex) { - logger.info("close exception",ex.getMessage()); - new Throwable(); - } - - } - @Test - public void testProxyConfig() { - RequestConfig rc = DingTalkUtils.getProxyConfig(); - Assert.assertEquals(80, rc.getProxy().getPort()); - Assert.assertEquals("proxy.com.cn", rc.getProxy().getHostName()); - } - - @Test - public void testDingTalkMsgToJson() { - String jsonString = DingTalkUtils.textToJsonString("this is test"); - - logger.info(jsonString); - String expect = "{\"text\":{\"content\":\"this is test\"},\"msgtype\":\"text\"}"; - Assert.assertEquals(expect, jsonString); - } - @Test - public void testDingTalkMsgUtf8() { - String msg = DingTalkUtils.textToJsonString("this is test:中文"); - - logger.info("test support utf8, actual:" + msg); - logger.info("test support utf8, actual:" + DingTalkUtils.IS_ENABLE_DING_TALK); - String expect = "{\"text\":{\"content\":\"this is test:中文\"},\"msgtype\":\"text\"}"; - Assert.assertEquals(expect, msg); - } - -} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtilsTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtilsTest.java deleted file mode 100644 index 1a70c5becb..0000000000 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtilsTest.java +++ /dev/null @@ -1,283 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.dolphinscheduler.alert.utils; - -import org.apache.dolphinscheduler.common.enums.AlertType; -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.apache.dolphinscheduler.common.utils.JSONUtils; -import org.apache.dolphinscheduler.dao.entity.Alert; -import org.apache.dolphinscheduler.plugin.model.AlertData; - -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; - -import org.junit.Assert; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.mockito.Mockito; -import org.powermock.api.mockito.PowerMockito; -import org.powermock.core.classloader.annotations.PrepareForTest; -import org.powermock.modules.junit4.PowerMockRunner; - -/** - * Please manually modify the configuration file before testing. - * file: alert.properties - * enterprise.wechat.corp.id - * enterprise.wechat.secret - * enterprise.wechat.token.url - * enterprise.wechat.push.url - * enterprise.wechat.send.msg - * enterprise.wechat.agent.id - * enterprise.wechat.users - */ -@PrepareForTest(PropertyUtils.class) -@RunWith(PowerMockRunner.class) -public class EnterpriseWeChatUtilsTest { - - private static final String toParty = "wwc99134b6fc1edb6"; - private static final String enterpriseWechatSecret = "Uuv2KFrkdf7SeKOsTDCpsTkpawXBMNRhFy6VKX5FV"; - private static final String enterpriseWechatAgentId = "1000004"; - private static final String enterpriseWechatUsers = "LiGang,journey"; - private static final String msg = "hello world"; - - private static final String enterpriseWechatTeamSendMsg = "{\\\"toparty\\\":\\\"{toParty}\\\",\\\"agentid\\\":\\\"{agentId}\\\"" - + - ",\\\"msgtype\\\":\\\"text\\\",\\\"text\\\":{\\\"content\\\":\\\"{msg}\\\"},\\\"safe\\\":\\\"0\\\"}"; - private static final String enterpriseWechatUserSendMsg = "{\\\"touser\\\":\\\"{toUser}\\\",\\\"agentid\\\":\\\"{agentId}\\\"" - + - ",\\\"msgtype\\\":\\\"markdown\\\",\\\"markdown\\\":{\\\"content\\\":\\\"{msg}\\\"}}"; - - @Before - public void init() { - PowerMockito.mockStatic(PropertyUtils.class); - Mockito.when(PropertyUtils.getBoolean(Constants.ENTERPRISE_WECHAT_ENABLE)).thenReturn(true); - Mockito.when(PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_USER_SEND_MSG)).thenReturn(enterpriseWechatUserSendMsg); - Mockito.when(PropertyUtils.getString(Constants.ENTERPRISE_WECHAT_TEAM_SEND_MSG)).thenReturn(enterpriseWechatTeamSendMsg); - } - - @Test - public void testIsEnable() { - Boolean weChartEnable = EnterpriseWeChatUtils.isEnable(); - Assert.assertTrue(weChartEnable); - } - - @Test - public void testMakeTeamSendMsg1() { - String sendMsg = EnterpriseWeChatUtils.makeTeamSendMsg(toParty, enterpriseWechatSecret, msg); - Assert.assertTrue(sendMsg.contains(toParty)); - Assert.assertTrue(sendMsg.contains(enterpriseWechatSecret)); - Assert.assertTrue(sendMsg.contains(msg)); - - } - - @Test - public void testMakeTeamSendMsg2() { - List parties = new ArrayList<>(); - parties.add(toParty); - parties.add("test1"); - - String sendMsg = EnterpriseWeChatUtils.makeTeamSendMsg(parties, enterpriseWechatSecret, msg); - Assert.assertTrue(sendMsg.contains(toParty)); - Assert.assertTrue(sendMsg.contains(enterpriseWechatSecret)); - Assert.assertTrue(sendMsg.contains(msg)); - } - - @Test - public void tesMakeUserSendMsg1() { - - String sendMsg = EnterpriseWeChatUtils.makeUserSendMsg(enterpriseWechatUsers, enterpriseWechatAgentId, msg); - Assert.assertTrue(sendMsg.contains(enterpriseWechatUsers)); - Assert.assertTrue(sendMsg.contains(enterpriseWechatAgentId)); - Assert.assertTrue(sendMsg.contains(msg)); - } - - @Test - public void tesMakeUserSendMsg2() { - List users = new ArrayList<>(); - users.add("user1"); - users.add("user2"); - - String sendMsg = EnterpriseWeChatUtils.makeUserSendMsg(users, enterpriseWechatAgentId, msg); - Assert.assertTrue(sendMsg.contains(users.get(0))); - Assert.assertTrue(sendMsg.contains(users.get(1))); - Assert.assertTrue(sendMsg.contains(enterpriseWechatAgentId)); - Assert.assertTrue(sendMsg.contains(msg)); - } - - @Test - public void testMarkdownByAlertForText() { - Alert alertForText = createAlertForText(); - AlertData alertData = new AlertData(); - alertData.setTitle(alertForText.getTitle()) - .setShowType(alertForText.getShowType().getDescp()) - .setContent(alertForText.getContent()); - String result = EnterpriseWeChatUtils.markdownByAlert(alertData); - Assert.assertNotNull(result); - } - - @Test - public void testMarkdownByAlertForTable() { - Alert alertForText = createAlertForTable(); - AlertData alertData = new AlertData(); - alertData.setTitle(alertForText.getTitle()) - .setShowType(alertForText.getShowType().getDescp()) - .setContent(alertForText.getContent()); - String result = EnterpriseWeChatUtils.markdownByAlert(alertData); - Assert.assertNotNull(result); - } - - private Alert createAlertForText() { - String content = "[{\"id\":\"69\"," - + - "\"name\":\"UserBehavior-0--1193959466\"," - + - "\"Job name\":\"Start workflow\"," - + - "\"State\":\"SUCCESS\"," - + - "\"Recovery\":\"NO\"," - + - "\"Run time\":\"1\"," - + - "\"Start time\": \"2018-08-06 10:31:34.0\"," - + - "\"End time\": \"2018-08-06 10:31:49.0\"," - + - "\"Host\": \"192.168.xx.xx\"," - + - "\"Notify group\" :\"4\"}]"; - - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TEXT); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(4); - - return alert; - } - - private String list2String() { - - LinkedHashMap map1 = new LinkedHashMap<>(); - map1.put("mysql service name", "mysql200"); - map1.put("mysql address", "192.168.xx.xx"); - map1.put("port", "3306"); - map1.put("no index of number", "80"); - map1.put("database client connections", "190"); - - LinkedHashMap map2 = new LinkedHashMap<>(); - map2.put("mysql service name", "mysql210"); - map2.put("mysql address", "192.168.xx.xx"); - map2.put("port", "3306"); - map2.put("no index of number", "10"); - map2.put("database client connections", "90"); - - List> maps = new ArrayList<>(); - maps.add(0, map1); - maps.add(1, map2); - String mapjson = JSONUtils.toJsonString(maps); - return mapjson; - } - - private Alert createAlertForTable() { - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TABLE); - String content = list2String(); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(1); - return alert; - } - - - // @Test - // public void testSendSingleTeamWeChat() { - // try { - // String token = EnterpriseWeChatUtils.getToken(); - // String msg = EnterpriseWeChatUtils.makeTeamSendMsg(partyId, agentId, "hello world"); - // String resp = EnterpriseWeChatUtils.sendEnterpriseWeChat("utf-8", msg, token); - // - // String errmsg = JSONUtils.parseObject(resp).getString("errmsg"); - // Assert.assertEquals("ok",errmsg); - // } catch (IOException e) { - // e.printStackTrace(); - // } - // } - // - // @Test - // public void testSendMultiTeamWeChat() { - // - // try { - // String token = EnterpriseWeChatUtils.getToken(); - // String msg = EnterpriseWeChatUtils.makeTeamSendMsg(listPartyId, agentId, "hello world"); - // String resp = EnterpriseWeChatUtils.sendEnterpriseWeChat("utf-8", msg, token); - // - // String errmsg = JSONUtils.parseObject(resp).getString("errmsg"); - // Assert.assertEquals("ok",errmsg); - // } catch (IOException e) { - // e.printStackTrace(); - // } - // } - // - // @Test - // public void testSendSingleUserWeChat() { - // try { - // String token = EnterpriseWeChatUtils.getToken(); - // String msg = EnterpriseWeChatUtils.makeUserSendMsg(listUserId.stream().findFirst().get(), agentId, "your meeting room has been booked and will be synced to the 'mailbox' later \n" + - // ">**matter details** \n" + - // ">matter：meeting
" + - // ">organizer：@miglioguan \n" + - // ">participant：@miglioguan、@kunliu、@jamdeezhou、@kanexiong、@kisonwang \n" + - // "> \n" + - // ">meeting room：Guangzhou TIT 1st Floor 301 \n" + - // ">date：May 18, 2018 \n" + - // ">time：9:00-11:00 am \n" + - // "> \n" + - // ">please attend the meeting on time\n" + - // "> \n" + - // ">to modify the meeting information, please click: [Modify Meeting Information](https://work.weixin.qq.com)\""); - // - // String resp = EnterpriseWeChatUtils.sendEnterpriseWeChat("utf-8", msg, token); - // - // String errmsg = JSONUtils.parseObject(resp).getString("errmsg"); - // Assert.assertEquals("ok",errmsg); - // } catch (IOException e) { - // e.printStackTrace(); - // } - // } - // - // @Test - // public void testSendMultiUserWeChat() { - // try { - // String token = EnterpriseWeChatUtils.getToken(); - // - // String msg = EnterpriseWeChatUtils.makeUserSendMsg(listUserId, agentId, "hello world"); - // String resp = EnterpriseWeChatUtils.sendEnterpriseWeChat("utf-8", msg, token); - // - // String errmsg = JSONUtils.parseObject(resp).getString("errmsg"); - // Assert.assertEquals("ok",errmsg); - // } catch (IOException e) { - // e.printStackTrace(); - // } - // } - -} diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/FuncUtilsTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/FuncUtilsTest.java index a4aeea9c0c..818fac98b6 100644 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/FuncUtilsTest.java +++ b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/FuncUtilsTest.java @@ -17,14 +17,14 @@ package org.apache.dolphinscheduler.alert.utils; -import org.junit.Test; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; import java.util.Arrays; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNull; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; public class FuncUtilsTest { diff --git a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/MailUtilsTest.java b/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/MailUtilsTest.java deleted file mode 100644 index 26a69c43ba..0000000000 --- a/dolphinscheduler-alert/src/test/java/org/apache/dolphinscheduler/alert/utils/MailUtilsTest.java +++ /dev/null @@ -1,190 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.alert.utils; - - -import org.apache.dolphinscheduler.common.enums.AlertType; -import org.apache.dolphinscheduler.common.enums.ShowType; -import org.apache.dolphinscheduler.dao.AlertDao; -import org.apache.dolphinscheduler.dao.DaoFactory; -import org.apache.dolphinscheduler.dao.entity.Alert; -import org.apache.dolphinscheduler.dao.entity.User; -import org.junit.Assert; -import org.junit.Test; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.*; -import org.apache.dolphinscheduler.common.utils.*; - - -/** - */ -public class MailUtilsTest { - private static final Logger logger = LoggerFactory.getLogger(MailUtilsTest.class); - @Test - public void testSendMails() { - String[] receivers = new String[]{"347801120@qq.com"}; - String[] receiversCc = new String[]{"347801120@qq.com"}; - - String content ="[\"id:69\"," + - "\"name:UserBehavior-0--1193959466\"," + - "\"Job name: Start workflow\"," + - "\"State: SUCCESS\"," + - "\"Recovery:NO\"," + - "\"Run time: 1\"," + - "\"Start time: 2018-08-06 10:31:34.0\"," + - "\"End time: 2018-08-06 10:31:49.0\"," + - "\"Host: 192.168.xx.xx\"," + - "\"Notify group :4\"]"; - - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TEXT); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(4); - - MailUtils.sendMails(Arrays.asList(receivers),Arrays.asList(receiversCc),alert.getTitle(),alert.getContent(), ShowType.TEXT.getDescp()); - } - - - @Test - public void testQuery(){ - AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); - List alerts = alertDao.listWaitExecutionAlert(); - - String[] mails = new String[]{"xx@xx.com"}; - - for(Alert alert : alerts){ - MailUtils.sendMails(Arrays.asList(mails),"gaojing", alert.getContent(), ShowType.TABLE.getDescp()); - } - - } - - public String list2String(){ - - LinkedHashMap map1 = new LinkedHashMap<>(); - map1.put("mysql service name","mysql200"); - map1.put("mysql address","192.168.xx.xx"); - map1.put("port","3306"); - map1.put("no index of number","80"); - map1.put("database client connections","190"); - - LinkedHashMap map2 = new LinkedHashMap<>(); - map2.put("mysql service name","mysql210"); - map2.put("mysql address","192.168.xx.xx"); - map2.put("port","3306"); - map2.put("no index of number","10"); - map2.put("database client connections","90"); - - List> maps = new ArrayList<>(); - maps.add(0,map1); - maps.add(1,map2); - String mapjson = JSONUtils.toJsonString(maps); - logger.info(mapjson); - - return mapjson; - - } - - @Test - public void testSendTableMail(){ - String[] mails = new String[]{"347801120@qq.com"}; - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TABLE); - String content= list2String(); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(1); - MailUtils.sendMails(Arrays.asList(mails),"gaojing", alert.getContent(), ShowType.TABLE.getDescp()); - } - - /** - * Used to test add alarm information, mail sent - * Text - */ - @Test - public void addAlertText(){ - AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TEXT); - alert.setContent("[\"alarm time：2018-02-05\", \"service name：MYSQL_ALTER\", \"alarm name：MYSQL_ALTER_DUMP\", " + - "\"get the alarm exception.！，interface error，exception information：timed out\", \"request address：http://blog.csdn.net/dreamInTheWorld/article/details/78539286\"]"); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(1); - alertDao.addAlert(alert); - } - - - /** - * Used to test add alarm information, mail sent - * Table - */ - @Test - public void testAddAlertTable(){ - logger.info("testAddAlertTable"); - AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); - Assert.assertNotNull(alertDao); - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TABLE); - - String content = list2String(); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(1); - alertDao.addAlert(alert); - logger.info("" +alert); - } - - @Test - public void testAlertDao(){ - AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); - List users = alertDao.listUserByAlertgroupId(3); - logger.info(users.toString()); - } - - @Test - public void testAttachmentFile()throws Exception{ - String[] mails = new String[]{"xx@xx.com"}; - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.ATTACHMENT); - String content = list2String(); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(1); - MailUtils.sendMails(Arrays.asList(mails),"gaojing",alert.getContent(),ShowType.ATTACHMENT.getDescp()); - } - - @Test - public void testTableAttachmentFile()throws Exception{ - String[] mails = new String[]{"xx@xx.com"}; - Alert alert = new Alert(); - alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TABLEATTACHMENT); - String content = list2String(); - alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(1); - MailUtils.sendMails(Arrays.asList(mails),"gaojing",alert.getContent(),ShowType.TABLEATTACHMENT.getDescp()); - } - -} diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java index 37c38d6b6c..10590529bf 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java @@ -14,31 +14,44 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.api.controller; +import static org.apache.dolphinscheduler.api.enums.Status.CREATE_ALERT_GROUP_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.DELETE_ALERT_GROUP_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.LIST_PAGING_ALERT_GROUP_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.QUERY_ALL_ALERTGROUP_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_ALERT_GROUP_ERROR; + +import org.apache.dolphinscheduler.api.enums.Status; import org.apache.dolphinscheduler.api.exceptions.ApiException; import org.apache.dolphinscheduler.api.service.AlertGroupService; import org.apache.dolphinscheduler.api.utils.Result; import org.apache.dolphinscheduler.common.Constants; -import org.apache.dolphinscheduler.common.enums.AlertType; import org.apache.dolphinscheduler.common.utils.ParameterUtils; +import org.apache.dolphinscheduler.common.utils.StringUtils; import org.apache.dolphinscheduler.dao.entity.User; -import io.swagger.annotations.Api; -import io.swagger.annotations.ApiImplicitParam; -import io.swagger.annotations.ApiImplicitParams; -import io.swagger.annotations.ApiOperation; -import org.apache.dolphinscheduler.api.enums.Status; + +import java.util.HashMap; +import java.util.Map; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.*; -import springfox.documentation.annotations.ApiIgnore; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestAttribute; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.ResponseStatus; +import org.springframework.web.bind.annotation.RestController; -import java.util.HashMap; -import java.util.Map; - -import static org.apache.dolphinscheduler.api.enums.Status.*; +import io.swagger.annotations.Api; +import io.swagger.annotations.ApiImplicitParam; +import io.swagger.annotations.ApiImplicitParams; +import io.swagger.annotations.ApiOperation; +import springfox.documentation.annotations.ApiIgnore; /** * alert group controller @@ -57,28 +70,31 @@ public class AlertGroupController extends BaseController { /** * create alert group * - * @param loginUser login user - * @param groupName group name - * @param groupType group type + * @param loginUser login user + * @param groupName group name * @param description description * @return create result code */ @ApiOperation(value = "createAlertgroup", notes = "CREATE_ALERT_GROUP_NOTES") @ApiImplicitParams({ @ApiImplicitParam(name = "groupName", value = "GROUP_NAME", required = true, dataType = "String"), - @ApiImplicitParam(name = "groupType", value = "GROUP_TYPE", required = true, dataType = "AlertType"), - @ApiImplicitParam(name = "description", value = "DESC", dataType = "String") + @ApiImplicitParam(name = "description", value = "DESC", dataType = "String"), + @ApiImplicitParam(name = "alertInstanceIds", value = "alertInstanceIds", dataType = "String") }) @PostMapping(value = "/create") @ResponseStatus(HttpStatus.CREATED) @ApiException(CREATE_ALERT_GROUP_ERROR) public Result createAlertgroup(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, @RequestParam(value = "groupName") String groupName, - @RequestParam(value = "groupType") AlertType groupType, - @RequestParam(value = "description", required = false) String description) { - logger.info("loginUser user {}, create alertgroup, groupName: {}, groupType: {}, desc: {}", - loginUser.getUserName(), groupName, groupType, description); - Map result = alertGroupService.createAlertgroup(loginUser, groupName, groupType, description); + @RequestParam(value = "description", required = false) String description, + @RequestParam(value = "alertInstanceIds") String alertInstanceIds) { + String strUserName = StringUtils.replaceNRTtoUnderline(loginUser.getUserName()); + String strGroupName = StringUtils.replaceNRTtoUnderline(groupName); + String strDescription = StringUtils.replaceNRTtoUnderline(description); + String strAlertInstanceIds = StringUtils.replaceNRTtoUnderline(alertInstanceIds); + logger.info("loginUser user {}, create alert group, groupName: {}, desc: {},alertInstanceIds:{}", + strUserName, strGroupName, strDescription, strAlertInstanceIds); + Map result = alertGroupService.createAlertgroup(loginUser, groupName, description, alertInstanceIds); return returnDataList(result); } @@ -103,9 +119,9 @@ public class AlertGroupController extends BaseController { * paging query alarm group list * * @param loginUser login user - * @param pageNo page number + * @param pageNo page number * @param searchVal search value - * @param pageSize page size + * @param pageSize page size * @return alert group list page */ @ApiOperation(value = "queryAlertGroupListPaging", notes = "QUERY_ALERT_GROUP_LIST_PAGING_NOTES") @@ -136,10 +152,9 @@ public class AlertGroupController extends BaseController { /** * updateProcessInstance alert group * - * @param loginUser login user - * @param id alert group id - * @param groupName group name - * @param groupType group type + * @param loginUser login user + * @param id alert group id + * @param groupName group name * @param description description * @return update result code */ @@ -147,8 +162,8 @@ public class AlertGroupController extends BaseController { @ApiImplicitParams({ @ApiImplicitParam(name = "id", value = "ALERT_GROUP_ID", required = true, dataType = "Int", example = "100"), @ApiImplicitParam(name = "groupName", value = "GROUP_NAME", required = true, dataType = "String"), - @ApiImplicitParam(name = "groupType", value = "GROUP_TYPE", required = true, dataType = "AlertType"), - @ApiImplicitParam(name = "description", value = "DESC", dataType = "String") + @ApiImplicitParam(name = "description", value = "DESC", dataType = "String"), + @ApiImplicitParam(name = "alertInstanceIds", value = "alertInstanceIds", dataType = "String") }) @PostMapping(value = "/update") @ResponseStatus(HttpStatus.OK) @@ -156,11 +171,13 @@ public class AlertGroupController extends BaseController { public Result updateAlertgroup(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, @RequestParam(value = "id") int id, @RequestParam(value = "groupName") String groupName, - @RequestParam(value = "groupType") AlertType groupType, - @RequestParam(value = "description", required = false) String description) { - logger.info("login user {}, updateProcessInstance alertgroup, groupName: {}, groupType: {}, desc: {}", - loginUser.getUserName(), groupName, groupType, description); - Map result = alertGroupService.updateAlertgroup(loginUser, id, groupName, groupType, description); + @RequestParam(value = "description", required = false) String description, + @RequestParam(value = "alertInstanceIds") String alertInstanceIds) { + logger.info("login user {}, updateProcessInstance alert group, groupName: {}, desc: {}", + StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), + StringUtils.replaceNRTtoUnderline(groupName), + StringUtils.replaceNRTtoUnderline(description)); + Map result = alertGroupService.updateAlertgroup(loginUser, id, groupName, description, alertInstanceIds); return returnDataList(result); } @@ -168,7 +185,7 @@ public class AlertGroupController extends BaseController { * delete alert group by id * * @param loginUser login user - * @param id alert group id + * @param id alert group id * @return delete result code */ @ApiOperation(value = "delAlertgroupById", notes = "DELETE_ALERT_GROUP_BY_ID_NOTES") @@ -215,28 +232,4 @@ public class AlertGroupController extends BaseController { } return result; } - - /** - * grant user - * - * @param loginUser login user - * @param userIds user ids in the group - * @param alertgroupId alert group id - * @return grant result code - */ - @ApiOperation(value = "grantUser", notes = "GRANT_ALERT_GROUP_NOTES") - @ApiImplicitParams({ - @ApiImplicitParam(name = "id", value = "ALERT_GROUP_ID", required = true, dataType = "Int", example = "100"), - @ApiImplicitParam(name = "userIds", value = "USER_IDS", required = true, dataType = "String") - }) - @PostMapping(value = "/grant-user") - @ResponseStatus(HttpStatus.OK) - @ApiException(ALERT_GROUP_GRANT_USER_ERROR) - public Result grantUser(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, - @RequestParam(value = "alertgroupId") int alertgroupId, - @RequestParam(value = "userIds") String userIds) { - logger.info("login user {}, grant user, alertGroupId: {},userIds : {}", loginUser.getUserName(), alertgroupId, userIds); - Map result = alertGroupService.grantUser(loginUser, alertgroupId, userIds); - return returnDataList(result); - } } diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertPluginInstanceController.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertPluginInstanceController.java new file mode 100644 index 0000000000..d34f42060f --- /dev/null +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertPluginInstanceController.java @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.controller; + +import static org.apache.dolphinscheduler.api.enums.Status.CREATE_ALERT_PLUGIN_INSTANCE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.DELETE_ALERT_PLUGIN_INSTANCE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.GET_ALERT_PLUGIN_INSTANCE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.LIST_PAGING_ALERT_PLUGIN_INSTANCE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.QUERY_ALL_ALERT_PLUGIN_INSTANCE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_ALERT_PLUGIN_INSTANCE_ERROR; + +import org.apache.dolphinscheduler.api.enums.Status; +import org.apache.dolphinscheduler.api.exceptions.ApiException; +import org.apache.dolphinscheduler.api.service.AlertPluginInstanceService; +import org.apache.dolphinscheduler.api.utils.Result; +import org.apache.dolphinscheduler.common.Constants; +import org.apache.dolphinscheduler.common.utils.StringUtils; +import org.apache.dolphinscheduler.dao.entity.User; + +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.http.HttpStatus; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestAttribute; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.ResponseStatus; +import org.springframework.web.bind.annotation.RestController; + +import io.swagger.annotations.Api; +import io.swagger.annotations.ApiImplicitParam; +import io.swagger.annotations.ApiImplicitParams; +import io.swagger.annotations.ApiOperation; +import springfox.documentation.annotations.ApiIgnore; + +/** + * alert plugin instance controller + */ +@Api(tags = "ALERT_PLUGIN_INSTANCE_TAG", position = 1) +@RestController +@RequestMapping("alert-plugin-instance") +public class AlertPluginInstanceController extends BaseController { + + private static final Logger logger = LoggerFactory.getLogger(AlertPluginInstanceController.class); + + @Autowired + private AlertPluginInstanceService alertPluginInstanceService; + + + /** + * create alert plugin instance + * + * @param loginUser login user + * @param pluginDefineId alert plugin define id + * @param instanceName instance name + * @param pluginInstanceParams instance params + * @return result + */ + @ApiOperation(value = "createAlertPluginInstance", notes = "CREATE_ALERT_PLUGIN_INSTANCE_NOTES") + @ApiImplicitParams({ + @ApiImplicitParam(name = "pluginDefineId", value = "ALERT_PLUGIN_DEFINE_ID", required = true, dataType = "Int", example = "100"), + @ApiImplicitParam(name = "instanceName", value = "ALERT_PLUGIN_INSTANCE_NAME", required = true, dataType = "String", example = "DING TALK"), + @ApiImplicitParam(name = "pluginInstanceParams", value = "ALERT_PLUGIN_INSTANCE_PARAMS", required = true, dataType = "String", example = "ALERT_PLUGIN_INSTANCE_PARAMS") + }) + @PostMapping(value = "/create") + @ResponseStatus(HttpStatus.CREATED) + @ApiException(CREATE_ALERT_PLUGIN_INSTANCE_ERROR) + public Result createAlertPluginInstance(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam(value = "pluginDefineId") int pluginDefineId, + @RequestParam(value = "instanceName") String instanceName, + @RequestParam(value = "pluginInstanceParams") String pluginInstanceParams) { + logger.info("login user {},create alert plugin instance, instanceName:{} ", + StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), + StringUtils.replaceNRTtoUnderline(instanceName)); + Map result = alertPluginInstanceService.create(loginUser, pluginDefineId, instanceName, pluginInstanceParams); + return returnDataList(result); + } + + /** + * updateAlertPluginInstance + * + * @param loginUser login user + * @param alertPluginInstanceId alert plugin instance id + * @param instanceName instance name + * @param pluginInstanceParams instance params + * @return result + */ + @ApiOperation(value = "update", notes = "UPDATE_ALERT_PLUGIN_INSTANCE_NOTES") + @ApiImplicitParams({ + @ApiImplicitParam(name = "alertPluginInstanceId", value = "ALERT_PLUGIN_INSTANCE_ID", required = true, dataType = "Int", example = "100"), + @ApiImplicitParam(name = "instanceName", value = "ALERT_PLUGIN_INSTANCE_NAME", required = true, dataType = "String", example = "DING TALK"), + @ApiImplicitParam(name = "pluginInstanceParams", value = "ALERT_PLUGIN_INSTANCE_PARAMS", required = true, dataType = "String", example = "ALERT_PLUGIN_INSTANCE_PARAMS") + }) + @GetMapping(value = "/update") + @ResponseStatus(HttpStatus.OK) + @ApiException(UPDATE_ALERT_PLUGIN_INSTANCE_ERROR) + public Result updateAlertPluginInstance(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam(value = "alertPluginInstanceId") int alertPluginInstanceId, + @RequestParam(value = "instanceName") String instanceName, + @RequestParam(value = "pluginInstanceParams") String pluginInstanceParams) { + logger.info("login user {},update alert plugin instance id {}", StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), alertPluginInstanceId); + Map result = alertPluginInstanceService.update(loginUser, alertPluginInstanceId, instanceName, pluginInstanceParams); + return returnDataList(result); + } + + /** + * deleteAlertPluginInstance + * + * @param loginUser login user + * @param id id + * @return result + */ + @ApiOperation(value = "delete", notes = "DELETE_ALERT_PLUGIN_INSTANCE_NOTES") + @ApiImplicitParams({ + @ApiImplicitParam(name = "id", value = "ALERT_PLUGIN_ID", required = true, dataType = "Int", example = "100") + }) + @GetMapping(value = "/delete") + @ResponseStatus(HttpStatus.OK) + @ApiException(DELETE_ALERT_PLUGIN_INSTANCE_ERROR) + public Result deleteAlertPluginInstance(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam(value = "id") int id) { + logger.info("login user {},delete alert plugin instance id {}", StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), id); + + Map result = alertPluginInstanceService.delete(loginUser, id); + return returnDataList(result); + } + + /** + * getAlertPluginInstance + * + * @param loginUser login user + * @param id alert plugin instance id + * @return result + */ + @ApiOperation(value = "get", notes = "GET_ALERT_PLUGIN_INSTANCE_NOTES") + @PostMapping(value = "/get") + @ResponseStatus(HttpStatus.OK) + @ApiException(GET_ALERT_PLUGIN_INSTANCE_ERROR) + public Result getAlertPluginInstance(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam(value = "id") int id) { + logger.info("login user {},get alert plugin instance, id {}", StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), id); + Map result = alertPluginInstanceService.get(loginUser, id); + return returnDataList(result); + } + + /** + * getAlertPluginInstance + * + * @param loginUser login user + * @return result + */ + @ApiOperation(value = "/queryAll", notes = "QUERY_ALL_ALERT_PLUGIN_INSTANCE_NOTES") + @PostMapping(value = "/queryAll") + @ResponseStatus(HttpStatus.OK) + @ApiException(QUERY_ALL_ALERT_PLUGIN_INSTANCE_ERROR) + public Result getAlertPluginInstance(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser) { + logger.info("login user {}, query all alert plugin instance", StringUtils.replaceNRTtoUnderline(loginUser.getUserName())); + Map result = alertPluginInstanceService.queryAll(); + return returnDataList(result); + } + + /** + * check alert group exist + * + * @param loginUser login user + * @param alertInstanceName alert instance name + * @return check result code + */ + @ApiOperation(value = "verifyAlertInstanceName", notes = "VERIFY_ALERT_INSTANCE_NAME_NOTES") + @ApiImplicitParams({ + @ApiImplicitParam(name = "groupName", value = "GROUP_NAME", required = true, dataType = "String"), + }) + @GetMapping(value = "/verify-alert-instance-name") + @ResponseStatus(HttpStatus.OK) + public Result verifyGroupName(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam(value = "alertInstanceName") String alertInstanceName) { + logger.info("login user {},verify alert instance name: {}", StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), StringUtils.replaceNRTtoUnderline(alertInstanceName)); + + boolean exist = alertPluginInstanceService.checkExistPluginInstanceName(alertInstanceName); + Result result = new Result(); + if (exist) { + logger.error("alert plugin instance {} has exist, can't create again.", alertInstanceName); + result.setCode(Status.PLUGIN_INSTANCE_ALREADY_EXIT.getCode()); + result.setMsg(Status.PLUGIN_INSTANCE_ALREADY_EXIT.getMsg()); + } else { + result.setCode(Status.SUCCESS.getCode()); + result.setMsg(Status.SUCCESS.getMsg()); + } + return result; + } + + /** + * paging query alert plugin instance group list + * + * @param loginUser login user + * @param pageNo page number + * @param pageSize page size + * @return alert plugin instance list page + */ + @ApiOperation(value = "queryAlertPluginInstanceListPaging", notes = "QUERY_ALERT_PLUGIN_INSTANCE_LIST_PAGING_NOTES") + @ApiImplicitParams({ + @ApiImplicitParam(name = "pageNo", value = "PAGE_NO", dataType = "Int", example = "1"), + @ApiImplicitParam(name = "pageSize", value = "PAGE_SIZE", dataType = "Int", example = "20") + }) + @GetMapping(value = "/list-paging") + @ResponseStatus(HttpStatus.OK) + @ApiException(LIST_PAGING_ALERT_PLUGIN_INSTANCE_ERROR) + public Result listPaging(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam("pageNo") Integer pageNo, + @RequestParam("pageSize") Integer pageSize) { + logger.info("login user {}, list paging, pageNo: {}, pageSize: {}",StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), pageNo, pageSize); + Map result = checkPageParams(pageNo, pageSize); + if (result.get(Constants.STATUS) != Status.SUCCESS) { + return returnDataListPaging(result); + } + + result = alertPluginInstanceService.queryPluginPage(pageNo, pageSize); + return returnDataListPaging(result); + } + +} diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java index b093483df1..c1e75a73f5 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java @@ -14,31 +14,49 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.api.controller; +import static org.apache.dolphinscheduler.api.enums.Status.CHECK_PROCESS_DEFINITION_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.EXECUTE_PROCESS_INSTANCE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.START_PROCESS_INSTANCE_ERROR; import org.apache.dolphinscheduler.api.enums.ExecuteType; import org.apache.dolphinscheduler.api.exceptions.ApiException; import org.apache.dolphinscheduler.api.service.ExecutorService; import org.apache.dolphinscheduler.api.utils.Result; import org.apache.dolphinscheduler.common.Constants; +import org.apache.dolphinscheduler.common.enums.CommandType; +import org.apache.dolphinscheduler.common.enums.FailureStrategy; +import org.apache.dolphinscheduler.common.enums.Priority; +import org.apache.dolphinscheduler.common.enums.RunMode; +import org.apache.dolphinscheduler.common.enums.TaskDependType; +import org.apache.dolphinscheduler.common.enums.WarningType; import org.apache.dolphinscheduler.common.utils.JSONUtils; import org.apache.dolphinscheduler.dao.entity.User; -import io.swagger.annotations.*; -import org.apache.dolphinscheduler.common.enums.*; + +import java.text.ParseException; +import java.util.Map; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.*; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestAttribute; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.ResponseStatus; +import org.springframework.web.bind.annotation.RestController; + +import io.swagger.annotations.Api; +import io.swagger.annotations.ApiImplicitParam; +import io.swagger.annotations.ApiImplicitParams; +import io.swagger.annotations.ApiOperation; +import io.swagger.annotations.ApiParam; import springfox.documentation.annotations.ApiIgnore; -import java.text.ParseException; -import java.util.Map; - -import static org.apache.dolphinscheduler.api.enums.Status.*; - - /** * execute process controller */ @@ -55,22 +73,20 @@ public class ExecutorController extends BaseController { /** * execute process instance * - * @param loginUser login user - * @param projectName project name - * @param processDefinitionId process definition id - * @param scheduleTime schedule time - * @param failureStrategy failure strategy - * @param startNodeList start nodes list - * @param taskDependType task depend type - * @param execType execute type - * @param warningType warning type - * @param warningGroupId warning group id - * @param receivers receivers - * @param receiversCc receivers cc - * @param runMode run mode + * @param loginUser login user + * @param projectName project name + * @param processDefinitionId process definition id + * @param scheduleTime schedule time + * @param failureStrategy failure strategy + * @param startNodeList start nodes list + * @param taskDependType task depend type + * @param execType execute type + * @param warningType warning type + * @param warningGroupId warning group id + * @param runMode run mode * @param processInstancePriority process instance priority - * @param workerGroup worker group - * @param timeout timeout + * @param workerGroup worker group + * @param timeout timeout * @return start process result code */ @ApiOperation(value = "startProcessInstance", notes = "RUN_PROCESS_INSTANCE_NOTES") @@ -83,8 +99,6 @@ public class ExecutorController extends BaseController { @ApiImplicitParam(name = "execType", value = "COMMAND_TYPE", dataType = "CommandType"), @ApiImplicitParam(name = "warningType", value = "WARNING_TYPE", required = true, dataType = "WarningType"), @ApiImplicitParam(name = "warningGroupId", value = "WARNING_GROUP_ID", required = true, dataType = "Int", example = "100"), - @ApiImplicitParam(name = "receivers", value = "RECEIVERS", dataType = "String"), - @ApiImplicitParam(name = "receiversCc", value = "RECEIVERS_CC", dataType = "String"), @ApiImplicitParam(name = "runMode", value = "RUN_MODE", dataType = "RunMode"), @ApiImplicitParam(name = "processInstancePriority", value = "PROCESS_INSTANCE_PRIORITY", required = true, dataType = "Priority"), @ApiImplicitParam(name = "workerGroup", value = "WORKER_GROUP", dataType = "String", example = "default"), @@ -103,19 +117,16 @@ public class ExecutorController extends BaseController { @RequestParam(value = "execType", required = false) CommandType execType, @RequestParam(value = "warningType", required = true) WarningType warningType, @RequestParam(value = "warningGroupId", required = false) int warningGroupId, - @RequestParam(value = "receivers", required = false) String receivers, - @RequestParam(value = "receiversCc", required = false) String receiversCc, @RequestParam(value = "runMode", required = false) RunMode runMode, @RequestParam(value = "processInstancePriority", required = false) Priority processInstancePriority, @RequestParam(value = "workerGroup", required = false, defaultValue = "default") String workerGroup, @RequestParam(value = "timeout", required = false) Integer timeout, @RequestParam(value = "startParams", required = false) String startParams) throws ParseException { - logger.info("login user {}, start process instance, project name: {}, process definition id: {}, schedule time: {}, " + logger.info("login user {}, start process instance, project name: {}, process definition id: {}, schedule time: {}, " + "failure policy: {}, node name: {}, node dep: {}, notify type: {}, " - + "notify group id: {},receivers:{},receiversCc:{}, run mode: {},process instance priority:{}, workerGroup: {}, timeout: {}, " - + "startParams: {}", + + "notify group id: {}, run mode: {},process instance priority:{}, workerGroup: {}, timeout: {}, startParams: {} ", loginUser.getUserName(), projectName, processDefinitionId, scheduleTime, - failureStrategy, startNodeList, taskDependType, warningType, workerGroup, receivers, receiversCc, runMode, processInstancePriority, + failureStrategy, startNodeList, taskDependType, warningType, workerGroup, runMode, processInstancePriority, workerGroup, timeout, startParams); if (timeout == null) { @@ -127,7 +138,7 @@ public class ExecutorController extends BaseController { } Map result = execService.execProcessInstance(loginUser, projectName, processDefinitionId, scheduleTime, execType, failureStrategy, startNodeList, taskDependType, warningType, - warningGroupId, receivers, receiversCc, runMode, processInstancePriority, workerGroup, timeout, startParamMap); + warningGroupId, runMode, processInstancePriority, workerGroup, timeout, startParamMap); return returnDataList(result); } @@ -135,10 +146,10 @@ public class ExecutorController extends BaseController { /** * do action to process instance：pause, stop, repeat, recover from pause, recover from stop * - * @param loginUser login user - * @param projectName project name + * @param loginUser login user + * @param projectName project name * @param processInstanceId process instance id - * @param executeType execute type + * @param executeType execute type * @return execute result code */ @ApiOperation(value = "execute", notes = "EXECUTE_ACTION_TO_PROCESS_INSTANCE_NOTES") @@ -163,7 +174,7 @@ public class ExecutorController extends BaseController { /** * check process definition and all of the son process definitions is on line. * - * @param loginUser login user + * @param loginUser login user * @param processDefinitionId process definition id * @return check result code */ @@ -180,32 +191,4 @@ public class ExecutorController extends BaseController { Map result = execService.startCheckByProcessDefinedId(processDefinitionId); return returnDataList(result); } - - /** - * query recipients and copyers by process definition ID - * - * @param loginUser login user - * @param processDefinitionId process definition id - * @param processInstanceId process instance id - * @return receivers cc list - */ - @ApiIgnore - @ApiOperation(value = "getReceiverCc", notes = "GET_RECEIVER_CC_NOTES") - @ApiImplicitParams({ - @ApiImplicitParam(name = "processDefinitionId", value = "PROCESS_DEFINITION_ID", required = true, dataType = "Int", example = "100"), - @ApiImplicitParam(name = "processInstanceId", value = "PROCESS_INSTANCE_ID", required = true, dataType = "Int", example = "100") - - }) - @GetMapping(value = "/get-receiver-cc") - @ResponseStatus(HttpStatus.OK) - @ApiException(QUERY_RECIPIENTS_AND_COPYERS_BY_PROCESS_DEFINITION_ERROR) - public Result getReceiverCc(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, - @RequestParam(value = "processDefinitionId", required = false) Integer processDefinitionId, - @RequestParam(value = "processInstanceId", required = false) Integer processInstanceId) { - logger.info("login user {}, get process definition receiver and cc", loginUser.getUserName()); - Map result = execService.getReceiverCc(processDefinitionId, processInstanceId); - return returnDataList(result); - } - - } diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java index 5713d7ffc4..ca57ad11a4 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java @@ -14,8 +14,18 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.api.controller; +import static org.apache.dolphinscheduler.api.enums.Status.CREATE_SCHEDULE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.DELETE_SCHEDULE_CRON_BY_ID_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.OFFLINE_SCHEDULE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.PREVIEW_SCHEDULE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.PUBLISH_SCHEDULE_ONLINE_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.QUERY_SCHEDULE_LIST_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.QUERY_SCHEDULE_LIST_PAGING_ERROR; +import static org.apache.dolphinscheduler.api.enums.Status.UPDATE_SCHEDULE_ERROR; +import static org.apache.dolphinscheduler.common.Constants.SESSION_USER; import org.apache.dolphinscheduler.api.exceptions.ApiException; import org.apache.dolphinscheduler.api.service.SchedulerService; @@ -25,20 +35,30 @@ import org.apache.dolphinscheduler.common.enums.Priority; import org.apache.dolphinscheduler.common.enums.ReleaseState; import org.apache.dolphinscheduler.common.enums.WarningType; import org.apache.dolphinscheduler.common.utils.ParameterUtils; +import org.apache.dolphinscheduler.common.utils.StringUtils; import org.apache.dolphinscheduler.dao.entity.User; -import io.swagger.annotations.*; + +import java.util.Map; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.*; -import springfox.documentation.annotations.ApiIgnore; - -import java.io.IOException; -import java.util.Map; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestAttribute; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.ResponseStatus; +import org.springframework.web.bind.annotation.RestController; -import static org.apache.dolphinscheduler.api.enums.Status.*; -import static org.apache.dolphinscheduler.common.Constants.SESSION_USER; +import io.swagger.annotations.Api; +import io.swagger.annotations.ApiImplicitParam; +import io.swagger.annotations.ApiImplicitParams; +import io.swagger.annotations.ApiOperation; +import io.swagger.annotations.ApiParam; +import springfox.documentation.annotations.ApiIgnore; /** * schedule controller @@ -61,17 +81,15 @@ public class SchedulerController extends BaseController { /** * create schedule * - * @param loginUser login user - * @param projectName project name - * @param processDefinitionId process definition id - * @param schedule scheduler - * @param warningType warning type - * @param warningGroupId warning group id - * @param failureStrategy failure strategy + * @param loginUser login user + * @param projectName project name + * @param processDefinitionId process definition id + * @param schedule scheduler + * @param warningType warning type + * @param warningGroupId warning group id + * @param failureStrategy failure strategy * @param processInstancePriority process instance priority - * @param receivers receivers - * @param receiversCc receivers cc - * @param workerGroup worker group + * @param workerGroup worker group * @return create result code */ @ApiOperation(value = "createSchedule", notes = "CREATE_SCHEDULE_NOTES") @@ -81,8 +99,6 @@ public class SchedulerController extends BaseController { @ApiImplicitParam(name = "warningType", value = "WARNING_TYPE", type = "WarningType"), @ApiImplicitParam(name = "warningGroupId", value = "WARNING_GROUP_ID", dataType = "Int", example = "100"), @ApiImplicitParam(name = "failureStrategy", value = "FAILURE_STRATEGY", type = "FailureStrategy"), - @ApiImplicitParam(name = "receivers", value = "RECEIVERS", type = "String"), - @ApiImplicitParam(name = "receiversCc", value = "RECEIVERS_CC", type = "String"), @ApiImplicitParam(name = "workerGroupId", value = "WORKER_GROUP_ID", dataType = "Int", example = "100"), @ApiImplicitParam(name = "processInstancePriority", value = "PROCESS_INSTANCE_PRIORITY", type = "Priority"), }) @@ -96,16 +112,14 @@ public class SchedulerController extends BaseController { @RequestParam(value = "warningType", required = false, defaultValue = DEFAULT_WARNING_TYPE) WarningType warningType, @RequestParam(value = "warningGroupId", required = false, defaultValue = DEFAULT_NOTIFY_GROUP_ID) int warningGroupId, @RequestParam(value = "failureStrategy", required = false, defaultValue = DEFAULT_FAILURE_POLICY) FailureStrategy failureStrategy, - @RequestParam(value = "receivers", required = false) String receivers, - @RequestParam(value = "receiversCc", required = false) String receiversCc, @RequestParam(value = "workerGroup", required = false, defaultValue = "default") String workerGroup, - @RequestParam(value = "processInstancePriority", required = false, defaultValue = DEFAULT_PROCESS_INSTANCE_PRIORITY) Priority processInstancePriority) throws IOException { - logger.info("login user {}, project name: {}, process name: {}, create schedule: {}, warning type: {}, warning group id: {}," + - "failure policy: {},receivers : {},receiversCc : {},processInstancePriority : {}, workGroupId:{}", - loginUser.getUserName(), projectName, processDefinitionId, schedule, warningType, warningGroupId, - failureStrategy, receivers, receiversCc, processInstancePriority, workerGroup); + @RequestParam(value = "processInstancePriority", required = false, defaultValue = DEFAULT_PROCESS_INSTANCE_PRIORITY) Priority processInstancePriority) { + logger.info("login user {},project name: {}, process name: {}, create schedule: {}, warning type: {}, warning group id: {}," + + "failure policy: {},processInstancePriority : {}, workGroupId:{}", + StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), StringUtils.replaceNRTtoUnderline(projectName), processDefinitionId, schedule, warningType, warningGroupId, + failureStrategy, processInstancePriority, workerGroup); Map result = schedulerService.insertSchedule(loginUser, projectName, processDefinitionId, schedule, - warningType, warningGroupId, failureStrategy, receivers, receiversCc, processInstancePriority, workerGroup); + warningType, warningGroupId, failureStrategy, processInstancePriority, workerGroup); return returnDataList(result); } @@ -113,17 +127,15 @@ public class SchedulerController extends BaseController { /** * updateProcessInstance schedule * - * @param loginUser login user - * @param projectName project name - * @param id scheduler id - * @param schedule scheduler - * @param warningType warning type - * @param warningGroupId warning group id - * @param failureStrategy failure strategy - * @param receivers receivers - * @param workerGroup worker group + * @param loginUser login user + * @param projectName project name + * @param id scheduler id + * @param schedule scheduler + * @param warningType warning type + * @param warningGroupId warning group id + * @param failureStrategy failure strategy + * @param workerGroup worker group * @param processInstancePriority process instance priority - * @param receiversCc receivers cc * @return update result code */ @ApiOperation(value = "updateSchedule", notes = "UPDATE_SCHEDULE_NOTES") @@ -133,8 +145,6 @@ public class SchedulerController extends BaseController { @ApiImplicitParam(name = "warningType", value = "WARNING_TYPE", type = "WarningType"), @ApiImplicitParam(name = "warningGroupId", value = "WARNING_GROUP_ID", dataType = "Int", example = "100"), @ApiImplicitParam(name = "failureStrategy", value = "FAILURE_STRATEGY", type = "FailureStrategy"), - @ApiImplicitParam(name = "receivers", value = "RECEIVERS", type = "String"), - @ApiImplicitParam(name = "receiversCc", value = "RECEIVERS_CC", type = "String"), @ApiImplicitParam(name = "workerGroupId", value = "WORKER_GROUP_ID", dataType = "Int", example = "100"), @ApiImplicitParam(name = "processInstancePriority", value = "PROCESS_INSTANCE_PRIORITY", type = "Priority"), }) @@ -147,26 +157,24 @@ public class SchedulerController extends BaseController { @RequestParam(value = "warningType", required = false, defaultValue = DEFAULT_WARNING_TYPE) WarningType warningType, @RequestParam(value = "warningGroupId", required = false) int warningGroupId, @RequestParam(value = "failureStrategy", required = false, defaultValue = "END") FailureStrategy failureStrategy, - @RequestParam(value = "receivers", required = false) String receivers, - @RequestParam(value = "receiversCc", required = false) String receiversCc, @RequestParam(value = "workerGroup", required = false, defaultValue = "default") String workerGroup, - @RequestParam(value = "processInstancePriority", required = false) Priority processInstancePriority) throws IOException { - logger.info("login user {}, project name: {},id: {}, updateProcessInstance schedule: {}, notify type: {}, notify mails: {}, " + - "failure policy: {},receivers : {},receiversCc : {},processInstancePriority : {},workerGroupId:{}", - loginUser.getUserName(), projectName, id, schedule, warningType, warningGroupId, failureStrategy, - receivers, receiversCc, processInstancePriority, workerGroup); + @RequestParam(value = "processInstancePriority", required = false) Priority processInstancePriority) { + logger.info("login user {},project name: {},id: {}, updateProcessInstance schedule: {}, notify type: {}, notify mails: {}, " + + "failure policy: {},processInstancePriority : {},workerGroupId:{}", + StringUtils.replaceNRTtoUnderline(loginUser.getUserName()), StringUtils.replaceNRTtoUnderline(projectName), id, schedule, warningType, warningGroupId, failureStrategy, + processInstancePriority, workerGroup); Map result = schedulerService.updateSchedule(loginUser, projectName, id, schedule, - warningType, warningGroupId, failureStrategy, receivers, receiversCc, null, processInstancePriority, workerGroup); + warningType, warningGroupId, failureStrategy, null, processInstancePriority, workerGroup); return returnDataList(result); } /** * publish schedule setScheduleState * - * @param loginUser login user + * @param loginUser login user * @param projectName project name - * @param id scheduler id + * @param id scheduler id * @return publish result code */ @ApiOperation(value = "online", notes = "ONLINE_SCHEDULE_NOTES") @@ -187,9 +195,9 @@ public class SchedulerController extends BaseController { /** * offline schedule * - * @param loginUser login user + * @param loginUser login user * @param projectName project name - * @param id schedule id + * @param id schedule id * @return operation result code */ @ApiOperation(value = "offline", notes = "OFFLINE_SCHEDULE_NOTES") @@ -211,12 +219,12 @@ public class SchedulerController extends BaseController { /** * query schedule list paging * - * @param loginUser login user - * @param projectName project name + * @param loginUser login user + * @param projectName project name * @param processDefinitionId process definition id - * @param pageNo page number - * @param pageSize page size - * @param searchVal search value + * @param pageNo page number + * @param pageSize page size + * @param searchVal search value * @return schedule list page */ @ApiOperation(value = "queryScheduleListPaging", notes = "QUERY_SCHEDULE_LIST_PAGING_NOTES") @@ -245,9 +253,9 @@ public class SchedulerController extends BaseController { /** * delete schedule by id * - * @param loginUser login user + * @param loginUser login user * @param projectName project name - * @param scheduleId scheule id + * @param scheduleId scheule id * @return delete result code */ @ApiOperation(value = "deleteScheduleById", notes = "OFFLINE_SCHEDULE_NOTES") @@ -270,7 +278,7 @@ public class SchedulerController extends BaseController { /** * query schedule list * - * @param loginUser login user + * @param loginUser login user * @param projectName project name * @return schedule list */ @@ -288,9 +296,9 @@ public class SchedulerController extends BaseController { /** * preview schedule * - * @param loginUser login user + * @param loginUser login user * @param projectName project name - * @param schedule schedule expression + * @param schedule schedule expression * @return the next five fire time */ @ApiOperation(value = "previewSchedule", notes = "PREVIEW_SCHEDULE_NOTES") diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UiPluginController.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UiPluginController.java new file mode 100644 index 0000000000..3589bdcdc0 --- /dev/null +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UiPluginController.java @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.controller; + +import static org.apache.dolphinscheduler.api.enums.Status.QUERY_PLUGINS_ERROR; + +import org.apache.dolphinscheduler.api.exceptions.ApiException; +import org.apache.dolphinscheduler.api.service.UiPluginService; +import org.apache.dolphinscheduler.api.utils.Result; +import org.apache.dolphinscheduler.common.Constants; +import org.apache.dolphinscheduler.common.enums.PluginType; +import org.apache.dolphinscheduler.dao.entity.User; + +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.http.HttpStatus; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestAttribute; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.ResponseStatus; +import org.springframework.web.bind.annotation.RestController; + +import io.swagger.annotations.Api; +import io.swagger.annotations.ApiImplicitParam; +import io.swagger.annotations.ApiImplicitParams; +import io.swagger.annotations.ApiOperation; +import springfox.documentation.annotations.ApiIgnore; + +/** + * UiPluginController + * Some plugins (such as alert plugin) need to provide UI interfaces to users. + * We use from-creat to dynamically generate UI interfaces. Related parameters are mainly provided by pluginParams. + * From-create can generate dynamic ui based on this parameter. + */ +@Api(tags = "UI_PLUGINS", position = 1) +@RestController +@RequestMapping("ui-plugins") +public class UiPluginController extends BaseController { + + private static final Logger logger = LoggerFactory.getLogger(UiPluginController.class); + + @Autowired + UiPluginService uiPluginService; + + @ApiOperation(value = "queryUiPluginsByType", notes = "QUERY_UI_PLUGINS_BY_TYPE") + @ApiImplicitParams({ + @ApiImplicitParam(name = "pluginType", value = "pluginType", required = true, dataType = "PluginType"), + }) + @PostMapping(value = "/queryUiPluginsByType") + @ResponseStatus(HttpStatus.CREATED) + @ApiException(QUERY_PLUGINS_ERROR) + public Result queryUiPluginsByType(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam(value = "pluginType") PluginType pluginType) { + + logger.info("query plugins by type , pluginType: {}", pluginType); + Map result = uiPluginService.queryUiPluginsByType(pluginType); + return returnDataList(result); + } + + @ApiOperation(value = "queryUiPluginDetailById", notes = "QUERY_UI_PLUGIN_DETAIL_BY_ID") + @ApiImplicitParams({ + @ApiImplicitParam(name = "id", value = "id", required = true, dataType = "PluginType"), + }) + @PostMapping(value = "/queryUiPluginDetailById") + @ResponseStatus(HttpStatus.CREATED) + @ApiException(QUERY_PLUGINS_ERROR) + public Result queryUiPluginDetailById(@ApiIgnore @RequestAttribute(value = Constants.SESSION_USER) User loginUser, + @RequestParam("pluginId") Integer pluginId) { + + logger.info("query plugin detail by id , pluginId: {}", pluginId); + Map result = uiPluginService.queryUiPluginDetailById(pluginId); + return returnDataList(result); + } +} diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java index c56f7d0b95..04ef02303d 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java @@ -276,13 +276,28 @@ public enum Status { QUEUE_COUNT_ERROR(90001, "queue count error", "查询队列数据错误"), KERBEROS_STARTUP_STATE(100001, "get kerberos startup state error", "获取kerberos启动状态错误"), + + //plugin + PLUGIN_NOT_A_UI_COMPONENT(110001, "query plugin error, this plugin has no UI component", "查询插件错误，此插件无UI组件"), + QUERY_PLUGINS_RESULT_IS_NULL(110002, "query plugins result is null", "查询插件为空"), + QUERY_PLUGINS_ERROR(110003, "query plugins error", "查询插件错误"), + QUERY_PLUGIN_DETAIL_RESULT_IS_NULL(110004, "query plugin detail result is null", "查询插件详情结果为空"), + + UPDATE_ALERT_PLUGIN_INSTANCE_ERROR(110005, "update alert plugin instance error", "更新告警组和告警组插件实例错误"), + DELETE_ALERT_PLUGIN_INSTANCE_ERROR(110006, "delete alert plugin instance error", "删除告警组和告警组插件实例错误"), + GET_ALERT_PLUGIN_INSTANCE_ERROR(110007, "get alert plugin instance error", "获取告警组和告警组插件实例错误"), + CREATE_ALERT_PLUGIN_INSTANCE_ERROR(110008, "create alert plugin instance error", "创建告警组和告警组插件实例错误"), + QUERY_ALL_ALERT_PLUGIN_INSTANCE_ERROR(110009, "query all alert plugin instance error", "查询所有告警实例失败"), + PLUGIN_INSTANCE_ALREADY_EXIT(110010,"plugin instance already exit","该告警插件实例已存在"), + LIST_PAGING_ALERT_PLUGIN_INSTANCE_ERROR(110011,"query plugin instance page error","分页查询告警实例失败"), + ; private final int code; private final String enMsg; private final String zhMsg; - private Status(int code, String enMsg, String zhMsg) { + Status(int code, String enMsg, String zhMsg) { this.code = code; this.enMsg = enMsg; this.zhMsg = zhMsg; diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertGroupService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertGroupService.java index e9a90d5903..e1e63a623c 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertGroupService.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertGroupService.java @@ -14,9 +14,9 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.api.service; -import java.util.*; import org.apache.dolphinscheduler.api.enums.Status; import org.apache.dolphinscheduler.api.utils.PageInfo; import org.apache.dolphinscheduler.common.Constants; @@ -25,29 +25,33 @@ import org.apache.dolphinscheduler.common.utils.CollectionUtils; import org.apache.dolphinscheduler.common.utils.StringUtils; import org.apache.dolphinscheduler.dao.entity.AlertGroup; import org.apache.dolphinscheduler.dao.entity.User; -import org.apache.dolphinscheduler.dao.entity.UserAlertGroup; import org.apache.dolphinscheduler.dao.mapper.AlertGroupMapper; -import com.baomidou.mybatisplus.core.metadata.IPage; -import com.baomidou.mybatisplus.extension.plugins.pagination.Page; + +import java.util.Date; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; +import com.baomidou.mybatisplus.core.metadata.IPage; +import com.baomidou.mybatisplus.extension.plugins.pagination.Page; + /** * alert group service */ @Service -public class AlertGroupService extends BaseService{ +public class AlertGroupService extends BaseService { private static final Logger logger = LoggerFactory.getLogger(AlertGroupService.class); @Autowired private AlertGroupMapper alertGroupMapper; - @Autowired - private UserAlertGroupService userAlertGroupService; /** * query alert group list * @@ -81,7 +85,7 @@ public class AlertGroupService extends BaseService{ Page page = new Page(pageNo, pageSize); IPage alertGroupIPage = alertGroupMapper.queryAlertGroupPage( - page, searchVal); + page, searchVal); PageInfo pageInfo = new PageInfo<>(pageNo, pageSize); pageInfo.setTotalCount((int) alertGroupIPage.getTotal()); pageInfo.setLists(alertGroupIPage.getRecords()); @@ -96,11 +100,11 @@ public class AlertGroupService extends BaseService{ * * @param loginUser login user * @param groupName group name - * @param groupType group type * @param desc description + * @param alertInstanceIds alertInstanceIds * @return create result code */ - public Map createAlertgroup(User loginUser, String groupName, AlertType groupType, String desc) { + public Map createAlertgroup(User loginUser, String groupName, String desc, String alertInstanceIds) { Map result = new HashMap<>(); //only admin can operate if (isNotAdmin(loginUser, result)) { @@ -111,10 +115,11 @@ public class AlertGroupService extends BaseService{ Date now = new Date(); alertGroup.setGroupName(groupName); - alertGroup.setGroupType(groupType); + alertGroup.setAlertInstanceIds(alertInstanceIds); alertGroup.setDescription(desc); alertGroup.setCreateTime(now); alertGroup.setUpdateTime(now); + alertGroup.setCreateUserId(loginUser.getId()); // insert int insert = alertGroupMapper.insert(alertGroup); @@ -133,18 +138,17 @@ public class AlertGroupService extends BaseService{ * @param loginUser login user * @param id alert group id * @param groupName group name - * @param groupType group type * @param desc description + * @param alertInstanceIds alertInstanceIds * @return update result code */ - public Map updateAlertgroup(User loginUser, int id, String groupName, AlertType groupType, String desc) { + public Map updateAlertgroup(User loginUser, int id, String groupName, String desc, String alertInstanceIds) { Map result = new HashMap<>(); if (isNotAdmin(loginUser, result)) { return result; } - AlertGroup alertGroup = alertGroupMapper.selectById(id); if (alertGroup == null) { @@ -158,12 +162,10 @@ public class AlertGroupService extends BaseService{ if (StringUtils.isNotEmpty(groupName)) { alertGroup.setGroupName(groupName); } - - if (groupType != null) { - alertGroup.setGroupType(groupType); - } alertGroup.setDescription(desc); alertGroup.setUpdateTime(now); + alertGroup.setCreateUserId(loginUser.getId()); + alertGroup.setAlertInstanceIds(alertInstanceIds); // updateProcessInstance alertGroupMapper.updateById(alertGroup); putMsg(result, Status.SUCCESS); @@ -192,57 +194,11 @@ public class AlertGroupService extends BaseService{ putMsg(result, Status.ALERT_GROUP_NOT_EXIST); return result; } - - userAlertGroupService.deleteByAlertGroupId(id); alertGroupMapper.deleteById(id); putMsg(result, Status.SUCCESS); return result; } - - /** - * grant user - * - * @param loginUser login user - * @param alertgroupId alert group id - * @param userIds user id list - * @return grant result code - */ - public Map grantUser(User loginUser, int alertgroupId, String userIds) { - Map result = new HashMap<>(); - result.put(Constants.STATUS, false); - - //only admin can operate - if (isNotAdmin(loginUser, result)) { - return result; - } - - userAlertGroupService.deleteByAlertGroupId(alertgroupId); - if (StringUtils.isEmpty(userIds)) { - putMsg(result, Status.SUCCESS); - return result; - } - - String[] userIdsArr = userIds.split(","); - Date now = new Date(); - List alertGroups = new ArrayList<>(userIds.length()); - for (String userId : userIdsArr) { - UserAlertGroup userAlertGroup = new UserAlertGroup(); - userAlertGroup.setAlertgroupId(alertgroupId); - userAlertGroup.setUserId(Integer.parseInt(userId)); - userAlertGroup.setCreateTime(now); - userAlertGroup.setUpdateTime(now); - alertGroups.add(userAlertGroup); - } - - if (CollectionUtils.isNotEmpty(alertGroups)) { - userAlertGroupService.saveBatch(alertGroups); - } - - putMsg(result, Status.SUCCESS); - return result; - } - /** * verify group name exists * diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertPluginInstanceService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertPluginInstanceService.java new file mode 100644 index 0000000000..d526a41d99 --- /dev/null +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/AlertPluginInstanceService.java @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.service; + +import org.apache.dolphinscheduler.dao.entity.User; + +import java.util.Map; + +/** + * alert plugin instance service + */ +public interface AlertPluginInstanceService { + + /** + * creat alert plugin instance + * + * @param loginUser login user + * @param pluginDefineId plugin define id + * @param instanceName instance name + * @param pluginInstanceParams plugin instance params + * @return result + */ + Map create(User loginUser,int pluginDefineId,String instanceName,String pluginInstanceParams); + + /** + * update + * @param loginUser login user + * @param alertPluginInstanceId plugin instance id + * @param instanceName instance name + * @param pluginInstanceParams plugin instance params + * @return result + */ + Map update(User loginUser, int alertPluginInstanceId,String instanceName,String pluginInstanceParams); + + /** + * delete alert plugin instance + * + * @param loginUser login user + * @param id id + * @return result + */ + Map delete(User loginUser, int id); + + /** + * get alert plugin instance + * + * @param loginUser login user + * @param id get id + * @return alert plugin + */ + Map get(User loginUser, int id); + + /** + * queryAll + * + * @return alert plugins + */ + Map queryAll(); + + /** + * checkExistPluginInstanceName + * @param pluginName plugin name + * @return isExist + */ + boolean checkExistPluginInstanceName(String pluginName); + + /** + * queryPluginPage + * @param pageIndex page index + * @param pageSize page size + * @return plugins + */ + Map queryPluginPage(int pageIndex,int pageSize); +} diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java index f53d9026f9..77be0a048c 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java @@ -107,8 +107,6 @@ public class ExecutorService extends BaseService { * @param taskDependType node dependency type * @param warningType warning type * @param warningGroupId notify group id - * @param receivers receivers - * @param receiversCc receivers cc * @param processInstancePriority process instance priority * @param workerGroup worker group name * @param runMode run mode @@ -121,7 +119,7 @@ public class ExecutorService extends BaseService { int processDefinitionId, String cronTime, CommandType commandType, FailureStrategy failureStrategy, String startNodeList, TaskDependType taskDependType, WarningType warningType, int warningGroupId, - String receivers, String receiversCc, RunMode runMode, + RunMode runMode, Priority processInstancePriority, String workerGroup, Integer timeout, Map startParams) throws ParseException { Map result = new HashMap<>(); @@ -161,12 +159,9 @@ public class ExecutorService extends BaseService { int create = this.createCommand(commandType, processDefinitionId, taskDependType, failureStrategy, startNodeList, cronTime, warningType, loginUser.getId(), warningGroupId, runMode, processInstancePriority, workerGroup, startParams); + if (create > 0) { - /** - * according to the process definition ID updateProcessInstance and CC recipient - */ - processDefinition.setReceivers(receivers); - processDefinition.setReceiversCc(receiversCc); + processDefinition.setWarningGroupId(warningGroupId); processDefinitionMapper.updateById(processDefinition); putMsg(result, Status.SUCCESS); } else { @@ -448,42 +443,6 @@ public class ExecutorService extends BaseService { return result; } - /** - * query recipients and copyers by process definition id or processInstanceId - * - * @param processDefineId process definition id - * @param processInstanceId process instance id - * @return receivers cc list - */ - public Map getReceiverCc(Integer processDefineId, Integer processInstanceId) { - Map result = new HashMap<>(); - logger.info("processInstanceId {}", processInstanceId); - if (processDefineId == null && processInstanceId == null) { - throw new RuntimeException("You must set values for parameters processDefineId or processInstanceId"); - } - if (processDefineId == null && processInstanceId != null) { - ProcessInstance processInstance = processInstanceMapper.selectById(processInstanceId); - if (processInstance == null) { - throw new RuntimeException("processInstanceId is not exists"); - } - processDefineId = processInstance.getProcessDefinitionId(); - } - ProcessDefinition processDefinition = processDefinitionMapper.selectById(processDefineId); - if (processDefinition == null) { - throw new RuntimeException(String.format("processDefineId %d is not exists", processDefineId)); - } - - String receivers = processDefinition.getReceivers(); - String receiversCc = processDefinition.getReceiversCc(); - Map dataMap = new HashMap<>(); - dataMap.put(Constants.RECEIVERS, receivers); - dataMap.put(Constants.RECEIVERS_CC, receiversCc); - - result.put(Constants.DATA_LIST, dataMap); - putMsg(result, Status.SUCCESS); - return result; - } - /** * create command * diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java index 3f55fae66f..1d6169b06e 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java @@ -189,8 +189,7 @@ public class ProcessInstanceService extends BaseService { ProcessInstance processInstance = processService.findProcessInstanceDetailById(processId); ProcessDefinition processDefinition = processService.findProcessDefineById(processInstance.getProcessDefinitionId()); - processInstance.setReceivers(processDefinition.getReceivers()); - processInstance.setReceiversCc(processDefinition.getReceiversCc()); + processInstance.setWarningGroupId(processDefinition.getWarningGroupId()); result.put(DATA_LIST, processInstance); putMsg(result, Status.SUCCESS); diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java index 93fa14872a..55880ad63c 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java @@ -14,18 +14,22 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.api.service; +package org.apache.dolphinscheduler.api.service; import org.apache.dolphinscheduler.api.dto.ScheduleParam; import org.apache.dolphinscheduler.api.enums.Status; import org.apache.dolphinscheduler.api.exceptions.ServiceException; import org.apache.dolphinscheduler.api.utils.PageInfo; import org.apache.dolphinscheduler.common.Constants; -import org.apache.dolphinscheduler.common.enums.*; +import org.apache.dolphinscheduler.common.enums.FailureStrategy; +import org.apache.dolphinscheduler.common.enums.Priority; +import org.apache.dolphinscheduler.common.enums.ReleaseState; +import org.apache.dolphinscheduler.common.enums.UserType; +import org.apache.dolphinscheduler.common.enums.WarningType; import org.apache.dolphinscheduler.common.model.Server; import org.apache.dolphinscheduler.common.utils.DateUtils; -import org.apache.dolphinscheduler.common.utils.*; +import org.apache.dolphinscheduler.common.utils.JSONUtils; import org.apache.dolphinscheduler.common.utils.StringUtils; import org.apache.dolphinscheduler.dao.entity.ProcessDefinition; import org.apache.dolphinscheduler.dao.entity.Project; @@ -34,12 +38,18 @@ import org.apache.dolphinscheduler.dao.entity.User; import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper; import org.apache.dolphinscheduler.dao.mapper.ProjectMapper; import org.apache.dolphinscheduler.dao.mapper.ScheduleMapper; -import com.baomidou.mybatisplus.core.metadata.IPage; -import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import org.apache.dolphinscheduler.service.process.ProcessService; import org.apache.dolphinscheduler.service.quartz.ProcessScheduleJob; import org.apache.dolphinscheduler.service.quartz.QuartzExecutors; import org.apache.dolphinscheduler.service.quartz.cron.CronUtils; + +import java.text.ParseException; +import java.util.ArrayList; +import java.util.Date; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + import org.quartz.CronExpression; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -47,8 +57,8 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; -import java.text.ParseException; -import java.util.*; +import com.baomidou.mybatisplus.core.metadata.IPage; +import com.baomidou.mybatisplus.extension.plugins.pagination.Page; /** * scheduler service @@ -86,12 +96,10 @@ public class SchedulerService extends BaseService { * @param projectName project name * @param processDefineId process definition id * @param schedule scheduler - * @param warningType warning type + * @param warningType warning type * @param warningGroupId warning group id * @param failureStrategy failure strategy * @param processInstancePriority process instance priority - * @param receivers receivers - * @param receiversCc receivers cc * @param workerGroup worker group * @return create result code */ @@ -102,12 +110,10 @@ public class SchedulerService extends BaseService { WarningType warningType, int warningGroupId, FailureStrategy failureStrategy, - String receivers, - String receiversCc, Priority processInstancePriority, String workerGroup) { - Map result = new HashMap(5); + Map result = new HashMap(); Project project = projectMapper.queryByName(projectName); @@ -132,9 +138,9 @@ public class SchedulerService extends BaseService { scheduleObj.setProcessDefinitionName(processDefinition.getName()); ScheduleParam scheduleParam = JSONUtils.parseObject(schedule, ScheduleParam.class); - if (DateUtils.differSec(scheduleParam.getStartTime(),scheduleParam.getEndTime()) == 0) { + if (DateUtils.differSec(scheduleParam.getStartTime(), scheduleParam.getEndTime()) == 0) { logger.warn("The start time must not be the same as the end"); - putMsg(result,Status.SCHEDULE_START_TIME_END_TIME_SAME); + putMsg(result, Status.SCHEDULE_START_TIME_END_TIME_SAME); return result; } scheduleObj.setStartTime(scheduleParam.getStartTime()); @@ -161,8 +167,7 @@ public class SchedulerService extends BaseService { /** * updateProcessInstance receivers and cc by process definition id */ - processDefinition.setReceivers(receivers); - processDefinition.setReceiversCc(receiversCc); + processDefinition.setWarningGroupId(warningGroupId); processDefinitionMapper.updateById(processDefinition); // return scheduler object with ID @@ -173,7 +178,6 @@ public class SchedulerService extends BaseService { return result; } - /** * updateProcessInstance schedule * @@ -186,8 +190,6 @@ public class SchedulerService extends BaseService { * @param failureStrategy failure strategy * @param workerGroup worker group * @param processInstancePriority process instance priority - * @param receiversCc receiver cc - * @param receivers receivers * @param scheduleStatus schedule status * @return update result code */ @@ -199,8 +201,6 @@ public class SchedulerService extends BaseService { WarningType warningType, int warningGroupId, FailureStrategy failureStrategy, - String receivers, - String receiversCc, ReleaseState scheduleStatus, Priority processInstancePriority, String workerGroup) { @@ -240,9 +240,9 @@ public class SchedulerService extends BaseService { // updateProcessInstance param if (StringUtils.isNotEmpty(scheduleExpression)) { ScheduleParam scheduleParam = JSONUtils.parseObject(scheduleExpression, ScheduleParam.class); - if (DateUtils.differSec(scheduleParam.getStartTime(),scheduleParam.getEndTime()) == 0) { + if (DateUtils.differSec(scheduleParam.getStartTime(), scheduleParam.getEndTime()) == 0) { logger.warn("The start time must not be the same as the end"); - putMsg(result,Status.SCHEDULE_START_TIME_END_TIME_SAME); + putMsg(result, Status.SCHEDULE_START_TIME_END_TIME_SAME); return result; } schedule.setStartTime(scheduleParam.getStartTime()); @@ -275,8 +275,8 @@ public class SchedulerService extends BaseService { /** * updateProcessInstance recipients and cc by process definition ID */ - processDefinition.setReceivers(receivers); - processDefinition.setReceiversCc(receiversCc); + processDefinition.setWarningGroupId(warningGroupId); + processDefinitionMapper.updateById(processDefinition); putMsg(result, Status.SUCCESS); @@ -290,7 +290,7 @@ public class SchedulerService extends BaseService { * @param loginUser login user * @param projectName project name * @param id scheduler id - * @param scheduleStatus schedule status + * @param scheduleStatus schedule status * @return publish result code */ @Transactional(rollbackFor = RuntimeException.class) @@ -316,7 +316,7 @@ public class SchedulerService extends BaseService { return result; } // check schedule release state - if(scheduleObj.getReleaseState() == scheduleStatus){ + if (scheduleObj.getReleaseState() == scheduleStatus) { logger.info("schedule release is already {},needn't to change schedule id: {} from {} to {}", scheduleObj.getReleaseState(), scheduleObj.getId(), scheduleObj.getReleaseState(), scheduleStatus); putMsg(result, Status.SCHEDULE_CRON_REALEASE_NEED_NOT_CHANGE, scheduleStatus); @@ -328,9 +328,9 @@ public class SchedulerService extends BaseService { return result; } - if(scheduleStatus == ReleaseState.ONLINE){ + if (scheduleStatus == ReleaseState.ONLINE) { // check process definition release state - if(processDefinition.getReleaseState() != ReleaseState.ONLINE){ + if (processDefinition.getReleaseState() != ReleaseState.ONLINE) { logger.info("not release process definition id: {} , name : {}", processDefinition.getId(), processDefinition.getName()); putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE, processDefinition.getName()); @@ -340,15 +340,15 @@ public class SchedulerService extends BaseService { List subProcessDefineIds = new ArrayList<>(); processService.recurseFindSubProcessId(scheduleObj.getProcessDefinitionId(), subProcessDefineIds); Integer[] idArray = subProcessDefineIds.toArray(new Integer[subProcessDefineIds.size()]); - if (subProcessDefineIds.size() > 0){ + if (subProcessDefineIds.size() > 0) { List subProcessDefinitionList = processDefinitionMapper.queryDefinitionListByIdList(idArray); - if (subProcessDefinitionList != null && subProcessDefinitionList.size() > 0){ - for (ProcessDefinition subProcessDefinition : subProcessDefinitionList){ + if (subProcessDefinitionList != null && subProcessDefinitionList.size() > 0) { + for (ProcessDefinition subProcessDefinition : subProcessDefinitionList) { /** * if there is no online process, exit directly */ - if (subProcessDefinition.getReleaseState() != ReleaseState.ONLINE){ + if (subProcessDefinition.getReleaseState() != ReleaseState.ONLINE) { logger.info("not release process definition id: {} , name : {}", subProcessDefinition.getId(), subProcessDefinition.getName()); putMsg(result, Status.PROCESS_DEFINE_NOT_RELEASE, subProcessDefinition.getId()); @@ -362,7 +362,6 @@ public class SchedulerService extends BaseService { // check master server exists List masterServers = monitorService.getServerListFromZK(true); - if (masterServers.size() == 0) { putMsg(result, Status.MASTER_NOT_EXISTS); return result; @@ -399,8 +398,6 @@ public class SchedulerService extends BaseService { return result; } - - /** * query schedule * @@ -408,7 +405,7 @@ public class SchedulerService extends BaseService { * @param projectName project name * @param processDefineId process definition id * @param pageNo page number - * @param pageSize page size + * @param pageSize page size * @param searchVal search value * @return schedule list page */ @@ -434,9 +431,8 @@ public class SchedulerService extends BaseService { page, processDefineId, searchVal ); - PageInfo pageInfo = new PageInfo(pageNo, pageSize); - pageInfo.setTotalCount((int)scheduleIPage.getTotal()); + pageInfo.setTotalCount((int) scheduleIPage.getTotal()); pageInfo.setLists(scheduleIPage.getRecords()); result.put(Constants.DATA_LIST, pageInfo); putMsg(result, Status.SUCCESS); @@ -500,8 +496,8 @@ public class SchedulerService extends BaseService { String jobName = QuartzExecutors.buildJobName(scheduleId); String jobGroupName = QuartzExecutors.buildJobGroupName(projectId); - if(!QuartzExecutors.getInstance().deleteJob(jobName, jobGroupName)){ - logger.warn("set offline failure:projectId:{},scheduleId:{}",projectId,scheduleId); + if (!QuartzExecutors.getInstance().deleteJob(jobName, jobGroupName)) { + logger.warn("set offline failure:projectId:{},scheduleId:{}", projectId, scheduleId); throw new ServiceException("set offline failure"); } @@ -551,19 +547,18 @@ public class SchedulerService extends BaseService { } // Determine if the login user is the owner of the schedule - if (loginUser.getId() != schedule.getUserId() && - loginUser.getUserType() != UserType.ADMIN_USER) { + if (loginUser.getId() != schedule.getUserId() + && loginUser.getUserType() != UserType.ADMIN_USER) { putMsg(result, Status.USER_NO_OPERATION_PERM); return result; } // check schedule is already online - if(schedule.getReleaseState() == ReleaseState.ONLINE){ - putMsg(result, Status.SCHEDULE_CRON_STATE_ONLINE,schedule.getId()); + if (schedule.getReleaseState() == ReleaseState.ONLINE) { + putMsg(result, Status.SCHEDULE_CRON_STATE_ONLINE, schedule.getId()); return result; } - int delete = scheduleMapper.deleteById(scheduleId); if (delete > 0) { @@ -582,7 +577,7 @@ public class SchedulerService extends BaseService { * @param schedule schedule expression * @return the next five fire time */ - public Map previewSchedule(User loginUser, String projectName, String schedule) { + public Map previewSchedule(User loginUser, String projectName, String schedule) { Map result = new HashMap<>(); CronExpression cronExpression; ScheduleParam scheduleParam = JSONUtils.parseObject(schedule, ScheduleParam.class); @@ -593,11 +588,11 @@ public class SchedulerService extends BaseService { try { cronExpression = CronUtils.parse2CronExpression(scheduleParam.getCrontab()); } catch (ParseException e) { - logger.error(e.getMessage(),e); - putMsg(result,Status.PARSE_TO_CRON_EXPRESSION_ERROR); + logger.error(e.getMessage(), e); + putMsg(result, Status.PARSE_TO_CRON_EXPRESSION_ERROR); return result; } - List selfFireDateList = CronUtils.getSelfFireDateList(startTime, endTime,cronExpression,Constants.PREVIEW_SCHEDULE_EXECUTE_COUNT); + List selfFireDateList = CronUtils.getSelfFireDateList(startTime, endTime, cronExpression, Constants.PREVIEW_SCHEDULE_EXECUTE_COUNT); result.put(Constants.DATA_LIST, selfFireDateList.stream().map(t -> DateUtils.dateToString(t))); putMsg(result, Status.SUCCESS); return result; diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginManager.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UiPluginService.java similarity index 74% rename from dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginManager.java rename to dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UiPluginService.java index f8078841e4..102d927673 100644 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginManager.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UiPluginService.java @@ -14,20 +14,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.common.plugin; -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; +package org.apache.dolphinscheduler.api.service; + +import org.apache.dolphinscheduler.common.enums.PluginType; import java.util.Map; /** - * PluginManager + * UiPluginService */ -public interface PluginManager { +public interface UiPluginService { - AlertPlugin findOne(String name); + Map queryUiPluginsByType(PluginType pluginType); - Map findAll(); + Map queryUiPluginDetailById(int id); - void addPlugin(AlertPlugin plugin); } diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UsersService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UsersService.java index 3fb5f64346..5fc5e1ee13 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UsersService.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UsersService.java @@ -131,7 +131,7 @@ public class UsersService extends BaseService { String queue, int state) throws Exception { - Map result = new HashMap<>(5); + Map result = new HashMap<>(); //check all user params String msg = this.checkUserParams(userName, userPassword, email, phone); @@ -773,7 +773,7 @@ public class UsersService extends BaseService { * @return user list */ public Map queryAllGeneralUsers(User loginUser) { - Map result = new HashMap<>(5); + Map result = new HashMap<>(); //only admin can operate if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) { return result; @@ -794,7 +794,7 @@ public class UsersService extends BaseService { * @return user list */ public Map queryUserList(User loginUser) { - Map result = new HashMap<>(5); + Map result = new HashMap<>(); //only admin can operate if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) { return result; @@ -838,7 +838,7 @@ public class UsersService extends BaseService { */ public Map unauthorizedUser(User loginUser, Integer alertgroupId) { - Map result = new HashMap<>(5); + Map result = new HashMap<>(); //only admin can operate if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) { return result; @@ -874,7 +874,7 @@ public class UsersService extends BaseService { * @return authorized result code */ public Map authorizedUser(User loginUser, Integer alertgroupId) { - Map result = new HashMap<>(5); + Map result = new HashMap<>(); //only admin can operate if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) { return result; diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/AlertPluginInstanceServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/AlertPluginInstanceServiceImpl.java new file mode 100644 index 0000000000..8004a90fae --- /dev/null +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/AlertPluginInstanceServiceImpl.java @@ -0,0 +1,208 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.service.impl; + +import org.apache.dolphinscheduler.api.enums.Status; +import org.apache.dolphinscheduler.api.service.AlertPluginInstanceService; +import org.apache.dolphinscheduler.api.service.BaseService; +import org.apache.dolphinscheduler.api.utils.PageInfo; +import org.apache.dolphinscheduler.api.vo.AlertPluginInstanceVO; +import org.apache.dolphinscheduler.common.Constants; +import org.apache.dolphinscheduler.common.utils.CollectionUtils; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.dao.entity.User; +import org.apache.dolphinscheduler.dao.mapper.AlertPluginInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Lazy; +import org.springframework.stereotype.Service; + +import com.baomidou.mybatisplus.core.metadata.IPage; +import com.baomidou.mybatisplus.extension.plugins.pagination.Page; + +/** + * alert plugin instance service impl + */ +@Service +@Lazy +public class AlertPluginInstanceServiceImpl extends BaseService implements AlertPluginInstanceService { + + @Autowired + private AlertPluginInstanceMapper alertPluginInstanceMapper; + + @Autowired + private PluginDefineMapper pluginDefineMapper; + + /** + * creat alert plugin instance + * + * @param loginUser login user + * @param pluginDefineId plugin define id + * @param instanceName instance name + * @param pluginInstanceParams plugin instance params + */ + @Override + public Map create(User loginUser, int pluginDefineId, String instanceName, String pluginInstanceParams) { + AlertPluginInstance alertPluginInstance = new AlertPluginInstance(); + alertPluginInstance.setPluginInstanceParams(pluginInstanceParams); + alertPluginInstance.setInstanceName(instanceName); + alertPluginInstance.setPluginDefineId(pluginDefineId); + + Map result = new HashMap<>(); + + if (CollectionUtils.isNotEmpty(alertPluginInstanceMapper.queryByInstanceName(alertPluginInstance.getInstanceName()))) { + putMsg(result, Status.PLUGIN_INSTANCE_ALREADY_EXIT); + return result; + } + + int i = alertPluginInstanceMapper.insert(alertPluginInstance); + + if (i > 0) { + putMsg(result, Status.SUCCESS); + } + return result; + } + + /** + * update alert plugin instance + * + * @param loginUser login user + * @param pluginInstanceId plugin instance id + * @param instanceName instance name + * @param pluginInstanceParams plugin instance params + */ + @Override + public Map update(User loginUser, int pluginInstanceId, String instanceName, String pluginInstanceParams) { + + AlertPluginInstance alertPluginInstance = new AlertPluginInstance(); + alertPluginInstance.setPluginInstanceParams(pluginInstanceParams); + alertPluginInstance.setInstanceName(instanceName); + alertPluginInstance.setId(pluginInstanceId); + Map result = new HashMap<>(); + int i = alertPluginInstanceMapper.updateById(alertPluginInstance); + + if (i > 0) { + putMsg(result, Status.SUCCESS); + } + + return result; + } + + /** + * delete alert plugin instance + * + * @param loginUser login user + * @param id id + * @return result + */ + @Override + public Map delete(User loginUser, int id) { + Map result = new HashMap<>(); + int i = alertPluginInstanceMapper.deleteById(id); + if (i > 0) { + putMsg(result, Status.SUCCESS); + } + + return result; + } + + /** + * get alert plugin instance + * + * @param loginUser login user + * @param id get id + * @return alert plugin + */ + @Override + public Map get(User loginUser, int id) { + Map result = new HashMap<>(); + AlertPluginInstance alertPluginInstance = alertPluginInstanceMapper.selectById(id); + + if (null != alertPluginInstance) { + putMsg(result, Status.SUCCESS); + result.put(Constants.DATA_LIST, alertPluginInstance); + } + + return result; + } + + @Override + public Map queryAll() { + Map result = new HashMap<>(); + List alertPluginInstances = alertPluginInstanceMapper.queryAllAlertPluginInstanceList(); + List alertPluginInstanceVOS = buildPluginInstanceVOList(alertPluginInstances); + if (null != alertPluginInstances) { + putMsg(result, Status.SUCCESS); + result.put(Constants.DATA_LIST, alertPluginInstanceVOS); + } + return result; + } + + @Override + public boolean checkExistPluginInstanceName(String pluginInstanceName) { + return CollectionUtils.isNotEmpty(alertPluginInstanceMapper.queryByInstanceName(pluginInstanceName)); + } + + @Override + public Map queryPluginPage(int pageIndex, int pageSize) { + IPage pluginInstanceIPage = new Page<>(pageIndex, pageSize); + pluginInstanceIPage = alertPluginInstanceMapper.selectPage(pluginInstanceIPage, null); + + PageInfo pageInfo = new PageInfo<>(pageIndex, pageSize); + pageInfo.setTotalCount((int) pluginInstanceIPage.getTotal()); + pageInfo.setLists(buildPluginInstanceVOList(pluginInstanceIPage.getRecords())); + Map result = new HashMap<>(); + result.put(Constants.DATA_LIST, pageInfo); + putMsg(result, Status.SUCCESS); + return result; + } + + private List buildPluginInstanceVOList(List alertPluginInstances) { + if (CollectionUtils.isEmpty(alertPluginInstances)) { + return null; + } + List pluginDefineList = pluginDefineMapper.queryAllPluginDefineList(); + if (CollectionUtils.isEmpty(pluginDefineList)) { + return null; + } + Map pluginDefineMap = pluginDefineList.stream().collect(Collectors.toMap(PluginDefine::getId, PluginDefine::getPluginName)); + List alertPluginInstanceVOS = new ArrayList<>(); + alertPluginInstances.forEach(alertPluginInstance -> { + AlertPluginInstanceVO alertPluginInstanceVO = new AlertPluginInstanceVO(); + alertPluginInstanceVO.setAlertPluginName(pluginDefineMap.get(alertPluginInstance.getPluginDefineId())); + alertPluginInstanceVO.setCreateTime(alertPluginInstance.getCreateTime()); + alertPluginInstanceVO.setUpdateTime(alertPluginInstance.getUpdateTime()); + alertPluginInstanceVO.setPluginDefineId(alertPluginInstance.getPluginDefineId()); + alertPluginInstanceVO.setInstanceName(alertPluginInstance.getInstanceName()); + alertPluginInstanceVO.setId(alertPluginInstance.getId()); + //todo List pages do not recommend returning this parameter + alertPluginInstanceVO.setPluginInstanceParams(alertPluginInstance.getPluginInstanceParams()); + alertPluginInstanceVOS.add(alertPluginInstanceVO); + }); + return alertPluginInstanceVOS; + + } +} diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java index 52385dd00f..92aa1f5f69 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java @@ -157,7 +157,6 @@ public class ProcessDefinitionServiceImpl extends BaseService implements * @param locations locations for nodes * @param connects connects for nodes * @return create result code - * @throws JsonProcessingException JsonProcessingException */ @Override public Map createProcessDefinition(User loginUser, @@ -1130,8 +1129,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements processDefine.setCreateTime(now); processDefine.setUpdateTime(now); processDefine.setFlag(subProcess.getFlag()); - processDefine.setReceivers(subProcess.getReceivers()); - processDefine.setReceiversCc(subProcess.getReceiversCc()); + processDefine.setWarningGroupId(subProcess.getWarningGroupId()); processDefineMapper.insert(processDefine); logger.info("create sub process, project: {}, process name: {}", targetProject.getName(), processDefine.getName()); @@ -1659,8 +1657,7 @@ public class ProcessDefinitionServiceImpl extends BaseService implements processDefinition.setTimeout(processDefinitionVersion.getTimeout()); processDefinition.setGlobalParams(processDefinitionVersion.getGlobalParams()); processDefinition.setUpdateTime(new Date()); - processDefinition.setReceivers(processDefinitionVersion.getReceivers()); - processDefinition.setReceiversCc(processDefinitionVersion.getReceiversCc()); + processDefinition.setWarningGroupId(processDefinitionVersion.getWarningGroupId()); processDefinition.setResourceIds(processDefinitionVersion.getResourceIds()); if (processDefineMapper.updateById(processDefinition) > 0) { diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java index 6364242190..08cedfee15 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java @@ -66,18 +66,17 @@ public class ProcessDefinitionVersionServiceImpl extends BaseService implements long version = this.queryMaxVersionByProcessDefinitionId(processDefinition.getId()) + 1; ProcessDefinitionVersion processDefinitionVersion = ProcessDefinitionVersion - .newBuilder() - .processDefinitionId(processDefinition.getId()) - .version(version) - .processDefinitionJson(processDefinition.getProcessDefinitionJson()) - .description(processDefinition.getDescription()) - .locations(processDefinition.getLocations()) - .connects(processDefinition.getConnects()) - .timeout(processDefinition.getTimeout()) - .globalParams(processDefinition.getGlobalParams()) - .createTime(processDefinition.getUpdateTime()) - .receivers(processDefinition.getReceivers()) - .receiversCc(processDefinition.getReceiversCc()) + .newBuilder() + .processDefinitionId(processDefinition.getId()) + .version(version) + .processDefinitionJson(processDefinition.getProcessDefinitionJson()) + .description(processDefinition.getDescription()) + .locations(processDefinition.getLocations()) + .connects(processDefinition.getConnects()) + .timeout(processDefinition.getTimeout()) + .globalParams(processDefinition.getGlobalParams()) + .createTime(processDefinition.getUpdateTime()) + .warningGroupId(processDefinition.getWarningGroupId()) .resourceIds(processDefinition.getResourceIds()) .build(); diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UiPluginServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UiPluginServiceImpl.java new file mode 100644 index 0000000000..0c3cb5dfdb --- /dev/null +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UiPluginServiceImpl.java @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.service.impl; + +import org.apache.dolphinscheduler.api.enums.Status; +import org.apache.dolphinscheduler.api.service.BaseService; +import org.apache.dolphinscheduler.api.service.UiPluginService; +import org.apache.dolphinscheduler.common.Constants; +import org.apache.dolphinscheduler.common.enums.PluginType; +import org.apache.dolphinscheduler.common.utils.CollectionUtils; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.stereotype.Service; + +/** + * UiPluginServiceImpl + */ +@Service +public class UiPluginServiceImpl extends BaseService implements UiPluginService { + + @Autowired + PluginDefineMapper pluginDefineMapper; + + private static final String LANGUAGE_REGEX = "\"([^\"]*)\""; + + private static final String LANGUAGE_SYMBOL = "$t"; + + private static final String ESCAPE_SYMBOL = "\\"; + + @Override + public Map queryUiPluginsByType(PluginType pluginType) { + Map result = new HashMap<>(); + if (!pluginType.getHasUi()) { + putMsg(result, Status.PLUGIN_NOT_A_UI_COMPONENT); + return result; + } + List pluginDefines = pluginDefineMapper.queryByPluginType(pluginType.getDesc()); + + if (CollectionUtils.isEmpty(pluginDefines)) { + putMsg(result, Status.QUERY_PLUGINS_RESULT_IS_NULL); + return result; + } + // pluginDefines=buildPluginParams(pluginDefines); + putMsg(result, Status.SUCCESS); + result.put(Constants.DATA_LIST, pluginDefines); + return result; + } + + @Override + public Map queryUiPluginDetailById(int id) { + Map result = new HashMap<>(); + PluginDefine pluginDefine = pluginDefineMapper.queryDetailById(id); + if (null == pluginDefine) { + putMsg(result, Status.QUERY_PLUGIN_DETAIL_RESULT_IS_NULL); + return result; + } + // String params=pluginDefine.getPluginParams(); + // pluginDefine.setPluginParams(parseParams(params)); + putMsg(result, Status.SUCCESS); + result.put(Constants.DATA_LIST, pluginDefine); + return result; + } + +} diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo/AlertPluginInstanceVO.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo/AlertPluginInstanceVO.java new file mode 100644 index 0000000000..d8b76f633d --- /dev/null +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo/AlertPluginInstanceVO.java @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.vo; + +import java.util.Date; + +/** + * AlertPluginInstanceVO + */ +public class AlertPluginInstanceVO { + + /** + * id + */ + private int id; + + /** + * plugin_define_id + */ + private int pluginDefineId; + + /** + * alert plugin instance name + */ + private String instanceName; + + /** + * plugin_instance_params + */ + private String pluginInstanceParams; + + /** + * create_time + */ + private Date createTime; + + /** + * update_time + */ + private Date updateTime; + + /** + * alert plugin name + */ + private String alertPluginName; + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public int getPluginDefineId() { + return pluginDefineId; + } + + public void setPluginDefineId(int pluginDefineId) { + this.pluginDefineId = pluginDefineId; + } + + public String getInstanceName() { + return instanceName; + } + + public void setInstanceName(String instanceName) { + this.instanceName = instanceName; + } + + public String getPluginInstanceParams() { + return pluginInstanceParams; + } + + public void setPluginInstanceParams(String pluginInstanceParams) { + this.pluginInstanceParams = pluginInstanceParams; + } + + public Date getCreateTime() { + return createTime; + } + + public void setCreateTime(Date createTime) { + this.createTime = createTime; + } + + public Date getUpdateTime() { + return updateTime; + } + + public void setUpdateTime(Date updateTime) { + this.updateTime = updateTime; + } + + public String getAlertPluginName() { + return alertPluginName; + } + + public void setAlertPluginName(String alertPluginName) { + this.alertPluginName = alertPluginName; + } +} diff --git a/dolphinscheduler-api/src/main/resources/i18n/messages.properties b/dolphinscheduler-api/src/main/resources/i18n/messages.properties index 5837cfa929..cae8e5fa59 100644 --- a/dolphinscheduler-api/src/main/resources/i18n/messages.properties +++ b/dolphinscheduler-api/src/main/resources/i18n/messages.properties @@ -38,6 +38,11 @@ VERIFY_ALERT_GROUP_NAME_NOTES=verify alert group name, check alert group exist o GRANT_ALERT_GROUP_NOTES=grant alert group USER_IDS=user id list ALERT_GROUP_TAG=alert group related operation +ALERT_PLUGIN_INSTANCE_TAG=alert plugin instance related operation +UPDATE_ALERT_PLUGIN_INSTANCE_NOTES=update alert plugin instance operation +CREATE_ALERT_PLUGIN_INSTANCE_NOTES=create alert plugin instance operation +DELETE_ALERT_PLUGIN_INSTANCE_NOTES=delete alert plugin instance operation +GET_ALERT_PLUGIN_INSTANCE_NOTES=get alert plugin instance operation CREATE_ALERT_GROUP_NOTES=create alert group WORKER_GROUP_TAG=worker group related operation SAVE_WORKER_GROUP_NOTES=create worker group diff --git a/dolphinscheduler-api/src/main/resources/i18n/messages_en_US.properties b/dolphinscheduler-api/src/main/resources/i18n/messages_en_US.properties index 6f4777f44d..18b7a10885 100644 --- a/dolphinscheduler-api/src/main/resources/i18n/messages_en_US.properties +++ b/dolphinscheduler-api/src/main/resources/i18n/messages_en_US.properties @@ -38,6 +38,11 @@ VERIFY_ALERT_GROUP_NAME_NOTES=verify alert group name, check alert group exist o GRANT_ALERT_GROUP_NOTES=grant alert group USER_IDS=user id list ALERT_GROUP_TAG=alert group related operation +ALERT_PLUGIN_INSTANCE_TAG=alert plugin instance related operation +UPDATE_ALERT_PLUGIN_INSTANCE_NOTES=update alert plugin instance operation +CREATE_ALERT_PLUGIN_INSTANCE_NOTES=create alert plugin instance operation +DELETE_ALERT_PLUGIN_INSTANCE_NOTES=delete alert plugin instance operation +GET_ALERT_PLUGIN_INSTANCE_NOTES=get alert plugin instance operation CREATE_ALERT_GROUP_NOTES=create alert group WORKER_GROUP_TAG=worker group related operation SAVE_WORKER_GROUP_NOTES=create worker group diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ExecutorControllerTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ExecutorControllerTest.java index 77d8f38df0..f0bd948766 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ExecutorControllerTest.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ExecutorControllerTest.java @@ -14,14 +14,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.api.controller; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + import org.apache.dolphinscheduler.api.enums.ExecuteType; import org.apache.dolphinscheduler.api.enums.Status; import org.apache.dolphinscheduler.api.utils.Result; import org.apache.dolphinscheduler.common.enums.FailureStrategy; import org.apache.dolphinscheduler.common.enums.WarningType; -import org.apache.dolphinscheduler.common.utils.*; +import org.apache.dolphinscheduler.common.utils.JSONUtils; + import org.junit.Assert; import org.junit.Ignore; import org.junit.Test; @@ -32,44 +38,39 @@ import org.springframework.test.web.servlet.MvcResult; import org.springframework.util.LinkedMultiValueMap; import org.springframework.util.MultiValueMap; -import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; -import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; - /** * executor controller test */ -public class ExecutorControllerTest extends AbstractControllerTest{ +public class ExecutorControllerTest extends AbstractControllerTest { private static Logger logger = LoggerFactory.getLogger(ExecutorControllerTest.class); @Ignore @Test public void testStartProcessInstance() throws Exception { MultiValueMap paramsMap = new LinkedMultiValueMap<>(); - paramsMap.add("processDefinitionId","40"); - paramsMap.add("scheduleTime",""); + paramsMap.add("processDefinitionId", "40"); + paramsMap.add("scheduleTime", ""); paramsMap.add("failureStrategy", String.valueOf(FailureStrategy.CONTINUE)); - paramsMap.add("startNodeList",""); - paramsMap.add("taskDependType",""); - paramsMap.add("execType",""); + paramsMap.add("startNodeList", ""); + paramsMap.add("taskDependType", ""); + paramsMap.add("execType", ""); paramsMap.add("warningType", String.valueOf(WarningType.NONE)); - paramsMap.add("warningGroupId",""); - paramsMap.add("receivers",""); - paramsMap.add("receiversCc",""); - paramsMap.add("runMode",""); - paramsMap.add("processInstancePriority",""); - paramsMap.add("workerGroupId",""); - paramsMap.add("timeout",""); + paramsMap.add("warningGroupId", ""); + paramsMap.add("receivers", ""); + paramsMap.add("receiversCc", ""); + paramsMap.add("runMode", ""); + paramsMap.add("processInstancePriority", ""); + paramsMap.add("workerGroupId", ""); + paramsMap.add("timeout", ""); - MvcResult mvcResult = mockMvc.perform(post("/projects/{projectName}/executors/start-process-instance","cxc_1113") - .header("sessionId", sessionId) - .params(paramsMap)) - .andExpect(status().isOk()) - .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) - .andReturn(); + MvcResult mvcResult = mockMvc.perform(post("/projects/{projectName}/executors/start-process-instance", "cxc_1113") + .header("sessionId", sessionId) + .params(paramsMap)) + .andExpect(status().isOk()) + .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) + .andReturn(); Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class); - Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue()); + Assert.assertEquals(Status.SUCCESS.getCode(), result.getCode().intValue()); logger.info(mvcResult.getResponse().getContentAsString()); } @@ -77,47 +78,32 @@ public class ExecutorControllerTest extends AbstractControllerTest{ @Test public void testExecute() throws Exception { MultiValueMap paramsMap = new LinkedMultiValueMap<>(); - paramsMap.add("processInstanceId","40"); - paramsMap.add("executeType",String.valueOf(ExecuteType.NONE)); + paramsMap.add("processInstanceId", "40"); + paramsMap.add("executeType", String.valueOf(ExecuteType.NONE)); - MvcResult mvcResult = mockMvc.perform(post("/projects/{projectName}/executors/execute","cxc_1113") - .header("sessionId", sessionId) - .params(paramsMap)) - .andExpect(status().isOk()) - .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) - .andReturn(); + MvcResult mvcResult = mockMvc.perform(post("/projects/{projectName}/executors/execute", "cxc_1113") + .header("sessionId", sessionId) + .params(paramsMap)) + .andExpect(status().isOk()) + .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) + .andReturn(); Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class); - Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue()); + Assert.assertEquals(Status.SUCCESS.getCode(), result.getCode().intValue()); logger.info(mvcResult.getResponse().getContentAsString()); } - @Test public void testStartCheckProcessDefinition() throws Exception { - MvcResult mvcResult = mockMvc.perform(post("/projects/{projectName}/executors/start-check","cxc_1113") - .header(SESSION_ID, sessionId) - .param("processDefinitionId","40")) - .andExpect(status().isOk()) - .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) - .andReturn(); + MvcResult mvcResult = mockMvc.perform(post("/projects/{projectName}/executors/start-check", "cxc_1113") + .header(SESSION_ID, sessionId) + .param("processDefinitionId", "40")) + .andExpect(status().isOk()) + .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) + .andReturn(); Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class); - Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue()); + Assert.assertEquals(Status.SUCCESS.getCode(), result.getCode().intValue()); logger.info(mvcResult.getResponse().getContentAsString()); } - @Test - public void testGetReceiverCc() throws Exception { - MultiValueMap paramsMap = new LinkedMultiValueMap<>(); - paramsMap.add("processInstanceId","13"); - MvcResult mvcResult = mockMvc.perform(get("/projects/{projectName}/executors/get-receiver-cc","cxc_1113") - .header(SESSION_ID, sessionId) - .params(paramsMap)) - .andExpect(status().isOk()) - .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8)) - .andReturn(); - Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class); - Assert.assertEquals(Status.SUCCESS.getCode(),result.getCode().intValue()); - logger.info(mvcResult.getResponse().getContentAsString()); - } } diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TaskInstanceControllerTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TaskInstanceControllerTest.java index 031d7819cb..ec47c5e4a1 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TaskInstanceControllerTest.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/TaskInstanceControllerTest.java @@ -99,4 +99,5 @@ public class TaskInstanceControllerTest extends AbstractControllerTest { Result result = JSONUtils.parseObject(mvcResult.getResponse().getContentAsString(), Result.class); Assert.assertEquals(Status.SUCCESS.getCode(), result.getCode().intValue()); } + } diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~HEAD b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~HEAD new file mode 100644 index 0000000000..a96cec9158 --- /dev/null +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~HEAD @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.security; + +import org.apache.dolphinscheduler.api.ApiApplicationServer; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.context.TestPropertySource; +import org.springframework.test.context.junit4.SpringRunner; + +@RunWith(SpringRunner.class) +@SpringBootTest(classes = ApiApplicationServer.class) +@TestPropertySource(properties = { + "security.authentication.type=LDAP", +}) +public class SecurityConfigLDAPTest { + + @Autowired + private SecurityConfig securityConfig; + + @Test + public void testAuthenticator() { + Authenticator authenticator = securityConfig.authenticator(); + Assert.assertNotNull(authenticator); + } +} diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~dev b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~dev new file mode 100644 index 0000000000..a96cec9158 --- /dev/null +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/SecurityConfigLDAPTest.java~dev @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.security; + +import org.apache.dolphinscheduler.api.ApiApplicationServer; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.context.TestPropertySource; +import org.springframework.test.context.junit4.SpringRunner; + +@RunWith(SpringRunner.class) +@SpringBootTest(classes = ApiApplicationServer.class) +@TestPropertySource(properties = { + "security.authentication.type=LDAP", +}) +public class SecurityConfigLDAPTest { + + @Autowired + private SecurityConfig securityConfig; + + @Test + public void testAuthenticator() { + Authenticator authenticator = securityConfig.authenticator(); + Assert.assertNotNull(authenticator); + } +} diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/AlertGroupServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/AlertGroupServiceTest.java index 1b93e86773..b7b6575ed1 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/AlertGroupServiceTest.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/AlertGroupServiceTest.java @@ -14,14 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.api.service; -import com.baomidou.mybatisplus.core.metadata.IPage; -import com.baomidou.mybatisplus.extension.plugins.pagination.Page; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; + import org.apache.dolphinscheduler.api.enums.Status; import org.apache.dolphinscheduler.api.utils.PageInfo; import org.apache.dolphinscheduler.common.Constants; @@ -31,15 +29,15 @@ import org.apache.dolphinscheduler.common.utils.CollectionUtils; import org.apache.dolphinscheduler.dao.entity.AlertGroup; import org.apache.dolphinscheduler.dao.entity.User; import org.apache.dolphinscheduler.dao.mapper.AlertGroupMapper; -import org.apache.dolphinscheduler.dao.mapper.UserAlertGroupMapper; -import org.junit.After; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + import org.junit.Assert; -import static org.junit.Assert.assertEquals; -import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; -import org.mockito.ArgumentCaptor; -import static org.mockito.ArgumentMatchers.*; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.Mockito; @@ -47,6 +45,9 @@ import org.mockito.junit.MockitoJUnitRunner; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import com.baomidou.mybatisplus.core.metadata.IPage; +import com.baomidou.mybatisplus.extension.plugins.pagination.Page; + @RunWith(MockitoJUnitRunner.class) public class AlertGroupServiceTest { @@ -56,128 +57,97 @@ public class AlertGroupServiceTest { private AlertGroupService alertGroupService; @Mock private AlertGroupMapper alertGroupMapper; - @Mock - private UserAlertGroupMapper userAlertGroupMapper; - @Mock - UserAlertGroupService userAlertGroupService; private String groupName = "AlertGroupServiceTest"; - @Before - public void setUp() { - } - - - @After - public void after(){ - - } - - - @Test - public void testQueryAlertGroup(){ + public void testQueryAlertGroup() { Mockito.when(alertGroupMapper.queryAllGroupList()).thenReturn(getList()); - HashMap result= alertGroupService.queryAlertgroup(); + HashMap result = alertGroupService.queryAlertgroup(); logger.info(result.toString()); List alertGroups = (List) result.get(Constants.DATA_LIST); Assert.assertTrue(CollectionUtils.isNotEmpty(alertGroups)); } + @Test - public void testListPaging(){ - IPage page = new Page<>(1,10); + public void testListPaging() { + IPage page = new Page<>(1, 10); page.setTotal(1L); page.setRecords(getList()); - Mockito.when(alertGroupMapper.queryAlertGroupPage(any(Page.class),eq(groupName))).thenReturn(page); + Mockito.when(alertGroupMapper.queryAlertGroupPage(any(Page.class), eq(groupName))).thenReturn(page); User user = new User(); // no operate - Map result = alertGroupService.listPaging(user,groupName,1,10); + Map result = alertGroupService.listPaging(user, groupName, 1, 10); logger.info(result.toString()); - Assert.assertEquals(Status.USER_NO_OPERATION_PERM,result.get(Constants.STATUS)); + Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS)); //success user.setUserType(UserType.ADMIN_USER); - result = alertGroupService.listPaging(user,groupName,1,10); + result = alertGroupService.listPaging(user, groupName, 1, 10); logger.info(result.toString()); PageInfo pageInfo = (PageInfo) result.get(Constants.DATA_LIST); Assert.assertTrue(CollectionUtils.isNotEmpty(pageInfo.getLists())); } - @Test - public void testCreateAlertgroup(){ + @Test + public void testCreateAlertgroup() { Mockito.when(alertGroupMapper.insert(any(AlertGroup.class))).thenReturn(2); User user = new User(); //no operate - Map result = alertGroupService.createAlertgroup(user,groupName, AlertType.EMAIL,groupName); + Map result = alertGroupService.createAlertgroup(user, groupName, groupName, null); logger.info(result.toString()); - Assert.assertEquals(Status.USER_NO_OPERATION_PERM,result.get(Constants.STATUS)); + Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS)); user.setUserType(UserType.ADMIN_USER); //success - result = alertGroupService.createAlertgroup(user,groupName, AlertType.EMAIL,groupName); + result = alertGroupService.createAlertgroup(user, groupName, groupName, null); logger.info(result.toString()); - Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS)); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); } + @Test - public void testUpdateAlertgroup(){ + public void testUpdateAlertgroup() { User user = new User(); // no operate - Map result = alertGroupService.updateAlertgroup(user,1,groupName, AlertType.SMS,groupName); + Map result = alertGroupService.updateAlertgroup(user, 1, groupName, groupName, null); logger.info(result.toString()); - Assert.assertEquals(Status.USER_NO_OPERATION_PERM,result.get(Constants.STATUS)); + Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS)); user.setUserType(UserType.ADMIN_USER); // not exist - result = alertGroupService.updateAlertgroup(user,1,groupName, AlertType.SMS,groupName); + result = alertGroupService.updateAlertgroup(user, 1, groupName, groupName, null); logger.info(result.toString()); - Assert.assertEquals(Status.ALERT_GROUP_NOT_EXIST,result.get(Constants.STATUS)); + Assert.assertEquals(Status.ALERT_GROUP_NOT_EXIST, result.get(Constants.STATUS)); //success Mockito.when(alertGroupMapper.selectById(2)).thenReturn(getEntity()); - result = alertGroupService.updateAlertgroup(user,2,groupName, AlertType.SMS,groupName); + result = alertGroupService.updateAlertgroup(user, 2, groupName, groupName, null); logger.info(result.toString()); - Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS)); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); } + @Test - public void testDelAlertgroupById(){ + public void testDelAlertgroupById() { User user = new User(); // no operate - Map result = alertGroupService.delAlertgroupById(user,1); + Map result = alertGroupService.delAlertgroupById(user, 1); logger.info(result.toString()); - Assert.assertEquals(Status.USER_NO_OPERATION_PERM,result.get(Constants.STATUS)); + Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS)); user.setUserType(UserType.ADMIN_USER); // not exist - result = alertGroupService.delAlertgroupById(user,2); + result = alertGroupService.delAlertgroupById(user, 2); logger.info(result.toString()); - Assert.assertEquals(Status.ALERT_GROUP_NOT_EXIST,result.get(Constants.STATUS)); + Assert.assertEquals(Status.ALERT_GROUP_NOT_EXIST, result.get(Constants.STATUS)); //success Mockito.when(alertGroupMapper.selectById(2)).thenReturn(getEntity()); - result = alertGroupService.delAlertgroupById(user,2); + result = alertGroupService.delAlertgroupById(user, 2); logger.info(result.toString()); - Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS)); - + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); } - @Test - public void testGrantUser() { - - Integer groupId = 1; - - ArgumentCaptor groupArgument = ArgumentCaptor.forClass(Integer.class); - - Mockito.when(userAlertGroupService.deleteByAlertGroupId(anyInt())).thenReturn(true); - - Map result = alertGroupService.grantUser(getLoginUser(), groupId, "123,321"); - Mockito.verify(userAlertGroupService).deleteByAlertGroupId(groupArgument.capture()); - - logger.info(result.toString()); - assertEquals(groupArgument.getValue(), groupId); - assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); - } - @Test public void testVerifyGroupName() { //group name not exist @@ -190,12 +160,10 @@ public class AlertGroupServiceTest { Assert.assertTrue(result); } - /** * create admin user - * @return */ - private User getLoginUser(){ + private User getLoginUser() { User loginUser = new User(); loginUser.setUserType(UserType.ADMIN_USER); @@ -205,9 +173,8 @@ public class AlertGroupServiceTest { /** * get list - * @return */ - private List getList(){ + private List getList() { List alertGroups = new ArrayList<>(); alertGroups.add(getEntity()); return alertGroups; @@ -215,13 +182,11 @@ public class AlertGroupServiceTest { /** * get entity - * @return */ - private AlertGroup getEntity(){ + private AlertGroup getEntity() { AlertGroup alertGroup = new AlertGroup(); alertGroup.setId(1); alertGroup.setGroupName(groupName); - alertGroup.setGroupType(AlertType.EMAIL); return alertGroup; } diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java index 93dc3bc727..3f25fb84d4 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java @@ -142,19 +142,17 @@ public class ExecutorService2Test { */ @Test public void testNoComplement() throws ParseException { - try { - Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); - Map result = executorService.execProcessInstance(loginUser, projectName, - processDefinitionId, cronTime, CommandType.START_PROCESS, - null, null, - null, null, 0, - "", "", RunMode.RUN_MODE_SERIAL, - Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); - Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); - verify(processService, times(1)).createCommand(any(Command.class)); - } catch (Exception e) { - //ignore - } + + Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); + Map result = executorService.execProcessInstance(loginUser, projectName, + processDefinitionId, cronTime, CommandType.START_PROCESS, + null, null, + null, null, 0, + RunMode.RUN_MODE_SERIAL, + Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); + verify(processService, times(1)).createCommand(any(Command.class)); + } /** @@ -162,19 +160,17 @@ public class ExecutorService2Test { */ @Test public void testComplementWithStartNodeList() throws ParseException { - try { - Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); - Map result = executorService.execProcessInstance(loginUser, projectName, - processDefinitionId, cronTime, CommandType.START_PROCESS, - null, "n1,n2", - null, null, 0, - "", "", RunMode.RUN_MODE_SERIAL, - Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); - Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); - verify(processService, times(1)).createCommand(any(Command.class)); - } catch (Exception e) { - //ignore - } + + Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); + Map result = executorService.execProcessInstance(loginUser, projectName, + processDefinitionId, cronTime, CommandType.START_PROCESS, + null, "n1,n2", + null, null, 0, + RunMode.RUN_MODE_SERIAL, + Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); + verify(processService, times(1)).createCommand(any(Command.class)); + } @@ -183,19 +179,16 @@ public class ExecutorService2Test { */ @Test public void testDateError() throws ParseException { - try { - Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); - Map result = executorService.execProcessInstance(loginUser, projectName, - processDefinitionId, "2020-01-31 23:00:00,2020-01-01 00:00:00", CommandType.COMPLEMENT_DATA, - null, null, - null, null, 0, - "", "", RunMode.RUN_MODE_SERIAL, - Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); - Assert.assertEquals(Status.START_PROCESS_INSTANCE_ERROR, result.get(Constants.STATUS)); - verify(processService, times(0)).createCommand(any(Command.class)); - } catch (Exception e) { - //ignore - } + + Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); + Map result = executorService.execProcessInstance(loginUser, projectName, + processDefinitionId, "2020-01-31 23:00:00,2020-01-01 00:00:00", CommandType.COMPLEMENT_DATA, + null, null, + null, null, 0, + RunMode.RUN_MODE_SERIAL, + Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); + Assert.assertEquals(Status.START_PROCESS_INSTANCE_ERROR, result.get(Constants.STATUS)); + verify(processService, times(0)).createCommand(any(Command.class)); } /** @@ -203,19 +196,17 @@ public class ExecutorService2Test { */ @Test public void testSerial() throws ParseException { - try { - Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); - Map result = executorService.execProcessInstance(loginUser, projectName, - processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, - null, null, - null, null, 0, - "", "", RunMode.RUN_MODE_SERIAL, - Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); - Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); - verify(processService, times(1)).createCommand(any(Command.class)); - } catch (Exception e) { - //ignore - } + + Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); + Map result = executorService.execProcessInstance(loginUser, projectName, + processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, + null, null, + null, null, 0, + RunMode.RUN_MODE_SERIAL, + Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); + verify(processService, times(1)).createCommand(any(Command.class)); + } /** @@ -223,19 +214,17 @@ public class ExecutorService2Test { */ @Test public void testParallelWithOutSchedule() throws ParseException { - try { - Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); - Map result = executorService.execProcessInstance(loginUser, projectName, - processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, - null, null, - null, null, 0, - "", "", RunMode.RUN_MODE_PARALLEL, - Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); - Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); - verify(processService, times(31)).createCommand(any(Command.class)); - } catch (Exception e) { - //ignore - } + + Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(zeroSchedulerList()); + Map result = executorService.execProcessInstance(loginUser, projectName, + processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, + null, null, + null, null, 0, + RunMode.RUN_MODE_PARALLEL, + Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); + verify(processService, times(31)).createCommand(any(Command.class)); + } /** @@ -243,19 +232,17 @@ public class ExecutorService2Test { */ @Test public void testParallelWithSchedule() throws ParseException { - try { - Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(oneSchedulerList()); - Map result = executorService.execProcessInstance(loginUser, projectName, - processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, - null, null, - null, null, 0, - "", "", RunMode.RUN_MODE_PARALLEL, - Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); - Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); - verify(processService, times(15)).createCommand(any(Command.class)); - } catch (Exception e) { - //ignore - } + + Mockito.when(processService.queryReleaseSchedulerListByProcessDefinitionId(processDefinitionId)).thenReturn(oneSchedulerList()); + Map result = executorService.execProcessInstance(loginUser, projectName, + processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, + null, null, + null, null, 0, + RunMode.RUN_MODE_PARALLEL, + Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); + Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS)); + verify(processService, times(15)).createCommand(any(Command.class)); + } @Test @@ -266,7 +253,7 @@ public class ExecutorService2Test { processDefinitionId, cronTime, CommandType.COMPLEMENT_DATA, null, null, null, null, 0, - "", "", RunMode.RUN_MODE_PARALLEL, + RunMode.RUN_MODE_PARALLEL, Priority.LOW, Constants.DEFAULT_WORKER_GROUP, 110, null); Assert.assertEquals(result.get(Constants.STATUS), Status.MASTER_NOT_EXISTS); @@ -290,7 +277,7 @@ public class ExecutorService2Test { } - private List zeroSchedulerList() { + private List zeroSchedulerList() { return Collections.EMPTY_LIST; } diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java index 169ef2bcfe..5a8d09fdc5 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java @@ -216,18 +216,17 @@ public class ProcessDefinitionVersionServiceTest { */ private ProcessDefinitionVersion getProcessDefinitionVersion(ProcessDefinition processDefinition) { return ProcessDefinitionVersion - .newBuilder() - .processDefinitionId(processDefinition.getId()) - .version(1) - .processDefinitionJson(processDefinition.getProcessDefinitionJson()) - .description(processDefinition.getDescription()) - .locations(processDefinition.getLocations()) - .connects(processDefinition.getConnects()) - .timeout(processDefinition.getTimeout()) - .globalParams(processDefinition.getGlobalParams()) - .createTime(processDefinition.getUpdateTime()) - .receivers(processDefinition.getReceivers()) - .receiversCc(processDefinition.getReceiversCc()) + .newBuilder() + .processDefinitionId(processDefinition.getId()) + .version(1) + .processDefinitionJson(processDefinition.getProcessDefinitionJson()) + .description(processDefinition.getDescription()) + .locations(processDefinition.getLocations()) + .connects(processDefinition.getConnects()) + .timeout(processDefinition.getTimeout()) + .globalParams(processDefinition.getGlobalParams()) + .createTime(processDefinition.getUpdateTime()) + .warningGroupId(processDefinition.getWarningGroupId()) .resourceIds(processDefinition.getResourceIds()) .build(); } diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java index 5511f69aeb..de23d7570e 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java @@ -211,8 +211,6 @@ public class ProcessInstanceServiceTest { //project auth success ProcessInstance processInstance = getProcessInstance(); - processInstance.setReceivers("xxx@qq.com"); - processInstance.setReceiversCc("xxx@qq.com"); processInstance.setProcessDefinitionId(46); putMsg(result, Status.SUCCESS, projectName); Project project = getProject(projectName); diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UiPluginServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UiPluginServiceTest.java new file mode 100644 index 0000000000..95e38b2ab9 --- /dev/null +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UiPluginServiceTest.java @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.api.service; + +import org.apache.dolphinscheduler.api.enums.Status; +import org.apache.dolphinscheduler.api.service.impl.UiPluginServiceImpl; +import org.apache.dolphinscheduler.common.enums.PluginType; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper; + +import java.util.Collections; +import java.util.Map; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.Mockito; +import org.mockito.junit.MockitoJUnitRunner; + +/** + * UiPluginServiceTest + */ +@RunWith(MockitoJUnitRunner.class) +public class UiPluginServiceTest { + + @InjectMocks + UiPluginServiceImpl uiPluginService; + + @Mock + PluginDefineMapper pluginDefineMapper; + + private PluginDefine pluginDefine; + + @Before + public void before() { + String pluginParams = "[{\"field\":\"receivers\",\"props\":null,\"type\"}]"; + pluginDefine = new PluginDefine("email-alert", "alert", pluginParams); + } + + @Test + public void testQueryPlugins1() { + Map result = uiPluginService.queryUiPluginsByType(PluginType.REGISTER); + Assert.assertEquals(Status.PLUGIN_NOT_A_UI_COMPONENT, result.get("status")); + } + + @Test + public void testQueryPlugins2() { + Map result = uiPluginService.queryUiPluginsByType(PluginType.ALERT); + Mockito.when(pluginDefineMapper.queryByPluginType(PluginType.ALERT.getDesc())).thenReturn(null); + Assert.assertEquals(Status.QUERY_PLUGINS_RESULT_IS_NULL, result.get("status")); + + Mockito.when(pluginDefineMapper.queryByPluginType(PluginType.ALERT.getDesc())).thenReturn(Collections.singletonList(pluginDefine)); + result = uiPluginService.queryUiPluginsByType(PluginType.ALERT); + Assert.assertEquals(Status.SUCCESS, result.get("status")); + } + + @Test + public void testQueryPluginDetailById() { + Mockito.when(pluginDefineMapper.queryDetailById(1)).thenReturn(null); + Map result = uiPluginService.queryUiPluginDetailById(1); + Assert.assertEquals(Status.QUERY_PLUGIN_DETAIL_RESULT_IS_NULL, result.get("status")); + + Mockito.when(pluginDefineMapper.queryDetailById(1)).thenReturn(pluginDefine); + result = uiPluginService.queryUiPluginDetailById(1); + Assert.assertEquals(Status.SUCCESS, result.get("status")); + } + +} diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UserAlertGroupServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UserAlertGroupServiceTest.java deleted file mode 100644 index 24b1d5a98b..0000000000 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UserAlertGroupServiceTest.java +++ /dev/null @@ -1,53 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.api.service; - -import org.apache.dolphinscheduler.dao.mapper.UserAlertGroupMapper; -import static org.junit.Assert.assertEquals; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.mockito.ArgumentCaptor; -import org.mockito.InjectMocks; -import org.mockito.Mock; -import org.mockito.Mockito; -import org.mockito.junit.MockitoJUnitRunner; - -/** - * - */ -@RunWith(MockitoJUnitRunner.class) -public class UserAlertGroupServiceTest { - - @InjectMocks - UserAlertGroupService userAlertGroupService; - - @Mock - UserAlertGroupMapper userAlertGroupMapper; - - @Test - public void deleteByAlertGroupId() { - - Integer groupId = 1; - userAlertGroupService.deleteByAlertGroupId(groupId); - ArgumentCaptor argumentCaptor = ArgumentCaptor.forClass(Integer.class); - - Mockito.verify(userAlertGroupMapper).deleteByAlertgroupId(argumentCaptor.capture()); - assertEquals(argumentCaptor.getValue(), groupId); - - } - -} \ No newline at end of file diff --git a/dolphinscheduler-common/pom.xml b/dolphinscheduler-common/pom.xml index 52f2361581..8aeca37e93 100644 --- a/dolphinscheduler-common/pom.xml +++ b/dolphinscheduler-common/pom.xml @@ -32,11 +32,6 @@ 3.1.0 - - org.apache.dolphinscheduler - dolphinscheduler-plugin-api - - org.apache.httpcomponents httpclient diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java index 17fa753a37..02580e6749 100644 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java +++ b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java @@ -401,13 +401,17 @@ public final class Constants { */ public static final int SEC_2_MINUTES_TIME_UNIT = 60; - /*** * * rpc port */ public static final int RPC_PORT = 50051; + /*** + * alert rpc port + */ + public static final int ALERT_RPC_PORT = 50052; + /** * forbid running task */ @@ -1008,6 +1012,8 @@ public final class Constants { public static final String DOLPHIN_SCHEDULER_PREFERRED_NETWORK_INTERFACE = "dolphin.scheduler.network.interface.preferred"; + public static final String EXCEL_SUFFIX_XLS = ".xls"; + /** * datasource encryption salt */ diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/PluginType.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/PluginType.java new file mode 100644 index 0000000000..958e4852d8 --- /dev/null +++ b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/PluginType.java @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.common.enums; + +import java.util.HashMap; + +import com.baomidou.mybatisplus.annotation.EnumValue; + +/** + * PluginType + */ +public enum PluginType { + + ALERT(1, "alert", true), + REGISTER(2, "register", false); + + PluginType(int code, String desc, boolean hasUi) { + this.code = code; + this.desc = desc; + this.hasUi = hasUi; + } + + @EnumValue + private final int code; + private final String desc; + private final boolean hasUi; + + public int getCode() { + return code; + } + + public String getDesc() { + return desc; + } + + public boolean getHasUi() { + return hasUi; + } + + + private static HashMap PLUGIN_TYPE_MAP = new HashMap<>(); + + static { + for (PluginType pluginType : PluginType.values()) { + PLUGIN_TYPE_MAP.put(pluginType.getCode(), pluginType); + } + } + + public static PluginType of(int type) { + if (PLUGIN_TYPE_MAP.containsKey(type)) { + return PLUGIN_TYPE_MAP.get(type); + } + throw new IllegalArgumentException("invalid type : " + type); + } +} diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/FilePluginManager.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/FilePluginManager.java deleted file mode 100644 index cde299f9dd..0000000000 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/FilePluginManager.java +++ /dev/null @@ -1,107 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.common.plugin; - -import org.apache.dolphinscheduler.common.Constants; -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; -import org.apache.dolphinscheduler.plugin.spi.AlertPluginProvider; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.File; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.Map; -import java.util.ServiceLoader; -import java.util.concurrent.ConcurrentHashMap; - -/** - * FilePluginManager - */ -public class FilePluginManager implements PluginManager { - - private static final Logger logger = LoggerFactory.getLogger(FilePluginManager.class); - - private Map pluginMap = new ConcurrentHashMap<>(); - - private Map> pluginLoaderMap = new ConcurrentHashMap<>(); - - private Map classLoaderMap = new ConcurrentHashMap<>(); - - private String[] whitePrefixes; - - private String[] excludePrefixes; - - public FilePluginManager(String dirPath, String[] whitePrefixes, String[] excludePrefixes) { - this.whitePrefixes = whitePrefixes; - this.excludePrefixes = excludePrefixes; - try { - load(dirPath); - } catch (MalformedURLException e) { - logger.error("load plugins failed.", e); - } - } - - private void load(String dirPath) throws MalformedURLException { - logger.info("start to load jar files in {}", dirPath); - if (dirPath == null) { - logger.error("not a valid path - {}", dirPath); - return; - } - File[] files = new File(dirPath).listFiles(); - if (files == null) { - logger.error("not a valid path - {}", dirPath); - return; - } - for (File file : files) { - if (file.isDirectory() && !file.getPath().endsWith(Constants.PLUGIN_JAR_SUFFIX)) { - continue; - } - String pluginName = file.getName() - .substring(0, file.getName().length() - Constants.PLUGIN_JAR_SUFFIX.length()); - URL[] urls = new URL[]{ file.toURI().toURL() }; - PluginClassLoader classLoader = - new PluginClassLoader(urls, Thread.currentThread().getContextClassLoader(), whitePrefixes, excludePrefixes); - classLoaderMap.put(pluginName, classLoader); - - ServiceLoader loader = ServiceLoader.load(AlertPluginProvider.class, classLoader); - pluginLoaderMap.put(pluginName, loader); - - loader.forEach(provider -> { - AlertPlugin plugin = provider.createPlugin(); - pluginMap.put(plugin.getId(), plugin); - logger.info("loaded plugin - {}", plugin.getId()); - }); - } - } - - @Override - public AlertPlugin findOne(String pluginId) { - return pluginMap.get(pluginId); - } - - @Override - public Map findAll() { - return pluginMap; - } - - @Override - public void addPlugin(AlertPlugin plugin) { - pluginMap.put(plugin.getId(), plugin); - } - -} diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoader.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoader.java deleted file mode 100644 index 528e83a59c..0000000000 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoader.java +++ /dev/null @@ -1,154 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.common.plugin; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.net.URL; -import java.net.URLClassLoader; -import java.util.Enumeration; -import java.util.Iterator; -import java.util.LinkedList; -import java.util.List; - -/** - * Plugin Class Loader - */ -public class PluginClassLoader extends URLClassLoader { - - private static final Logger logger = LoggerFactory.getLogger(PluginClassLoader.class); - - private static final String JAVA_PACKAGE_PREFIX = "java."; - private static final String JAVAX_PACKAGE_PREFIX = "javax."; - - private final String[] whitePrefixes; - - private final String[] excludePrefixes; - - public PluginClassLoader(URL[] urls, ClassLoader parent, String[] whitePrefix, String[] excludePreifx) { - super(urls, parent); - this.whitePrefixes = whitePrefix; - this.excludePrefixes = excludePreifx; - } - - @Override - public Class loadClass(String name) throws ClassNotFoundException { - logger.trace("Received request to load class '{}'", name); - synchronized (getClassLoadingLock(name)) { - if (name.startsWith(JAVA_PACKAGE_PREFIX) || name.startsWith(JAVAX_PACKAGE_PREFIX)) { - return findSystemClass(name); - } - - boolean isWhitePrefixes = fromWhitePrefix(name); - boolean isExcludePrefixed = fromExcludePrefix(name); - - // if the class is part of the plugin engine use parent class loader - if (!isWhitePrefixes && isExcludePrefixed) { - return getParent().loadClass(name); - } - - // check whether it's already been loaded - Class loadedClass = findLoadedClass(name); - if (loadedClass != null) { - logger.debug("Found loaded class '{}'", name); - return loadedClass; - } - - // nope, try to load locally - try { - loadedClass = findClass(name); - logger.debug("Found class '{}' in plugin classpath", name); - return loadedClass; - } catch (ClassNotFoundException e) { - // try next step - } - - // use the standard ClassLoader (which follows normal parent delegation) - return super.loadClass(name); - } - } - - private boolean fromWhitePrefix(String name) { - if (this.whitePrefixes == null) { - return false; - } - for (String whitePrefix : this.whitePrefixes) { - if (name.startsWith(whitePrefix)) { - return true; - } - } - return false; - } - - private boolean fromExcludePrefix(String name) { - if (this.excludePrefixes == null) { - return false; - } - for (String excludePrefix : this.excludePrefixes) { - if (name.startsWith(excludePrefix)) { - return true; - } - } - return false; - } - - @Override - public Enumeration getResources(String name) throws IOException { - List allRes = new LinkedList<>(); - - Enumeration thisRes = findResources(name); - if (thisRes != null) { - while (thisRes.hasMoreElements()) { - allRes.add(thisRes.nextElement()); - } - } - - Enumeration parentRes = super.findResources(name); - if (parentRes != null) { - while (parentRes.hasMoreElements()) { - allRes.add(parentRes.nextElement()); - } - } - - return new Enumeration() { - Iterator it = allRes.iterator(); - - @Override - public boolean hasMoreElements() { - return it.hasNext(); - } - - @Override - public URL nextElement() { - return it.next(); - } - }; - } - - @Override - public URL getResource(String name) { - URL res = null; - - res = findResource(name); - if (res == null) { - res = super.getResource(name); - } - return res; - } -} diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/task/sql/SqlParameters.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/task/sql/SqlParameters.java index 4604234e8f..3b5d39f274 100644 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/task/sql/SqlParameters.java +++ b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/task/sql/SqlParameters.java @@ -14,11 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.common.task.sql; import org.apache.dolphinscheduler.common.process.ResourceInfo; import org.apache.dolphinscheduler.common.task.AbstractParameters; -import org.apache.commons.lang.StringUtils; +import org.apache.dolphinscheduler.common.utils.StringUtils; import java.util.ArrayList; import java.util.List; @@ -75,19 +76,13 @@ public class SqlParameters extends AbstractParameters { private List postStatements; /** - * title - */ - private String title; - - /** - * receivers + * groupId */ - private String receivers; - + private int groupId; /** - * receivers cc + * title */ - private String receiversCc; + private String title; public String getType() { return type; @@ -153,21 +148,6 @@ public class SqlParameters extends AbstractParameters { this.title = title; } - public String getReceivers() { - return receivers; - } - - public void setReceivers(String receivers) { - this.receivers = receivers; - } - - public String getReceiversCc() { - return receiversCc; - } - - public void setReceiversCc(String receiversCc) { - this.receiversCc = receiversCc; - } public List getPreStatements() { return preStatements; } @@ -184,6 +164,14 @@ public class SqlParameters extends AbstractParameters { this.postStatements = postStatements; } + public int getGroupId() { + return groupId; + } + + public void setGroupId(int groupId) { + this.groupId = groupId; + } + @Override public boolean checkParameters() { return datasource != 0 && StringUtils.isNotEmpty(type) && StringUtils.isNotEmpty(sql); @@ -196,19 +184,18 @@ public class SqlParameters extends AbstractParameters { @Override public String toString() { - return "SqlParameters{" + - "type='" + type + '\'' + - ", datasource=" + datasource + - ", sql='" + sql + '\'' + - ", sqlType=" + sqlType + - ", udfs='" + udfs + '\'' + - ", showType='" + showType + '\'' + - ", connParams='" + connParams + '\'' + - ", title='" + title + '\'' + - ", receivers='" + receivers + '\'' + - ", receiversCc='" + receiversCc + '\'' + - ", preStatements=" + preStatements + - ", postStatements=" + postStatements + - '}'; + return "SqlParameters{" + + "type='" + type + '\'' + + ", datasource=" + datasource + + ", sql='" + sql + '\'' + + ", sqlType=" + sqlType + + ", udfs='" + udfs + '\'' + + ", showType='" + showType + '\'' + + ", connParams='" + connParams + '\'' + + ", groupId='" + groupId + '\'' + + ", title='" + title + '\'' + + ", preStatements=" + preStatements + + ", postStatements=" + postStatements + + '}'; } } diff --git a/dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/FilePluginManagerTest.java b/dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/FilePluginManagerTest.java deleted file mode 100644 index 1a57cb10fa..0000000000 --- a/dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/FilePluginManagerTest.java +++ /dev/null @@ -1,72 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.common.plugin; - -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; -import org.apache.dolphinscheduler.plugin.model.PluginName; -import org.junit.Before; -import org.junit.Test; - -import java.util.HashMap; -import java.util.Map; - -import static org.junit.Assert.*; - -public class FilePluginManagerTest { - - private FilePluginManager filePluginManager; - private AlertPlugin alertPlugin; - - @Before - public void before() { - filePluginManager = new FilePluginManager(null, null, null); - alertPlugin = new AlertPlugin() { - @Override - public String getId() { - return "test"; - } - - @Override - public PluginName getName() { - return new PluginName().setChinese("ch").setEnglish("en"); - } - - @Override - public Map process(AlertInfo info) { - return new HashMap<>(); - } - }; - } - - @Test - public void findOne() { - filePluginManager.addPlugin(alertPlugin); - assertEquals(alertPlugin, filePluginManager.findOne(alertPlugin.getId())); - } - - @Test - public void findAll() { - assertNotNull(filePluginManager.findAll()); - } - - @Test - public void addPlugin() { - filePluginManager.addPlugin(alertPlugin); - assertNotNull(filePluginManager.findAll()); - } -} \ No newline at end of file diff --git a/dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoaderTest.java b/dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoaderTest.java deleted file mode 100644 index 8a6bfaee13..0000000000 --- a/dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/plugin/PluginClassLoaderTest.java +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.common.plugin; - -import org.junit.Before; -import org.junit.Test; - -import java.net.URL; - -import static org.junit.Assert.*; - -public class PluginClassLoaderTest { - - private PluginClassLoader pluginClassLoader; - private ClassLoader parent; - - @Before - public void setUp() { - parent = Thread.currentThread().getContextClassLoader(); - pluginClassLoader = new PluginClassLoader( - new URL[]{}, parent, - null, null); - } - - @Test - public void loadClassNull() { - Class clazz = null; - try { - clazz = pluginClassLoader.loadClass("java.lang.Object"); - } catch (ClassNotFoundException e) { - e.printStackTrace(); - } - assertEquals(null, clazz.getClassLoader()); - } - - @Test - public void loadClassApp() { - Class clazz = null; - try { - clazz = pluginClassLoader.loadClass("org.apache.dolphinscheduler.common.Constants"); - } catch (ClassNotFoundException e) { - e.printStackTrace(); - } - assertEquals(parent, clazz.getClassLoader()); - } - -} \ No newline at end of file diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/AlertDao.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/AlertDao.java index cd101f06b6..5b66932a36 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/AlertDao.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/AlertDao.java @@ -19,24 +19,25 @@ package org.apache.dolphinscheduler.dao; import org.apache.dolphinscheduler.common.enums.AlertEvent; import org.apache.dolphinscheduler.common.enums.AlertStatus; -import org.apache.dolphinscheduler.common.enums.AlertType; import org.apache.dolphinscheduler.common.enums.AlertWarnLevel; -import org.apache.dolphinscheduler.common.enums.ShowType; import org.apache.dolphinscheduler.common.utils.JSONUtils; import org.apache.dolphinscheduler.common.utils.StringUtils; import org.apache.dolphinscheduler.dao.datasource.ConnectionFactory; import org.apache.dolphinscheduler.dao.entity.Alert; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; import org.apache.dolphinscheduler.dao.entity.ProcessAlertContent; import org.apache.dolphinscheduler.dao.entity.ProcessDefinition; import org.apache.dolphinscheduler.dao.entity.ProcessInstance; import org.apache.dolphinscheduler.dao.entity.ServerAlertContent; -import org.apache.dolphinscheduler.dao.entity.User; +import org.apache.dolphinscheduler.dao.mapper.AlertGroupMapper; import org.apache.dolphinscheduler.dao.mapper.AlertMapper; -import org.apache.dolphinscheduler.dao.mapper.UserAlertGroupMapper; +import org.apache.dolphinscheduler.dao.mapper.AlertPluginInstanceMapper; import java.util.ArrayList; +import java.util.Arrays; import java.util.Date; import java.util.List; +import java.util.stream.Collectors; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -52,12 +53,16 @@ public class AlertDao extends AbstractBaseDao { private AlertMapper alertMapper; @Autowired - private UserAlertGroupMapper userAlertGroupMapper; + private AlertPluginInstanceMapper alertPluginInstanceMapper; + + @Autowired + private AlertGroupMapper alertGroupMapper; @Override protected void init() { alertMapper = ConnectionFactory.getInstance().getMapper(AlertMapper.class); - userAlertGroupMapper = ConnectionFactory.getInstance().getMapper(UserAlertGroupMapper.class); + alertPluginInstanceMapper = ConnectionFactory.getInstance().getMapper(AlertPluginInstanceMapper.class); + alertGroupMapper = ConnectionFactory.getInstance().getMapper(AlertGroupMapper.class); } /** @@ -86,25 +91,14 @@ public class AlertDao extends AbstractBaseDao { return alertMapper.updateById(alert); } - /** - * query user list by alert group id - * - * @param alerGroupId alerGroupId - * @return user list - */ - public List queryUserByAlertGroupId(int alerGroupId) { - - return userAlertGroupMapper.listUserByAlertgroupId(alerGroupId); - } - /** * MasterServer or WorkerServer stoped * - * @param alertgroupId alertgroupId + * @param alertGroupId alertGroupId * @param host host * @param serverType serverType */ - public void sendServerStopedAlert(int alertgroupId, String host, String serverType) { + public void sendServerStopedAlert(int alertGroupId, String host, String serverType) { Alert alert = new Alert(); List serverAlertContents = new ArrayList<>(1); ServerAlertContent serverStopAlertContent = ServerAlertContent.newBuilder(). @@ -113,7 +107,7 @@ public class AlertDao extends AbstractBaseDao { serverAlertContents.add(serverStopAlertContent); String content = JSONUtils.toJsonString(serverAlertContents); alert.setTitle("Fault tolerance warning"); - saveTaskTimeoutAlert(alert, content, alertgroupId, null, null); + saveTaskTimeoutAlert(alert, content, alertGroupId); } /** @@ -123,9 +117,7 @@ public class AlertDao extends AbstractBaseDao { * @param processDefinition processDefinition */ public void sendProcessTimeoutAlert(ProcessInstance processInstance, ProcessDefinition processDefinition) { - int alertgroupId = processInstance.getWarningGroupId(); - String receivers = processDefinition.getReceivers(); - String receiversCc = processDefinition.getReceiversCc(); + int alertGroupId = processInstance.getWarningGroupId(); Alert alert = new Alert(); List processAlertContentList = new ArrayList<>(1); ProcessAlertContent processAlertContent = ProcessAlertContent.newBuilder() @@ -137,21 +129,12 @@ public class AlertDao extends AbstractBaseDao { processAlertContentList.add(processAlertContent); String content = JSONUtils.toJsonString(processAlertContentList); alert.setTitle("Process Timeout Warn"); - saveTaskTimeoutAlert(alert, content, alertgroupId, receivers, receiversCc); + saveTaskTimeoutAlert(alert, content, alertGroupId); } - private void saveTaskTimeoutAlert(Alert alert, String content, int alertgroupId, - String receivers, String receiversCc) { - alert.setShowType(ShowType.TABLE); + private void saveTaskTimeoutAlert(Alert alert, String content, int alertGroupId) { + alert.setAlertGroupId(alertGroupId); alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); - alert.setAlertGroupId(alertgroupId); - if (StringUtils.isNotEmpty(receivers)) { - alert.setReceivers(receivers); - } - if (StringUtils.isNotEmpty(receiversCc)) { - alert.setReceiversCc(receiversCc); - } alert.setCreateTime(new Date()); alert.setUpdateTime(new Date()); alertMapper.insert(alert); @@ -160,15 +143,13 @@ public class AlertDao extends AbstractBaseDao { /** * task timeout warn * - * @param alertgroupId alertgroupId - * @param receivers receivers - * @param receiversCc receiversCc + * @param alertGroupId alertGroupId * @param processInstanceId processInstanceId * @param processInstanceName processInstanceName * @param taskId taskId * @param taskName taskName */ - public void sendTaskTimeoutAlert(int alertgroupId, String receivers, String receiversCc, int processInstanceId, + public void sendTaskTimeoutAlert(int alertGroupId, int processInstanceId, String processInstanceName, int taskId, String taskName) { Alert alert = new Alert(); List processAlertContentList = new ArrayList<>(1); @@ -183,7 +164,7 @@ public class AlertDao extends AbstractBaseDao { processAlertContentList.add(processAlertContent); String content = JSONUtils.toJsonString(processAlertContentList); alert.setTitle("Task Timeout Warn"); - saveTaskTimeoutAlert(alert, content, alertgroupId, receivers, receiversCc); + saveTaskTimeoutAlert(alert, content, alertGroupId); } /** @@ -196,22 +177,45 @@ public class AlertDao extends AbstractBaseDao { } /** - * list user information by alert group id + * for test * - * @param alertgroupId alertgroupId - * @return user list + * @return AlertMapper */ - public List listUserByAlertgroupId(int alertgroupId) { - return userAlertGroupMapper.listUserByAlertgroupId(alertgroupId); + public AlertMapper getAlertMapper() { + return alertMapper; } /** - * for test + * list all alert plugin instance by alert group id * - * @return AlertMapper + * @param alertGroupId alert group id + * @return AlertPluginInstance list */ - public AlertMapper getAlertMapper() { - return alertMapper; + public List listInstanceByAlertGroupId(int alertGroupId) { + String alertInstanceIdsParam = alertGroupMapper.queryAlertGroupInstanceIdsById(alertGroupId); + if (StringUtils.isNotBlank(alertInstanceIdsParam)) { + String[] idsArray = alertInstanceIdsParam.split(","); + List ids = Arrays.stream(idsArray) + .map(s -> Integer.parseInt(s.trim())) + .collect(Collectors.toList()); + return alertPluginInstanceMapper.queryByIds(ids); + } + return null; } + public AlertPluginInstanceMapper getAlertPluginInstanceMapper() { + return alertPluginInstanceMapper; + } + + public void setAlertPluginInstanceMapper(AlertPluginInstanceMapper alertPluginInstanceMapper) { + this.alertPluginInstanceMapper = alertPluginInstanceMapper; + } + + public AlertGroupMapper getAlertGroupMapper() { + return alertGroupMapper; + } + + public void setAlertGroupMapper(AlertGroupMapper alertGroupMapper) { + this.alertGroupMapper = alertGroupMapper; + } } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/PluginDao.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/PluginDao.java new file mode 100644 index 0000000000..ab82997bc2 --- /dev/null +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/PluginDao.java @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.dao; + +import static java.util.Objects.requireNonNull; + +import org.apache.dolphinscheduler.dao.datasource.ConnectionFactory; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; +import org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper; + +import java.util.List; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.stereotype.Component; + +@Component +public class PluginDao extends AbstractBaseDao { + + private final Logger logger = LoggerFactory.getLogger(getClass()); + + @Autowired + private PluginDefineMapper pluginDefineMapper; + + @Override + protected void init() { + pluginDefineMapper = ConnectionFactory.getInstance().getMapper(PluginDefineMapper.class); + } + + /** + * add pluginDefine + * + * @param pluginDefine plugin define entiy + * @return plugin define id + */ + public int addPluginDefine(PluginDefine pluginDefine) { + return pluginDefineMapper.insert(pluginDefine); + } + + /** + * add or update plugin define + * + * @param pluginDefine new pluginDefine + */ + public void addOrUpdatePluginDefine(PluginDefine pluginDefine) { + requireNonNull(pluginDefine, "pluginDefine is null"); + requireNonNull(pluginDefine.getPluginName(), "pluginName is null"); + requireNonNull(pluginDefine.getPluginType(), "pluginType is null"); + + List pluginDefineList = pluginDefineMapper.queryByNameAndType(pluginDefine.getPluginName(), pluginDefine.getPluginType()); + if (pluginDefineList == null || pluginDefineList.size() == 0) { + pluginDefineMapper.insert(pluginDefine); + } else { + PluginDefine currPluginDefine = pluginDefineList.get(0); + if (!currPluginDefine.getPluginParams().equals(pluginDefine.getPluginParams())) { + currPluginDefine.setUpdateTime(pluginDefine.getUpdateTime()); + currPluginDefine.setPluginParams(pluginDefine.getPluginParams()); + pluginDefineMapper.updateById(currPluginDefine); + } + } + } + + /** + * query plugin define by id + * + * @param pluginDefineId plugin define id + * @return PluginDefine + */ + public PluginDefine getPluginDefineById(int pluginDefineId) { + return pluginDefineMapper.selectById(pluginDefineId); + } + + public PluginDefineMapper getPluginDefineMapper() { + return pluginDefineMapper; + } + + public void setPluginDefineMapper(PluginDefineMapper pluginDefineMapper) { + this.pluginDefineMapper = pluginDefineMapper; + } +} diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Alert.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Alert.java index cfd4995a2b..752ab9d8f9 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Alert.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Alert.java @@ -14,20 +14,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.entity; -import com.baomidou.mybatisplus.annotation.IdType; -import com.baomidou.mybatisplus.annotation.TableField; -import com.baomidou.mybatisplus.annotation.TableId; -import com.baomidou.mybatisplus.annotation.TableName; import org.apache.dolphinscheduler.common.enums.AlertStatus; -import org.apache.dolphinscheduler.common.enums.AlertType; -import org.apache.dolphinscheduler.common.enums.ShowType; import java.util.Date; import java.util.HashMap; import java.util.Map; +import com.baomidou.mybatisplus.annotation.IdType; +import com.baomidou.mybatisplus.annotation.TableField; +import com.baomidou.mybatisplus.annotation.TableId; +import com.baomidou.mybatisplus.annotation.TableName; + @TableName("t_ds_alert") public class Alert { /** @@ -40,21 +40,13 @@ public class Alert { */ @TableField(value = "title") private String title; - /** - * show_type - */ - @TableField(value = "show_type") - private ShowType showType; + /** * content */ @TableField(value = "content") private String content; - /** - * alert_type - */ - @TableField(value = "alert_type") - private AlertType alertType; + /** * alert_status */ @@ -70,16 +62,7 @@ public class Alert { */ @TableField("alertgroup_id") private int alertGroupId; - /** - * receivers - */ - @TableField("receivers") - private String receivers; - /** - * receivers_cc - */ - @TableField("receivers_cc") - private String receiversCc; + /** * create_time */ @@ -112,14 +95,6 @@ public class Alert { this.title = title; } - public ShowType getShowType() { - return showType; - } - - public void setShowType(ShowType showType) { - this.showType = showType; - } - public String getContent() { return content; } @@ -128,14 +103,6 @@ public class Alert { this.content = content; } - public AlertType getAlertType() { - return alertType; - } - - public void setAlertType(AlertType alertType) { - this.alertType = alertType; - } - public AlertStatus getAlertStatus() { return alertStatus; } @@ -160,22 +127,6 @@ public class Alert { this.alertGroupId = alertGroupId; } - public String getReceivers() { - return receivers; - } - - public void setReceivers(String receivers) { - this.receivers = receivers; - } - - public String getReceiversCc() { - return receiversCc; - } - - public void setReceiversCc(String receiversCc) { - this.receiversCc = receiversCc; - } - public Date getCreateTime() { return createTime; } @@ -220,27 +171,15 @@ public class Alert { if (!title.equals(alert.title)) { return false; } - if (showType != alert.showType) { - return false; - } if (!content.equals(alert.content)) { return false; } - if (alertType != alert.alertType) { - return false; - } if (alertStatus != alert.alertStatus) { return false; } if (!log.equals(alert.log)) { return false; } - if (!receivers.equals(alert.receivers)) { - return false; - } - if (!receiversCc.equals(alert.receiversCc)) { - return false; - } if (!createTime.equals(alert.createTime)) { return false; } @@ -252,14 +191,10 @@ public class Alert { public int hashCode() { int result = id; result = 31 * result + title.hashCode(); - result = 31 * result + showType.hashCode(); result = 31 * result + content.hashCode(); - result = 31 * result + alertType.hashCode(); result = 31 * result + alertStatus.hashCode(); result = 31 * result + log.hashCode(); result = 31 * result + alertGroupId; - result = 31 * result + receivers.hashCode(); - result = 31 * result + receiversCc.hashCode(); result = 31 * result + createTime.hashCode(); result = 31 * result + updateTime.hashCode(); result = 31 * result + info.hashCode(); @@ -268,20 +203,28 @@ public class Alert { @Override public String toString() { - return "Alert{" + - "id=" + id + - ", title='" + title + '\'' + - ", showType=" + showType + - ", content='" + content + '\'' + - ", alertType=" + alertType + - ", alertStatus=" + alertStatus + - ", log='" + log + '\'' + - ", alertGroupId=" + alertGroupId + - ", receivers='" + receivers + '\'' + - ", receiversCc='" + receiversCc + '\'' + - ", createTime=" + createTime + - ", updateTime=" + updateTime + - ", info=" + info + - '}'; + return "Alert{" + + "id=" + + id + + ", title='" + + title + '\'' + + ", content='" + + content + + '\'' + + ", alertStatus=" + + alertStatus + + ", log='" + + log + + '\'' + + ", alertGroupId=" + + alertGroupId + + '\'' + + ", createTime=" + + createTime + + ", updateTime=" + + updateTime + + ", info=" + + info + + '}'; } } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertGroup.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertGroup.java index e35e2f759c..174eb55a3a 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertGroup.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertGroup.java @@ -14,16 +14,16 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.entity; -import com.baomidou.mybatisplus.annotation.TableField; -import org.apache.dolphinscheduler.common.enums.AlertType; +import java.util.Date; + import com.baomidou.mybatisplus.annotation.IdType; +import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableName; -import java.util.Date; - @TableName("t_ds_alertgroup") public class AlertGroup { /** @@ -36,11 +36,10 @@ public class AlertGroup { */ @TableField(value = "group_name") private String groupName; - /** - * group_type - */ - @TableField(value = "group_type") - private AlertType groupType; + + @TableField(value = "alert_instance_ids") + private String alertInstanceIds; + /** * description */ @@ -57,6 +56,12 @@ public class AlertGroup { @TableField(value = "update_time") private Date updateTime; + /** + * create_user_id + */ + @TableField(value = "create_user_id") + private int createUserId; + public int getId() { return id; } @@ -73,14 +78,6 @@ public class AlertGroup { this.groupName = groupName; } - public AlertType getGroupType() { - return groupType; - } - - public void setGroupType(AlertType groupType) { - this.groupType = groupType; - } - public Date getCreateTime() { return createTime; } @@ -105,6 +102,22 @@ public class AlertGroup { this.description = description; } + public int getCreateUserId() { + return createUserId; + } + + public void setCreateUserId(int createUserId) { + this.createUserId = createUserId; + } + + public String getAlertInstanceIds() { + return alertInstanceIds; + } + + public void setAlertInstanceIds(String alertInstanceIds) { + this.alertInstanceIds = alertInstanceIds; + } + @Override public boolean equals(Object o) { if (this == o) { @@ -119,10 +132,13 @@ public class AlertGroup { if (id != that.id) { return false; } + if (createUserId != that.createUserId) { + return false; + } if (groupName != null ? !groupName.equals(that.groupName) : that.groupName != null) { return false; } - if (groupType != that.groupType) { + if (alertInstanceIds != null ? !alertInstanceIds.equals(that.alertInstanceIds) : that.alertInstanceIds != null) { return false; } if (description != null ? !description.equals(that.description) : that.description != null) { @@ -135,8 +151,9 @@ public class AlertGroup { @Override public int hashCode() { int result = id; + result = 31 * result + createUserId; result = 31 * result + (groupName != null ? groupName.hashCode() : 0); - result = 31 * result + (groupType != null ? groupType.hashCode() : 0); + result = 31 * result + (alertInstanceIds != null ? alertInstanceIds.hashCode() : 0); result = 31 * result + (description != null ? description.hashCode() : 0); result = 31 * result + (createTime != null ? createTime.hashCode() : 0); result = 31 * result + (updateTime != null ? updateTime.hashCode() : 0); @@ -145,13 +162,13 @@ public class AlertGroup { @Override public String toString() { - return "AlertGroup{" + - "id=" + id + - ", groupName='" + groupName + '\'' + - ", groupType=" + groupType + - ", description='" + description + '\'' + - ", createTime=" + createTime + - ", updateTime=" + updateTime + - '}'; + return "AlertGroup{" + + "id=" + id + + "createUserId=" + createUserId + + ", groupName='" + groupName + '\'' + + ", description='" + description + '\'' + + ", createTime=" + createTime + + ", updateTime=" + updateTime + + '}'; } } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/UserAlertGroup.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertPluginInstance.java similarity index 51% rename from dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/UserAlertGroup.java rename to dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertPluginInstance.java index 902196d186..5993697a56 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/UserAlertGroup.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertPluginInstance.java @@ -14,63 +14,71 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.entity; +import java.util.Date; + import com.baomidou.mybatisplus.annotation.IdType; import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableName; -import com.fasterxml.jackson.annotation.JsonFormat; - -import java.util.Date; /** - * user alert group + * t_ds_alert_plugin_instance */ -@TableName("t_ds_relation_user_alertgroup") -public class UserAlertGroup { +@TableName("t_ds_alert_plugin_instance") +public class AlertPluginInstance { /** * id */ - @TableId(value="id", type=IdType.AUTO) + @TableId(value = "id", type = IdType.AUTO) private int id; /** - * id + * plugin_define_id */ - @TableField("alertgroup_id") - private int alertgroupId; + @TableField("plugin_define_id") + private int pluginDefineId; /** - * alert group name + * alert plugin instance name */ - @TableField(exist = false) - private String alertgroupName; + @TableField("instance_name") + private String instanceName; /** - * user id + * plugin_instance_params */ - private int userId; + @TableField("plugin_instance_params") + private String pluginInstanceParams; /** - * user name + * create_time */ - @TableField(exist = false) - private String userName; - - /** - * create time - */ - @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss",timezone="GMT+8") + @TableField("create_time") private Date createTime; /** - * update time + * update_time */ - @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss",timezone="GMT+8") + @TableField("update_time") private Date updateTime; + public AlertPluginInstance() { + this.createTime = new Date(); + this.updateTime = new Date(); + } + + public AlertPluginInstance(int pluginDefineId, String pluginInstanceParams, String instanceName) { + this.pluginDefineId = pluginDefineId; + this.pluginInstanceParams = pluginInstanceParams; + this.createTime = new Date(); + this.updateTime = new Date(); + this.instanceName = instanceName; + } + public int getId() { return id; } @@ -79,20 +87,20 @@ public class UserAlertGroup { this.id = id; } - public int getAlertgroupId() { - return alertgroupId; + public int getPluginDefineId() { + return pluginDefineId; } - public void setAlertgroupId(int alertgroupId) { - this.alertgroupId = alertgroupId; + public void setPluginDefineId(int pluginDefineId) { + this.pluginDefineId = pluginDefineId; } - public int getUserId() { - return userId; + public String getPluginInstanceParams() { + return pluginInstanceParams; } - public void setUserId(int userId) { - this.userId = userId; + public void setPluginInstanceParams(String pluginInstanceParams) { + this.pluginInstanceParams = pluginInstanceParams; } public Date getCreateTime() { @@ -111,32 +119,12 @@ public class UserAlertGroup { this.updateTime = updateTime; } - public String getAlertgroupName() { - return alertgroupName; - } - - public void setAlertgroupName(String alertgroupName) { - this.alertgroupName = alertgroupName; + public String getInstanceName() { + return instanceName; } - public String getUserName() { - return userName; - } - - public void setUserName(String userName) { - this.userName = userName; - } - - @Override - public String toString() { - return "UserAlertGroup{" + - "id=" + id + - ", alertgroupId=" + alertgroupId + - ", alertgroupName='" + alertgroupName + '\'' + - ", userId=" + userId + - ", userName='" + userName + '\'' + - ", createTime=" + createTime + - ", updateTime=" + updateTime + - '}'; + public void setInstanceName(String instanceName) { + this.instanceName = instanceName; } } + diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/PluginDefine.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/PluginDefine.java new file mode 100644 index 0000000000..2be8988a08 --- /dev/null +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/PluginDefine.java @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.dao.entity; + +import java.util.Date; + +import com.baomidou.mybatisplus.annotation.IdType; +import com.baomidou.mybatisplus.annotation.TableField; +import com.baomidou.mybatisplus.annotation.TableId; +import com.baomidou.mybatisplus.annotation.TableName; + +/** + * t_ds_plugin_define + */ +@TableName("t_ds_plugin_define") +public class PluginDefine { + + /** + * id + */ + @TableId(value = "id", type = IdType.AUTO) + private int id; + + /** + * plugin name + */ + @TableField("plugin_name") + private String pluginName; + + /** + * plugin_type + */ + @TableField("plugin_type") + private String pluginType; + + /** + * plugin_params + */ + @TableField("plugin_params") + private String pluginParams; + + /** + * create_time + */ + @TableField("create_time") + private Date createTime; + + /** + * update_time + */ + @TableField("update_time") + private Date updateTime; + + public PluginDefine(String pluginName, String pluginType, String pluginParams) { + this.pluginName = pluginName; + this.pluginType = pluginType; + this.pluginParams = pluginParams; + this.createTime = new Date(); + this.updateTime = new Date(); + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public String getPluginName() { + return pluginName; + } + + public void setPluginName(String pluginName) { + this.pluginName = pluginName; + } + + public String getPluginType() { + return pluginType; + } + + public void setPluginType(String pluginType) { + this.pluginType = pluginType; + } + + public String getPluginParams() { + return pluginParams; + } + + public void setPluginParams(String pluginParams) { + this.pluginParams = pluginParams; + } + + public Date getCreateTime() { + return createTime; + } + + public void setCreateTime(Date createTime) { + this.createTime = createTime; + } + + public Date getUpdateTime() { + return updateTime; + } + + public void setUpdateTime(Date updateTime) { + this.updateTime = updateTime; + } +} + diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java index 56f6cfe905..0c2b844fd1 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java @@ -139,16 +139,6 @@ public class ProcessDefinition { */ private String connects; - /** - * receivers - */ - private String receivers; - - /** - * receivers cc - */ - private String receiversCc; - /** * schedule release state : online/offline */ @@ -175,6 +165,11 @@ public class ProcessDefinition { */ private String resourceIds; + /** + * warningGroupId + */ + @TableField(exist = false) + private int warningGroupId; public String getName() { return name; @@ -272,7 +267,6 @@ public class ProcessDefinition { this.projectName = projectName; } - public String getGlobalParams() { return globalParams; } @@ -324,22 +318,6 @@ public class ProcessDefinition { this.connects = connects; } - public String getReceivers() { - return receivers; - } - - public void setReceivers(String receivers) { - this.receivers = receivers; - } - - public String getReceiversCc() { - return receiversCc; - } - - public void setReceiversCc(String receiversCc) { - this.receiversCc = receiversCc; - } - public ReleaseState getScheduleReleaseState() { return scheduleReleaseState; } @@ -388,35 +366,42 @@ public class ProcessDefinition { this.modifyBy = modifyBy; } + public int getWarningGroupId() { + return warningGroupId; + } + + public void setWarningGroupId(int warningGroupId) { + this.warningGroupId = warningGroupId; + } + @Override public String toString() { - return "ProcessDefinition{" + - "id=" + id + - ", name='" + name + '\'' + - ", version=" + version + - ", releaseState=" + releaseState + - ", projectId=" + projectId + - ", processDefinitionJson='" + processDefinitionJson + '\'' + - ", description='" + description + '\'' + - ", globalParams='" + globalParams + '\'' + - ", globalParamList=" + globalParamList + - ", globalParamMap=" + globalParamMap + - ", createTime=" + createTime + - ", updateTime=" + updateTime + - ", flag=" + flag + - ", userId=" + userId + - ", userName='" + userName + '\'' + - ", projectName='" + projectName + '\'' + - ", locations='" + locations + '\'' + - ", connects='" + connects + '\'' + - ", receivers='" + receivers + '\'' + - ", receiversCc='" + receiversCc + '\'' + - ", scheduleReleaseState=" + scheduleReleaseState + - ", timeout=" + timeout + - ", tenantId=" + tenantId + - ", modifyBy='" + modifyBy + '\'' + - ", resourceIds='" + resourceIds + '\'' + - '}'; + return "ProcessDefinition{" + + "id=" + id + + ", name='" + name + '\'' + + ", version=" + version + + ", releaseState=" + releaseState + + ", projectId=" + projectId + + ", processDefinitionJson='" + processDefinitionJson + '\'' + + ", description='" + description + '\'' + + ", globalParams='" + globalParams + '\'' + + ", globalParamList=" + globalParamList + + ", globalParamMap=" + globalParamMap + + ", createTime=" + createTime + + ", updateTime=" + updateTime + + ", flag=" + flag + + ", userId=" + userId + + ", userName='" + userName + '\'' + + ", projectName='" + projectName + '\'' + + ", locations='" + locations + '\'' + + ", connects='" + connects + '\'' + + ", scheduleReleaseState=" + scheduleReleaseState + + ", timeout=" + timeout + + ", warningGroupId=" + warningGroupId + + ", tenantId=" + tenantId + + ", modifyBy='" + modifyBy + '\'' + + ", resourceIds='" + resourceIds + '\'' + + '}'; } } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinitionVersion.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinitionVersion.java index 26779ba925..1c4d979415 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinitionVersion.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinitionVersion.java @@ -20,6 +20,7 @@ package org.apache.dolphinscheduler.dao.entity; import java.util.Date; import com.baomidou.mybatisplus.annotation.IdType; +import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableName; import com.fasterxml.jackson.annotation.JsonFormat; @@ -57,16 +58,6 @@ public class ProcessDefinitionVersion { */ private String description; - /** - * receivers - */ - private String receivers; - - /** - * receivers cc - */ - private String receiversCc; - /** * process warning time out. unit: minute */ @@ -98,6 +89,13 @@ public class ProcessDefinitionVersion { */ private String connects; + + /** + * warningGroupId + */ + @TableField(exist = false) + private int warningGroupId; + public String getGlobalParams() { return globalParams; } @@ -170,22 +168,6 @@ public class ProcessDefinitionVersion { this.connects = connects; } - public String getReceivers() { - return receivers; - } - - public void setReceivers(String receivers) { - this.receivers = receivers; - } - - public String getReceiversCc() { - return receiversCc; - } - - public void setReceiversCc(String receiversCc) { - this.receiversCc = receiversCc; - } - public int getTimeout() { return timeout; } @@ -202,23 +184,30 @@ public class ProcessDefinitionVersion { this.resourceIds = resourceIds; } + public int getWarningGroupId() { + return warningGroupId; + } + + public void setWarningGroupId(int warningGroupId) { + this.warningGroupId = warningGroupId; + } + @Override public String toString() { return "ProcessDefinitionVersion{" - + "id=" + id - + ", processDefinitionId=" + processDefinitionId - + ", version=" + version - + ", processDefinitionJson='" + processDefinitionJson + '\'' - + ", description='" + description + '\'' - + ", globalParams='" + globalParams + '\'' - + ", createTime=" + createTime - + ", locations='" + locations + '\'' - + ", connects='" + connects + '\'' - + ", receivers='" + receivers + '\'' - + ", receiversCc='" + receiversCc + '\'' - + ", timeout=" + timeout - + ", resourceIds='" + resourceIds + '\'' - + '}'; + + "id=" + id + + ", processDefinitionId=" + processDefinitionId + + ", version=" + version + + ", processDefinitionJson='" + processDefinitionJson + '\'' + + ", description='" + description + '\'' + + ", globalParams='" + globalParams + '\'' + + ", createTime=" + createTime + + ", locations='" + locations + '\'' + + ", connects='" + connects + '\'' + + ", timeout=" + timeout + + ", warningGroupId=" + warningGroupId + + ", resourceIds='" + resourceIds + '\'' + + '}'; } public static Builder newBuilder() { @@ -235,9 +224,8 @@ public class ProcessDefinitionVersion { private Date createTime; private String locations; private String connects; - private String receivers; - private String receiversCc; private int timeout; + private int warningGroupId; private String resourceIds; private Builder() { @@ -288,18 +276,13 @@ public class ProcessDefinitionVersion { return this; } - public Builder receivers(String receivers) { - this.receivers = receivers; - return this; - } - - public Builder receiversCc(String receiversCc) { - this.receiversCc = receiversCc; + public Builder timeout(int timeout) { + this.timeout = timeout; return this; } - public Builder timeout(int timeout) { - this.timeout = timeout; + public Builder warningGroupId(int warningGroupId) { + this.warningGroupId = warningGroupId; return this; } @@ -319,9 +302,8 @@ public class ProcessDefinitionVersion { processDefinitionVersion.setCreateTime(createTime); processDefinitionVersion.setLocations(locations); processDefinitionVersion.setConnects(connects); - processDefinitionVersion.setReceivers(receivers); - processDefinitionVersion.setReceiversCc(receiversCc); processDefinitionVersion.setTimeout(timeout); + processDefinitionVersion.setWarningGroupId(warningGroupId); processDefinitionVersion.setResourceIds(resourceIds); return processDefinitionVersion; } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java index e3a3f11386..f1d43a353b 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java @@ -228,18 +228,6 @@ public class ProcessInstance { * varPool string */ private String varPool; - - /** - * receivers for api - */ - @TableField(exist = false) - private String receivers; - - /** - * receivers cc for api - */ - @TableField(exist = false) - private String receiversCc; public ProcessInstance() { @@ -268,7 +256,7 @@ public class ProcessInstance { public void setVarPool(String varPool) { this.varPool = varPool; } - + public ProcessDefinition getProcessDefinition() { return processDefinition; } @@ -591,22 +579,6 @@ public class ProcessInstance { this.tenantId = tenantId; } - public String getReceivers() { - return receivers; - } - - public void setReceivers(String receivers) { - this.receivers = receivers; - } - - public String getReceiversCc() { - return receiversCc; - } - - public void setReceiversCc(String receiversCc) { - this.receiversCc = receiversCc; - } - @Override public String toString() { return "ProcessInstance{" @@ -679,12 +651,6 @@ public class ProcessInstance { + timeout + ", tenantId=" + tenantId - + ", receivers='" - + receivers - + '\'' - + ", receiversCc='" - + receiversCc - + '\'' + '}'; } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.java index 0fd26b9f79..b206a787b6 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.java @@ -55,17 +55,16 @@ public interface AlertGroupMapper extends BaseMapper { */ List queryByUserId(@Param("userId") int userId); - /** - * query by alert type - * @param alertType alertType + * query all group list * @return alertgroup list */ - List queryByAlertType(@Param("alertType") AlertType alertType); + List queryAllGroupList(); /** - * query all group list - * @return alertgroup list + * queryAlertGroupInstanceIdsById + * @param alertGroupId + * @return */ - List queryAllGroupList(); + String queryAlertGroupInstanceIdsById(@Param("alertGroupId") int alertGroupId); } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapper.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapper.java similarity index 62% rename from dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapper.java rename to dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapper.java index f374c202cd..618a8774c7 100644 --- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapper.java +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapper.java @@ -14,32 +14,34 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.mapper; -import org.apache.dolphinscheduler.dao.entity.User; -import org.apache.dolphinscheduler.dao.entity.UserAlertGroup; -import com.baomidou.mybatisplus.core.mapper.BaseMapper; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; + import org.apache.ibatis.annotations.Param; import java.util.List; -/** - * user alertgroup relation mapper interface - */ -public interface UserAlertGroupMapper extends BaseMapper { +import com.baomidou.mybatisplus.core.mapper.BaseMapper; + +public interface AlertPluginInstanceMapper extends BaseMapper { /** - * delete user alertgroup relation by alertgroupId - * @param alertgroupId alertgroupId - * @return delete result + * query all alert plugin instance + * + * @return AlertPluginInstance list */ - int deleteByAlertgroupId(@Param("alertgroupId") int alertgroupId); + List queryAllAlertPluginInstanceList(); /** - * list user by alertgroupId - * @param alertgroupId alertgroupId - * @return user list + * query by alert group id + * + * @param ids + * @return AlertPluginInstance list */ - List listUserByAlertgroupId(@Param("alertgroupId") int alertgroupId); + List queryByIds(@Param("ids") List ids); + + List queryByInstanceName(@Param("instanceName")String instanceName); } diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.java new file mode 100644 index 0000000000..e2d08b5222 --- /dev/null +++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.java @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.dao.mapper; + +import org.apache.dolphinscheduler.dao.entity.PluginDefine; + +import org.apache.ibatis.annotations.Param; + +import java.util.List; + +import com.baomidou.mybatisplus.core.mapper.BaseMapper; + +public interface PluginDefineMapper extends BaseMapper { + + /** + * query all plugin define + * + * @return PluginDefine list + */ + List queryAllPluginDefineList(); + + /** + * query by plugin type + * + * @param pluginType pluginType + * @return PluginDefine list + */ + List queryByPluginType(@Param("pluginType") String pluginType); + + /** + * query detail by id + * + * @param id id + * @return PluginDefineDetail + */ + PluginDefine queryDetailById(@Param("id") int id); + + /** + * query by name and type + * + * @param pluginName + * @param pluginType + * @return + */ + List queryByNameAndType(@Param("pluginName") String pluginName, @Param("pluginType") String pluginType); +} diff --git a/dolphinscheduler-dao/src/main/resources/datasource.properties b/dolphinscheduler-dao/src/main/resources/datasource.properties index f8fe97bcea..fa07c6a6d3 100644 --- a/dolphinscheduler-dao/src/main/resources/datasource.properties +++ b/dolphinscheduler-dao/src/main/resources/datasource.properties @@ -15,9 +15,15 @@ # limitations under the License. # +# mysql +#spring.datasource.driver-class-name=com.mysql.jdbc.Driver +#spring.datasource.url=jdbc:mysql://localhost:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8 +#spring.datasource.username=root +#spring.datasource.password=123456 + # postgresql spring.datasource.driver-class-name=org.postgresql.Driver -spring.datasource.url=jdbc:postgresql://localhost:5432/dolphinscheduler +spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/dolphinscheduler spring.datasource.username=test spring.datasource.password=test diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.xml index 08d1ea365f..e60dedfd6c 100644 --- a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.xml +++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapper.xml @@ -19,7 +19,8 @@ - id, group_name, group_type, description, create_time, update_time + id + , group_name, description,alert_instance_ids, create_user_id,create_time, update_time - + - + + + \ No newline at end of file diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertMapper.xml index 036e0ed0fe..9be5c7c784 100644 --- a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertMapper.xml +++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/AlertMapper.xml @@ -19,8 +19,9 @@ - id, title, show_type, content, alert_type, alert_status, log, - alertgroup_id, receivers, receivers_cc, create_time, update_time + id + , title, content, alert_status, log, + alertgroup_id, create_time, update_time - SELECT - - - - FROM t_ds_relation_user_alertgroup g_u - JOIN t_ds_user u on g_u.user_id = u.id - WHERE g_u.alertgroup_id = #{alertgroupId} + + + + + - + + + + \ No newline at end of file diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.xml new file mode 100644 index 0000000000..49abe7cb2d --- /dev/null +++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.xml @@ -0,0 +1,45 @@ + + + + + + + + + + + + + + \ No newline at end of file diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapper.xml index f1179c2b33..6a9165f076 100644 --- a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapper.xml +++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapper.xml @@ -19,13 +19,14 @@ - id, name, version, release_state, project_id, user_id, process_definition_json, description, - global_params, flag, locations, connects, receivers, receivers_cc, create_time, timeout, + id + , name, version, release_state, project_id, user_id, process_definition_json, description, + global_params, flag, locations, connects, warning_group_id, create_time, timeout, tenant_id, update_time, modify_by, resource_ids SELECT td.id, td.name, td.version, td.release_state, td.project_id, td.user_id, td.description, td.global_params, - td.flag, td.receivers, td.receivers_cc, td.timeout, td.tenant_id, td.modify_by, td.update_time, td.create_time, + td.flag, td.warning_group_id, td.timeout, td.tenant_id, td.modify_by, td.update_time, td.create_time, sc.schedule_release_state, tu.user_name FROM t_ds_process_definition td left join (select process_definition_id,release_state as schedule_release_state from t_ds_schedules group by @@ -101,7 +102,7 @@ select max(version) diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml index 0ec03846d0..a6d72d0d2d 100644 --- a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml +++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml @@ -19,6 +19,7 @@ + id, name, description, user_id, flag, create_time, update_time diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/AlertDaoTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/AlertDaoTest.java index ef3f0ffbb9..0137bd5639 100644 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/AlertDaoTest.java +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/AlertDaoTest.java @@ -14,35 +14,30 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao; import org.apache.dolphinscheduler.common.enums.AlertStatus; -import org.apache.dolphinscheduler.common.enums.AlertType; -import org.apache.dolphinscheduler.common.enums.ShowType; import org.apache.dolphinscheduler.dao.entity.Alert; -import org.junit.Assert; -import org.junit.Test; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import java.util.Arrays; import java.util.List; +import org.junit.Assert; +import org.junit.Test; + public class AlertDaoTest { + @Test - public void testAlertDao(){ + public void testAlertDao() { AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class); Alert alert = new Alert(); alert.setTitle("Mysql Exception"); - alert.setShowType(ShowType.TEXT); - alert.setContent("[\"alarm time：2018-02-05\", \"service name：MYSQL_ALTER\", \"alarm name：MYSQL_ALTER_DUMP\", " + - "\"get the alarm exception.！，interface error，exception information：timed out\", \"request address：http://blog.csdn.net/dreamInTheWorld/article/details/78539286\"]"); - alert.setAlertType(AlertType.EMAIL); + alert.setContent("[\"alarm time：2018-02-05\", \"service name：MYSQL_ALTER\", \"alarm name：MYSQL_ALTER_DUMP\", " + + "\"get the alarm exception.！，interface error，exception information：timed out\", \"request address：http://blog.csdn.net/dreamInTheWorld/article/details/78539286\"]"); alert.setAlertGroupId(1); alert.setAlertStatus(AlertStatus.WAIT_EXECUTION); alertDao.addAlert(alert); - List alerts = alertDao.listWaitExecutionAlert(); Assert.assertNotNull(alerts); Assert.assertNotEquals(0, alerts.size()); diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapperTest.java index c8d0953bc6..db42a2f57b 100644 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapperTest.java +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertGroupMapperTest.java @@ -14,38 +14,36 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.mapper; +import static org.hamcrest.Matchers.greaterThan; +import static org.hamcrest.Matchers.greaterThanOrEqualTo; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThat; -import org.apache.dolphinscheduler.common.enums.AlertType; import org.apache.dolphinscheduler.common.utils.DateUtils; -import org.apache.dolphinscheduler.dao.entity.AccessToken; import org.apache.dolphinscheduler.dao.entity.AlertGroup; -import org.apache.dolphinscheduler.dao.entity.UserAlertGroup; -import com.baomidou.mybatisplus.core.metadata.IPage; -import com.baomidou.mybatisplus.extension.plugins.pagination.Page; -import org.junit.Assert; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.test.annotation.Rollback; -import org.springframework.test.context.junit4.SpringRunner; -import java.util.Date; import java.util.HashMap; import java.util.List; import java.util.Map; - import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.annotation.Rollback; +import org.springframework.test.context.junit4.SpringRunner; import org.springframework.transaction.annotation.Transactional; -import static org.hamcrest.Matchers.*; -import static org.junit.Assert.*; +import com.baomidou.mybatisplus.core.metadata.IPage; +import com.baomidou.mybatisplus.extension.plugins.pagination.Page; /** - * AlertGroup mapper test + * AlertGroup mapper test */ @RunWith(SpringRunner.class) @SpringBootTest @@ -53,22 +51,17 @@ import static org.junit.Assert.*; @Rollback(true) public class AlertGroupMapperTest { - @Autowired AlertGroupMapper alertGroupMapper; - @Autowired - UserAlertGroupMapper userAlertGroupMapper; - - /** * test insert */ @Test - public void testInsert(){ + public void testInsert() { AlertGroup alertGroup = createAlertGroup(); assertNotNull(alertGroup); - assertThat(alertGroup.getId(),greaterThan(0)); + assertThat(alertGroup.getId(), greaterThan(0)); } @@ -99,7 +92,7 @@ public class AlertGroupMapperTest { Integer offset = 2; Integer size = 2; - Map alertGroupMap = createAlertGroups(count,groupName); + Map alertGroupMap = createAlertGroups(count, groupName); Page page = new Page(offset, size); @@ -109,25 +102,23 @@ public class AlertGroupMapperTest { assertEquals(alertGroupList.size(), size.intValue()); - for (AlertGroup alertGroup : alertGroupList){ + for (AlertGroup alertGroup : alertGroupList) { AlertGroup resultAlertGroup = alertGroupMap.get(alertGroup.getId()); - if (resultAlertGroup != null){ - assertEquals(alertGroup,resultAlertGroup); + if (resultAlertGroup != null) { + assertEquals(alertGroup, resultAlertGroup); } } - } /** * test update */ @Test - public void testUpdate(){ + public void testUpdate() { AlertGroup alertGroup = createAlertGroup(); alertGroup.setGroupName("modify GroupName"); - alertGroup.setGroupType(AlertType.SMS); alertGroup.setDescription("modify GroupName"); alertGroup.setUpdateTime(DateUtils.getCurrentDate()); @@ -135,16 +126,15 @@ public class AlertGroupMapperTest { AlertGroup resultAlertGroup = alertGroupMapper.selectById(alertGroup.getId()); - assertEquals(alertGroup,resultAlertGroup); + assertEquals(alertGroup, resultAlertGroup); } - /** * test delete */ @Test - public void testDelete(){ + public void testDelete() { AlertGroup alertGroup = createAlertGroup(); @@ -156,7 +146,6 @@ public class AlertGroupMapperTest { } - /** * test query by groupname */ @@ -169,42 +158,9 @@ public class AlertGroupMapperTest { List alertGroupList = alertGroupMapper.queryByGroupName("testGroup"); - compareAlertGroups(alertGroupMap, alertGroupList); } - /** - * test query by userId - */ - @Test - public void testQueryByUserId() { - Integer count = 4; - Integer userId = 1; - - Map alertGroupMap = - createAlertGroups(count, userId); - - List alertGroupList = - alertGroupMapper.queryByUserId(userId); - - compareAlertGroups(alertGroupMap,alertGroupList); - - } - - /** - * test query by alert type - */ - @Test - public void testQueryByAlertType() { - Integer count = 4; - - Map alertGroupMap = createAlertGroups(count); - List alertGroupList = alertGroupMapper.queryByAlertType(AlertType.EMAIL); - - compareAlertGroups(alertGroupMap,alertGroupList); - - } - /** * test query all group list */ @@ -215,37 +171,38 @@ public class AlertGroupMapperTest { List alertGroupList = alertGroupMapper.queryAllGroupList(); - compareAlertGroups(alertGroupMap,alertGroupList); + compareAlertGroups(alertGroupMap, alertGroupList); } - /** * compare AlertGruops - * @param alertGroupMap alertGroupMap + * + * @param alertGroupMap alertGroupMap * @param alertGroupList alertGroupList */ - private void compareAlertGroups(Map alertGroupMap, - List alertGroupList){ + private void compareAlertGroups(Map alertGroupMap, + List alertGroupList) { // greaterThanOrEqualTo，beacuse maybe db have already exists - assertThat(alertGroupList.size(),greaterThanOrEqualTo(alertGroupMap.size())); + assertThat(alertGroupList.size(), greaterThanOrEqualTo(alertGroupMap.size())); - for (AlertGroup alertGroup : alertGroupList){ + for (AlertGroup alertGroup : alertGroupList) { AlertGroup resultAlertGroup = alertGroupMap.get(alertGroup.getId()); - if (resultAlertGroup != null){ - assertEquals(alertGroup,resultAlertGroup); + if (resultAlertGroup != null) { + assertEquals(alertGroup, resultAlertGroup); } } } + /** * insert + * * @return AlertGroup */ - private AlertGroup createAlertGroup(String groupName){ + private AlertGroup createAlertGroup(String groupName) { AlertGroup alertGroup = new AlertGroup(); alertGroup.setGroupName(groupName); alertGroup.setDescription("alert group 1"); - alertGroup.setGroupType(AlertType.EMAIL); alertGroup.setCreateTime(DateUtils.getCurrentDate()); alertGroup.setUpdateTime(DateUtils.getCurrentDate()); @@ -257,13 +214,13 @@ public class AlertGroupMapperTest { /** * insert + * * @return AlertGroup */ - private AlertGroup createAlertGroup(){ + private AlertGroup createAlertGroup() { AlertGroup alertGroup = new AlertGroup(); alertGroup.setGroupName("testGroup"); alertGroup.setDescription("testGroup"); - alertGroup.setGroupType(AlertType.EMAIL); alertGroup.setCreateTime(DateUtils.getCurrentDate()); alertGroup.setUpdateTime(DateUtils.getCurrentDate()); @@ -275,81 +232,41 @@ public class AlertGroupMapperTest { /** * create AlertGroups - * @param count create AlertGroup count + * + * @param count create AlertGroup count * @param groupName groupName * @return AlertGroup map */ - private Map createAlertGroups( - Integer count,String groupName){ - Map alertGroupMap = new HashMap<>(); + private Map createAlertGroups( + Integer count, String groupName) { + Map alertGroupMap = new HashMap<>(); - AlertGroup alertGroup = null; - for (int i = 0 ; i < count; i++){ + AlertGroup alertGroup = null; + for (int i = 0; i < count; i++) { alertGroup = createAlertGroup(groupName); - alertGroupMap.put(alertGroup.getId(),alertGroup); - } - - return alertGroupMap; - } - - - /** - * create AlertGroups - * @param count create AlertGroup count - * @return AlertGroup map - */ - private Map createAlertGroups( - Integer count){ - Map alertGroupMap = new HashMap<>(); - - AlertGroup alertGroup = null; - for (int i = 0 ; i < count; i++){ - alertGroup = createAlertGroup(); - alertGroupMap.put(alertGroup.getId(),alertGroup); + alertGroupMap.put(alertGroup.getId(), alertGroup); } return alertGroupMap; } - /** * create AlertGroups + * * @param count create AlertGroup count * @return AlertGroup map */ - private Map createAlertGroups( - Integer count,Integer userId){ - Map alertGroupMap = new HashMap<>(); + private Map createAlertGroups( + Integer count) { + Map alertGroupMap = new HashMap<>(); - AlertGroup alertGroup = null; - for (int i = 0 ; i < count; i++){ + AlertGroup alertGroup = null; + for (int i = 0; i < count; i++) { alertGroup = createAlertGroup(); - - createUserAlertGroup(userId,alertGroup.getId()); - - alertGroupMap.put(alertGroup.getId(),alertGroup); + alertGroupMap.put(alertGroup.getId(), alertGroup); } return alertGroupMap; } - /** - * create AlertGroup - * @param userId userId - * @param alertgroupId alertgroupId - * @return UserAlertGroup - */ - private UserAlertGroup createUserAlertGroup( - Integer userId,Integer alertgroupId){ - UserAlertGroup userAlertGroup = new UserAlertGroup(); - userAlertGroup.setAlertgroupId(alertgroupId); - userAlertGroup.setUserId(userId); - userAlertGroup.setCreateTime(DateUtils.getCurrentDate()); - userAlertGroup.setUpdateTime(DateUtils.getCurrentDate()); - - userAlertGroupMapper.insert(userAlertGroup); - - return userAlertGroup; - } - } \ No newline at end of file diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertMapperTest.java index 064305d33b..63fcca2973 100644 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertMapperTest.java +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertMapperTest.java @@ -14,14 +14,22 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.mapper; +import static org.hamcrest.Matchers.greaterThan; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThat; + import org.apache.dolphinscheduler.common.enums.AlertStatus; -import org.apache.dolphinscheduler.common.enums.AlertType; -import org.apache.dolphinscheduler.common.enums.ShowType; import org.apache.dolphinscheduler.common.utils.DateUtils; import org.apache.dolphinscheduler.dao.entity.Alert; -import org.junit.Assert; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; + import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; @@ -30,13 +38,8 @@ import org.springframework.test.annotation.Rollback; import org.springframework.test.context.junit4.SpringRunner; import org.springframework.transaction.annotation.Transactional; -import java.util.*; - -import static org.hamcrest.Matchers.*; -import static org.junit.Assert.*; - /** - * alert mapper test + * alert mapper test */ @RunWith(SpringRunner.class) @SpringBootTest @@ -49,21 +52,22 @@ public class AlertMapperTest { /** * test insert + * * @return */ @Test - public void testInsert(){ + public void testInsert() { Alert expectedAlert = createAlert(); assertThat(expectedAlert.getId(), greaterThan(0)); } - /** * test select by id + * * @return */ @Test - public void testSelectById(){ + public void testSelectById() { Alert expectedAlert = createAlert(); Alert actualAlert = alertMapper.selectById(expectedAlert.getId()); assertEquals(expectedAlert, actualAlert); @@ -73,7 +77,7 @@ public class AlertMapperTest { * test update */ @Test - public void testUpdate(){ + public void testUpdate() { Alert expectedAlert = createAlert(); @@ -92,7 +96,7 @@ public class AlertMapperTest { * test delete */ @Test - public void testDelete(){ + public void testDelete() { Alert expectedAlert = createAlert(); alertMapper.deleteById(expectedAlert.getId()); @@ -102,7 +106,6 @@ public class AlertMapperTest { assertNull(actualAlert); } - /** * test list alert by status */ @@ -111,61 +114,57 @@ public class AlertMapperTest { Integer count = 10; AlertStatus waitExecution = AlertStatus.WAIT_EXECUTION; - Map expectedAlertMap = createAlertMap(count, waitExecution); + Map expectedAlertMap = createAlertMap(count, waitExecution); List actualAlerts = alertMapper.listAlertByStatus(waitExecution); - for (Alert actualAlert : actualAlerts){ + for (Alert actualAlert : actualAlerts) { Alert expectedAlert = expectedAlertMap.get(actualAlert.getId()); - if (expectedAlert != null){ - assertEquals(expectedAlert,actualAlert); + if (expectedAlert != null) { + assertEquals(expectedAlert, actualAlert); } } } /** - * create alert map - * @param count alert count + * create alert map + * + * @param count alert count * @param alertStatus alert status * @return alert map */ - private Map createAlertMap(Integer count,AlertStatus alertStatus){ - Map alertMap = new HashMap<>(); + private Map createAlertMap(Integer count, AlertStatus alertStatus) { + Map alertMap = new HashMap<>(); - for (int i = 0 ; i < count ;i++){ + for (int i = 0; i < count; i++) { Alert alert = createAlert(alertStatus); - alertMap.put(alert.getId(),alert); + alertMap.put(alert.getId(), alert); } - return alertMap; - } - /** * create alert + * * @return alert * @throws Exception */ - private Alert createAlert(){ + private Alert createAlert() { return createAlert(AlertStatus.WAIT_EXECUTION); } /** * create alert + * * @param alertStatus alert status * @return alert */ - private Alert createAlert(AlertStatus alertStatus){ + private Alert createAlert(AlertStatus alertStatus) { Alert alert = new Alert(); - alert.setShowType(ShowType.TABLE); alert.setTitle("test alert"); alert.setContent("[{'type':'WORKER','host':'192.168.xx.xx','event':'server down','warning level':'serious'}]"); - alert.setAlertType(AlertType.EMAIL); alert.setAlertStatus(alertStatus); alert.setLog("success"); - alert.setReceivers("aa@aa.com"); - alert.setReceiversCc("bb@aa.com"); alert.setCreateTime(DateUtils.getCurrentDate()); alert.setUpdateTime(DateUtils.getCurrentDate()); diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapperTest.java new file mode 100644 index 0000000000..6ebcde39f9 --- /dev/null +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AlertPluginInstanceMapperTest.java @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.dao.mapper; + +import org.apache.dolphinscheduler.common.utils.DateUtils; +import org.apache.dolphinscheduler.dao.entity.AlertGroup; +import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance; +import org.apache.dolphinscheduler.dao.entity.PluginDefine; + +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.annotation.Rollback; +import org.springframework.test.context.junit4.SpringRunner; +import org.springframework.transaction.annotation.Transactional; + +/** + * AlertPluginInstanceMapper mapper test + */ +@RunWith(SpringRunner.class) +@SpringBootTest +@Transactional +@Rollback(true) +public class AlertPluginInstanceMapperTest { + + @Autowired + AlertPluginInstanceMapper alertPluginInstanceMapper; + + @Autowired + PluginDefineMapper pluginDefineMapper; + + @Autowired + AlertGroupMapper alertGroupMapper; + + @Test + public void testQueryAllAlertPluginInstanceList() { + AlertPluginInstance alertPluginInstance = createAlertPluginInstance(); + List alertPluginInstanceList = alertPluginInstanceMapper.queryAllAlertPluginInstanceList(); + Assert.assertTrue(alertPluginInstanceList.size() > 0); + } + + @Test + public void testQueryByAlertGroupId() { + createAlertPluginInstance(); + List testAlertGroupList = alertGroupMapper.queryByGroupName("test_group_01"); + Assert.assertNotNull(testAlertGroupList); + Assert.assertTrue(testAlertGroupList.size() > 0); + AlertGroup alertGroup = testAlertGroupList.get(0); + } + + /** + * insert + * + * @return AlertPluginInstance + */ + private AlertPluginInstance createAlertPluginInstance() { + + PluginDefine pluginDefine = createPluginDefine(); + AlertGroup alertGroup = createAlertGroup("test_group_01"); + AlertPluginInstance alertPluginInstance = new AlertPluginInstance(pluginDefine.getId(), "", "test_instance"); + alertPluginInstanceMapper.insert(alertPluginInstance); + return alertPluginInstance; + } + + /** + * insert + * + * @return PluginDefine + */ + private PluginDefine createPluginDefine() { + PluginDefine pluginDefine = new PluginDefine("test plugin", "alert", ""); + pluginDefineMapper.insert(pluginDefine); + return pluginDefine; + } + + /** + * insert + * + * @return AlertGroup + */ + private AlertGroup createAlertGroup(String groupName) { + AlertGroup alertGroup = new AlertGroup(); + alertGroup.setGroupName(groupName); + alertGroup.setDescription("alert group 1"); + + alertGroup.setCreateTime(DateUtils.getCurrentDate()); + alertGroup.setUpdateTime(DateUtils.getCurrentDate()); + + alertGroupMapper.insert(alertGroup); + + return alertGroup; + } +} diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineTest.java new file mode 100644 index 0000000000..d8636a6fbc --- /dev/null +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/PluginDefineTest.java @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.dao.mapper; + +import org.apache.dolphinscheduler.dao.entity.PluginDefine; + +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.annotation.Rollback; +import org.springframework.test.context.junit4.SpringRunner; +import org.springframework.transaction.annotation.Transactional; + +@RunWith(SpringRunner.class) +@SpringBootTest +@Transactional +@Rollback(true) + +public class PluginDefineTest { + + @Autowired + PluginDefineMapper pluginDefineMapper; + + @Test + public void testQueryAllPluginDefineList() { + createPluginDefine(); + List pluginDefines = pluginDefineMapper.queryAllPluginDefineList(); + Assert.assertTrue(pluginDefines.size() > 0); + } + + @Test + public void testQeryByPluginType() { + PluginDefine pluginDefine = createPluginDefine(); + List pluginDefines = pluginDefineMapper.queryByPluginType(pluginDefine.getPluginType()); + Assert.assertTrue(pluginDefines.size() > 0); + Assert.assertEquals(pluginDefines.get(0).getPluginType(), pluginDefine.getPluginType()); + } + + @Test + public void testQueryByNameAndType() { + PluginDefine pluginDefine = createPluginDefine(); + List pluginDefines = pluginDefineMapper.queryByNameAndType(pluginDefine.getPluginName(), pluginDefine.getPluginType()); + Assert.assertTrue(pluginDefines.size() > 0); + Assert.assertEquals(pluginDefines.get(0).getPluginType(), pluginDefine.getPluginType()); + Assert.assertEquals(pluginDefines.get(0).getPluginName(), pluginDefine.getPluginName()); + } + + /** + * insert + * + * @return PluginDefine + */ + private PluginDefine createPluginDefine() { + PluginDefine pluginDefine = new PluginDefine("test plugin", "alert", ""); + pluginDefineMapper.insert(pluginDefine); + return pluginDefine; + } +} diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapperTest.java index 34dd4fa96f..2046735221 100644 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapperTest.java +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionMapperTest.java @@ -77,8 +77,6 @@ public class ProcessDefinitionMapperTest { processDefinition.setUserId(101); processDefinition.setUpdateTime(new Date()); processDefinition.setCreateTime(new Date()); -// processDefinition.setGlobalParams("[{\"prop\":\"selenium_global_parameters_1\",\"direct\":\"IN\",\"type\":\"VARCHAR\",\"value\":\"selenium_global_parameters_value_1\"}]"); - processDefinitionMapper.insert(processDefinition); return processDefinition; } @@ -207,8 +205,6 @@ public class ProcessDefinitionMapperTest { processDefinition.setCreateTime(new Date()); processDefinition.setTenantId(tenant.getId()); processDefinition.setUserId(user.getId()); -// processDefinition.setGlobalParams("[{\"prop\":\"selenium_global_parameters_1\",\"direct\":\"IN\",\"type\":\"VARCHAR\",\"value\":\"selenium_global_parameters_value_1\"}]"); -// processDefinitionMapper.insert(processDefinition); ProcessDefinition processDefinition1 = processDefinitionMapper.queryByDefineName(project.getId(), "def 1"); diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionVersionMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionVersionMapperTest.java index e825e33847..938e26dd6c 100644 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionVersionMapperTest.java +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionVersionMapperTest.java @@ -76,8 +76,6 @@ public class ProcessDefinitionVersionMapperTest { processDefinitionVersion.setCreateTime(new Date()); processDefinitionVersion.setLocations(StringUtils.EMPTY); processDefinitionVersion.setConnects(StringUtils.EMPTY); - processDefinitionVersion.setReceivers(StringUtils.EMPTY); - processDefinitionVersion.setReceiversCc(StringUtils.EMPTY); processDefinitionVersion.setTimeout(10); processDefinitionVersion.setResourceIds("1,2"); processDefinitionVersionMapper.insert(processDefinitionVersion); @@ -101,8 +99,6 @@ public class ProcessDefinitionVersionMapperTest { processDefinitionVersion.setCreateTime(new Date()); processDefinitionVersion.setLocations(StringUtils.EMPTY); processDefinitionVersion.setConnects(StringUtils.EMPTY); - processDefinitionVersion.setReceivers(StringUtils.EMPTY); - processDefinitionVersion.setReceiversCc(StringUtils.EMPTY); processDefinitionVersion.setTimeout(10); processDefinitionVersion.setResourceIds("1,2"); processDefinitionVersionMapper.insert(processDefinitionVersion); diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapperTest.java deleted file mode 100644 index 2c5024f2ee..0000000000 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserAlertGroupMapperTest.java +++ /dev/null @@ -1,196 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.dao.mapper; - - -import org.apache.dolphinscheduler.common.enums.AlertType; -import org.apache.dolphinscheduler.common.enums.UserType; -import org.apache.dolphinscheduler.dao.entity.AlertGroup; -import org.apache.dolphinscheduler.dao.entity.User; -import org.apache.dolphinscheduler.dao.entity.UserAlertGroup; -import org.junit.Assert; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.test.annotation.Rollback; -import org.springframework.test.context.junit4.SpringRunner; -import org.springframework.transaction.annotation.Transactional; - -import java.util.Date; -import java.util.List; - -@RunWith(SpringRunner.class) -@SpringBootTest -@Transactional -@Rollback(true) -public class UserAlertGroupMapperTest { - - @Autowired - private UserMapper userMapper; - - @Autowired - AlertGroupMapper alertGroupMapper; - - @Autowired - private UserAlertGroupMapper userAlertGroupMapper; - - /** - * insert one UserAlertGroup - * @param user user - * @param alertGroup alertGroup - * @return UserAlertGroup - */ - private UserAlertGroup insertOne(User user,AlertGroup alertGroup){ - UserAlertGroup userAlertGroup = new UserAlertGroup(); - userAlertGroup.setAlertgroupName(alertGroup.getGroupName()); - userAlertGroup.setAlertgroupId(alertGroup.getId()); - userAlertGroup.setUserId(user.getId()); - userAlertGroup.setCreateTime(new Date()); - userAlertGroup.setUpdateTime(new Date()); - userAlertGroupMapper.insert(userAlertGroup); - return userAlertGroup; - } - - /** - * insert one UserAlertGroup - * @return UserAlertGroup - */ - private UserAlertGroup insertOne(){ - UserAlertGroup userAlertGroup = new UserAlertGroup(); - userAlertGroup.setAlertgroupName("dolphin_alert_group"); - userAlertGroup.setAlertgroupId(10); - userAlertGroup.setUserId(4); - userAlertGroup.setCreateTime(new Date()); - userAlertGroup.setUpdateTime(new Date()); - userAlertGroupMapper.insert(userAlertGroup); - return userAlertGroup; - } - - /** - * insert one user - * @return User - */ - private User insertOneUser(){ - User user = new User(); - user.setUserName("user1"); - user.setUserPassword("1"); - user.setEmail("xx@123.com"); - user.setUserType(UserType.GENERAL_USER); - user.setCreateTime(new Date()); - user.setTenantId(1); - user.setQueue("dolphin"); - user.setUpdateTime(new Date()); - userMapper.insert(user); - return user; - } - - /** - * insert one AlertGroup - * @return AlertGroup - */ - private AlertGroup insertOneAlertGroup(){ - //insertOne - AlertGroup alertGroup = new AlertGroup(); - alertGroup.setGroupName("alert group 1"); - alertGroup.setDescription("alert test1"); - alertGroup.setGroupType(AlertType.EMAIL); - - alertGroup.setCreateTime(new Date()); - alertGroup.setUpdateTime(new Date()); - alertGroupMapper.insert(alertGroup); - return alertGroup; - } - - /** - * test update - */ - @Test - public void testUpdate(){ - //insertOneUser - User user = insertOneUser(); - //insertOneAlertGroup - AlertGroup alertGroup = insertOneAlertGroup(); - - //insertOne - UserAlertGroup userAlertGroup = insertOne(); - //update - userAlertGroup.setUserId(user.getId()); - userAlertGroup.setAlertgroupId(alertGroup.getId()); - userAlertGroup.setUpdateTime(new Date()); - - int update = userAlertGroupMapper.updateById(userAlertGroup); - Assert.assertEquals(update, 1); - } - - /** - * test delete - */ - @Test - public void testDelete(){ - //insertOne - UserAlertGroup userAlertGroup = insertOne(); - //delete - int delete = userAlertGroupMapper.deleteById(userAlertGroup.getId()); - Assert.assertEquals(delete, 1); - } - - /** - * test query - */ - @Test - public void testQuery() { - //insertOne - UserAlertGroup userAlertGroup = insertOne(); - //query - List userAlertGroupList = userAlertGroupMapper.selectList(null); - Assert.assertNotEquals(userAlertGroupList.size(), 0); - } - - /** - * test delete by alertgroupId - */ - @Test - public void testDeleteByAlertgroupId() { - //insertOneUser - User user = insertOneUser(); - //insertOneAlertGroup - AlertGroup alertGroup = insertOneAlertGroup(); - - //insertOne - UserAlertGroup userAlertGroup = insertOne(user,alertGroup); - int delete = userAlertGroupMapper.deleteByAlertgroupId(alertGroup.getId()); - Assert.assertEquals(delete, 1); - } - - /** - * test list user by alertgroupId - */ - @Test - public void testListUserByAlertgroupId() { - //insertOneUser - User user = insertOneUser(); - //insertOneAlertGroup - AlertGroup alertGroup = insertOneAlertGroup(); - - //insertOne - UserAlertGroup userAlertGroup = insertOne(user,alertGroup); - List userList = userAlertGroupMapper.listUserByAlertgroupId(alertGroup.getId()); - Assert.assertNotEquals(userList.size(), 0); - - } -} \ No newline at end of file diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserMapperTest.java index d1d5ffe925..7f7ceb480a 100644 --- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserMapperTest.java +++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UserMapperTest.java @@ -14,14 +14,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.dao.mapper; -import com.baomidou.mybatisplus.core.metadata.IPage; -import com.baomidou.mybatisplus.extension.plugins.pagination.Page; -import org.apache.dolphinscheduler.common.enums.AlertType; import org.apache.dolphinscheduler.common.enums.UserType; import org.apache.dolphinscheduler.common.utils.DateUtils; -import org.apache.dolphinscheduler.dao.entity.*; +import org.apache.dolphinscheduler.dao.entity.AccessToken; +import org.apache.dolphinscheduler.dao.entity.AlertGroup; +import org.apache.dolphinscheduler.dao.entity.Queue; +import org.apache.dolphinscheduler.dao.entity.Tenant; +import org.apache.dolphinscheduler.dao.entity.User; + +import java.util.Date; +import java.util.List; + import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; @@ -31,8 +37,8 @@ import org.springframework.test.annotation.Rollback; import org.springframework.test.context.junit4.SpringRunner; import org.springframework.transaction.annotation.Transactional; -import java.util.Date; -import java.util.List; +import com.baomidou.mybatisplus.core.metadata.IPage; +import com.baomidou.mybatisplus.extension.plugins.pagination.Page; @RunWith(SpringRunner.class) @SpringBootTest @@ -45,9 +51,6 @@ public class UserMapperTest { @Autowired AlertGroupMapper alertGroupMapper; - @Autowired - private UserAlertGroupMapper userAlertGroupMapper; - @Autowired AccessTokenMapper accessTokenMapper; @@ -59,9 +62,10 @@ public class UserMapperTest { /** * insert one user + * * @return User */ - private User insertOne(){ + private User insertOne() { User user = new User(); user.setUserName("user1"); user.setUserPassword("1"); @@ -76,10 +80,11 @@ public class UserMapperTest { /** * insert one user + * * @param tenant tenant * @return User */ - private User insertOne(Tenant tenant){ + private User insertOne(Tenant tenant) { User user = new User(); user.setUserName("user1"); user.setUserPassword("1"); @@ -94,11 +99,12 @@ public class UserMapperTest { /** * insert one user - * @param queue queue + * + * @param queue queue * @param tenant tenant * @return User */ - private User insertOne(Queue queue,Tenant tenant){ + private User insertOne(Queue queue, Tenant tenant) { User user = new User(); user.setUserName("user1"); user.setUserPassword("1"); @@ -114,14 +120,14 @@ public class UserMapperTest { /** * insert one AlertGroup + * * @return AlertGroup */ - private AlertGroup insertOneAlertGroup(){ + private AlertGroup insertOneAlertGroup() { //insertOne AlertGroup alertGroup = new AlertGroup(); alertGroup.setGroupName("alert group 1"); alertGroup.setDescription("alert test1"); - alertGroup.setGroupType(AlertType.EMAIL); alertGroup.setCreateTime(new Date()); alertGroup.setUpdateTime(new Date()); @@ -129,45 +135,30 @@ public class UserMapperTest { return alertGroup; } - /** - * insert one UserAlertGroup - * @param user user - * @param alertGroup alertGroup - * @return UserAlertGroup - */ - private UserAlertGroup insertOneUserAlertGroup(User user,AlertGroup alertGroup){ - UserAlertGroup userAlertGroup = new UserAlertGroup(); - userAlertGroup.setAlertgroupName(alertGroup.getGroupName()); - userAlertGroup.setAlertgroupId(alertGroup.getId()); - userAlertGroup.setUserId(user.getId()); - userAlertGroup.setCreateTime(new Date()); - userAlertGroup.setUpdateTime(new Date()); - userAlertGroupMapper.insert(userAlertGroup); - return userAlertGroup; - } - /** * insert one AccessToken + * * @param user user * @return AccessToken */ - private AccessToken insertOneAccessToken(User user){ + private AccessToken insertOneAccessToken(User user) { //insertOne AccessToken accessToken = new AccessToken(); accessToken.setUserId(user.getId()); accessToken.setToken("secrettoken"); accessToken.setCreateTime(new Date()); accessToken.setUpdateTime(new Date()); - accessToken.setExpireTime(DateUtils.getSomeHourOfDay(new Date(),1)); + accessToken.setExpireTime(DateUtils.getSomeHourOfDay(new Date(), 1)); accessTokenMapper.insert(accessToken); return accessToken; } /** * insert one Tenant + * * @return Tenant */ - private Tenant insertOneTenant(){ + private Tenant insertOneTenant() { Tenant tenant = new Tenant(); tenant.setTenantCode("dolphin"); tenant.setDescription("dolphin user use"); @@ -180,9 +171,10 @@ public class UserMapperTest { /** * insert one Tenant + * * @return Tenant */ - private Tenant insertOneTenant(Queue queue){ + private Tenant insertOneTenant(Queue queue) { Tenant tenant = new Tenant(); tenant.setTenantCode("dolphin"); tenant.setDescription("dolphin user use"); @@ -196,9 +188,10 @@ public class UserMapperTest { /** * insert one Queue + * * @return Queue */ - private Queue insertOneQueue(){ + private Queue insertOneQueue() { Queue queue = new Queue(); queue.setQueue("dolphin"); queue.setQueueName("dolphin queue"); @@ -212,7 +205,7 @@ public class UserMapperTest { * test update */ @Test - public void testUpdate(){ + public void testUpdate() { //insertOne User user = insertOne(); //update @@ -227,7 +220,7 @@ public class UserMapperTest { * test delete */ @Test - public void testDelete(){ + public void testDelete() { //insertOne User user = insertOne(); //delete @@ -259,30 +252,6 @@ public class UserMapperTest { Assert.assertNotEquals(userList.size(), 0); } -// /** -// * test query by username -// */ -// @Test -// public void testQueryByUserNameAccurately() { -// //insertOne -// User user = insertOne(); -// //queryByUserNameAccurately -// User queryUser = userMapper.queryByUserNameAccurately(user.getUserName()); -// Assert.assertEquals(queryUser.getUserName(), user.getUserName()); -// } - -// /** -// * test query by username and password -// */ -// @Test -// public void testQueryUserByNamePassword() { -// //insertOne -// User user = insertOne(); -// //queryUserByNamePassword -// User queryUser = userMapper.queryUserByNamePassword(user.getUserName(),user.getUserPassword()); -// Assert.assertEquals(queryUser.getUserName(),user.getUserName()); -// Assert.assertEquals(queryUser.getUserPassword(), user.getUserPassword()); -// } /** * test page @@ -294,9 +263,9 @@ public class UserMapperTest { //insertOneTenant Tenant tenant = insertOneTenant(); //insertOne - User user = insertOne(queue,tenant); + User user = insertOne(queue, tenant); //queryUserPaging - Page page = new Page(1,3); + Page page = new Page(1, 3); IPage userIPage = userMapper.queryUserPaging(page, user.getUserName()); Assert.assertNotEquals(userIPage.getTotal(), 0); } @@ -309,29 +278,12 @@ public class UserMapperTest { //insertOneQueue and insertOneTenant Queue queue = insertOneQueue(); Tenant tenant = insertOneTenant(queue); - User user = insertOne(queue,tenant); + User user = insertOne(queue, tenant); //queryDetailsById User queryUser = userMapper.queryDetailsById(user.getId()); Assert.assertEquals(user.getUserName(), queryUser.getUserName()); } - /** - * test query user list by alertgroupId - */ - @Test - public void testQueryUserListByAlertGroupId() { - //insertOne - User user = insertOne(); - //insertOneAlertGroup - AlertGroup alertGroup = insertOneAlertGroup(); - //insertOneUserAlertGroup - UserAlertGroup userAlertGroup = insertOneUserAlertGroup(user, alertGroup); - //queryUserListByAlertGroupId - List userList = userMapper.queryUserListByAlertGroupId(userAlertGroup.getAlertgroupId()); - Assert.assertNotEquals(userList.size(), 0); - - } - /** * test query tenant code by userId */ @@ -343,7 +295,7 @@ public class UserMapperTest { User user = insertOne(tenant); //queryTenantCodeByUserId User queryUser = userMapper.queryTenantCodeByUserId(user.getId()); - Assert.assertEquals(queryUser,user); + Assert.assertEquals(queryUser, user); } /** @@ -357,7 +309,7 @@ public class UserMapperTest { AccessToken accessToken = insertOneAccessToken(user); //queryUserByToken User userToken = userMapper.queryUserByToken(accessToken.getToken()); - Assert.assertEquals(userToken,user); + Assert.assertEquals(userToken, user); } } diff --git a/dolphinscheduler-dist/pom.xml b/dolphinscheduler-dist/pom.xml index c461564dbf..1ec74fd409 100644 --- a/dolphinscheduler-dist/pom.xml +++ b/dolphinscheduler-dist/pom.xml @@ -25,7 +25,7 @@ 4.0.0 dolphinscheduler-dist - pom + provisio ${project.artifactId} true diff --git a/dolphinscheduler-dist/release-docs/LICENSE b/dolphinscheduler-dist/release-docs/LICENSE index c2d5b7d698..4f8afe6741 100644 --- a/dolphinscheduler-dist/release-docs/LICENSE +++ b/dolphinscheduler-dist/release-docs/LICENSE @@ -236,7 +236,6 @@ The text of each license is also included at licenses/LICENSE-[project].txt. commons-configuration 1.10: https://mvnrepository.com/artifact/commons-configuration/commons-configuration/1.10, Apache 2.0 commons-daemon 1.0.13 https://mvnrepository.com/artifact/commons-daemon/commons-daemon/1.0.13, Apache 2.0 commons-dbcp 1.4: https://github.com/apache/commons-dbcp, Apache 2.0 - commons-email 1.5: https://github.com/apache/commons-email, Apache 2.0 commons-httpclient 3.0.1: https://mvnrepository.com/artifact/commons-httpclient/commons-httpclient/3.0.1, Apache 2.0 commons-io 2.4: https://github.com/apache/commons-io, Apache 2.0 commons-lang 2.6: https://github.com/apache/commons-lang, Apache 2.0 @@ -253,8 +252,9 @@ The text of each license is also included at licenses/LICENSE-[project].txt. datanucleus-rdbms 4.1.7: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-rdbms/4.1.7, Apache 2.0 derby 10.14.2.0: https://github.com/apache/derby, Apache 2.0 druid 1.1.14: https://mvnrepository.com/artifact/com.alibaba/druid/1.1.14, Apache 2.0 + error_prone_annotations 2.1.3 https://mvnrepository.com/artifact/com.google.errorprone/error_prone_annotations/2.1.3, Apache 2.0 gson 2.8.5: https://github.com/google/gson, Apache 2.0 - guava 20.0: https://mvnrepository.com/artifact/com.google.guava/guava/20.0, Apache 2.0 + guava 24.1-jre: https://mvnrepository.com/artifact/com.google.guava/guava/24.1-jre, Apache 2.0 guice 3.0: https://mvnrepository.com/artifact/com.google.inject/guice/3.0, Apache 2.0 guice-servlet 3.0: https://mvnrepository.com/artifact/com.google.inject.extensions/guice-servlet/3.0, Apache 2.0 hadoop-annotations 2.7.3:https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-annotations/2.7.3, Apache 2.0 @@ -321,6 +321,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt. jpam 1.1: https://mvnrepository.com/artifact/net.sf.jpam/jpam/1.1, Apache 2.0 jsqlparser 2.1: https://github.com/JSQLParser/JSqlParser, Apache 2.0 or LGPL 2.1 jsr305 3.0.0: https://mvnrepository.com/artifact/com.google.code.findbugs/jsr305, Apache 2.0 + j2objc-annotations 1.1 https://mvnrepository.com/artifact/com.google.j2objc/j2objc-annotations/1.1, Apache 2.0 libfb303 0.9.3: https://mvnrepository.com/artifact/org.apache.thrift/libfb303/0.9.3, Apache 2.0 libthrift 0.9.3: https://mvnrepository.com/artifact/org.apache.thrift/libthrift/0.9.3, Apache 2.0 log4j-api 2.11.2: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api/2.11.2, Apache 2.0 @@ -411,10 +412,8 @@ CDDL licenses The following components are provided under the CDDL License. See project link for details. The text of each license is also included at licenses/LICENSE-[project].txt. - activation 1.1: https://mvnrepository.com/artifact/javax.activation/activation/1.1 CDDL 1.0 javax.activation-api 1.2.0: https://mvnrepository.com/artifact/javax.activation/javax.activation-api/1.2.0, CDDL and LGPL 2.0 javax.annotation-api 1.3.2: https://mvnrepository.com/artifact/javax.annotation/javax.annotation-api/1.3.2, CDDL + GPLv2 - javax.mail 1.6.2: https://mvnrepository.com/artifact/com.sun.mail/javax.mail/1.6.2, CDDL/GPLv2 javax.servlet-api 3.1.0: https://mvnrepository.com/artifact/javax.servlet/javax.servlet-api/3.1.0, CDDL + GPLv2 jaxb-api 2.3.1: https://mvnrepository.com/artifact/javax.xml.bind/jaxb-api/2.3.1, CDDL 1.1 jaxb-impl 2.2.3-1: https://mvnrepository.com/artifact/com.sun.xml.bind/jaxb-impl/2.2.3-1, CDDL and GPL 1.1 @@ -451,6 +450,8 @@ The text of each license is also included at licenses/LICENSE-[project].txt. jul-to-slf4j 1.7.25: https://mvnrepository.com/artifact/org.slf4j/jul-to-slf4j/1.7.25, MIT mssql-jdbc 6.1.0.jre8: https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc/6.1.0.jre8, MIT slf4j-api 1.7.5: https://mvnrepository.com/artifact/org.slf4j/slf4j-api/1.7.5, MIT + animal-sniffer-annotations 1.14 https://mvnrepository.com/artifact/org.codehaus.mojo/animal-sniffer-annotations/1.14, MIT + checker-compat-qual 2.0.0 https://mvnrepository.com/artifact/org.checkerframework/checker-compat-qual/2.0.0, MIT + GPLv2 ======================================================================== MPL 1.1 licenses @@ -479,6 +480,7 @@ The text of each license is also included at licenses/ui-licenses/LICENSE-[proje ======================================== MIT licenses ======================================== + @form-create/element-ui 1.0.18: https://github.com/xaboy/form-create MIT axios 0.16.2: https://github.com/axios/axios MIT bootstrap 3.3.7: https://github.com/twbs/bootstrap MIT canvg 1.5.1: https://github.com/canvg/canvg MIT @@ -509,4 +511,4 @@ Apache 2.0 licenses ======================================== BSD licenses ======================================== - d3 3.5.17: https://github.com/d3/d3 BSD-3-Clause + d3 3.5.17: https://github.com/d3/d3 BSD-3-Clause \ No newline at end of file diff --git a/dolphinscheduler-dist/release-docs/NOTICE b/dolphinscheduler-dist/release-docs/NOTICE index 66abc45977..dd9eab04b1 100644 --- a/dolphinscheduler-dist/release-docs/NOTICE +++ b/dolphinscheduler-dist/release-docs/NOTICE @@ -1917,26 +1917,3 @@ ANT NOTICE Please read the different LICENSE files present in the root directory of this distribution. - - - - - - - - - - - - - - - - - - - - - - - diff --git a/dolphinscheduler-dist/release-docs/licenses/LICENSE-aether-api.txt b/dolphinscheduler-dist/release-docs/licenses/LICENSE-aether-api.txt new file mode 100644 index 0000000000..3fa00836fa --- /dev/null +++ b/dolphinscheduler-dist/release-docs/licenses/LICENSE-aether-api.txt @@ -0,0 +1,86 @@ +Eclipse Public License - v 1.0 +THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT. + +1. DEFINITIONS + +"Contribution" means: + +a) in the case of the initial Contributor, the initial code and documentation distributed under this Agreement, and + +b) in the case of each subsequent Contributor: + +i) changes to the Program, and + +ii) additions to the Program; + +where such changes and/or additions to the Program originate from and are distributed by that particular Contributor. A Contribution 'originates' from a Contributor if it was added to the Program by such Contributor itself or anyone acting on such Contributor's behalf. Contributions do not include additions to the Program which: (i) are separate modules of software distributed in conjunction with the Program under their own license agreement, and (ii) are not derivative works of the Program. + +"Contributor" means any person or entity that distributes the Program. + +"Licensed Patents" mean patent claims licensable by a Contributor which are necessarily infringed by the use or sale of its Contribution alone or when combined with the Program. + +"Program" means the Contributions distributed in accordance with this Agreement. + +"Recipient" means anyone who receives the Program under this Agreement, including all Contributors. + +2. GRANT OF RIGHTS + +a) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, distribute and sublicense the Contribution of such Contributor, if any, and such derivative works, in source code and object code form. + +b) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free patent license under Licensed Patents to make, use, sell, offer to sell, import and otherwise transfer the Contribution of such Contributor, if any, in source code and object code form. This patent license shall apply to the combination of the Contribution and the Program if, at the time the Contribution is added by the Contributor, such addition of the Contribution causes such combination to be covered by the Licensed Patents. The patent license shall not apply to any other combinations which include the Contribution. No hardware per se is licensed hereunder. + +c) Recipient understands that although each Contributor grants the licenses to its Contributions set forth herein, no assurances are provided by any Contributor that the Program does not infringe the patent or other intellectual property rights of any other entity. Each Contributor disclaims any liability to Recipient for claims brought by any other entity based on infringement of intellectual property rights or otherwise. As a condition to exercising the rights and licenses granted hereunder, each Recipient hereby assumes sole responsibility to secure any other intellectual property rights needed, if any. For example, if a third party patent license is required to allow Recipient to distribute the Program, it is Recipient's responsibility to acquire that license before distributing the Program. + +d) Each Contributor represents that to its knowledge it has sufficient copyright rights in its Contribution, if any, to grant the copyright license set forth in this Agreement. + +3. REQUIREMENTS + +A Contributor may choose to distribute the Program in object code form under its own license agreement, provided that: + +a) it complies with the terms and conditions of this Agreement; and + +b) its license agreement: + +i) effectively disclaims on behalf of all Contributors all warranties and conditions, express and implied, including warranties or conditions of title and non-infringement, and implied warranties or conditions of merchantability and fitness for a particular purpose; + +ii) effectively excludes on behalf of all Contributors all liability for damages, including direct, indirect, special, incidental and consequential damages, such as lost profits; + +iii) states that any provisions which differ from this Agreement are offered by that Contributor alone and not by any other party; and + +iv) states that source code for the Program is available from such Contributor, and informs licensees how to obtain it in a reasonable manner on or through a medium customarily used for software exchange. + +When the Program is made available in source code form: + +a) it must be made available under this Agreement; and + +b) a copy of this Agreement must be included with each copy of the Program. + +Contributors may not remove or alter any copyright notices contained within the Program. + +Each Contributor must identify itself as the originator of its Contribution, if any, in a manner that reasonably allows subsequent Recipients to identify the originator of the Contribution. + +4. COMMERCIAL DISTRIBUTION + +Commercial distributors of software may accept certain responsibilities with respect to end users, business partners and the like. While this license is intended to facilitate the commercial use of the Program, the Contributor who includes the Program in a commercial product offering should do so in a manner which does not create potential liability for other Contributors. Therefore, if a Contributor includes the Program in a commercial product offering, such Contributor ("Commercial Contributor") hereby agrees to defend and indemnify every other Contributor ("Indemnified Contributor") against any losses, damages and costs (collectively "Losses") arising from claims, lawsuits and other legal actions brought by a third party against the Indemnified Contributor to the extent caused by the acts or omissions of such Commercial Contributor in connection with its distribution of the Program in a commercial product offering. The obligations in this section do not apply to any claims or Losses relating to any actual or alleged intellectual property infringement. In order to qualify, an Indemnified Contributor must: a) promptly notify the Commercial Contributor in writing of such claim, and b) allow the Commercial Contributor to control, and cooperate with the Commercial Contributor in, the defense and any related settlement negotiations. The Indemnified Contributor may participate in any such claim at its own expense. + +For example, a Contributor might include the Program in a commercial product offering, Product X. That Contributor is then a Commercial Contributor. If that Commercial Contributor then makes performance claims, or offers warranties related to Product X, those performance claims and warranties are such Commercial Contributor's responsibility alone. Under this section, the Commercial Contributor would have to defend claims against the other Contributors related to those performance claims and warranties, and if a court requires any other Contributor to pay any damages as a result, the Commercial Contributor must pay those damages. + +5. NO WARRANTY + +EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the appropriateness of using and distributing the Program and assumes all risks associated with its exercise of rights under this Agreement , including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and unavailability or interruption of operations. + +6. DISCLAIMER OF LIABILITY + +EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. + +7. GENERAL + +If any provision of this Agreement is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this Agreement, and without further action by the parties hereto, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable. + +If Recipient institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Program itself (excluding combinations of the Program with other software or hardware) infringes such Recipient's patent(s), then such Recipient's rights granted under Section 2(b) shall terminate as of the date such litigation is filed. + +All Recipient's rights under this Agreement shall terminate if it fails to comply with any of the material terms or conditions of this Agreement and does not cure such failure in a reasonable period of time after becoming aware of such noncompliance. If all Recipient's rights under this Agreement terminate, Recipient agrees to cease use and distribution of the Program as soon as reasonably practicable. However, Recipient's obligations under this Agreement and any licenses granted by Recipient relating to the Program shall continue and survive. + +Everyone is permitted to copy and distribute copies of this Agreement, but in order to avoid inconsistency the Agreement is copyrighted and may only be modified in the following manner. The Agreement Steward reserves the right to publish new versions (including revisions) of this Agreement from time to time. No one other than the Agreement Steward has the right to modify this Agreement. The Eclipse Foundation is the initial Agreement Steward. The Eclipse Foundation may assign the responsibility to serve as the Agreement Steward to a suitable separate entity. Each new version of the Agreement will be given a distinguishing version number. The Program (including Contributions) may always be distributed subject to the version of the Agreement under which it was received. In addition, after a new version of the Agreement is published, Contributor may elect to distribute the Program (including its Contributions) under the new version. Except as expressly stated in Sections 2(a) and 2(b) above, Recipient receives no rights or licenses to the intellectual property of any Contributor under this Agreement, whether expressly, by implication, estoppel or otherwise. All rights in the Program not expressly granted under this Agreement are reserved. + +This Agreement is governed by the laws of the State of New York and the intellectual property laws of the United States of America. No party to this Agreement will bring a legal action under this Agreement more than one year after the cause of action arose. Each party waives its rights to a jury trial in any resulting litigation. \ No newline at end of file diff --git a/dolphinscheduler-dist/release-docs/licenses/LICENSE-animal-sniffer-annotations.txt b/dolphinscheduler-dist/release-docs/licenses/LICENSE-animal-sniffer-annotations.txt new file mode 100644 index 0000000000..2062eb88b4 --- /dev/null +++ b/dolphinscheduler-dist/release-docs/licenses/LICENSE-animal-sniffer-annotations.txt @@ -0,0 +1,21 @@ +The MIT License + +Copyright (c) 2009 codehaus.org. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. \ No newline at end of file diff --git a/dolphinscheduler-dist/release-docs/licenses/LICENSE-checker-compat-qual.txt b/dolphinscheduler-dist/release-docs/licenses/LICENSE-checker-compat-qual.txt new file mode 100644 index 0000000000..7b59b5c982 --- /dev/null +++ b/dolphinscheduler-dist/release-docs/licenses/LICENSE-checker-compat-qual.txt @@ -0,0 +1,22 @@ +Checker Framework qualifiers +Copyright 2004-present by the Checker Framework developers + +MIT License: + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. \ No newline at end of file diff --git a/dolphinscheduler-dist/release-docs/licenses/LICENSE-j2objc-annotations.txt b/dolphinscheduler-dist/release-docs/licenses/LICENSE-j2objc-annotations.txt new file mode 100644 index 0000000000..989e2c59e9 --- /dev/null +++ b/dolphinscheduler-dist/release-docs/licenses/LICENSE-j2objc-annotations.txt @@ -0,0 +1,201 @@ +Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/dolphinscheduler-dist/release-docs/licenses/LICENSE-javax.mail.txt b/dolphinscheduler-dist/release-docs/licenses/LICENSE-javax.mail.txt deleted file mode 100644 index 5ad62c442b..0000000000 --- a/dolphinscheduler-dist/release-docs/licenses/LICENSE-javax.mail.txt +++ /dev/null @@ -1,759 +0,0 @@ -COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL) Version 1.1 - -1. Definitions. - - 1.1. "Contributor" means each individual or entity that creates or - contributes to the creation of Modifications. - - 1.2. "Contributor Version" means the combination of the Original - Software, prior Modifications used by a Contributor (if any), and - the Modifications made by that particular Contributor. - - 1.3. "Covered Software" means (a) the Original Software, or (b) - Modifications, or (c) the combination of files containing Original - Software with files containing Modifications, in each case including - portions thereof. - - 1.4. "Executable" means the Covered Software in any form other than - Source Code. - - 1.5. "Initial Developer" means the individual or entity that first - makes Original Software available under this License. - - 1.6. "Larger Work" means a work which combines Covered Software or - portions thereof with code not governed by the terms of this License. - - 1.7. "License" means this document. - - 1.8. "Licensable" means having the right to grant, to the maximum - extent possible, whether at the time of the initial grant or - subsequently acquired, any and all of the rights conveyed herein. - - 1.9. "Modifications" means the Source Code and Executable form of - any of the following: - - A. Any file that results from an addition to, deletion from or - modification of the contents of a file containing Original Software - or previous Modifications; - - B. Any new file that contains any part of the Original Software or - previous Modification; or - - C. Any new file that is contributed or otherwise made available - under the terms of this License. - - 1.10. "Original Software" means the Source Code and Executable form - of computer software code that is originally released under this - License. - - 1.11. "Patent Claims" means any patent claim(s), now owned or - hereafter acquired, including without limitation, method, process, - and apparatus claims, in any patent Licensable by grantor. - - 1.12. "Source Code" means (a) the common form of computer software - code in which modifications are made and (b) associated - documentation included in or with such code. - - 1.13. "You" (or "Your") means an individual or a legal entity - exercising rights under, and complying with all of the terms of, - this License. For legal entities, "You" includes any entity which - controls, is controlled by, or is under common control with You. For - purposes of this definition, "control" means (a) the power, direct - or indirect, to cause the direction or management of such entity, - whether by contract or otherwise, or (b) ownership of more than - fifty percent (50%) of the outstanding shares or beneficial - ownership of such entity. - -2. License Grants. - - 2.1. The Initial Developer Grant. - - Conditioned upon Your compliance with Section 3.1 below and subject - to third party intellectual property claims, the Initial Developer - hereby grants You a world-wide, royalty-free, non-exclusive license: - - (a) under intellectual property rights (other than patent or - trademark) Licensable by Initial Developer, to use, reproduce, - modify, display, perform, sublicense and distribute the Original - Software (or portions thereof), with or without Modifications, - and/or as part of a Larger Work; and - - (b) under Patent Claims infringed by the making, using or selling of - Original Software, to make, have made, use, practice, sell, and - offer for sale, and/or otherwise dispose of the Original Software - (or portions thereof). - - (c) The licenses granted in Sections 2.1(a) and (b) are effective on - the date Initial Developer first distributes or otherwise makes the - Original Software available to a third party under the terms of this - License. - - (d) Notwithstanding Section 2.1(b) above, no patent license is - granted: (1) for code that You delete from the Original Software, or - (2) for infringements caused by: (i) the modification of the - Original Software, or (ii) the combination of the Original Software - with other software or devices. - - 2.2. Contributor Grant. - - Conditioned upon Your compliance with Section 3.1 below and subject - to third party intellectual property claims, each Contributor hereby - grants You a world-wide, royalty-free, non-exclusive license: - - (a) under intellectual property rights (other than patent or - trademark) Licensable by Contributor to use, reproduce, modify, - display, perform, sublicense and distribute the Modifications - created by such Contributor (or portions thereof), either on an - unmodified basis, with other Modifications, as Covered Software - and/or as part of a Larger Work; and - - (b) under Patent Claims infringed by the making, using, or selling - of Modifications made by that Contributor either alone and/or in - combination with its Contributor Version (or portions of such - combination), to make, use, sell, offer for sale, have made, and/or - otherwise dispose of: (1) Modifications made by that Contributor (or - portions thereof); and (2) the combination of Modifications made by - that Contributor with its Contributor Version (or portions of such - combination). - - (c) The licenses granted in Sections 2.2(a) and 2.2(b) are effective - on the date Contributor first distributes or otherwise makes the - Modifications available to a third party. - - (d) Notwithstanding Section 2.2(b) above, no patent license is - granted: (1) for any code that Contributor has deleted from the - Contributor Version; (2) for infringements caused by: (i) third - party modifications of Contributor Version, or (ii) the combination - of Modifications made by that Contributor with other software - (except as part of the Contributor Version) or other devices; or (3) - under Patent Claims infringed by Covered Software in the absence of - Modifications made by that Contributor. - -3. Distribution Obligations. - - 3.1. Availability of Source Code. - - Any Covered Software that You distribute or otherwise make available - in Executable form must also be made available in Source Code form - and that Source Code form must be distributed only under the terms - of this License. You must include a copy of this License with every - copy of the Source Code form of the Covered Software You distribute - or otherwise make available. You must inform recipients of any such - Covered Software in Executable form as to how they can obtain such - Covered Software in Source Code form in a reasonable manner on or - through a medium customarily used for software exchange. - - 3.2. Modifications. - - The Modifications that You create or to which You contribute are - governed by the terms of this License. You represent that You - believe Your Modifications are Your original creation(s) and/or You - have sufficient rights to grant the rights conveyed by this License. - - 3.3. Required Notices. - - You must include a notice in each of Your Modifications that - identifies You as the Contributor of the Modification. You may not - remove or alter any copyright, patent or trademark notices contained - within the Covered Software, or any notices of licensing or any - descriptive text giving attribution to any Contributor or the - Initial Developer. - - 3.4. Application of Additional Terms. - - You may not offer or impose any terms on any Covered Software in - Source Code form that alters or restricts the applicable version of - this License or the recipients' rights hereunder. You may choose to - offer, and to charge a fee for, warranty, support, indemnity or - liability obligations to one or more recipients of Covered Software. - However, you may do so only on Your own behalf, and not on behalf of - the Initial Developer or any Contributor. You must make it - absolutely clear that any such warranty, support, indemnity or - liability obligation is offered by You alone, and You hereby agree - to indemnify the Initial Developer and every Contributor for any - liability incurred by the Initial Developer or such Contributor as a - result of warranty, support, indemnity or liability terms You offer. - - 3.5. Distribution of Executable Versions. - - You may distribute the Executable form of the Covered Software under - the terms of this License or under the terms of a license of Your - choice, which may contain terms different from this License, - provided that You are in compliance with the terms of this License - and that the license for the Executable form does not attempt to - limit or alter the recipient's rights in the Source Code form from - the rights set forth in this License. If You distribute the Covered - Software in Executable form under a different license, You must make - it absolutely clear that any terms which differ from this License - are offered by You alone, not by the Initial Developer or - Contributor. You hereby agree to indemnify the Initial Developer and - every Contributor for any liability incurred by the Initial - Developer or such Contributor as a result of any such terms You offer. - - 3.6. Larger Works. - - You may create a Larger Work by combining Covered Software with - other code not governed by the terms of this License and distribute - the Larger Work as a single product. In such a case, You must make - sure the requirements of this License are fulfilled for the Covered - Software. - -4. Versions of the License. - - 4.1. New Versions. - - Oracle is the initial license steward and may publish revised and/or - new versions of this License from time to time. Each version will be - given a distinguishing version number. Except as provided in Section - 4.3, no one other than the license steward has the right to modify - this License. - - 4.2. Effect of New Versions. - - You may always continue to use, distribute or otherwise make the - Covered Software available under the terms of the version of the - License under which You originally received the Covered Software. If - the Initial Developer includes a notice in the Original Software - prohibiting it from being distributed or otherwise made available - under any subsequent version of the License, You must distribute and - make the Covered Software available under the terms of the version - of the License under which You originally received the Covered - Software. Otherwise, You may also choose to use, distribute or - otherwise make the Covered Software available under the terms of any - subsequent version of the License published by the license steward. - - 4.3. Modified Versions. - - When You are an Initial Developer and You want to create a new - license for Your Original Software, You may create and use a - modified version of this License if You: (a) rename the license and - remove any references to the name of the license steward (except to - note that the license differs from this License); and (b) otherwise - make it clear that the license contains terms which differ from this - License. - -5. DISCLAIMER OF WARRANTY. - - COVERED SOFTWARE IS PROVIDED UNDER THIS LICENSE ON AN "AS IS" BASIS, - WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, - INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT THE COVERED SOFTWARE - IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR - NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF - THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE - DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY - OTHER CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, - REPAIR OR CORRECTION. THIS DISCLAIMER OF WARRANTY CONSTITUTES AN - ESSENTIAL PART OF THIS LICENSE. NO USE OF ANY COVERED SOFTWARE IS - AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER. - -6. TERMINATION. - - 6.1. This License and the rights granted hereunder will terminate - automatically if You fail to comply with terms herein and fail to - cure such breach within 30 days of becoming aware of the breach. - Provisions which, by their nature, must remain in effect beyond the - termination of this License shall survive. - - 6.2. If You assert a patent infringement claim (excluding - declaratory judgment actions) against Initial Developer or a - Contributor (the Initial Developer or Contributor against whom You - assert such claim is referred to as "Participant") alleging that the - Participant Software (meaning the Contributor Version where the - Participant is a Contributor or the Original Software where the - Participant is the Initial Developer) directly or indirectly - infringes any patent, then any and all rights granted directly or - indirectly to You by such Participant, the Initial Developer (if the - Initial Developer is not the Participant) and all Contributors under - Sections 2.1 and/or 2.2 of this License shall, upon 60 days notice - from Participant terminate prospectively and automatically at the - expiration of such 60 day notice period, unless if within such 60 - day period You withdraw Your claim with respect to the Participant - Software against such Participant either unilaterally or pursuant to - a written agreement with Participant. - - 6.3. If You assert a patent infringement claim against Participant - alleging that the Participant Software directly or indirectly - infringes any patent where such claim is resolved (such as by - license or settlement) prior to the initiation of patent - infringement litigation, then the reasonable value of the licenses - granted by such Participant under Sections 2.1 or 2.2 shall be taken - into account in determining the amount or value of any payment or - license. - - 6.4. In the event of termination under Sections 6.1 or 6.2 above, - all end user licenses that have been validly granted by You or any - distributor hereunder prior to termination (excluding licenses - granted to You by any distributor) shall survive termination. - -7. LIMITATION OF LIABILITY. - - UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT - (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE - INITIAL DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF - COVERED SOFTWARE, OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE - TO ANY PERSON FOR ANY INDIRECT, SPECIAL, INCIDENTAL, OR - CONSEQUENTIAL DAMAGES OF ANY CHARACTER INCLUDING, WITHOUT - LIMITATION, DAMAGES FOR LOSS OF GOODWILL, WORK STOPPAGE, COMPUTER - FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER COMMERCIAL DAMAGES OR - LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN INFORMED OF THE - POSSIBILITY OF SUCH DAMAGES. THIS LIMITATION OF LIABILITY SHALL NOT - APPLY TO LIABILITY FOR DEATH OR PERSONAL INJURY RESULTING FROM SUCH - PARTY'S NEGLIGENCE TO THE EXTENT APPLICABLE LAW PROHIBITS SUCH - LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR - LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS EXCLUSION - AND LIMITATION MAY NOT APPLY TO YOU. - -8. U.S. GOVERNMENT END USERS. - - The Covered Software is a "commercial item," as that term is defined - in 48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial computer - software" (as that term is defined at 48 C.F.R. - 252.227-7014(a)(1)) and "commercial computer software documentation" - as such terms are used in 48 C.F.R. 12.212 (Sept. 1995). Consistent - with 48 C.F.R. 12.212 and 48 C.F.R. 227.7202-1 through 227.7202-4 - (June 1995), all U.S. Government End Users acquire Covered Software - with only those rights set forth herein. This U.S. Government Rights - clause is in lieu of, and supersedes, any other FAR, DFAR, or other - clause or provision that addresses Government rights in computer - software under this License. - -9. MISCELLANEOUS. - - This License represents the complete agreement concerning subject - matter hereof. If any provision of this License is held to be - unenforceable, such provision shall be reformed only to the extent - necessary to make it enforceable. This License shall be governed by - the law of the jurisdiction specified in a notice contained within - the Original Software (except to the extent applicable law, if any, - provides otherwise), excluding such jurisdiction's conflict-of-law - provisions. Any litigation relating to this License shall be subject - to the jurisdiction of the courts located in the jurisdiction and - venue specified in a notice contained within the Original Software, - with the losing party responsible for costs, including, without - limitation, court costs and reasonable attorneys' fees and expenses. - The application of the United Nations Convention on Contracts for - the International Sale of Goods is expressly excluded. Any law or - regulation which provides that the language of a contract shall be - construed against the drafter shall not apply to this License. You - agree that You alone are responsible for compliance with the United - States export administration regulations (and the export control - laws and regulation of any other countries) when You use, distribute - or otherwise make available any Covered Software. - -10. RESPONSIBILITY FOR CLAIMS. - - As between Initial Developer and the Contributors, each party is - responsible for claims and damages arising, directly or indirectly, - out of its utilization of rights under this License and You agree to - work with Initial Developer and Contributors to distribute such - responsibility on an equitable basis. Nothing herein is intended or - shall be deemed to constitute any admission of liability. - ------------------------------------------------------------------------- - -NOTICE PURSUANT TO SECTION 9 OF THE COMMON DEVELOPMENT AND DISTRIBUTION -LICENSE (CDDL) - -The code released under the CDDL shall be governed by the laws of the -State of California (excluding conflict-of-law provisions). Any -litigation relating to this License shall be subject to the jurisdiction -of the Federal Courts of the Northern District of California and the -state courts of the State of California, with venue lying in Santa Clara -County, California. - - - - The GNU General Public License (GPL) Version 2, June 1991 - -Copyright (C) 1989, 1991 Free Software Foundation, Inc. -51 Franklin Street, Fifth Floor -Boston, MA 02110-1335 -USA - -Everyone is permitted to copy and distribute verbatim copies -of this license document, but changing it is not allowed. - -Preamble - -The licenses for most software are designed to take away your freedom to -share and change it. By contrast, the GNU General Public License is -intended to guarantee your freedom to share and change free software--to -make sure the software is free for all its users. This General Public -License applies to most of the Free Software Foundation's software and -to any other program whose authors commit to using it. (Some other Free -Software Foundation software is covered by the GNU Library General -Public License instead.) You can apply it to your programs, too. - -When we speak of free software, we are referring to freedom, not price. -Our General Public Licenses are designed to make sure that you have the -freedom to distribute copies of free software (and charge for this -service if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs; and that you know you can do these things. - -To protect your rights, we need to make restrictions that forbid anyone -to deny you these rights or to ask you to surrender the rights. These -restrictions translate to certain responsibilities for you if you -distribute copies of the software, or if you modify it. - -For example, if you distribute copies of such a program, whether gratis -or for a fee, you must give the recipients all the rights that you have. -You must make sure that they, too, receive or can get the source code. -And you must show them these terms so they know their rights. - -We protect your rights with two steps: (1) copyright the software, and -(2) offer you this license which gives you legal permission to copy, -distribute and/or modify the software. - -Also, for each author's protection and ours, we want to make certain -that everyone understands that there is no warranty for this free -software. If the software is modified by someone else and passed on, we -want its recipients to know that what they have is not the original, so -that any problems introduced by others will not reflect on the original -authors' reputations. - -Finally, any free program is threatened constantly by software patents. -We wish to avoid the danger that redistributors of a free program will -individually obtain patent licenses, in effect making the program -proprietary. To prevent this, we have made it clear that any patent must -be licensed for everyone's free use or not licensed at all. - -The precise terms and conditions for copying, distribution and -modification follow. - -TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION - -0. This License applies to any program or other work which contains a -notice placed by the copyright holder saying it may be distributed under -the terms of this General Public License. The "Program", below, refers -to any such program or work, and a "work based on the Program" means -either the Program or any derivative work under copyright law: that is -to say, a work containing the Program or a portion of it, either -verbatim or with modifications and/or translated into another language. -(Hereinafter, translation is included without limitation in the term -"modification".) Each licensee is addressed as "you". - -Activities other than copying, distribution and modification are not -covered by this License; they are outside its scope. The act of running -the Program is not restricted, and the output from the Program is -covered only if its contents constitute a work based on the Program -(independent of having been made by running the Program). Whether that -is true depends on what the Program does. - -1. You may copy and distribute verbatim copies of the Program's source -code as you receive it, in any medium, provided that you conspicuously -and appropriately publish on each copy an appropriate copyright notice -and disclaimer of warranty; keep intact all the notices that refer to -this License and to the absence of any warranty; and give any other -recipients of the Program a copy of this License along with the Program. - -You may charge a fee for the physical act of transferring a copy, and -you may at your option offer warranty protection in exchange for a fee. - -2. You may modify your copy or copies of the Program or any portion of -it, thus forming a work based on the Program, and copy and distribute -such modifications or work under the terms of Section 1 above, provided -that you also meet all of these conditions: - - a) You must cause the modified files to carry prominent notices - stating that you changed the files and the date of any change. - - b) You must cause any work that you distribute or publish, that in - whole or in part contains or is derived from the Program or any part - thereof, to be licensed as a whole at no charge to all third parties - under the terms of this License. - - c) If the modified program normally reads commands interactively - when run, you must cause it, when started running for such - interactive use in the most ordinary way, to print or display an - announcement including an appropriate copyright notice and a notice - that there is no warranty (or else, saying that you provide a - warranty) and that users may redistribute the program under these - conditions, and telling the user how to view a copy of this License. - (Exception: if the Program itself is interactive but does not - normally print such an announcement, your work based on the Program - is not required to print an announcement.) - -These requirements apply to the modified work as a whole. If -identifiable sections of that work are not derived from the Program, and -can be reasonably considered independent and separate works in -themselves, then this License, and its terms, do not apply to those -sections when you distribute them as separate works. But when you -distribute the same sections as part of a whole which is a work based on -the Program, the distribution of the whole must be on the terms of this -License, whose permissions for other licensees extend to the entire -whole, and thus to each and every part regardless of who wrote it. - -Thus, it is not the intent of this section to claim rights or contest -your rights to work written entirely by you; rather, the intent is to -exercise the right to control the distribution of derivative or -collective works based on the Program. - -In addition, mere aggregation of another work not based on the Program -with the Program (or with a work based on the Program) on a volume of a -storage or distribution medium does not bring the other work under the -scope of this License. - -3. You may copy and distribute the Program (or a work based on it, -under Section 2) in object code or executable form under the terms of -Sections 1 and 2 above provided that you also do one of the following: - - a) Accompany it with the complete corresponding machine-readable - source code, which must be distributed under the terms of Sections 1 - and 2 above on a medium customarily used for software interchange; or, - - b) Accompany it with a written offer, valid for at least three - years, to give any third party, for a charge no more than your cost - of physically performing source distribution, a complete - machine-readable copy of the corresponding source code, to be - distributed under the terms of Sections 1 and 2 above on a medium - customarily used for software interchange; or, - - c) Accompany it with the information you received as to the offer to - distribute corresponding source code. (This alternative is allowed - only for noncommercial distribution and only if you received the - program in object code or executable form with such an offer, in - accord with Subsection b above.) - -The source code for a work means the preferred form of the work for -making modifications to it. For an executable work, complete source code -means all the source code for all modules it contains, plus any -associated interface definition files, plus the scripts used to control -compilation and installation of the executable. However, as a special -exception, the source code distributed need not include anything that is -normally distributed (in either source or binary form) with the major -components (compiler, kernel, and so on) of the operating system on -which the executable runs, unless that component itself accompanies the -executable. - -If distribution of executable or object code is made by offering access -to copy from a designated place, then offering equivalent access to copy -the source code from the same place counts as distribution of the source -code, even though third parties are not compelled to copy the source -along with the object code. - -4. You may not copy, modify, sublicense, or distribute the Program -except as expressly provided under this License. Any attempt otherwise -to copy, modify, sublicense or distribute the Program is void, and will -automatically terminate your rights under this License. However, parties -who have received copies, or rights, from you under this License will -not have their licenses terminated so long as such parties remain in -full compliance. - -5. You are not required to accept this License, since you have not -signed it. However, nothing else grants you permission to modify or -distribute the Program or its derivative works. These actions are -prohibited by law if you do not accept this License. Therefore, by -modifying or distributing the Program (or any work based on the -Program), you indicate your acceptance of this License to do so, and all -its terms and conditions for copying, distributing or modifying the -Program or works based on it. - -6. Each time you redistribute the Program (or any work based on the -Program), the recipient automatically receives a license from the -original licensor to copy, distribute or modify the Program subject to -these terms and conditions. You may not impose any further restrictions -on the recipients' exercise of the rights granted herein. You are not -responsible for enforcing compliance by third parties to this License. - -7. If, as a consequence of a court judgment or allegation of patent -infringement or for any other reason (not limited to patent issues), -conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot distribute -so as to satisfy simultaneously your obligations under this License and -any other pertinent obligations, then as a consequence you may not -distribute the Program at all. For example, if a patent license would -not permit royalty-free redistribution of the Program by all those who -receive copies directly or indirectly through you, then the only way you -could satisfy both it and this License would be to refrain entirely from -distribution of the Program. - -If any portion of this section is held invalid or unenforceable under -any particular circumstance, the balance of the section is intended to -apply and the section as a whole is intended to apply in other -circumstances. - -It is not the purpose of this section to induce you to infringe any -patents or other property right claims or to contest validity of any -such claims; this section has the sole purpose of protecting the -integrity of the free software distribution system, which is implemented -by public license practices. Many people have made generous -contributions to the wide range of software distributed through that -system in reliance on consistent application of that system; it is up to -the author/donor to decide if he or she is willing to distribute -software through any other system and a licensee cannot impose that choice. - -This section is intended to make thoroughly clear what is believed to be -a consequence of the rest of this License. - -8. If the distribution and/or use of the Program is restricted in -certain countries either by patents or by copyrighted interfaces, the -original copyright holder who places the Program under this License may -add an explicit geographical distribution limitation excluding those -countries, so that distribution is permitted only in or among countries -not thus excluded. In such case, this License incorporates the -limitation as if written in the body of this License. - -9. The Free Software Foundation may publish revised and/or new -versions of the General Public License from time to time. Such new -versions will be similar in spirit to the present version, but may -differ in detail to address new problems or concerns. - -Each version is given a distinguishing version number. If the Program -specifies a version number of this License which applies to it and "any -later version", you have the option of following the terms and -conditions either of that version or of any later version published by -the Free Software Foundation. If the Program does not specify a version -number of this License, you may choose any version ever published by the -Free Software Foundation. - -10. If you wish to incorporate parts of the Program into other free -programs whose distribution conditions are different, write to the -author to ask for permission. For software which is copyrighted by the -Free Software Foundation, write to the Free Software Foundation; we -sometimes make exceptions for this. Our decision will be guided by the -two goals of preserving the free status of all derivatives of our free -software and of promoting the sharing and reuse of software generally. - -NO WARRANTY - -11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO -WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. -EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR -OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, -EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE -ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH -YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL -NECESSARY SERVICING, REPAIR OR CORRECTION. - -12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN -WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY -AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR -DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL -DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM -(INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED -INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF -THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR -OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. - -END OF TERMS AND CONDITIONS - -How to Apply These Terms to Your New Programs - -If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - -To do so, attach the following notices to the program. It is safest to -attach them to the start of each source file to most effectively convey -the exclusion of warranty; and each file should have at least the -"copyright" line and a pointer to where the full notice is found. - - One line to give the program's name and a brief idea of what it does. - Copyright (C) - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, but - WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA - -Also add information on how to contact you by electronic and paper mail. - -If the program is interactive, make it output a short notice like this -when it starts in an interactive mode: - - Gnomovision version 69, Copyright (C) year name of author - Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type - `show w'. This is free software, and you are welcome to redistribute - it under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the -appropriate parts of the General Public License. Of course, the commands -you use may be called something other than `show w' and `show c'; they -could even be mouse-clicks or menu items--whatever suits your program. - -You should also get your employer (if you work as a programmer) or your -school, if any, to sign a "copyright disclaimer" for the program, if -necessary. Here is a sample; alter the names: - - Yoyodyne, Inc., hereby disclaims all copyright interest in the - program `Gnomovision' (which makes passes at compilers) written by - James Hacker. - - signature of Ty Coon, 1 April 1989 - Ty Coon, President of Vice - -This General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications -with the library. If this is what you want to do, use the GNU Library -General Public License instead of this License. - -# - -Certain source files distributed by Oracle America, Inc. and/or its -affiliates are subject to the following clarification and special -exception to the GPLv2, based on the GNU Project exception for its -Classpath libraries, known as the GNU Classpath Exception, but only -where Oracle has expressly included in the particular source file's -header the words "Oracle designates this particular file as subject to -the "Classpath" exception as provided by Oracle in the LICENSE file -that accompanied this code." - -You should also note that Oracle includes multiple, independent -programs in this software package. Some of those programs are provided -under licenses deemed incompatible with the GPLv2 by the Free Software -Foundation and others. For example, the package includes programs -licensed under the Apache License, Version 2.0. Such programs are -licensed to you under their original licenses. - -Oracle facilitates your further distribution of this package by adding -the Classpath Exception to the necessary parts of its GPLv2 code, which -permits you to use that code in combination with other independent -modules not licensed under the GPLv2. However, note that this would -not permit you to commingle code under an incompatible license with -Oracle's GPLv2 licensed code by, for example, cutting and pasting such -code into a file also containing Oracle's GPLv2 licensed code and then -distributing the result. Additionally, if you were to remove the -Classpath Exception from any of the files to which it applies and -distribute the result, you would likely be required to license some or -all of the other code in that distribution under the GPLv2 as well, and -since the GPLv2 is incompatible with the license terms of some items -included in the distribution by Oracle, removing the Classpath -Exception could therefore effectively compromise your ability to -further distribute the package. - -Proceed with caution and we recommend that you obtain the advice of a -lawyer skilled in open source matters before removing the Classpath -Exception or making modifications to this package which may -subsequently be redistributed and/or involve the use of third party -software. - -CLASSPATH EXCEPTION -Linking this library statically or dynamically with other modules is -making a combined work based on this library. Thus, the terms and -conditions of the GNU General Public License version 2 cover the whole -combination. - -As a special exception, the copyright holders of this library give you -permission to link this library with independent modules to produce an -executable, regardless of the license terms of these independent -modules, and to copy and distribute the resulting executable under -terms of your choice, provided that you also meet, for each linked -independent module, the terms and conditions of the license of that -module. An independent module is a module which is not derived from or -based on this library. If you modify this library, you may extend this -exception to your version of the library, but you are not obligated to -do so. If you do not wish to do so, delete this exception statement -from your version. diff --git a/dolphinscheduler-dist/release-docs/licenses/LICENSE-resolver.txt b/dolphinscheduler-dist/release-docs/licenses/LICENSE-resolver.txt new file mode 100644 index 0000000000..d645695673 --- /dev/null +++ b/dolphinscheduler-dist/release-docs/licenses/LICENSE-resolver.txt @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/dolphinscheduler-dist/release-docs/licenses/ui-licenses/LICENSE-@form-create-element-ui b/dolphinscheduler-dist/release-docs/licenses/ui-licenses/LICENSE-@form-create-element-ui new file mode 100644 index 0000000000..5609421fb8 --- /dev/null +++ b/dolphinscheduler-dist/release-docs/licenses/ui-licenses/LICENSE-@form-create-element-ui @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2018 xaboy + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/dolphinscheduler-dist/src/main/provisio/dolphinscheduler.xml b/dolphinscheduler-dist/src/main/provisio/dolphinscheduler.xml new file mode 100644 index 0000000000..615965cf9a --- /dev/null +++ b/dolphinscheduler-dist/src/main/provisio/dolphinscheduler.xml @@ -0,0 +1,67 @@ + + + + + + + + DISCLAIMER + install.sh + LICENSE + NOTICE + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtils.java b/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtils.java deleted file mode 100644 index a244dd491e..0000000000 --- a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtils.java +++ /dev/null @@ -1,190 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.plugin.utils; - -import org.apache.commons.io.IOUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -/** - * property utils - * single instance - */ -public class PropertyUtils { - - private static final Logger logger = LoggerFactory.getLogger(PropertyUtils.class); - - private static final Properties properties = new Properties(); - - private PropertyUtils() { - throw new IllegalStateException("PropertyUtils class"); - } - - static { - String propertyFiles = "/plugin.properties"; - InputStream fis = null; - try { - fis = PropertyUtils.class.getResourceAsStream(propertyFiles); - properties.load(fis); - } catch (IOException e) { - logger.error(e.getMessage(), e); - if (fis != null) { - IOUtils.closeQuietly(fis); - } - } finally { - IOUtils.closeQuietly(fis); - } - } - - /** - * get property value - * - * @param key property name - * @param defaultVal default value - * @return property value - */ - public static String getString(String key, String defaultVal) { - String val = properties.getProperty(key.trim()); - return val == null ? defaultVal : val; - } - - /** - * get property value - * - * @param key property name - * @return property value - */ - public static String getString(String key) { - if (key == null) { - return null; - } - return properties.getProperty(key.trim()); - } - - /** - * get property value - * - * @param key property name - * @return get property int value , if key == null, then return -1 - */ - public static int getInt(String key) { - return getInt(key, -1); - } - - /** - * get int - * - * @param key key - * @param defaultValue default value - * @return property value - */ - public static int getInt(String key, int defaultValue) { - String value = properties.getProperty(key.trim()); - if (value == null) { - return defaultValue; - } - try { - return Integer.parseInt(value); - } catch (NumberFormatException e) { - logger.info(e.getMessage(), e); - } - return defaultValue; - } - - /** - * get property value - * - * @param key property name - * @return property value - */ - public static boolean getBoolean(String key) { - String value = properties.getProperty(key.trim()); - if (value == null) { - return false; - } - - return Boolean.parseBoolean(value); - } - - /** - * get property value - * - * @param key property name - * @param defaultValue default value - * @return property value - */ - public static Boolean getBoolean(String key, boolean defaultValue) { - String value = properties.getProperty(key.trim()); - if (value == null) { - return defaultValue; - } - - return Boolean.parseBoolean(value); - } - - /** - * get property long value - * - * @param key key - * @param defaultVal default value - * @return property value - */ - public static long getLong(String key, long defaultVal) { - String val = getString(key); - return val == null ? defaultVal : Long.parseLong(val); - } - - /** - * get long - * - * @param key key - * @return property value - */ - public static long getLong(String key) { - return getLong(key, -1); - } - - /** - * get double - * - * @param key key - * @param defaultVal default value - * @return property value - */ - public static double getDouble(String key, double defaultVal) { - String val = properties.getProperty(key.trim()); - return val == null ? defaultVal : Double.parseDouble(val); - } - - /** - * @param key key - * @param type type - * @param defaultValue default value - * @param T - * @return get enum value - */ - public > T getEnum(String key, Class type, - T defaultValue) { - String val = properties.getProperty(key.trim()); - return val == null ? defaultValue : Enum.valueOf(type, val); - } - -} diff --git a/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertDataTest.java b/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertDataTest.java deleted file mode 100644 index c19b5bc29a..0000000000 --- a/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/model/AlertDataTest.java +++ /dev/null @@ -1,80 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.plugin.model; - -import org.junit.Before; -import org.junit.Test; - -import static org.junit.Assert.*; - -public class AlertDataTest { - - private AlertData alertData; - - @Before - public void before() { - alertData = new AlertData(); - alertData.setId(1) - .setContent("content") - .setShowType("email") - .setTitle("title") - .setReceivers("receivers") - .setReceiversCc("cc") - .setLog("log") - .setAlertGroupId(1); - } - - @Test - public void getId() { - assertEquals(1, alertData.getId()); - } - - @Test - public void getTitle() { - assertEquals("title", alertData.getTitle()); - } - - @Test - public void getContent() { - assertEquals("content", alertData.getContent()); - } - - @Test - public void getLog() { - assertEquals("log", alertData.getLog()); - } - - @Test - public void getAlertGroupId() { - assertEquals(1, alertData.getAlertGroupId()); - } - - @Test - public void getReceivers() { - assertEquals("receivers", alertData.getReceivers()); - } - - @Test - public void getReceiversCc() { - assertEquals("cc", alertData.getReceiversCc()); - } - - @Test - public void getShowType() { - assertEquals("email", alertData.getShowType()); - } -} \ No newline at end of file diff --git a/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtilsTest.java b/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtilsTest.java deleted file mode 100644 index 614a7c009c..0000000000 --- a/dolphinscheduler-plugin-api/src/test/java/org/apache/dolphinscheduler/plugin/utils/PropertyUtilsTest.java +++ /dev/null @@ -1,83 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.dolphinscheduler.plugin.utils; - -import org.junit.Test; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import static org.junit.Assert.*; - -public class PropertyUtilsTest { - - private static final Logger logger = LoggerFactory.getLogger(PropertyUtilsTest.class); - - /** - * Test getString - */ - @Test - public void testGetString() { - - String result = PropertyUtils.getString("test.string"); - logger.info(result); - assertEquals("teststring", result); - - //If key is null, then return null - result = PropertyUtils.getString(null); - assertNull(result); - } - - - /** - * Test getBoolean - */ - @Test - public void testGetBoolean() { - - //Expected true - Boolean result = PropertyUtils.getBoolean("test.true"); - assertTrue(result); - - //Expected false - result = PropertyUtils.getBoolean("test.false"); - assertFalse(result); - } - - /** - * Test getLong - */ - @Test - public void testGetLong() { - long result = PropertyUtils.getLong("test.long"); - assertSame(100L, result); - } - - /** - * Test getDouble - */ - @Test - public void testGetDouble() { - - //If key is undefine in alert.properties, and there is a defaultval, then return defaultval - double result = PropertyUtils.getDouble("abc", 5.0); - assertEquals(5.0, result, 0); - - result = PropertyUtils.getDouble("cba", 5.0); - assertEquals(3.1, result, 0.01); - } - -} \ No newline at end of file diff --git a/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/CommandType.java b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/CommandType.java index 77c6064e5b..6c7377db17 100644 --- a/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/CommandType.java +++ b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/CommandType.java @@ -111,5 +111,15 @@ public enum CommandType { /** * pong */ - PONG; + PONG, + + /** + * alert send request + */ + ALERT_SEND_REQUEST, + + /** + * alert send response + */ + ALERT_SEND_RESPONSE; } diff --git a/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommand.java b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommand.java new file mode 100644 index 0000000000..da56b0dc6b --- /dev/null +++ b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommand.java @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.remote.command.alert; + +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.CommandType; +import org.apache.dolphinscheduler.remote.utils.JsonSerializer; + +import java.io.Serializable; + +public class AlertSendRequestCommand implements Serializable { + + private int groupId; + + private String title; + + private String content; + + public int getGroupId() { + return groupId; + } + + public void setGroupId(int groupId) { + this.groupId = groupId; + } + + public String getTitle() { + return title; + } + + public void setTitle(String title) { + this.title = title; + } + + public String getContent() { + return content; + } + + public void setContent(String content) { + this.content = content; + } + + public AlertSendRequestCommand(){ + + } + + public AlertSendRequestCommand(int groupId, String title, String content) { + this.groupId = groupId; + this.title = title; + this.content = content; + } + + /** + * package request command + * + * @return command + */ + public Command convert2Command() { + Command command = new Command(); + command.setType(CommandType.ALERT_SEND_REQUEST); + byte[] body = JsonSerializer.serialize(this); + command.setBody(body); + return command; + } +} diff --git a/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommand.java b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommand.java new file mode 100644 index 0000000000..984cc43c94 --- /dev/null +++ b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommand.java @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.remote.command.alert; + +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.CommandType; +import org.apache.dolphinscheduler.remote.utils.JsonSerializer; + +import java.io.Serializable; +import java.util.List; + +public class AlertSendResponseCommand implements Serializable { + + /** + * true:All alert are successful, + * false:As long as one alert fails + */ + private boolean resStatus; + + private List resResults; + + public boolean getResStatus() { + return resStatus; + } + + public void setResStatus(boolean resStatus) { + this.resStatus = resStatus; + } + + public List getResResults() { + return resResults; + } + + public void setResResults(List resResults) { + this.resResults = resResults; + } + + public AlertSendResponseCommand() { + + } + + public AlertSendResponseCommand(boolean resStatus, List resResults) { + this.resStatus = resStatus; + this.resResults = resResults; + } + + /** + * package response command + * + * @param opaque request unique identification + * @return command + */ + public Command convert2Command(long opaque) { + Command command = new Command(opaque); + command.setType(CommandType.ALERT_SEND_RESPONSE); + byte[] body = JsonSerializer.serialize(this); + command.setBody(body); + return command; + } +} diff --git a/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseResult.java b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseResult.java new file mode 100644 index 0000000000..1263b83a73 --- /dev/null +++ b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseResult.java @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.remote.command.alert; + +import java.io.Serializable; + +public class AlertSendResponseResult implements Serializable { + + private boolean status; + + private String message; + + public boolean getStatus() { + return status; + } + + public void setStatus(boolean status) { + this.status = status; + } + + public String getMessage() { + return message; + } + + public void setMessage(String message) { + this.message = message; + } + + public AlertSendResponseResult() { + + } + + public AlertSendResponseResult(boolean status, String message) { + this.status = status; + this.message = message; + } +} diff --git a/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/JsonSerializer.java b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/JsonSerializer.java new file mode 100644 index 0000000000..e183289f73 --- /dev/null +++ b/dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/JsonSerializer.java @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.remote.utils; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; + +/** + * json serialize or deserialize + */ +public class JsonSerializer { + private static final ObjectMapper objectMapper = new ObjectMapper(); + private static final Logger logger = LoggerFactory.getLogger(JsonSerializer.class); + + private JsonSerializer() { + + } + + /** + * serialize to byte + * + * @param obj object + * @param object type + * @return byte array + */ + public static byte[] serialize(T obj) { + String json = ""; + try { + json = objectMapper.writeValueAsString(obj); + } catch (JsonProcessingException e) { + logger.error("serializeToString exception!", e); + } + + return json.getBytes(Constants.UTF8); + } + + /** + * serialize to string + * + * @param obj object + * @param object type + * @return string + */ + public static String serializeToString(T obj) { + String json = ""; + try { + json = objectMapper.writeValueAsString(obj); + } catch (JsonProcessingException e) { + logger.error("serializeToString exception!", e); + } + + return json; + } + + /** + * deserialize + * + * @param src byte array + * @param clazz class + * @param deserialize type + * @return deserialize type + */ + public static T deserialize(byte[] src, Class clazz) { + + String json = new String(src, StandardCharsets.UTF_8); + try { + return objectMapper.readValue(json, clazz); + } catch (IOException e) { + logger.error("deserialize exception!", e); + return null; + } + + } + +} diff --git a/dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommandTest.java b/dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommandTest.java new file mode 100644 index 0000000000..79d21316f8 --- /dev/null +++ b/dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendRequestCommandTest.java @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.remote.command.alert; + +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.CommandType; + +import org.junit.Assert; +import org.junit.Test; + +public class AlertSendRequestCommandTest { + + @Test + public void testConvert2Command() { + int groupId = 1; + String title = "test-title"; + String content = "test-content"; + AlertSendRequestCommand requestCommand = new AlertSendRequestCommand(groupId,title,content); + Command command = requestCommand.convert2Command(); + Assert.assertEquals(CommandType.ALERT_SEND_REQUEST,command.getType()); + AlertSendRequestCommand verifyCommand = new AlertSendRequestCommand(); + verifyCommand.setGroupId(groupId); + verifyCommand.setContent(content); + verifyCommand.setTitle(title); + + } +} diff --git a/dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommandTest.java b/dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommandTest.java new file mode 100644 index 0000000000..41265a5339 --- /dev/null +++ b/dolphinscheduler-remote/src/test/java/org/apache/dolphinscheduler/remote/command/alert/AlertSendResponseCommandTest.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.remote.command.alert; + +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.CommandType; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.Assert; +import org.junit.Test; + +public class AlertSendResponseCommandTest { + + @Test + public void testConvert2Command() { + AlertSendResponseCommand alertSendResponseCommand = new AlertSendResponseCommand(); + alertSendResponseCommand.setResStatus(false); + List responseResults = new ArrayList<>(); + AlertSendResponseResult responseResult1 = new AlertSendResponseResult(); + responseResult1.setStatus(false); + responseResult1.setMessage("fail"); + responseResults.add(responseResult1); + + AlertSendResponseResult responseResult2 = new AlertSendResponseResult(true,"success"); + responseResults.add(responseResult2); + alertSendResponseCommand.setResResults(responseResults); + + Command command = alertSendResponseCommand.convert2Command(1); + Assert.assertEquals(CommandType.ALERT_SEND_RESPONSE,command.getType()); + } +} diff --git a/dolphinscheduler-server/pom.xml b/dolphinscheduler-server/pom.xml index 10fa58faa3..9655b290c5 100644 --- a/dolphinscheduler-server/pom.xml +++ b/dolphinscheduler-server/pom.xml @@ -16,177 +16,182 @@ ~ limitations under the License. --> - - 4.0.0 - - org.apache.dolphinscheduler - dolphinscheduler - 1.3.4-SNAPSHOT - - dolphinscheduler-server - dolphinscheduler-server + + 4.0.0 + + org.apache.dolphinscheduler + dolphinscheduler + 1.3.4-SNAPSHOT + + dolphinscheduler-server + dolphinscheduler-server - jar - - UTF-8 - + jar + + UTF-8 + - - - org.apache.dolphinscheduler - dolphinscheduler-common - - - io.netty - netty - - - io.netty - netty-all - - - com.google - netty - - - log4j-slf4j-impl - org.apache.logging.log4j - - - - - org.apache.dolphinscheduler - dolphinscheduler-dao - - - spring-boot-starter-logging - org.springframework.boot - - - + + + org.apache.dolphinscheduler + dolphinscheduler-spi + + + org.apache.dolphinscheduler + dolphinscheduler-common + + + io.netty + netty + + + io.netty + netty-all + + + com.google + netty + + + log4j-slf4j-impl + org.apache.logging.log4j + + + + + org.apache.dolphinscheduler + dolphinscheduler-dao + + + spring-boot-starter-logging + org.springframework.boot + + + - - org.apache.dolphinscheduler - dolphinscheduler-service - - - org.apache.curator - curator-framework - - - org.apache.zookeeper - zookeeper - - - - - org.apache.curator - curator-recipes - - - org.apache.zookeeper - zookeeper - - - - - org.apache.zookeeper - zookeeper - + + org.apache.dolphinscheduler + dolphinscheduler-service + + + org.apache.curator + curator-framework + + + org.apache.zookeeper + zookeeper + + + + + org.apache.curator + curator-recipes + + + org.apache.zookeeper + zookeeper + + + + + org.apache.zookeeper + zookeeper + - - org.apache.httpcomponents - httpclient - - - org.apache.httpcomponents - httpcore - - - junit - junit - test - + + org.apache.httpcomponents + httpclient + + + org.apache.httpcomponents + httpcore + + + junit + junit + test + - - org.apache.dolphinscheduler - dolphinscheduler-alert - + + org.apache.dolphinscheduler + dolphinscheduler-alert + - - org.powermock - powermock-module-junit4 - test - - - org.powermock - powermock-api-mockito2 - test - - - org.mockito - mockito-core - - - - - org.mockito - mockito-core - test - + + org.powermock + powermock-module-junit4 + test + + + org.powermock + powermock-api-mockito2 + test + + + org.mockito + mockito-core + + + + + org.mockito + mockito-core + test + org.springframework spring-test - + - - - - - - - - - - - - - - - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - - ${java.version} - ${java.version} - ${project.build.sourceEncoding} - - - - + + + + + + + + + + + + + + + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + ${java.version} + ${java.version} + ${project.build.sourceEncoding} + + + + diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/log/TaskLogAppender.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/log/TaskLogAppender.java old mode 100644 new mode 100755 index 0819b9fce3..9c0361e8e6 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/log/TaskLogAppender.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/log/TaskLogAppender.java @@ -14,13 +14,15 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.server.log; -import ch.qos.logback.classic.spi.ILoggingEvent; -import ch.qos.logback.core.FileAppender; +import static ch.qos.logback.classic.ClassicConstants.FINALIZE_SESSION_MARKER; + import org.slf4j.Marker; -import static ch.qos.logback.classic.ClassicConstants.FINALIZE_SESSION_MARKER; +import ch.qos.logback.classic.spi.ILoggingEvent; +import ch.qos.logback.core.FileAppender; /** * Task log appender diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java index fcff67f15b..be666aed70 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java @@ -14,6 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.server.master.runner; import org.apache.dolphinscheduler.common.enums.ExecutionStatus; @@ -25,19 +26,18 @@ import org.apache.dolphinscheduler.dao.AlertDao; import org.apache.dolphinscheduler.dao.entity.ProcessDefinition; import org.apache.dolphinscheduler.dao.entity.ProcessInstance; import org.apache.dolphinscheduler.dao.entity.TaskInstance; -import org.apache.dolphinscheduler.service.queue.TaskPriority; import org.apache.dolphinscheduler.server.master.config.MasterConfig; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; import org.apache.dolphinscheduler.service.process.ProcessService; +import org.apache.dolphinscheduler.service.queue.TaskPriority; import org.apache.dolphinscheduler.service.queue.TaskPriorityQueue; import org.apache.dolphinscheduler.service.queue.TaskPriorityQueueImpl; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - import java.util.Date; import java.util.concurrent.Callable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * master task exec base class @@ -97,7 +97,8 @@ public class MasterBaseTaskExecThread implements Callable { /** * constructor of MasterBaseTaskExecThread - * @param taskInstance task instance + * + * @param taskInstance task instance */ public MasterBaseTaskExecThread(TaskInstance taskInstance) { this.processService = SpringApplicationContext.getBean(ProcessService.class); @@ -124,7 +125,7 @@ public class MasterBaseTaskExecThread implements Callable { TaskNode taskNode = JSONUtils.parseObject(taskJson, TaskNode.class); taskTimeoutParameter = taskNode.getTaskTimeoutParameter(); - if(taskTimeoutParameter.getEnable()){ + if (taskTimeoutParameter.getEnable()) { checkTimeoutFlag = true; } } @@ -147,6 +148,7 @@ public class MasterBaseTaskExecThread implements Callable { /** * submit master base task exec thread + * * @return TaskInstance */ protected TaskInstance submit() { @@ -157,30 +159,30 @@ public class MasterBaseTaskExecThread implements Callable { boolean submitDB = false; boolean submitTask = false; TaskInstance task = null; - while (retryTimes <= commitRetryTimes){ + while (retryTimes <= commitRetryTimes) { try { - if(!submitDB){ + if (!submitDB) { // submit task to db task = processService.submitTask(taskInstance); - if(task != null && task.getId() != 0){ + if (task != null && task.getId() != 0) { submitDB = true; } } - if(submitDB && !submitTask){ + if (submitDB && !submitTask) { // dispatch task submitTask = dispatchTask(task); } - if(submitDB && submitTask){ + if (submitDB && submitTask) { return task; } - if(!submitDB){ + if (!submitDB) { logger.error("task commit to db failed , taskId {} has already retry {} times, please check the database", taskInstance.getId(), retryTimes); - }else if(!submitTask){ + } else if (!submitTask) { logger.error("task commit failed , taskId {} has already retry {} times, please check", taskInstance.getId(), retryTimes); } Thread.sleep(commitRetryInterval); } catch (Exception e) { - logger.error("task commit to mysql and dispatcht task failed",e); + logger.error("task commit to mysql and dispatcht task failed", e); } retryTimes += 1; } @@ -189,18 +191,19 @@ public class MasterBaseTaskExecThread implements Callable { /** * dispatcht task + * * @param taskInstance taskInstance * @return whether submit task success */ public Boolean dispatchTask(TaskInstance taskInstance) { - try{ - if(taskInstance.isConditionsTask() + try { + if (taskInstance.isConditionsTask() || taskInstance.isDependTask() - || taskInstance.isSubProcess()){ + || taskInstance.isSubProcess()) { return true; } - if(taskInstance.getState().typeIsFinished()){ + if (taskInstance.getState().typeIsFinished()) { logger.info(String.format("submit task , but task [%s] state [%s] is already finished. ", taskInstance.getName(), taskInstance.getState().toString())); return true; } @@ -221,9 +224,9 @@ public class MasterBaseTaskExecThread implements Callable { taskInstance.getId(), org.apache.dolphinscheduler.common.Constants.DEFAULT_WORKER_GROUP); taskUpdateQueue.put(taskPriority); - logger.info(String.format("master submit success, task : %s", taskInstance.getName()) ); + logger.info(String.format("master submit success, task : %s", taskInstance.getName())); return true; - }catch (Exception e){ + } catch (Exception e) { logger.error("submit task Exception: ", e); logger.error("task error : %s", JSONUtils.toJsonString(taskInstance)); return false; @@ -251,6 +254,7 @@ public class MasterBaseTaskExecThread implements Callable { /** * submit wait complete + * * @return true */ protected Boolean submitWaitComplete() { @@ -259,6 +263,7 @@ public class MasterBaseTaskExecThread implements Callable { /** * call + * * @return boolean * @throws Exception exception */ @@ -270,27 +275,25 @@ public class MasterBaseTaskExecThread implements Callable { /** * alert time out - * @return */ - protected boolean alertTimeout(){ - if( TaskTimeoutStrategy.FAILED == this.taskTimeoutParameter.getStrategy()){ + protected boolean alertTimeout() { + if (TaskTimeoutStrategy.FAILED == this.taskTimeoutParameter.getStrategy()) { return true; } logger.warn("process id:{} process name:{} task id: {},name:{} execution time out", processInstance.getId(), processInstance.getName(), taskInstance.getId(), taskInstance.getName()); // send warn mail ProcessDefinition processDefine = processService.findProcessDefineById(processInstance.getProcessDefinitionId()); - alertDao.sendTaskTimeoutAlert(processInstance.getWarningGroupId(),processDefine.getReceivers(), - processDefine.getReceiversCc(), processInstance.getId(), processInstance.getName(), - taskInstance.getId(),taskInstance.getName()); + alertDao.sendTaskTimeoutAlert(processInstance.getWarningGroupId(), processInstance.getId(), processInstance.getName(), + taskInstance.getId(), taskInstance.getName()); return true; } /** * handle time out for time out strategy warn&&failed */ - protected void handleTimeoutFailed(){ - if(TaskTimeoutStrategy.WARN == this.taskTimeoutParameter.getStrategy()){ + protected void handleTimeoutFailed() { + if (TaskTimeoutStrategy.WARN == this.taskTimeoutParameter.getStrategy()) { return; } logger.info("process id:{} name:{} task id:{} name:{} cancel because of timeout.", @@ -300,10 +303,9 @@ public class MasterBaseTaskExecThread implements Callable { /** * check task remain time valid - * @return */ - protected boolean checkTaskTimeout(){ - if (!checkTimeoutFlag || taskInstance.getStartTime() == null){ + protected boolean checkTaskTimeout() { + if (!checkTimeoutFlag || taskInstance.getStartTime() == null) { return false; } long remainTime = getRemainTime(taskTimeoutParameter.getInterval() * 60L); diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterTaskExecThread.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterTaskExecThread.java index b23ad45774..1ed2aaabf9 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterTaskExecThread.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterTaskExecThread.java @@ -14,8 +14,8 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.server.master.runner; +package org.apache.dolphinscheduler.server.master.runner; import org.apache.dolphinscheduler.common.Constants; import org.apache.dolphinscheduler.common.enums.ExecutionStatus; @@ -42,7 +42,6 @@ import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; import java.util.Date; import java.util.Set; - /** * master task exec thread */ @@ -64,9 +63,10 @@ public class MasterTaskExecThread extends MasterBaseTaskExecThread { /** * constructor of MasterTaskExecThread - * @param taskInstance task instance + * + * @param taskInstance task instance */ - public MasterTaskExecThread(TaskInstance taskInstance){ + public MasterTaskExecThread(TaskInstance taskInstance) { super(taskInstance); this.taskInstanceCacheManager = SpringApplicationContext.getBean(TaskInstanceCacheManagerImpl.class); this.nettyExecutorManager = SpringApplicationContext.getBean(NettyExecutorManager.class); @@ -75,10 +75,11 @@ public class MasterTaskExecThread extends MasterBaseTaskExecThread { /** * get task instance + * * @return TaskInstance */ @Override - public TaskInstance getTaskInstance(){ + public TaskInstance getTaskInstance() { return this.taskInstance; } @@ -96,47 +97,48 @@ public class MasterTaskExecThread extends MasterBaseTaskExecThread { public Boolean submitWaitComplete() { Boolean result = false; this.taskInstance = submit(); - if(this.taskInstance == null){ + if (this.taskInstance == null) { logger.error("submit task instance to mysql and queue failed , please check and fix it"); return result; } - if(!this.taskInstance.getState().typeIsFinished()) { + if (!this.taskInstance.getState().typeIsFinished()) { result = waitTaskQuit(); } taskInstance.setEndTime(new Date()); processService.updateTaskInstance(taskInstance); logger.info("task :{} id:{}, process id:{}, exec thread completed ", - this.taskInstance.getName(),taskInstance.getId(), processInstance.getId() ); + this.taskInstance.getName(), taskInstance.getId(), processInstance.getId()); return result; } /** * polling db - * + *

* wait task quit + * * @return true if task quit success */ - public Boolean waitTaskQuit(){ + public Boolean waitTaskQuit() { // query new state taskInstance = processService.findTaskInstanceById(taskInstance.getId()); logger.info("wait task: process id: {}, task id:{}, task name:{} complete", this.taskInstance.getProcessInstanceId(), this.taskInstance.getId(), this.taskInstance.getName()); - while (Stopper.isRunning()){ + while (Stopper.isRunning()) { try { - if(this.processInstance == null){ + if (this.processInstance == null) { logger.error("process instance not exists , master task exec thread exit"); return true; } // task instance add queue , waiting worker to kill - if(this.cancel || this.processInstance.getState() == ExecutionStatus.READY_STOP){ + if (this.cancel || this.processInstance.getState() == ExecutionStatus.READY_STOP) { cancelTaskInstance(); } - if(processInstance.getState() == ExecutionStatus.READY_PAUSE){ + if (processInstance.getState() == ExecutionStatus.READY_PAUSE) { pauseTask(); } // task instance finished - if (taskInstance.getState().typeIsFinished()){ + if (taskInstance.getState().typeIsFinished()) { // if task is final result , then remove taskInstance from cache taskInstanceCacheManager.removeByTaskInstanceId(taskInstance.getId()); break; @@ -149,10 +151,10 @@ public class MasterTaskExecThread extends MasterBaseTaskExecThread { processInstance = processService.findProcessInstanceById(processInstance.getId()); Thread.sleep(Constants.SLEEP_TIME_MILLIS); } catch (Exception e) { - logger.error("exception",e); + logger.error("exception", e); if (processInstance != null) { logger.error("wait task quit failed, instance id:{}, task id:{}", - processInstance.getId(), taskInstance.getId()); + processInstance.getId(), taskInstance.getId()); } } } @@ -161,31 +163,29 @@ public class MasterTaskExecThread extends MasterBaseTaskExecThread { /** * pause task if task have not been dispatched to worker, do not dispatch anymore. - * */ public void pauseTask() { taskInstance = processService.findTaskInstanceById(taskInstance.getId()); - if(taskInstance == null){ + if (taskInstance == null) { return; } - if(StringUtils.isBlank(taskInstance.getHost())){ + if (StringUtils.isBlank(taskInstance.getHost())) { taskInstance.setState(ExecutionStatus.PAUSE); taskInstance.setEndTime(new Date()); processService.updateTaskInstance(taskInstance); } } - /** - * task instance add queue , waiting worker to kill + * task instance add queue , waiting worker to kill */ - private void cancelTaskInstance() throws Exception{ - if(alreadyKilled){ + private void cancelTaskInstance() throws Exception { + if (alreadyKilled) { return; } alreadyKilled = true; taskInstance = processService.findTaskInstanceById(taskInstance.getId()); - if(StringUtils.isBlank(taskInstance.getHost())){ + if (StringUtils.isBlank(taskInstance.getHost())) { taskInstance.setState(ExecutionStatus.KILL); taskInstance.setEndTime(new Date()); processService.updateTaskInstance(taskInstance); @@ -203,23 +203,24 @@ public class MasterTaskExecThread extends MasterBaseTaskExecThread { nettyExecutorManager.executeDirectly(executionContext); logger.info("master kill taskInstance name :{} taskInstance id:{}", - taskInstance.getName(), taskInstance.getId() ); + taskInstance.getName(), taskInstance.getId()); } /** * whether exists valid worker group + * * @param taskInstanceWorkerGroup taskInstanceWorkerGroup * @return whether exists */ - public Boolean existsValidWorkerGroup(String taskInstanceWorkerGroup){ + public Boolean existsValidWorkerGroup(String taskInstanceWorkerGroup) { Set workerGroups = zookeeperRegistryCenter.getWorkerGroupDirectly(); // not worker group - if (CollectionUtils.isEmpty(workerGroups)){ + if (CollectionUtils.isEmpty(workerGroups)) { return false; } // has worker group , but not taskInstance assigned worker group - if (!workerGroups.contains(taskInstanceWorkerGroup)){ + if (!workerGroups.contains(taskInstanceWorkerGroup)) { return false; } Set workers = zookeeperRegistryCenter.getWorkerGroupNodesDirectly(taskInstanceWorkerGroup); diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/AlertManager.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/AlertManager.java index 6cfbc5b650..576a3b1801 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/AlertManager.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/AlertManager.java @@ -17,10 +17,8 @@ package org.apache.dolphinscheduler.server.utils; -import org.apache.dolphinscheduler.common.enums.AlertType; import org.apache.dolphinscheduler.common.enums.CommandType; import org.apache.dolphinscheduler.common.enums.Flag; -import org.apache.dolphinscheduler.common.enums.ShowType; import org.apache.dolphinscheduler.common.enums.WarningType; import org.apache.dolphinscheduler.common.utils.JSONUtils; import org.apache.dolphinscheduler.dao.AlertDao; @@ -180,14 +178,10 @@ public class AlertManager { try { Alert alert = new Alert(); alert.setTitle("worker fault tolerance"); - alert.setShowType(ShowType.TABLE); String content = getWorkerToleranceContent(processInstance, toleranceTaskList); alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); alert.setCreateTime(new Date()); alert.setAlertGroupId(processInstance.getWarningGroupId() == null ? 1 : processInstance.getWarningGroupId()); - alert.setReceivers(processInstance.getProcessDefinition().getReceivers()); - alert.setReceiversCc(processInstance.getProcessDefinition().getReceiversCc()); alertDao.addAlert(alert); logger.info("add alert to db , alert : {}", alert.toString()); @@ -238,16 +232,10 @@ public class AlertManager { String cmdName = getCommandCnName(processInstance.getCommandType()); String success = processInstance.getState().typeIsSuccess() ? "success" : "failed"; alert.setTitle(cmdName + " " + success); - ShowType showType = processInstance.getState().typeIsSuccess() ? ShowType.TEXT : ShowType.TABLE; - alert.setShowType(showType); - String content = getContentProcessInstance(processInstance, taskInstances, projectUser); + String content = getContentProcessInstance(processInstance, taskInstances,projectUser); alert.setContent(content); - alert.setAlertType(AlertType.EMAIL); alert.setAlertGroupId(processInstance.getWarningGroupId()); alert.setCreateTime(new Date()); - alert.setReceivers(processInstance.getProcessDefinition().getReceivers()); - alert.setReceiversCc(processInstance.getProcessDefinition().getReceiversCc()); - alertDao.addAlert(alert); logger.info("add alert to db , alert: {}", alert.toString()); } diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java index 6895de3d4a..bcc082846b 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java @@ -14,6 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.server.worker; import org.apache.dolphinscheduler.common.Constants; @@ -28,7 +29,11 @@ import org.apache.dolphinscheduler.server.worker.processor.TaskExecuteProcessor; import org.apache.dolphinscheduler.server.worker.processor.TaskKillProcessor; import org.apache.dolphinscheduler.server.worker.registry.WorkerRegistry; import org.apache.dolphinscheduler.server.worker.runner.RetryReportTaskStatusThread; +import org.apache.dolphinscheduler.service.alert.AlertClientService; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; + +import javax.annotation.PostConstruct; + import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; @@ -36,8 +41,6 @@ import org.springframework.boot.WebApplicationType; import org.springframework.boot.builder.SpringApplicationBuilder; import org.springframework.context.annotation.ComponentScan; -import javax.annotation.PostConstruct; - /** * worker server */ @@ -73,6 +76,11 @@ public class WorkerServer { @Autowired private SpringApplicationContext springApplicationContext; + /** + * alert model netty remote server + */ + private AlertClientService alertClientService; + @Autowired private RetryReportTaskStatusThread retryReportTaskStatusThread; @@ -92,7 +100,7 @@ public class WorkerServer { * worker server run */ @PostConstruct - public void run(){ + public void run() { logger.info("start worker server..."); //init remoting server @@ -108,6 +116,9 @@ public class WorkerServer { // worker registry this.workerRegistry.registry(); + //alert-server client registry + alertClientService = new AlertClientService(workerConfig.getAlertListenHost(),Constants.ALERT_RPC_PORT); + // retry report task status this.retryReportTaskStatusThread.start(); @@ -126,7 +137,7 @@ public class WorkerServer { try { //execute only once - if(Stopper.isStopped()){ + if (Stopper.isStopped()) { return; } @@ -138,13 +149,15 @@ public class WorkerServer { try { //thread sleep 3 seconds for thread quitely stop Thread.sleep(3000L); - }catch (Exception e){ + } catch (Exception e) { logger.warn("thread sleep exception", e); } this.nettyRemotingServer.close(); this.workerRegistry.unRegistry(); + this.alertClientService.close(); + } catch (Exception e) { logger.error("worker server stop exception ", e); System.exit(-1); diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/config/WorkerConfig.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/config/WorkerConfig.java index fa97403527..a32d4c8ff3 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/config/WorkerConfig.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/config/WorkerConfig.java @@ -1,4 +1,3 @@ - /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with @@ -15,11 +14,13 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + package org.apache.dolphinscheduler.server.worker.config; +import org.apache.dolphinscheduler.common.Constants; + import java.util.Set; -import org.apache.dolphinscheduler.common.Constants; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.PropertySource; import org.springframework.stereotype.Component; @@ -52,6 +53,9 @@ public class WorkerConfig { @Value("${worker.weight:100}") private int weight; + @Value("${alert.listen.host:localhost}") + private String alertListenHost; + public int getListenPort() { return listenPort; } @@ -101,7 +105,7 @@ public class WorkerConfig { } public int getWorkerMaxCpuloadAvg() { - if (workerMaxCpuloadAvg == -1){ + if (workerMaxCpuloadAvg == -1) { return Constants.DEFAULT_WORKER_CPU_LOAD; } return workerMaxCpuloadAvg; @@ -111,7 +115,6 @@ public class WorkerConfig { this.workerMaxCpuloadAvg = workerMaxCpuloadAvg; } - public int getWeight() { return weight; } @@ -119,4 +122,12 @@ public class WorkerConfig { public void setWeight(int weight) { this.weight = weight; } + + public String getAlertListenHost() { + return alertListenHost; + } + + public void setAlertListenHost(String alertListenHost) { + this.alertListenHost = alertListenHost; + } } \ No newline at end of file diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/processor/TaskExecuteProcessor.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/processor/TaskExecuteProcessor.java index e43a913e8c..d8a6e2bfd8 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/processor/TaskExecuteProcessor.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/processor/TaskExecuteProcessor.java @@ -38,6 +38,7 @@ import org.apache.dolphinscheduler.server.worker.cache.TaskExecutionContextCache import org.apache.dolphinscheduler.server.worker.cache.impl.TaskExecutionContextCacheManagerImpl; import org.apache.dolphinscheduler.server.worker.config.WorkerConfig; import org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread; +import org.apache.dolphinscheduler.service.alert.AlertClientService; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; import java.util.Date; @@ -71,6 +72,11 @@ public class TaskExecuteProcessor implements NettyRequestProcessor { */ private final TaskCallbackService taskCallbackService; + /** + * alert client service + */ + private AlertClientService alertClientService; + /** * taskExecutionContextCacheManager */ @@ -94,6 +100,15 @@ public class TaskExecuteProcessor implements NettyRequestProcessor { taskExecutionContextCacheManager.cacheTaskExecutionContext(taskExecutionContext); } + public TaskExecuteProcessor(AlertClientService alertClientService) { + this.taskCallbackService = SpringApplicationContext.getBean(TaskCallbackService.class); + this.workerConfig = SpringApplicationContext.getBean(WorkerConfig.class); + this.workerExecService = ThreadUtils.newDaemonFixedThreadExecutor("Worker-Execute-Thread", workerConfig.getWorkerExecThreads()); + this.taskExecutionContextCacheManager = SpringApplicationContext.getBean(TaskExecutionContextCacheManagerImpl.class); + + this.alertClientService = alertClientService; + } + @Override public void process(Channel channel, Command command) { Preconditions.checkArgument(CommandType.TASK_EXECUTE_REQUEST == command.getType(), @@ -150,7 +165,7 @@ public class TaskExecuteProcessor implements NettyRequestProcessor { this.doAck(taskExecutionContext); // submit task - workerExecService.submit(new TaskExecuteThread(taskExecutionContext, taskCallbackService, taskLogger)); + workerExecService.submit(new TaskExecuteThread(taskExecutionContext, taskCallbackService, taskLogger, alertClientService)); } private void doAck(TaskExecutionContext taskExecutionContext) { @@ -162,7 +177,6 @@ public class TaskExecuteProcessor implements NettyRequestProcessor { /** * build ack command - * * @param taskExecutionContext taskExecutionContext * @return TaskExecuteAckCommand */ diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java index c1d03e366e..721656730d 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java @@ -40,6 +40,7 @@ import org.apache.dolphinscheduler.server.worker.cache.impl.TaskExecutionContext import org.apache.dolphinscheduler.server.worker.processor.TaskCallbackService; import org.apache.dolphinscheduler.server.worker.task.AbstractTask; import org.apache.dolphinscheduler.server.worker.task.TaskManager; +import org.apache.dolphinscheduler.service.alert.AlertClientService; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; import org.apache.commons.collections.MapUtils; @@ -93,6 +94,11 @@ public class TaskExecuteThread implements Runnable { */ private Logger taskLogger; + /** + * alert client server + */ + private AlertClientService alertClientService; + /** * constructor * @param taskExecutionContext taskExecutionContext @@ -100,11 +106,12 @@ public class TaskExecuteThread implements Runnable { */ public TaskExecuteThread(TaskExecutionContext taskExecutionContext , TaskCallbackService taskCallbackService - , Logger taskLogger) { + , Logger taskLogger, AlertClientService alertClientService) { this.taskExecutionContext = taskExecutionContext; this.taskCallbackService = taskCallbackService; this.taskExecutionContextCacheManager = SpringApplicationContext.getBean(TaskExecutionContextCacheManagerImpl.class); this.taskLogger = taskLogger; + this.alertClientService = alertClientService; } @Override @@ -151,7 +158,7 @@ public class TaskExecuteThread implements Runnable { taskExecutionContext.getProcessInstanceId(), taskExecutionContext.getTaskInstanceId())); - task = TaskManager.newTask(taskExecutionContext, taskLogger); + task = TaskManager.newTask(taskExecutionContext, taskLogger, alertClientService); // task init task.init(); diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/TaskManager.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/TaskManager.java index a9463336b4..b89c8d4ca6 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/TaskManager.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/TaskManager.java @@ -30,6 +30,8 @@ import org.apache.dolphinscheduler.server.worker.task.shell.ShellTask; import org.apache.dolphinscheduler.server.worker.task.spark.SparkTask; import org.apache.dolphinscheduler.server.worker.task.sql.SqlTask; import org.apache.dolphinscheduler.server.worker.task.sqoop.SqoopTask; +import org.apache.dolphinscheduler.service.alert.AlertClientService; + import org.slf4j.Logger; /** @@ -44,7 +46,7 @@ public class TaskManager { * @return AbstractTask * @throws IllegalArgumentException illegal argument exception */ - public static AbstractTask newTask(TaskExecutionContext taskExecutionContext, Logger logger) throws IllegalArgumentException { + public static AbstractTask newTask(TaskExecutionContext taskExecutionContext, Logger logger, AlertClientService alertClientService) throws IllegalArgumentException { TaskType anEnum = EnumUtils.getEnum(TaskType.class, taskExecutionContext.getTaskType()); if (anEnum == null) { logger.error("not support task type: {}", taskExecutionContext.getTaskType()); @@ -57,7 +59,7 @@ public class TaskManager { case PROCEDURE: return new ProcedureTask(taskExecutionContext, logger); case SQL: - return new SqlTask(taskExecutionContext, logger); + return new SqlTask(taskExecutionContext, logger, alertClientService); case MR: return new MapReduceTask(taskExecutionContext, logger); case SPARK: diff --git a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTask.java b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTask.java index 523d2e0848..7aabfce49d 100644 --- a/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTask.java +++ b/dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTask.java @@ -14,46 +14,62 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.server.worker.task.sql; -import com.fasterxml.jackson.databind.node.ArrayNode; -import com.fasterxml.jackson.databind.node.ObjectNode; +package org.apache.dolphinscheduler.server.worker.task.sql; -import org.apache.commons.lang.StringUtils; +import static org.apache.dolphinscheduler.common.Constants.HIVE_CONF; +import static org.apache.dolphinscheduler.common.Constants.PASSWORD; +import static org.apache.dolphinscheduler.common.Constants.SEMICOLON; +import static org.apache.dolphinscheduler.common.Constants.USER; +import static org.apache.dolphinscheduler.common.enums.DbType.HIVE; -import org.apache.dolphinscheduler.alert.utils.MailUtils; import org.apache.dolphinscheduler.common.Constants; -import org.apache.dolphinscheduler.common.enums.*; +import org.apache.dolphinscheduler.common.enums.CommandType; import org.apache.dolphinscheduler.common.enums.DbType; -import org.apache.dolphinscheduler.common.enums.ShowType; import org.apache.dolphinscheduler.common.enums.TaskTimeoutStrategy; import org.apache.dolphinscheduler.common.process.Property; import org.apache.dolphinscheduler.common.task.AbstractParameters; import org.apache.dolphinscheduler.common.task.sql.SqlBinds; import org.apache.dolphinscheduler.common.task.sql.SqlParameters; import org.apache.dolphinscheduler.common.task.sql.SqlType; -import org.apache.dolphinscheduler.common.utils.*; +import org.apache.dolphinscheduler.common.utils.CollectionUtils; +import org.apache.dolphinscheduler.common.utils.CommonUtils; +import org.apache.dolphinscheduler.common.utils.JSONUtils; +import org.apache.dolphinscheduler.common.utils.ParameterUtils; +import org.apache.dolphinscheduler.common.utils.StringUtils; import org.apache.dolphinscheduler.dao.AlertDao; import org.apache.dolphinscheduler.dao.datasource.BaseDataSource; import org.apache.dolphinscheduler.dao.datasource.DataSourceFactory; -import org.apache.dolphinscheduler.dao.entity.User; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseCommand; import org.apache.dolphinscheduler.server.entity.SQLTaskExecutionContext; import org.apache.dolphinscheduler.server.entity.TaskExecutionContext; import org.apache.dolphinscheduler.server.utils.ParamUtils; import org.apache.dolphinscheduler.server.utils.UDFUtils; import org.apache.dolphinscheduler.server.worker.task.AbstractTask; +import org.apache.dolphinscheduler.service.alert.AlertClientService; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; -import org.slf4j.Logger; - -import java.sql.*; -import java.util.*; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.ResultSetMetaData; +import java.sql.SQLException; +import java.sql.Statement; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Properties; import java.util.regex.Matcher; import java.util.regex.Pattern; import java.util.stream.Collectors; -import static org.apache.dolphinscheduler.common.Constants.*; -import static org.apache.dolphinscheduler.common.enums.DbType.HIVE; +import org.slf4j.Logger; + +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.ObjectNode; /** * sql task @@ -83,7 +99,10 @@ public class SqlTask extends AbstractTask { */ private static final int LIMIT = 10000; - public SqlTask(TaskExecutionContext taskExecutionContext, Logger logger) { + + private AlertClientService alertClientService; + + public SqlTask(TaskExecutionContext taskExecutionContext, Logger logger, AlertClientService alertClientService) { super(taskExecutionContext, logger); this.taskExecutionContext = taskExecutionContext; @@ -95,6 +114,7 @@ public class SqlTask extends AbstractTask { throw new RuntimeException("sql task params is not valid"); } + this.alertClientService = alertClientService; this.alertDao = SpringApplicationContext.getBean(AlertDao.class); } @@ -161,7 +181,6 @@ public class SqlTask extends AbstractTask { // find process instance by task id - Map paramsMap = ParamUtils.convert(ParamUtils.getUserDefParamsMap(taskExecutionContext.getDefinedParams()), taskExecutionContext.getDefinedParams(), sqlParameters.getLocalParametersMap(), @@ -292,8 +311,7 @@ public class SqlTask extends AbstractTask { String result = JSONUtils.toJsonString(resultJSONArray); logger.debug("execute sql : {}", result); - sendAttachment(StringUtils.isNotEmpty(sqlParameters.getTitle()) ? - sqlParameters.getTitle() : taskExecutionContext.getTaskName() + " query result sets", + sendAttachment(sqlParameters.getGroupId(), StringUtils.isNotEmpty(sqlParameters.getTitle()) ? sqlParameters.getTitle() : taskExecutionContext.getTaskName() + " query result sets", JSONUtils.toJsonString(resultJSONArray)); } @@ -419,8 +437,8 @@ public class SqlTask extends AbstractTask { */ private PreparedStatement prepareStatementAndBind(Connection connection, SqlBinds sqlBinds) throws Exception { // is the timeout set - boolean timeoutFlag = TaskTimeoutStrategy.of(taskExecutionContext.getTaskTimeoutStrategy()) == TaskTimeoutStrategy.FAILED || - TaskTimeoutStrategy.of(taskExecutionContext.getTaskTimeoutStrategy()) == TaskTimeoutStrategy.WARNFAILED; + boolean timeoutFlag = TaskTimeoutStrategy.of(taskExecutionContext.getTaskTimeoutStrategy()) == TaskTimeoutStrategy.FAILED + || TaskTimeoutStrategy.of(taskExecutionContext.getTaskTimeoutStrategy()) == TaskTimeoutStrategy.WARNFAILED; PreparedStatement stmt = connection.prepareStatement(sqlBinds.getSql()); if (timeoutFlag) { stmt.setQueryTimeout(taskExecutionContext.getTaskTimeout()); @@ -442,45 +460,10 @@ public class SqlTask extends AbstractTask { * @param title title * @param content content */ - public void sendAttachment(String title, String content) { - - List users = alertDao.queryUserByAlertGroupId(taskExecutionContext.getSqlTaskExecutionContext().getWarningGroupId()); - - // receiving group list - List receiversList = new ArrayList<>(); - for (User user : users) { - receiversList.add(user.getEmail().trim()); - } - // custom receiver - String receivers = sqlParameters.getReceivers(); - if (StringUtils.isNotEmpty(receivers)) { - String[] splits = receivers.split(COMMA); - for (String receiver : splits) { - receiversList.add(receiver.trim()); - } - } - - // copy list - List receiversCcList = new ArrayList<>(); - // Custom Copier - String receiversCc = sqlParameters.getReceiversCc(); - if (StringUtils.isNotEmpty(receiversCc)) { - String[] splits = receiversCc.split(COMMA); - for (String receiverCc : splits) { - receiversCcList.add(receiverCc.trim()); - } - } - - String showTypeName = sqlParameters.getShowType().replace(COMMA, "").trim(); - if (EnumUtils.isValidEnum(ShowType.class, showTypeName)) { - Map mailResult = MailUtils.sendMails(receiversList, - receiversCcList, title, content, ShowType.valueOf(showTypeName).getDescp()); - if (!(boolean) mailResult.get(STATUS)) { - throw new RuntimeException("send mail failed!"); - } - } else { - logger.error("showType: {} is not valid ", showTypeName); - throw new RuntimeException(String.format("showType: %s is not valid ", showTypeName)); + public void sendAttachment(int groupId, String title, String content) { + AlertSendResponseCommand alertSendResponseCommand = alertClientService.sendAlert(groupId, title, content); + if (!alertSendResponseCommand.getResStatus()) { + throw new RuntimeException("send mail failed!"); } } diff --git a/dolphinscheduler-server/src/main/resources/worker.properties b/dolphinscheduler-server/src/main/resources/worker.properties index 9fba30c147..cea1b4ea6c 100644 --- a/dolphinscheduler-server/src/main/resources/worker.properties +++ b/dolphinscheduler-server/src/main/resources/worker.properties @@ -35,3 +35,6 @@ # default worker weight #work.weight=100 + +# alert server listener host +alert.listen.host=localhost diff --git a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java index 30f1053d3a..d51d64d68d 100644 --- a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java +++ b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java @@ -18,10 +18,25 @@ package org.apache.dolphinscheduler.server.registry; import org.apache.dolphinscheduler.dao.AlertDao; -import org.apache.dolphinscheduler.dao.mapper.*; +import org.apache.dolphinscheduler.dao.mapper.AlertGroupMapper; +import org.apache.dolphinscheduler.dao.mapper.AlertMapper; +import org.apache.dolphinscheduler.dao.mapper.AlertPluginInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.CommandMapper; +import org.apache.dolphinscheduler.dao.mapper.DataSourceMapper; +import org.apache.dolphinscheduler.dao.mapper.ErrorCommandMapper; +import org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper; +import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper; +import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapMapper; +import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.ProjectMapper; +import org.apache.dolphinscheduler.dao.mapper.ResourceMapper; +import org.apache.dolphinscheduler.dao.mapper.ScheduleMapper; +import org.apache.dolphinscheduler.dao.mapper.TaskInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.TenantMapper; +import org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper; +import org.apache.dolphinscheduler.dao.mapper.UserMapper; import org.apache.dolphinscheduler.server.master.cache.impl.TaskInstanceCacheManagerImpl; import org.apache.dolphinscheduler.server.master.config.MasterConfig; -import org.apache.dolphinscheduler.server.master.dispatch.ExecutorDispatcher; import org.apache.dolphinscheduler.server.master.dispatch.host.HostManager; import org.apache.dolphinscheduler.server.master.dispatch.host.RandomHostManager; import org.apache.dolphinscheduler.server.master.processor.queue.TaskResponseService; @@ -29,6 +44,7 @@ import org.apache.dolphinscheduler.server.worker.processor.TaskCallbackService; import org.apache.dolphinscheduler.service.process.ProcessService; import org.apache.dolphinscheduler.service.queue.TaskPriorityQueue; import org.apache.dolphinscheduler.service.queue.TaskPriorityQueueImpl; + import org.mockito.Mockito; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @@ -50,108 +66,118 @@ public class DependencyConfig { } @Bean - public UserAlertGroupMapper userAlertGroupMapper() { - return Mockito.mock(UserAlertGroupMapper.class); - } - - @Bean - public TaskInstanceCacheManagerImpl taskInstanceCacheManagerImpl(){ + public TaskInstanceCacheManagerImpl taskInstanceCacheManagerImpl() { return Mockito.mock(TaskInstanceCacheManagerImpl.class); } @Bean - public ProcessService processService(){ + public ProcessService processService() { return Mockito.mock(ProcessService.class); } @Bean - public MasterConfig masterConfig(){ + public MasterConfig masterConfig() { return Mockito.mock(MasterConfig.class); } + @Bean - public UserMapper userMapper(){ + public UserMapper userMapper() { return Mockito.mock(UserMapper.class); } @Bean - public ProcessDefinitionMapper processDefineMapper(){ + public ProcessDefinitionMapper processDefineMapper() { return Mockito.mock(ProcessDefinitionMapper.class); } @Bean - public ProcessInstanceMapper processInstanceMapper(){ + public ProcessInstanceMapper processInstanceMapper() { return Mockito.mock(ProcessInstanceMapper.class); } @Bean - public DataSourceMapper dataSourceMapper(){ + public DataSourceMapper dataSourceMapper() { return Mockito.mock(DataSourceMapper.class); } @Bean - public ProcessInstanceMapMapper processInstanceMapMapper(){ + public ProcessInstanceMapMapper processInstanceMapMapper() { return Mockito.mock(ProcessInstanceMapMapper.class); } @Bean - public TaskInstanceMapper taskInstanceMapper(){ + public TaskInstanceMapper taskInstanceMapper() { return Mockito.mock(TaskInstanceMapper.class); } @Bean - public CommandMapper commandMapper(){ + public CommandMapper commandMapper() { return Mockito.mock(CommandMapper.class); } @Bean - public ScheduleMapper scheduleMapper(){ + public ScheduleMapper scheduleMapper() { return Mockito.mock(ScheduleMapper.class); } @Bean - public UdfFuncMapper udfFuncMapper(){ + public UdfFuncMapper udfFuncMapper() { return Mockito.mock(UdfFuncMapper.class); } @Bean - public ResourceMapper resourceMapper(){ + public ResourceMapper resourceMapper() { return Mockito.mock(ResourceMapper.class); } - - @Bean - public ErrorCommandMapper errorCommandMapper(){ + public ErrorCommandMapper errorCommandMapper() { return Mockito.mock(ErrorCommandMapper.class); } @Bean - public TenantMapper tenantMapper(){ + public TenantMapper tenantMapper() { return Mockito.mock(TenantMapper.class); } @Bean - public ProjectMapper projectMapper(){ + public ProjectMapper projectMapper() { return Mockito.mock(ProjectMapper.class); } @Bean - public TaskCallbackService taskCallbackService(){ + public TaskCallbackService taskCallbackService() { return Mockito.mock(TaskCallbackService.class); } @Bean - public HostManager hostManager(){ + public HostManager hostManager() { return new RandomHostManager(); } @Bean - public TaskResponseService taskResponseService(){ + public TaskResponseService taskResponseService() { return Mockito.mock(TaskResponseService.class); } @Bean - public TaskPriorityQueue taskPriorityQueue(){ + public TaskPriorityQueue taskPriorityQueue() { return new TaskPriorityQueueImpl(); } + + @Bean + public AlertPluginInstanceMapper alertPluginInstanceMapper() { + return Mockito.mock(AlertPluginInstanceMapper.class); + } + + @Bean + public AlertGroupMapper alertGroupMapper() { + return Mockito.mock(AlertGroupMapper.class); + } + + @Bean + public PluginDefineMapper pluginDefineMapper() { + return Mockito.mock(PluginDefineMapper.class); + } + } diff --git a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/processor/TaskCallbackServiceTestConfig.java b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/processor/TaskCallbackServiceTestConfig.java index 942a2d52bb..224764c882 100644 --- a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/processor/TaskCallbackServiceTestConfig.java +++ b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/processor/TaskCallbackServiceTestConfig.java @@ -18,9 +18,26 @@ package org.apache.dolphinscheduler.server.worker.processor; import org.apache.dolphinscheduler.dao.AlertDao; -import org.apache.dolphinscheduler.dao.mapper.*; +import org.apache.dolphinscheduler.dao.mapper.AlertGroupMapper; +import org.apache.dolphinscheduler.dao.mapper.AlertMapper; +import org.apache.dolphinscheduler.dao.mapper.AlertPluginInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.CommandMapper; +import org.apache.dolphinscheduler.dao.mapper.DataSourceMapper; +import org.apache.dolphinscheduler.dao.mapper.ErrorCommandMapper; +import org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper; +import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper; +import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapMapper; +import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.ProjectMapper; +import org.apache.dolphinscheduler.dao.mapper.ResourceMapper; +import org.apache.dolphinscheduler.dao.mapper.ScheduleMapper; +import org.apache.dolphinscheduler.dao.mapper.TaskInstanceMapper; +import org.apache.dolphinscheduler.dao.mapper.TenantMapper; +import org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper; +import org.apache.dolphinscheduler.dao.mapper.UserMapper; import org.apache.dolphinscheduler.server.master.cache.impl.TaskInstanceCacheManagerImpl; import org.apache.dolphinscheduler.service.process.ProcessService; + import org.mockito.Mockito; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @@ -31,7 +48,6 @@ import org.springframework.context.annotation.Configuration; @Configuration public class TaskCallbackServiceTestConfig { - @Bean public AlertDao alertDao() { return new AlertDao(); @@ -43,84 +59,93 @@ public class TaskCallbackServiceTestConfig { } @Bean - public UserAlertGroupMapper userAlertGroupMapper() { - return Mockito.mock(UserAlertGroupMapper.class); - } - - @Bean - public TaskInstanceCacheManagerImpl taskInstanceCacheManagerImpl(){ + public TaskInstanceCacheManagerImpl taskInstanceCacheManagerImpl() { return Mockito.mock(TaskInstanceCacheManagerImpl.class); } @Bean - public ProcessService processService(){ + public ProcessService processService() { return Mockito.mock(ProcessService.class); } @Bean - public UserMapper userMapper(){ + public UserMapper userMapper() { return Mockito.mock(UserMapper.class); } @Bean - public ProcessDefinitionMapper processDefineMapper(){ + public ProcessDefinitionMapper processDefineMapper() { return Mockito.mock(ProcessDefinitionMapper.class); } @Bean - public ProcessInstanceMapper processInstanceMapper(){ + public ProcessInstanceMapper processInstanceMapper() { return Mockito.mock(ProcessInstanceMapper.class); } @Bean - public DataSourceMapper dataSourceMapper(){ + public DataSourceMapper dataSourceMapper() { return Mockito.mock(DataSourceMapper.class); } @Bean - public ProcessInstanceMapMapper processInstanceMapMapper(){ + public ProcessInstanceMapMapper processInstanceMapMapper() { return Mockito.mock(ProcessInstanceMapMapper.class); } @Bean - public TaskInstanceMapper taskInstanceMapper(){ + public TaskInstanceMapper taskInstanceMapper() { return Mockito.mock(TaskInstanceMapper.class); } @Bean - public CommandMapper commandMapper(){ + public CommandMapper commandMapper() { return Mockito.mock(CommandMapper.class); } @Bean - public ScheduleMapper scheduleMapper(){ + public ScheduleMapper scheduleMapper() { return Mockito.mock(ScheduleMapper.class); } @Bean - public UdfFuncMapper udfFuncMapper(){ + public UdfFuncMapper udfFuncMapper() { return Mockito.mock(UdfFuncMapper.class); } @Bean - public ResourceMapper resourceMapper(){ + public ResourceMapper resourceMapper() { return Mockito.mock(ResourceMapper.class); } - @Bean - public ErrorCommandMapper errorCommandMapper(){ + public ErrorCommandMapper errorCommandMapper() { return Mockito.mock(ErrorCommandMapper.class); } @Bean - public TenantMapper tenantMapper(){ + public TenantMapper tenantMapper() { return Mockito.mock(TenantMapper.class); } @Bean - public ProjectMapper projectMapper(){ + public ProjectMapper projectMapper() { return Mockito.mock(ProjectMapper.class); } + @Bean + public AlertPluginInstanceMapper alertPluginInstanceMapper() { + return Mockito.mock(AlertPluginInstanceMapper.class); + } + + @Bean + public AlertGroupMapper alertGroupMapper() { + return Mockito.mock(AlertGroupMapper.class); + } + + @Bean + public PluginDefineMapper pluginDefineMapper() { + return Mockito.mock(PluginDefineMapper.class); + } + } diff --git a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThreadTest.java b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThreadTest.java index 86d3f8832c..30b4c4fe92 100644 --- a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThreadTest.java +++ b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThreadTest.java @@ -32,9 +32,10 @@ import org.apache.dolphinscheduler.server.worker.cache.impl.TaskExecutionContext import org.apache.dolphinscheduler.server.worker.processor.TaskCallbackService; import org.apache.dolphinscheduler.server.worker.task.AbstractTask; import org.apache.dolphinscheduler.server.worker.task.TaskManager; +import org.apache.dolphinscheduler.service.alert.AlertClientService; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; -import java.util.ArrayList; +import java.util.Collections; import java.util.Date; import java.util.List; @@ -67,6 +68,8 @@ public class TaskExecuteThreadTest { private TaskExecutionContextCacheManagerImpl taskExecutionContextCacheManager; + private AlertClientService alertClientService; + @Before public void before() { // init task execution context, logger @@ -103,8 +106,10 @@ public class TaskExecuteThreadTest { PowerMockito.when(SpringApplicationContext.getBean(TaskExecutionContextCacheManagerImpl.class)) .thenReturn(taskExecutionContextCacheManager); + alertClientService = PowerMockito.mock(AlertClientService.class); + PowerMockito.mockStatic(TaskManager.class); - PowerMockito.when(TaskManager.newTask(taskExecutionContext, taskLogger)) + PowerMockito.when(TaskManager.newTask(taskExecutionContext, taskLogger, alertClientService)) .thenReturn(new SimpleTask(taskExecutionContext, taskLogger)); PowerMockito.mockStatic(JSONUtils.class); @@ -114,9 +119,7 @@ public class TaskExecuteThreadTest { PowerMockito.mockStatic(CommonUtils.class); PowerMockito.when(CommonUtils.getSystemEnvPath()).thenReturn("/user_home/.bash_profile"); - List osUserList = new ArrayList() {{ - add("test"); - }}; + List osUserList = Collections.singletonList("test"); PowerMockito.mockStatic(OSUtils.class); PowerMockito.when(OSUtils.getUserList()).thenReturn(osUserList); } @@ -127,7 +130,7 @@ public class TaskExecuteThreadTest { taskExecutionContext.setStartTime(new Date()); taskExecutionContext.setCurrentExecutionStatus(ExecutionStatus.RUNNING_EXECUTION); taskExecutionContext.setTenantCode("test"); - TaskExecuteThread taskExecuteThread = new TaskExecuteThread(taskExecutionContext, taskCallbackService, taskLogger); + TaskExecuteThread taskExecuteThread = new TaskExecuteThread(taskExecutionContext, taskCallbackService, taskLogger, alertClientService); taskExecuteThread.run(); taskExecutionContext.getCurrentExecutionStatus(); @@ -143,7 +146,7 @@ public class TaskExecuteThreadTest { taskExecutionContext.setDelayTime(1); taskExecutionContext.setCurrentExecutionStatus(ExecutionStatus.DELAY_EXECUTION); taskExecutionContext.setTenantCode("test"); - TaskExecuteThread taskExecuteThread = new TaskExecuteThread(taskExecutionContext, taskCallbackService, taskLogger); + TaskExecuteThread taskExecuteThread = new TaskExecuteThread(taskExecutionContext, taskCallbackService, taskLogger, alertClientService); taskExecuteThread.run(); Assert.assertEquals(ExecutionStatus.RUNNING_EXECUTION, taskExecutionContext.getCurrentExecutionStatus()); diff --git a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/TaskManagerTest.java b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/TaskManagerTest.java index eb0383979c..6acfd180c2 100644 --- a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/TaskManagerTest.java +++ b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/TaskManagerTest.java @@ -20,6 +20,7 @@ package org.apache.dolphinscheduler.server.worker.task; import org.apache.dolphinscheduler.common.utils.LoggerUtils; import org.apache.dolphinscheduler.server.entity.TaskExecutionContext; import org.apache.dolphinscheduler.server.worker.cache.impl.TaskExecutionContextCacheManagerImpl; +import org.apache.dolphinscheduler.service.alert.AlertClientService; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; import java.util.Date; @@ -46,6 +47,8 @@ public class TaskManagerTest { private TaskExecutionContextCacheManagerImpl taskExecutionContextCacheManager; + private AlertClientService alertClientService; + @Before public void before() { // init task execution context, logger @@ -74,41 +77,43 @@ public class TaskManagerTest { PowerMockito.mockStatic(SpringApplicationContext.class); PowerMockito.when(SpringApplicationContext.getBean(TaskExecutionContextCacheManagerImpl.class)) .thenReturn(taskExecutionContextCacheManager); + + alertClientService = PowerMockito.mock(AlertClientService.class); } @Test public void testNewTask() { taskExecutionContext.setTaskType("SHELL"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("WATERDROP"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("HTTP"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("MR"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("SPARK"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("FLINK"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("PYTHON"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("DATAX"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); taskExecutionContext.setTaskType("SQOOP"); - Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger)); + Assert.assertNotNull(TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService)); } @Test(expected = IllegalArgumentException.class) public void testNewTaskIsNull() { taskExecutionContext.setTaskType(null); - TaskManager.newTask(taskExecutionContext,taskLogger); + TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService); } @Test(expected = IllegalArgumentException.class) public void testNewTaskIsNotExists() { taskExecutionContext.setTaskType("XXX"); - TaskManager.newTask(taskExecutionContext,taskLogger); + TaskManager.newTask(taskExecutionContext,taskLogger,alertClientService); } } diff --git a/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTaskTest.java b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTaskTest.java new file mode 100644 index 0000000000..64db568916 --- /dev/null +++ b/dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTaskTest.java @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.server.worker.task.sql; + +import org.apache.dolphinscheduler.common.Constants; +import org.apache.dolphinscheduler.common.utils.ParameterUtils; +import org.apache.dolphinscheduler.server.entity.SQLTaskExecutionContext; +import org.apache.dolphinscheduler.server.entity.TaskExecutionContext; +import org.apache.dolphinscheduler.server.worker.task.TaskProps; +import org.apache.dolphinscheduler.service.alert.AlertClientService; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.PreparedStatement; +import java.util.Date; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.powermock.api.mockito.PowerMockito; +import org.powermock.core.classloader.annotations.PrepareForTest; +import org.powermock.modules.junit4.PowerMockRunner; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * sql task test + */ +@RunWith(PowerMockRunner.class) +@PrepareForTest(value = {SqlTask.class, DriverManager.class}) +public class SqlTaskTest { + + private static final Logger logger = LoggerFactory.getLogger(SqlTaskTest.class); + + private static final String CONNECTION_PARAMS = "{\"user\":\"root\",\"password\":\"123456\",\"address\":\"jdbc:mysql://127.0.0.1:3306\"," + + "\"database\":\"test\",\"jdbcUrl\":\"jdbc:mysql://127.0.0.1:3306/test\"}"; + + private SqlTask sqlTask; + + private TaskExecutionContext taskExecutionContext; + + private AlertClientService alertClientService; + @Before + public void before() throws Exception { + taskExecutionContext = new TaskExecutionContext(); + + TaskProps props = new TaskProps(); + props.setExecutePath("/tmp"); + props.setTaskAppId(String.valueOf(System.currentTimeMillis())); + props.setTaskInstanceId(1); + props.setTenantCode("1"); + props.setEnvFile(".dolphinscheduler_env.sh"); + props.setTaskStartTime(new Date()); + props.setTaskTimeout(0); + props.setTaskParams( + "{\"localParams\":[],\"type\":\"POSTGRESQL\",\"datasource\":1,\"sql\":\"insert into tb_1 values('1','2')\",\"sqlType\":1}"); + + taskExecutionContext = PowerMockito.mock(TaskExecutionContext.class); + PowerMockito.when(taskExecutionContext.getTaskParams()).thenReturn(props.getTaskParams()); + PowerMockito.when(taskExecutionContext.getExecutePath()).thenReturn("/tmp"); + PowerMockito.when(taskExecutionContext.getTaskAppId()).thenReturn("1"); + PowerMockito.when(taskExecutionContext.getTenantCode()).thenReturn("root"); + PowerMockito.when(taskExecutionContext.getStartTime()).thenReturn(new Date()); + PowerMockito.when(taskExecutionContext.getTaskTimeout()).thenReturn(10000); + PowerMockito.when(taskExecutionContext.getLogPath()).thenReturn("/tmp/dx"); + + SQLTaskExecutionContext sqlTaskExecutionContext = new SQLTaskExecutionContext(); + sqlTaskExecutionContext.setConnectionParams(CONNECTION_PARAMS); + PowerMockito.when(taskExecutionContext.getSqlTaskExecutionContext()).thenReturn(sqlTaskExecutionContext); + + alertClientService = PowerMockito.mock(AlertClientService.class); + sqlTask = new SqlTask(taskExecutionContext, logger, alertClientService); + sqlTask.init(); + } + + @Test + public void testGetParameters() { + Assert.assertNotNull(sqlTask.getParameters()); + } + + @Test(expected = Exception.class) + public void testHandle() throws Exception { + Connection connection = PowerMockito.mock(Connection.class); + PowerMockito.mockStatic(DriverManager.class); + PowerMockito.when(DriverManager.getConnection(Mockito.any(), Mockito.any(), Mockito.any())).thenReturn(connection); + PreparedStatement preparedStatement = PowerMockito.mock(PreparedStatement.class); + PowerMockito.when(connection.prepareStatement(Mockito.any())).thenReturn(preparedStatement); + PowerMockito.mockStatic(ParameterUtils.class); + PowerMockito.when(ParameterUtils.replaceScheduleTime(Mockito.any(), Mockito.any())).thenReturn("insert into tb_1 values('1','2')"); + + sqlTask.handle(); + Assert.assertEquals(Constants.EXIT_CODE_SUCCESS,sqlTask.getExitStatusCode()); + } +} diff --git a/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/alert/AlertClientService.java b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/alert/AlertClientService.java new file mode 100644 index 0000000000..49977fa969 --- /dev/null +++ b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/alert/AlertClientService.java @@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.service.alert; + +import org.apache.dolphinscheduler.remote.NettyRemotingClient; +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendRequestCommand; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseCommand; +import org.apache.dolphinscheduler.remote.config.NettyClientConfig; +import org.apache.dolphinscheduler.remote.utils.Host; +import org.apache.dolphinscheduler.remote.utils.JsonSerializer; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class AlertClientService { + + private static final Logger logger = LoggerFactory.getLogger(AlertClientService.class); + + private final NettyClientConfig clientConfig; + + private final NettyRemotingClient client; + + private volatile boolean isRunning; + + private String host; + + private int port; + + /** + * request time out + */ + private static final long ALERT_REQUEST_TIMEOUT = 10 * 1000L; + + /** + * alert client + */ + public AlertClientService() { + this.clientConfig = new NettyClientConfig(); + this.client = new NettyRemotingClient(clientConfig); + this.isRunning = true; + } + + /** + * alert client + */ + public AlertClientService(String host, int port) { + this.clientConfig = new NettyClientConfig(); + this.client = new NettyRemotingClient(clientConfig); + this.isRunning = true; + this.host = host; + this.port = port; + } + + /** + * close + */ + public void close() { + this.client.close(); + this.isRunning = false; + logger.info("alter client closed"); + } + + /** + * alert sync send data + * @param groupId + * @param title + * @param content + * @return + */ + public AlertSendResponseCommand sendAlert(int groupId, String title, String content) { + return this.sendAlert(this.host,this.port,groupId,title,content); + } + + /** + * alert sync send data + * @param host host + * @param port port + * @param groupId groupId + * @param title title + * @param content content + * @return AlertSendResponseCommand + */ + public AlertSendResponseCommand sendAlert(String host, int port, int groupId, String title, String content) { + logger.info("sync alert send, host : {}, port : {}, groupId : {}, title : {} ", host, port, groupId, title); + AlertSendRequestCommand request = new AlertSendRequestCommand(groupId, title, content); + final Host address = new Host(host, port); + try { + Command command = request.convert2Command(); + Command response = this.client.sendSync(address, command, ALERT_REQUEST_TIMEOUT); + if (response != null) { + return JsonSerializer.deserialize(response.getBody(), AlertSendResponseCommand.class); + } + } catch (Exception e) { + logger.error("sync alert send error", e); + } finally { + this.client.closeChannel(address); + } + return null; + } + + public boolean isRunning() { + return isRunning; + } +} diff --git a/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java index 9501e7f371..cfe649d74f 100644 --- a/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java +++ b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java @@ -1023,7 +1023,7 @@ public class ProcessService { * create sub work process command * * @param parentProcessInstance parentProcessInstance - * @param task task + * @param task task */ public void createSubWorkProcess(ProcessInstance parentProcessInstance, TaskInstance task) { if (!task.isSubProcess()) { @@ -1142,8 +1142,7 @@ public class ProcessService { ProcessDefinition fatherDefinition = this.findProcessDefineById(parentProcessInstance.getProcessDefinitionId()); ProcessDefinition childDefinition = this.findProcessDefineById(childDefinitionId); if (childDefinition != null && fatherDefinition != null) { - childDefinition.setReceivers(fatherDefinition.getReceivers()); - childDefinition.setReceiversCc(fatherDefinition.getReceiversCc()); + childDefinition.setWarningGroupId(fatherDefinition.getWarningGroupId()); processDefineMapper.updateById(childDefinition); } } diff --git a/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/alert/AlertClientServiceTest.java b/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/alert/AlertClientServiceTest.java new file mode 100644 index 0000000000..ba44191b42 --- /dev/null +++ b/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/alert/AlertClientServiceTest.java @@ -0,0 +1,152 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.service.alert; + +import org.apache.dolphinscheduler.remote.NettyRemotingClient; +import org.apache.dolphinscheduler.remote.command.Command; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendRequestCommand; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseCommand; +import org.apache.dolphinscheduler.remote.command.alert.AlertSendResponseResult; + +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mockito; +import org.powermock.api.mockito.PowerMockito; +import org.powermock.core.classloader.annotations.PrepareForTest; +import org.powermock.modules.junit4.PowerMockRunner; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * alert client service test + */ +@RunWith(PowerMockRunner.class) +@PrepareForTest({AlertClientService.class}) +public class AlertClientServiceTest { + + private static final Logger logger = LoggerFactory.getLogger(AlertClientServiceTest.class); + + private NettyRemotingClient client; + + private AlertClientService alertClient; + + @Before + public void before() throws Exception { + client = PowerMockito.mock(NettyRemotingClient.class); + PowerMockito.whenNew(NettyRemotingClient.class).withAnyArguments().thenReturn(client); + alertClient = new AlertClientService(); + } + + @Test + public void testSendAlert() throws Exception { + String host = "127.0.0.1"; + int port = 50501; + int groupId = 1; + String title = "test-title"; + String content = "test-content"; + + //1.alter server does not exist + AlertSendResponseCommand alertSendResponseCommand = alertClient.sendAlert(host, port, groupId, title, content); + Assert.assertNull(alertSendResponseCommand); + + AlertSendRequestCommand alertSendRequestCommand = new AlertSendRequestCommand(groupId,title,content); + Command reqCommand = alertSendRequestCommand.convert2Command(); + boolean sendResponseStatus; + List sendResponseResults = new ArrayList<>(); + + //2.alter instance does not exist + sendResponseStatus = false; + AlertSendResponseResult alertResult = new AlertSendResponseResult(); + String message = String.format("Alert GroupId %s send error : not found alert instance",groupId); + alertResult.setStatus(false); + alertResult.setMessage(message); + sendResponseResults.add(alertResult); + AlertSendResponseCommand alertSendResponseCommandData = new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + Command resCommand = alertSendResponseCommandData.convert2Command(reqCommand.getOpaque()); + + PowerMockito.when(client.sendSync(Mockito.any(), Mockito.any(), Mockito.anyLong())).thenReturn(resCommand); + alertSendResponseCommand = alertClient.sendAlert(host, port, groupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //3.alter plugin does not exist + sendResponseStatus = false; + String pluginInstanceName = "alert-mail"; + message = String.format("Alert Plugin %s send error : return value is null",pluginInstanceName); + alertResult.setStatus(false); + alertResult.setMessage(message); + alertSendResponseCommandData = new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + resCommand = alertSendResponseCommandData.convert2Command(reqCommand.getOpaque()); + PowerMockito.when(client.sendSync(Mockito.any(), Mockito.any(), Mockito.anyLong())).thenReturn(resCommand); + alertSendResponseCommand = alertClient.sendAlert(host, port, groupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //4.alter result is null + sendResponseStatus = false; + message = String.format("Alert Plugin %s send error : return result value is null",pluginInstanceName); + alertResult.setStatus(false); + alertResult.setMessage(message); + alertSendResponseCommandData = new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + resCommand = alertSendResponseCommandData.convert2Command(reqCommand.getOpaque()); + PowerMockito.when(client.sendSync(Mockito.any(), Mockito.any(), Mockito.anyLong())).thenReturn(resCommand); + alertSendResponseCommand = alertClient.sendAlert(host, port, groupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //5.abnormal information inside the alert plug-in code + sendResponseStatus = false; + alertResult.setStatus(false); + alertResult.setMessage("Abnormal information inside the alert plug-in code"); + alertSendResponseCommandData = new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + resCommand = alertSendResponseCommandData.convert2Command(reqCommand.getOpaque()); + PowerMockito.when(client.sendSync(Mockito.any(), Mockito.any(), Mockito.anyLong())).thenReturn(resCommand); + alertSendResponseCommand = alertClient.sendAlert(host, port, groupId, title, content); + Assert.assertFalse(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + //6.alert plugin send success + sendResponseStatus = true; + message = String.format("Alert Plugin %s send success",pluginInstanceName); + alertResult.setStatus(true); + alertResult.setMessage(message); + alertSendResponseCommandData = new AlertSendResponseCommand(sendResponseStatus, sendResponseResults); + resCommand = alertSendResponseCommandData.convert2Command(reqCommand.getOpaque()); + PowerMockito.when(client.sendSync(Mockito.any(), Mockito.any(), Mockito.anyLong())).thenReturn(resCommand); + alertSendResponseCommand = alertClient.sendAlert(host, port, groupId, title, content); + Assert.assertTrue(alertSendResponseCommand.getResStatus()); + alertSendResponseCommand.getResResults().forEach(result -> + logger.info("alert send response result, status:{}, message:{}",result.getStatus(),result.getMessage())); + + if (Objects.nonNull(alertClient) && alertClient.isRunning()) { + alertClient.close(); + } + + } + +} diff --git a/dolphinscheduler-plugin-api/pom.xml b/dolphinscheduler-spi/pom.xml similarity index 64% rename from dolphinscheduler-plugin-api/pom.xml rename to dolphinscheduler-spi/pom.xml index 0c2547d137..3f18c48b0e 100644 --- a/dolphinscheduler-plugin-api/pom.xml +++ b/dolphinscheduler-spi/pom.xml @@ -1,4 +1,4 @@ - + - - + 4.0.0 org.apache.dolphinscheduler dolphinscheduler 1.3.4-SNAPSHOT - dolphinscheduler-plugin-api + dolphinscheduler-spi ${project.artifactId} - jar - UTF-8 + + + + com.fasterxml.jackson.core + jackson-annotations + + + + com.fasterxml.jackson.core + jackson-databind + + + + com.fasterxml.jackson.core + jackson-core + + org.slf4j slf4j-api + provided + junit junit test - - commons-io - commons-io - - + \ No newline at end of file diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/DolphinSchedulerPlugin.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/DolphinSchedulerPlugin.java new file mode 100644 index 0000000000..157c1af6bc --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/DolphinSchedulerPlugin.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi; + +import static java.util.Collections.emptyList; + +import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory; + +/** + * Dolphinscheduler plugin interface + * All plugin need implements this interface. + * Each plugin needs a factory. This factory has at least two methods. + * one called AlertChannelFactory#getId(), used to return the name of the plugin implementation, + * so that the 'PluginLoad' module can find the plugin implementation class by the name in the configuration file. + * The other method is called create(Map config). This method contains at least one parameter Map config. + * Config contains custom parameters read from the plug-in configuration file. + */ +public interface DolphinSchedulerPlugin { + + default Iterable getAlertChannelFactorys() { + return emptyList(); + } +} diff --git a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/spi/AlertPluginProvider.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannel.java similarity index 74% rename from dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/spi/AlertPluginProvider.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannel.java index 594636f4eb..a429a4ca02 100644 --- a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/spi/AlertPluginProvider.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannel.java @@ -14,20 +14,15 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.plugin.spi; -import org.apache.dolphinscheduler.plugin.api.AlertPlugin; +package org.apache.dolphinscheduler.spi.alert; /** - * PluginProvider + * alert channel interface . + * + * @author gaojun */ -public interface AlertPluginProvider { - - /** - * create an alert plugin - * - * @return an alert plugin - */ - AlertPlugin createPlugin(); +public interface AlertChannel { + AlertResult process(AlertInfo info); } diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannelFactory.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannelFactory.java new file mode 100644 index 0000000000..d23aa379a1 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertChannelFactory.java @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.alert; + +import org.apache.dolphinscheduler.spi.params.base.PluginParams; + +import java.util.List; + +/** + * Each AlertPlugin need implement this interface + */ +public interface AlertChannelFactory { + + /** + * plugin name + * Must be UNIQUE . + * This alert plugin name eg: email , message ... + * Name can often be displayed on the page ui eg : email , message , MR , spark , hive ... + * + * @return this alert plugin name + */ + String getName(); + + /** + * Returns the configurable parameters that this plugin needs to display on the web ui + * + * @return this alert plugin params + */ + List getParams(); + + /** + * The parameters configured in the alert / xxx.properties file will be in the config map + * + * @return AlertChannel + */ + AlertChannel create(); +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertConstants.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertConstants.java new file mode 100644 index 0000000000..b1eee2a0ba --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertConstants.java @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.alert; + +public class AlertConstants { + + /** the field name of alert show type **/ + public static final String SHOW_TYPE = "show_type"; +} diff --git a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertData.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertData.java similarity index 61% rename from dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertData.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertData.java index 89ab5c4279..5e0abf299c 100644 --- a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/AlertData.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertData.java @@ -14,7 +14,8 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.plugin.model; + +package org.apache.dolphinscheduler.spi.alert; /** * AlertData @@ -37,22 +38,6 @@ public class AlertData { * log */ private String log; - /** - * alertgroup_id - */ - private int alertGroupId; - /** - * receivers - */ - private String receivers; - /** - * show_type - */ - private String showType; - /** - * receivers_cc - */ - private String receiversCc; public int getId() { return id; @@ -89,41 +74,4 @@ public class AlertData { this.log = log; return this; } - - public int getAlertGroupId() { - return alertGroupId; - } - - public AlertData setAlertGroupId(int alertGroupId) { - this.alertGroupId = alertGroupId; - return this; - } - - public String getReceivers() { - return receivers; - } - - public AlertData setReceivers(String receivers) { - this.receivers = receivers; - return this; - } - - public String getReceiversCc() { - return receiversCc; - } - - public AlertData setReceiversCc(String receiversCc) { - this.receiversCc = receiversCc; - return this; - } - - public String getShowType() { - return showType; - } - - public AlertData setShowType(String showType) { - this.showType = showType; - return this; - } - } diff --git a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/api/AlertPlugin.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertInfo.java similarity index 60% rename from dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/api/AlertPlugin.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertInfo.java index deb7ff6aa4..c91428ce12 100644 --- a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/api/AlertPlugin.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertInfo.java @@ -14,32 +14,37 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.plugin.api; -import org.apache.dolphinscheduler.plugin.model.AlertInfo; -import org.apache.dolphinscheduler.plugin.model.PluginName; - -import java.util.Map; +package org.apache.dolphinscheduler.spi.alert; /** - * Plugin + * AlertInfo */ -public interface AlertPlugin { +public class AlertInfo { /** - * Get alert plugin id - * - * @return alert plugin id, which should be unique + * all params this plugin need is in alertProps */ - String getId(); + private String alertParams; /** - * Get alert plugin name, which will show in front end portal - * - * @return plugin name + * the alert content */ - PluginName getName(); + private AlertData alertData; + + public String getAlertParams() { + return alertParams; + } + + public void setAlertParams(String alertParams) { + this.alertParams = alertParams; + } - Map process(AlertInfo info); + public AlertData getAlertData() { + return alertData; + } + public void setAlertData(AlertData alertData) { + this.alertData = alertData; + } } diff --git a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/PluginName.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertResult.java similarity index 63% rename from dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/PluginName.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertResult.java index 8066e45f1d..a327d09403 100644 --- a/dolphinscheduler-plugin-api/src/main/java/org/apache/dolphinscheduler/plugin/model/PluginName.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/AlertResult.java @@ -14,32 +14,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.plugin.model; -/** - * PluginName - */ -public class PluginName { +package org.apache.dolphinscheduler.spi.alert; + +public class AlertResult { - private String chinese; + private String status; - private String english; + private String message; - public String getChinese() { - return chinese; + public String getStatus() { + return status; } - public PluginName setChinese(String chinese) { - this.chinese = chinese; - return this; + public void setStatus(String status) { + this.status = status; } - public String getEnglish() { - return english; + public String getMessage() { + return message; } - public PluginName setEnglish(String english) { - this.english = english; - return this; + public void setMessage(String message) { + this.message = message; } } diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/ShowType.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/ShowType.java similarity index 89% rename from dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/ShowType.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/ShowType.java index 19e552d765..a95e73ff7f 100644 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/ShowType.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/alert/ShowType.java @@ -14,12 +14,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.common.enums; -import com.baomidou.mybatisplus.annotation.EnumValue; +package org.apache.dolphinscheduler.spi.alert; /** * show type for email + * all alert plugin can use ShowType , so let it in spi package */ public enum ShowType { /** @@ -33,13 +33,11 @@ public enum ShowType { ATTACHMENT(2, "attachment"), TABLEATTACHMENT(3, "table attachment"); - - ShowType(int code, String descp){ + ShowType(int code, String descp) { this.code = code; this.descp = descp; } - @EnumValue private final int code; private final String descp; diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UserAlertGroupService.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/classloader/ThreadContextClassLoader.java similarity index 58% rename from dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UserAlertGroupService.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/classloader/ThreadContextClassLoader.java index 502185709f..b905ef72b7 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/UserAlertGroupService.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/classloader/ThreadContextClassLoader.java @@ -14,25 +14,22 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.dolphinscheduler.api.service; -import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; -import org.apache.dolphinscheduler.dao.entity.UserAlertGroup; -import org.apache.dolphinscheduler.dao.mapper.UserAlertGroupMapper; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.stereotype.Service; +package org.apache.dolphinscheduler.spi.classloader; -/** - * - */ -@Service -public class UserAlertGroupService extends ServiceImpl { +import java.io.Closeable; - @Autowired - private UserAlertGroupMapper userAlertGroupMapper; +public class ThreadContextClassLoader + implements Closeable { + private final ClassLoader threadContextClassLoader; - boolean deleteByAlertGroupId(Integer groupId) { - return userAlertGroupMapper.deleteByAlertgroupId(groupId) >= 1; + public ThreadContextClassLoader(ClassLoader newThreadContextClassLoader) { + this.threadContextClassLoader = Thread.currentThread().getContextClassLoader(); + Thread.currentThread().setContextClassLoader(newThreadContextClassLoader); } + @Override + public void close() { + Thread.currentThread().setContextClassLoader(threadContextClassLoader); + } } diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/InputParam.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/InputParam.java new file mode 100644 index 0000000000..8553dec18c --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/InputParam.java @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params; + +import org.apache.dolphinscheduler.spi.params.base.FormType; +import org.apache.dolphinscheduler.spi.params.base.ParamsProps; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.ArrayList; +import java.util.List; + +/** + * Text param + */ +public class InputParam extends PluginParams { + + private InputParam(Builder builder) { + super(builder); + } + + public static Builder newBuilder(String name, String title) { + return new InputParam.Builder(name, title); + } + + public static class Builder extends PluginParams.Builder { + + public Builder(String name, String title) { + super(name, FormType.INPUT, title); + } + + public Builder setPlaceholder(String placeholder) { + if (this.props == null) { + this.setProps(new ParamsProps()); + } + + this.props.setPlaceholder(placeholder); + return this; + } + + public Builder addValidate(Validate validate) { + if (this.validateList == null) { + this.validateList = new ArrayList<>(); + } + this.validateList.add(validate); + return this; + } + + public Builder setName(String name) { + this.name = name; + return this; + } + + public Builder setProps(ParamsProps props) { + this.props = props; + return this; + } + + public Builder setTitle(String title) { + this.title = title; + return this; + } + + public Builder setValue(Object value) { + this.value = value; + return this; + } + + public Builder setValidateList(List validateList) { + this.validateList = validateList; + return this; + } + + @Override + public InputParam build() { + return new InputParam(this); + } + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PasswordParam.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PasswordParam.java new file mode 100644 index 0000000000..a8420492bd --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PasswordParam.java @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params; + +import org.apache.dolphinscheduler.spi.params.base.FormType; +import org.apache.dolphinscheduler.spi.params.base.ParamsProps; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.ArrayList; +import java.util.List; + +/** + * password param + */ +public class PasswordParam extends PluginParams { + + private PasswordParam(Builder builder) { + super(builder); + } + + public static Builder newBuilder(String name, String title) { + return new PasswordParam.Builder(name, title); + } + + public static class Builder extends PluginParams.Builder { + + public Builder(String name, String title) { + super(name, FormType.INPUT, title); + ParamsProps paramsProps = new ParamsProps(); + this.props = paramsProps; + } + + public Builder setPlaceholder(String placeholder) { + this.props.setPlaceholder(placeholder); + return this; + } + + public Builder addValidate(Validate validate) { + if (this.validateList == null) { + this.validateList = new ArrayList<>(); + } + this.validateList.add(validate); + return this; + } + + public Builder setName(String name) { + this.name = name; + return this; + } + + public Builder setTitle(String title) { + this.title = title; + return this; + } + + public Builder setValue(Object value) { + this.value = value; + return this; + } + + public Builder setValidateList(List validateList) { + this.validateList = validateList; + return this; + } + + @Override + public PasswordParam build() { + return new PasswordParam(this); + } + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransfer.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransfer.java new file mode 100644 index 0000000000..c9b14b9f8e --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransfer.java @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params; + +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.utils.JSONUtils; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** + * plugin params pojo and json transfer tool + */ +public class PluginParamsTransfer { + + public static String transferParamsToJson(List list) { + return JSONUtils.toJsonString(list); + } + + public static List transferJsonToParamsList(String str) { + return JSONUtils.toList(str, PluginParams.class); + } + + /** + * return the plugin params map + */ + public static Map getPluginParamsMap(String paramsJsonStr) { + List pluginParams = transferJsonToParamsList(paramsJsonStr); + Map paramsMap = new HashMap<>(); + for (PluginParams param : pluginParams) { + paramsMap.put(param.getName(), null != param.getValue() ? param.getValue().toString() : null); + } + return paramsMap; + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/RadioParam.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/RadioParam.java new file mode 100644 index 0000000000..824c7fe6f3 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/RadioParam.java @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params; + +import org.apache.dolphinscheduler.spi.params.base.FormType; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.ArrayList; +import java.util.List; + +import com.fasterxml.jackson.annotation.JsonProperty; + +/** + * radio + */ +public class RadioParam extends PluginParams { + + @JsonProperty("options") + private List paramsOptionsList; + + private RadioParam(Builder builder) { + super(builder); + this.paramsOptionsList = builder.paramsOptionsList; + } + + public static Builder newBuilder(String name, String title) { + return new RadioParam.Builder(name, title); + } + + public static class Builder extends PluginParams.Builder { + + private List paramsOptionsList; + + public Builder(String name, String title) { + super(name, FormType.RADIO, title); + } + + public Builder addValidate(Validate validate) { + if (this.validateList == null) { + this.validateList = new ArrayList<>(); + } + this.validateList.add(validate); + return this; + } + + public Builder setName(String name) { + this.name = name; + return this; + } + + public Builder setTitle(String title) { + this.title = title; + return this; + } + + public Builder setValue(Object value) { + this.value = value; + return this; + } + + public Builder setValidateList(List validateList) { + this.validateList = validateList; + return this; + } + + public Builder setParamsOptionsList(List paramsOptionsList) { + this.paramsOptionsList = paramsOptionsList; + return this; + } + + public Builder addParamsOptions(ParamsOptions paramsOptions) { + if (this.paramsOptionsList == null) { + this.paramsOptionsList = new ArrayList<>(); + } + + this.paramsOptionsList.add(paramsOptions); + return this; + } + + @Override + public RadioParam build() { + return new RadioParam(this); + } + } + + public List getParamsOptionsList() { + return paramsOptionsList; + } +} diff --git a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactory.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/DataType.java similarity index 64% rename from dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactory.java rename to dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/DataType.java index d38463123e..2b94ac1a73 100644 --- a/dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/template/AlertTemplateFactory.java +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/DataType.java @@ -15,24 +15,25 @@ * limitations under the License. */ -package org.apache.dolphinscheduler.alert.template; - -import org.apache.dolphinscheduler.alert.template.impl.DefaultHTMLTemplate; +package org.apache.dolphinscheduler.spi.params.base; /** - * the alert template factory + * param datetype */ -public class AlertTemplateFactory { +public enum DataType { + + STRING("string"), + + NUMBER("number"); - private AlertTemplateFactory() { + private String dataType; + + DataType(String dataType) { + this.dataType = dataType; } - /** - * get a template from alert.properties conf file - * - * @return a template, default is DefaultHTMLTemplate - */ - public static AlertTemplate getMessageTemplate() { - return new DefaultHTMLTemplate(); + public String getDataType() { + return this.dataType; } + } diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/FormType.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/FormType.java new file mode 100644 index 0000000000..5d6234e753 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/FormType.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +import com.fasterxml.jackson.annotation.JsonValue; + +public enum FormType { + + INPUT("input"), + + RADIO("radio"); + + private String formType; + + FormType(String formType) { + this.formType = formType; + } + + @JsonValue + public String getFormType() { + return this.formType; + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsOptions.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsOptions.java new file mode 100644 index 0000000000..6b3fc25eb1 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsOptions.java @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +import com.fasterxml.jackson.annotation.JsonProperty; + +/** + * The options field in form-create`s json rule + * Set radio, select, checkbox and other component option options + */ +public class ParamsOptions { + + private String label; + + private Object value; + + /** + * is can be select + */ + private boolean disabled; + + public ParamsOptions(String label, Object value, boolean disabled) { + this.label = label; + this.value = value; + this.disabled = disabled; + } + + @JsonProperty("label") + public String getLabel() { + return label; + } + + public ParamsOptions setLabel(String label) { + this.label = label; + return this; + } + + @JsonProperty("value") + public Object getValue() { + return value; + } + + public ParamsOptions setValue(Object value) { + this.value = value; + return this; + } + + @JsonProperty("disabled") + public boolean isDisabled() { + return disabled; + } + + public ParamsOptions setDisabled(boolean disabled) { + this.disabled = disabled; + return this; + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsProps.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsProps.java new file mode 100644 index 0000000000..cab65ba645 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/ParamsProps.java @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +import com.fasterxml.jackson.annotation.JsonProperty; + +/** + * the props field in form-create`s json rule + */ +public class ParamsProps { + + private String placeholder; + + private String size = "small"; + + public void setSize(String size) { + this.size = size; + } + + @JsonProperty("size") + public String getSize() { + return size; + } + + @JsonProperty("placeholder") + public String getPlaceholder() { + return placeholder; + } + + public ParamsProps setPlaceholder(String placeholder) { + this.placeholder = placeholder; + return this; + } + +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PluginParams.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PluginParams.java new file mode 100644 index 0000000000..1fdf0e0792 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PluginParams.java @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +import static java.util.Objects.requireNonNull; + +import java.util.List; + +import com.fasterxml.jackson.annotation.JsonCreator; +import com.fasterxml.jackson.annotation.JsonProperty; +import com.fasterxml.jackson.databind.annotation.JsonDeserialize; +import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder; + +/** + * plugin params + */ +@JsonDeserialize(builder = PluginParams.Builder.class) +public class PluginParams { + + /** + * param name + */ + @JsonProperty("field") + protected String name; + + /** + * param name + */ + @JsonProperty("name") + protected String fieldName; + + @JsonProperty("props") + protected ParamsProps props; + + @JsonProperty("type") + protected String formType; + + /** + * Name displayed on the page + */ + @JsonProperty("title") + protected String title; + + /** + * default value or value input by user in the page + */ + @JsonProperty("value") + protected Object value; + + @JsonProperty("validate") + protected List validateList; + + protected PluginParams(Builder builder) { + + requireNonNull(builder, "builder is null"); + requireNonNull(builder.name, "name is null"); + requireNonNull(builder.formType, "formType is null"); + requireNonNull(builder.title, "title is null"); + + this.name = builder.name; + this.formType = builder.formType.getFormType(); + this.title = builder.title; + if (null == builder.props) { + builder.props = new ParamsProps(); + } + this.fieldName = builder.title; + this.props = builder.props; + this.value = builder.value; + this.validateList = builder.validateList; + + } + + @JsonPOJOBuilder(buildMethodName = "build", withPrefix = "set") + public static class Builder { + //Must have + protected String name; + + protected FormType formType; + + protected String title; + + protected String fieldName; + + //option params + protected ParamsProps props; + + protected Object value; + + protected List validateList; + + public Builder(String name, + FormType formType, + String title) { + requireNonNull(name, "name is null"); + requireNonNull(formType, "formType is null"); + requireNonNull(title, "title is null"); + this.name = name; + this.formType = formType; + this.title = title; + this.fieldName = title; + } + + //for json deserialize to POJO + @JsonCreator + public Builder(@JsonProperty("field") String name, + @JsonProperty("type") FormType formType, + @JsonProperty("title") String title, + @JsonProperty("props") ParamsProps props, + @JsonProperty("value") Object value, + @JsonProperty("name") String fieldName, + @JsonProperty("validate") List validateList + ) { + requireNonNull(name, "name is null"); + requireNonNull(formType, "formType is null"); + requireNonNull(title, "title is null"); + this.name = name; + this.formType = formType; + this.title = title; + this.props = props; + this.value = value; + this.validateList = validateList; + this.fieldName = fieldName; + } + + public PluginParams build() { + return new PluginParams(this); + } + } + + public String getName() { + return name; + } + + public ParamsProps getProps() { + return props; + } + + public String getFormType() { + return formType; + } + + public String getTitle() { + return title; + } + + public Object getValue() { + return value; + } + + public List getValidateList() { + return validateList; + } +} + + diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PropsType.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PropsType.java new file mode 100644 index 0000000000..00e8770490 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/PropsType.java @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +public enum PropsType { + + INPUT("input"), + + PASSWORD("password"), + + TEXTAREA("textarea"); + + private String propsType; + + PropsType(String propsType) { + this.propsType = propsType; + } + + public String getPropsType() { + return this.propsType; + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/TriggerType.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/TriggerType.java new file mode 100644 index 0000000000..ad964263fc --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/TriggerType.java @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +public enum TriggerType { + + BLUR("blur"), + + CHANGE("change"); + + private String triggerType; + + TriggerType(String triggerType) { + this.triggerType = triggerType; + } + + public String getTriggerType() { + return this.triggerType; + } + +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/Validate.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/Validate.java new file mode 100644 index 0000000000..289f5b3f2e --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params/base/Validate.java @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params.base; + +import com.fasterxml.jackson.annotation.JsonProperty; +import com.fasterxml.jackson.databind.annotation.JsonDeserialize; +import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder; + +/** + * form validate + */ +@JsonDeserialize(builder = Validate.Builder.class) +public class Validate { + + @JsonProperty("required") + private boolean required; + + @JsonProperty("message") + private String message; + + @JsonProperty("type") + private String type; + + @JsonProperty("trigger") + private String trigger; + + @JsonProperty("min") + private Double min; + + @JsonProperty("max") + private Double max; + + private Validate() { + + } + + private Validate(Builder builder) { + this.required = builder.required; + this.message = builder.message; + this.type = builder.type; + this.trigger = builder.trigger; + this.min = builder.min; + this.max = builder.max; + } + + public static Builder newBuilder() { + return new Validate.Builder(); + } + + @JsonPOJOBuilder(buildMethodName = "build", withPrefix = "set") + public static class Builder { + private boolean required = false; + + private String message; + + private String type = DataType.STRING.getDataType(); + + private String trigger = TriggerType.BLUR.getTriggerType(); + + private Double min; + + private Double max; + + public Builder setRequired(boolean required) { + this.required = required; + return this; + } + + public Builder setMessage(String message) { + this.message = message; + return this; + } + + public Builder setTrigger(String trigger) { + this.trigger = trigger; + return this; + } + + public Builder setMin(Double min) { + this.min = min; + return this; + } + + public Builder setMax(Double max) { + this.max = max; + return this; + } + + public Builder setType(String type) { + this.type = type; + return this; + } + + public Validate build() { + return new Validate(this); + } + } + + public boolean isRequired() { + return required; + } + + public String getMessage() { + return message; + } + + public String getType() { + return type; + } + + public String getTrigger() { + return trigger; + } + + public Double getMin() { + return min; + } + + public Double getMax() { + return max; + } +} diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/JSONUtils.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/JSONUtils.java new file mode 100644 index 0000000000..6542ef86a4 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/JSONUtils.java @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.utils; + +import static com.fasterxml.jackson.databind.DeserializationFeature.ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT; +import static com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES; +import static com.fasterxml.jackson.databind.DeserializationFeature.READ_UNKNOWN_ENUM_VALUES_AS_NULL; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.TimeZone; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.ObjectWriter; +import com.fasterxml.jackson.databind.SerializationFeature; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.ObjectNode; +import com.fasterxml.jackson.databind.type.CollectionType; + +/** + * json utils + */ +public class JSONUtils { + + private static final Logger logger = LoggerFactory.getLogger(JSONUtils.class); + + /** + * can use static singleton, inject: just make sure to reuse! + */ + private static final ObjectMapper objectMapper = new ObjectMapper() + .configure(FAIL_ON_UNKNOWN_PROPERTIES, false) + .configure(ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT, true) + .configure(READ_UNKNOWN_ENUM_VALUES_AS_NULL, true) + .setTimeZone(TimeZone.getDefault()); + + private JSONUtils() { + } + + /** + * json representation of object + * + * @param object object + * @param feature feature + * @return object to json string + */ + public static String toJsonString(Object object, SerializationFeature feature) { + try { + ObjectWriter writer = objectMapper.writer(feature); + return writer.writeValueAsString(object); + } catch (Exception e) { + logger.error("object to json exception!", e); + } + + return null; + } + + /** + * This method deserializes the specified Json into an object of the specified class. It is not + * suitable to use if the specified class is a generic type since it will not have the generic + * type information because of the Type Erasure feature of Java. Therefore, this method should not + * be used if the desired type is a generic type. Note that this method works fine if the any of + * the fields of the specified object are generics, just the object itself should not be a + * generic type. + * + * @param json the string from which the object is to be deserialized + * @param clazz the class of T + * @param T + * @return an object of type T from the string + * classOfT + */ + public static T parseObject(String json, Class clazz) { + if (StringUtils.isEmpty(json)) { + return null; + } + + try { + return objectMapper.readValue(json, clazz); + } catch (Exception e) { + logger.error("parse object exception!", e); + } + return null; + } + + /** + * json to list + * + * @param json json string + * @param clazz class + * @param T + * @return list + */ + public static List toList(String json, Class clazz) { + if (StringUtils.isEmpty(json)) { + return Collections.emptyList(); + } + + try { + + CollectionType listType = objectMapper.getTypeFactory().constructCollectionType(ArrayList.class, clazz); + return objectMapper.readValue(json, listType); + } catch (Exception e) { + logger.error("parse list exception!", e); + } + + return Collections.emptyList(); + } + + /** + * object to json string + * + * @param object object + * @return json string + */ + public static String toJsonString(Object object) { + try { + return objectMapper.writeValueAsString(object); + } catch (Exception e) { + throw new RuntimeException("Object json deserialization exception.", e); + } + } + + public static ObjectNode parseObject(String text) { + try { + return (ObjectNode) objectMapper.readTree(text); + } catch (Exception e) { + throw new RuntimeException("String json deserialization exception.", e); + } + } + + public static ArrayNode parseArray(String text) { + try { + return (ArrayNode) objectMapper.readTree(text); + } catch (Exception e) { + throw new RuntimeException("Json deserialization exception.", e); + } + } +} \ No newline at end of file diff --git a/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/StringUtils.java b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/StringUtils.java new file mode 100644 index 0000000000..fe0306b169 --- /dev/null +++ b/dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/utils/StringUtils.java @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.utils; + +public class StringUtils { + public static final String EMPTY = ""; + + public static boolean isEmpty(final CharSequence cs) { + return cs == null || cs.length() == 0; + } + + public static boolean isNotEmpty(final CharSequence cs) { + return !isEmpty(cs); + } + + public static boolean isBlank(String s) { + if (isEmpty(s)) { + return true; + } + return s.trim().length() == 0; + } + + public static boolean isNotBlank(String s) { + return !isBlank(s); + } +} diff --git a/dolphinscheduler-spi/src/test/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransferTest.java b/dolphinscheduler-spi/src/test/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransferTest.java new file mode 100644 index 0000000000..a4d93d6cd0 --- /dev/null +++ b/dolphinscheduler-spi/src/test/java/org/apache/dolphinscheduler/spi/params/PluginParamsTransferTest.java @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.dolphinscheduler.spi.params; + +import org.apache.dolphinscheduler.spi.params.base.DataType; +import org.apache.dolphinscheduler.spi.params.base.ParamsOptions; +import org.apache.dolphinscheduler.spi.params.base.PluginParams; +import org.apache.dolphinscheduler.spi.params.base.Validate; + +import java.util.ArrayList; +import java.util.List; + +import org.junit.After; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; + +/** + * PluginParamsTransfer Tester. + */ +public class PluginParamsTransferTest { + + @Before + public void before() throws Exception { + } + + @After + public void after() throws Exception { + } + + /** + * Method: getAlpacajsJson(List pluginParamsList) + */ + @Test + public void testGetParamsJson() { + List paramsList = new ArrayList<>(); + InputParam receivesParam = InputParam.newBuilder("field1", "field1") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam receiveCcsParam = new InputParam.Builder("field2", "field2").build(); + + InputParam mailSmtpHost = new InputParam.Builder("field3", "field3") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam mailSmtpPort = new InputParam.Builder("field4", "field4") + .addValidate(Validate.newBuilder() + .setRequired(true) + .setType(DataType.NUMBER.getDataType()) + .build()) + .build(); + + InputParam mailSender = new InputParam.Builder("field5", "field5") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + RadioParam enableSmtpAuth = new RadioParam.Builder("field6", "field6") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(true) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam mailUser = new InputParam.Builder("field7", "field7") + .setPlaceholder("if enable use authentication, you need input user") + .build(); + + PasswordParam mailPassword = new PasswordParam.Builder("field8", "field8") + .setPlaceholder("if enable use authentication, you need input password") + .build(); + + RadioParam enableTls = new RadioParam.Builder("field9", "field9") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(false) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + RadioParam enableSsl = new RadioParam.Builder("field10", "field10") + .addParamsOptions(new ParamsOptions("YES", true, false)) + .addParamsOptions(new ParamsOptions("NO", false, false)) + .setValue(false) + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + InputParam sslTrust = new InputParam.Builder("field11", "field11") + .setValue("*") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + List emailShowTypeList = new ArrayList<>(); + emailShowTypeList.add(new ParamsOptions("table", "table", false)); + emailShowTypeList.add(new ParamsOptions("text", "text", false)); + emailShowTypeList.add(new ParamsOptions("attachment", "attachment", false)); + emailShowTypeList.add(new ParamsOptions("tableattachment", "tableattachment", false)); + RadioParam showType = new RadioParam.Builder("showType", "showType") + .setParamsOptionsList(emailShowTypeList) + .setValue("table") + .addValidate(Validate.newBuilder().setRequired(true).build()) + .build(); + + paramsList.add(receivesParam); + paramsList.add(receiveCcsParam); + paramsList.add(mailSmtpHost); + paramsList.add(mailSmtpPort); + paramsList.add(mailSender); + paramsList.add(enableSmtpAuth); + paramsList.add(mailUser); + paramsList.add(mailPassword); + paramsList.add(enableTls); + paramsList.add(enableSsl); + paramsList.add(sslTrust); + paramsList.add(showType); + + String paramsJson = PluginParamsTransfer.transferParamsToJson(paramsList); + + String paramsJsonAssert = "[{\"field\":\"field1\",\"name\":\"field1\",\"props\":{\"placeholder\":null,\"size\":\"small\"},\"type\":\"input\"," + + "\"title\":\"field1\",\"value\":null,\"validate\":[{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"m" + + "in\":null,\"max\":null}]},{\"field\":\"field2\",\"name\":\"field2\",\"props\":{\"placeholder\":null,\"size\":\"small\"},\"type\":\"inp" + + "ut\",\"title\":\"field2\",\"value\":null,\"validate\":null},{\"field\":\"field3\",\"name\":\"field3\",\"props\":{\"placeholder\":nu" + + "ll,\"size\":\"small\"},\"type\":\"input\",\"title\":\"field3\",\"value\":null,\"validate\":[{\"required\":true,\"message\":null,\"typ" + + "e\":\"string\",\"trigger\":\"blur\",\"min\":null,\"max\":null}]},{\"field\":\"field4\",\"name\":\"field4\",\"props\":{\"placeholder\":nul" + + "l,\"size\":\"small\"},\"type\":\"input\",\"title\":\"field4\",\"value\":null,\"validate\":[{\"required\":true,\"message\":null,\"type\":\"nu" + + "mber\",\"trigger\":\"blur\",\"min\":null,\"max\":null}]},{\"field\":\"field5\",\"name\":\"field5\",\"props\":{\"placeholder\":null,\"size\":\"sma" + + "ll\"},\"type\":\"input\",\"title\":\"field5\",\"value\":null,\"validate\":[{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"bl" + + "ur\",\"min\":null,\"max\":null}]},{\"field\":\"field6\",\"name\":\"field6\",\"props\":{\"placeholder\":null,\"size\":\"small\"},\"type\":\"radio\",\"ti" + + "tle\":\"field6\",\"value\":true,\"validate\":[{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":null,\"max\":null}],\"o" + + "ptions\":[{\"label\":\"YES\",\"value\":true,\"disabled\":false},{\"label\":\"NO\",\"value\":false,\"disabled\":false}]},{\"field\":\"field7\",\"name\":\"fi" + + "eld7\",\"props\":{\"placeholder\":\"if enable use authentication, you need input user\",\"size\":\"small\"},\"type\":\"input\",\"title\":\"field7\",\"v" + + "alue\":null,\"validate\":null},{\"field\":\"field8\",\"name\":\"field8\",\"props\":{\"placeholder\":\"if enable use authentication, you need input p" + + "assword\",\"size\":\"small\"},\"type\":\"input\",\"title\":\"field8\",\"value\":null,\"validate\":null},{\"field\":\"field9\",\"name\":\"field9\",\"pr" + + "ops\":{\"placeholder\":null,\"size\":\"small\"},\"type\":\"radio\",\"title\":\"field9\",\"value\":false,\"validate\":[{\"required\":true,\"mes" + + "sage\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":null,\"max\":null}],\"options\":[{\"label\":\"YES\",\"value\":true,\"disa" + + "bled\":false},{\"label\":\"NO\",\"value\":false,\"disabled\":false}]},{\"field\":\"field10\",\"name\":\"field10\",\"props\":{\"placeh" + + "older\":null,\"size\":\"small\"},\"type\":\"radio\",\"title\":\"field10\",\"value\":false,\"validate\":[{\"required\":true,\"mes" + + "sage\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":null,\"max\":null}],\"options\":[{\"label\":\"YES\",\"value\":true,\"di" + + "sabled\":false},{\"label\":\"NO\",\"value\":false,\"disabled\":false}]},{\"field\":\"field11\",\"name\":\"field11\",\"props\":{\"pl" + + "aceholder\":null,\"size\":\"small\"},\"type\":\"input\",\"title\":\"field11\",\"value\":\"*\",\"validate\":[{\"required\":true,\"me" + + "ssage\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":null,\"max\":null}]},{\"field\":\"showType\",\"name\":\"showType\",\"p" + + "rops\":{\"placeholder\":null,\"size\":\"small\"},\"type\":\"radio\",\"title\":\"showType\",\"value\":\"table\",\"validate\":[{\"requ" + + "ired\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":null,\"max\":null}],\"options\":[{\"label\":\"ta" + + "ble\",\"value\":\"table\",\"disabled\":false},{\"label\":\"text\",\"value\":\"text\",\"disabled\":false},{\"label\":\"att" + + "achment\",\"value\":\"attachment\",\"disabled\":false},{\"label\":\"tableattachment\",\"value\":\"tableattachment\",\"disabled\":false}]}]"; + Assert.assertEquals(paramsJsonAssert, paramsJson); + } + + @Test + public void testGetPluginParams() { + String paramsJsonAssert = "[{\"field\":\"field1\",\"props\":null,\"type\":\"input\",\"title\":\"field1\",\"value\":\"v1\",\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}]}," + + "{\"field\":\"field2\",\"props\":null,\"type\":\"input\",\"title\":\"field2\",\"value\":\"v2\",\"validate\":null}," + + "{\"field\":\"field3\",\"props\":null,\"type\":\"input\",\"title\":\"field3\",\"value\":\"v3\",\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}]}," + + "{\"field\":\"field4\",\"props\":null,\"type\":\"input\",\"title\":\"field4\",\"value\":\"v4\",\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"number\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}]}," + + "{\"field\":\"field5\",\"props\":null,\"type\":\"input\",\"title\":\"field5\",\"value\":\"v5\",\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}]}," + + "{\"field\":\"field6\",\"props\":null,\"type\":\"radio\",\"title\":\"field6\",\"value\":true,\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}],\"options\":[" + + "{\"label\":\"YES\",\"value\":true,\"disabled\":false},{\"label\":\"NO\",\"value\":false,\"disabled\":false}]}," + + "{\"field\":\"field7\",\"props\":{\"type\":null,\"placeholder\":\"if enable use authentication, you need input user\",\"rows\":0}," + + "\"type\":\"input\",\"title\":\"field7\",\"value\":\"v6\",\"validate\":null},{\"field\":\"field8\",\"props\":{" + + "\"type\":\"PASSWORD\",\"placeholder\":\"if enable use authentication, you need input password\",\"rows\":0}," + + "\"type\":\"input\",\"title\":\"field8\",\"value\":\"v7\",\"validate\":null},{\"field\":\"field9\"," + + "\"props\":null,\"type\":\"radio\",\"title\":\"field9\",\"value\":false,\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}],\"options\":[" + + "{\"label\":\"YES\",\"value\":true,\"disabled\":false},{\"label\":\"NO\",\"value\":false,\"disabled\":false}]}," + + "{\"field\":\"field10\",\"props\":null,\"type\":\"radio\",\"title\":\"field10\",\"value\":false,\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}]," + + "\"options\":[{\"label\":\"YES\",\"value\":true,\"disabled\":false},{\"label\":\"NO\",\"value\":false,\"disabled\":false}]}," + + "{\"field\":\"field11\",\"props\":null,\"type\":\"input\",\"title\":\"field11\",\"value\":\"*\",\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}]}," + + "{\"field\":\"showType\",\"props\":null,\"type\":\"radio\",\"title\":\"showType\",\"value\":\"table\",\"validate\":[" + + "{\"required\":true,\"message\":null,\"type\":\"string\",\"trigger\":\"blur\",\"min\":0.0,\"max\":0.0}],\"options\":[" + + "{\"label\":\"table\",\"value\":\"table\",\"disabled\":false},{\"label\":\"text\",\"value\":\"text\",\"disabled\":false}," + + "{\"label\":\"attachment\",\"value\":\"attachment\",\"disabled\":false},{\"label\":\"tableattachment\",\"value\":\"tableattachment\",\"disabled\":false}]}]"; + List pluginParams = PluginParamsTransfer.transferJsonToParamsList(paramsJsonAssert); + String[] results = new String[]{"v1", "v2", "v3", "v4", "v5", "true", "v6", "v7", "false", "false", "*", "table", "v1"}; + Assert.assertEquals(12, pluginParams.size()); + for (int i = 0; i < pluginParams.size(); i++) { + PluginParams param = pluginParams.get(i); + Assert.assertEquals(param.getValue().toString(), results[i]); + } + } +} diff --git a/install.sh b/install.sh index da21085d14..7991275a01 100755 --- a/install.sh +++ b/install.sh @@ -64,6 +64,8 @@ sed -i ${txt} "s#mail.smtp.starttls.enable.*#mail.smtp.starttls.enable=${starttl sed -i ${txt} "s#mail.smtp.ssl.trust.*#mail.smtp.ssl.trust=${sslTrust}#g" conf/alert.properties sed -i ${txt} "s#mail.smtp.ssl.enable.*#mail.smtp.ssl.enable=${sslEnable}#g" conf/alert.properties +sed -i ${txt} "s#alert.listen.host.*#alert.listen.host=${alertServer}#g" conf/worker.properties + # 2.create directory echo "2.create directory" diff --git a/pom.xml b/pom.xml index cb71c4cc37..5c8dd67613 100644 --- a/pom.xml +++ b/pom.xml @@ -87,7 +87,7 @@ 3.17 3.1.0 4.1 - 20.0 + 24.1-jre 42.1.4 2.1.0 2.4 @@ -118,6 +118,8 @@ 2.9.2 1.5.24 2.0.0 + 0.184 + ${dep.airlift.version} @@ -201,12 +203,12 @@ org.apache.dolphinscheduler - dolphinscheduler-plugin-api + dolphinscheduler-common ${project.version} org.apache.dolphinscheduler - dolphinscheduler-common + dolphinscheduler-alert-plugin ${project.version} @@ -234,6 +236,12 @@ dolphinscheduler-alert ${project.version} + + org.apache.dolphinscheduler + dolphinscheduler-spi + ${project.version} + + org.apache.curator curator-framework @@ -550,6 +558,40 @@ guava-retrying ${guava-retry.version} + + + org.sonatype.aether + aether-api + 1.13.1 + provided + + + + io.airlift.resolver + resolver + 1.5 + provided + + + + org.ow2.asm + asm + 6.2.1 + provided + + + + javax.activation + activation + 1.1 + + + + com.sun.mail + javax.mail + 1.6.2 + + @@ -560,7 +602,19 @@ + + org.apache.dolphinscheduler + dolphinscheduler-maven-plugin + 1.0.0 + true + + + ca.vanzyl.maven.plugins + provisio-maven-plugin + 1.0.4 + true + org.codehaus.mojo @@ -628,6 +682,22 @@ + + org.apache.dolphinscheduler + dolphinscheduler-maven-plugin + true + + + + + + + + + ca.vanzyl.maven.plugins + provisio-maven-plugin + true + org.apache.maven.plugins maven-source-plugin @@ -698,15 +768,6 @@ ${maven-surefire-plugin.version} - **/alert/utils/DingTalkUtilsTest.java - **/alert/template/AlertTemplateFactoryTest.java - **/alert/template/impl/DefaultHTMLTemplateTest.java - **/alert/utils/EnterpriseWeChatUtilsTest.java - **/alert/utils/ExcelUtilsTest.java - **/alert/utils/FuncUtilsTest.java - **/alert/utils/JSONUtilsTest.java - - **/alert/plugin/EmailAlertPluginTest.java **/api/controller/ProcessDefinitionControllerTest.java **/api/controller/TenantControllerTest.java **/api/dto/resources/filter/ResourceFilterTest.java @@ -741,6 +802,7 @@ **/api/service/TaskInstanceServiceTest.java **/api/service/TenantServiceTest.java **/api/service/UdfFuncServiceTest.java + **/api/service/UiPluginServiceTest.java **/api/service/UserAlertGroupServiceTest.java **/api/service/UsersServiceTest.java **/api/service/WorkerGroupServiceTest.java @@ -810,6 +872,8 @@ **/remote/NettyRemotingClientTest.java **/remote/NettyUtilTest.java **/remote/ResponseFutureTest.java + **/remote/command/alert/AlertSendRequestCommandTest.java + **/remote/command/alert/AlertSendResponseCommandTest.java **/server/log/LoggerServerTest.java **/server/entity/SQLTaskExecutionContextTest.java **/server/log/MasterLogFilterTest.java @@ -866,6 +930,7 @@ **/service/queue/TaskUpdateQueueTest.java **/service/queue/PeerTaskInstancePriorityQueueTest.java + **/service/alert/AlertClientServiceTest.java **/dao/mapper/DataSourceUserMapperTest.java **/dao/mapper/ProcessDefinitionMapperTest.java @@ -882,8 +947,9 @@ **/dao/mapper/TenantMapperTest.java **/dao/mapper/UdfFuncMapperTest.java **/dao/mapper/UDFUserMapperTest.java - **/dao/mapper/UserAlertGroupMapperTest.java **/dao/mapper/UserMapperTest.java + **/dao/mapper/AlertPluginInstanceMapperTest.java + **/dao/mapper/PluginDefineTest.java **/dao/utils/DagHelperTest.java **/dao/AlertDaoTest.java **/dao/datasource/OracleDataSourceTest.java @@ -891,9 +957,32 @@ **/dao/upgrade/ProcessDefinitionDaoTest.java **/dao/upgrade/WokrerGrouopDaoTest.java **/dao/upgrade/UpgradeDaoTest.java - **/plugin/model/AlertDataTest.java - **/plugin/model/AlertInfoTest.java - **/plugin/utils/PropertyUtilsTest.java + **/plugin/alert/email/EmailAlertChannelFactoryTest.java + **/plugin/alert/email/EmailAlertChannelTest.java + **/plugin/alert/email/ExcelUtilsTest.java + **/plugin/alert/email/MailUtilsTest.java + **/plugin/alert/email/template/DefaultHTMLTemplateTest.java + **/plugin/alert/dingtalk/DingTalkSenderTest.java + **/plugin/alert/dingtalk/DingTalkAlertChannelFactoryTest.java + **/plugin/alert/wechat/WeChatSenderTest.java + **/plugin/alert/wechat/WeChatAlertChannelFactoryTest.java + **/plugin/alert/script/ProcessUtilsTest.java + **/plugin/alert/script/ScriptAlertChannelFactoryTest.java + **/plugin/alert/script/ScriptSenderTest.java + **/plugin/alert/http/HttpAlertChannelFactoryTest.java + **/plugin/alert/http/HttpAlertChannelTest.java + **/plugin/alert/http/HttpAlertPluginTest.java + **/plugin/alert/http/HttpSenderTest.java + **/spi/params/PluginParamsTransferTest.java + + **/alert/plugin/AlertPluginManagerTest.java + **/alert/plugin/DolphinPluginLoaderTest.java + **/alert/utils/DingTalkUtilsTest.java + **/alert/utils/EnterpriseWeChatUtilsTest.java + **/alert/utils/FuncUtilsTest.java + **/alert/processor/AlertRequestProcessorTest.java + **/alert/runner/AlertSenderTest.java + **/alert/AlertServerTest.java @@ -1000,6 +1089,7 @@ + dolphinscheduler-alert-plugin dolphinscheduler-ui dolphinscheduler-server dolphinscheduler-common @@ -1009,7 +1099,7 @@ dolphinscheduler-dist dolphinscheduler-remote dolphinscheduler-service - dolphinscheduler-plugin-api + dolphinscheduler-spi dolphinscheduler-microbench diff --git a/sql/dolphinscheduler-postgre.sql b/sql/dolphinscheduler-postgre.sql index e91527e2d3..8ca6cd8338 100644 --- a/sql/dolphinscheduler-postgre.sql +++ b/sql/dolphinscheduler-postgre.sql @@ -187,14 +187,10 @@ DROP TABLE IF EXISTS t_ds_alert; CREATE TABLE t_ds_alert ( id int NOT NULL , title varchar(64) DEFAULT NULL , - show_type int DEFAULT NULL , content text , - alert_type int DEFAULT NULL , alert_status int DEFAULT '0' , log text , alertgroup_id int DEFAULT NULL , - receivers text , - receivers_cc text , create_time timestamp DEFAULT NULL , update_time timestamp DEFAULT NULL , PRIMARY KEY (id) @@ -204,14 +200,15 @@ CREATE TABLE t_ds_alert ( -- DROP TABLE IF EXISTS t_ds_alertgroup; -CREATE TABLE t_ds_alertgroup ( - id int NOT NULL , - group_name varchar(255) DEFAULT NULL , - group_type int DEFAULT NULL , - description varchar(255) DEFAULT NULL , - create_time timestamp DEFAULT NULL , - update_time timestamp DEFAULT NULL , - PRIMARY KEY (id) +CREATE TABLE t_ds_alertgroup( + id int NOT NULL, + alert_instance_ids varchar (255) DEFAULT NULL, + create_user_id int4 DEFAULT NULL, + group_name varchar(255) DEFAULT NULL, + description varchar(255) DEFAULT NULL, + create_time timestamp DEFAULT NULL, + update_time timestamp DEFAULT NULL, + PRIMARY KEY (id) ) ; -- @@ -301,8 +298,7 @@ CREATE TABLE t_ds_process_definition ( flag int DEFAULT NULL , locations text , connects text , - receivers text , - receivers_cc text , + warning_group_id int4 DEFAULT NULL , create_time timestamp DEFAULT NULL , timeout int DEFAULT '0' , tenant_id int NOT NULL DEFAULT '-1' , @@ -329,8 +325,7 @@ CREATE TABLE t_ds_process_definition_version ( global_params text , locations text , connects text , - receivers text , - receivers_cc text , + warning_group_id int4 DEFAULT NULL, create_time timestamp DEFAULT NULL , timeout int DEFAULT '0' , resource_ids varchar(64), @@ -492,20 +487,6 @@ CREATE TABLE t_ds_relation_udfs_user ( ) ; ; --- --- Table structure for table t_ds_relation_user_alertgroup --- - -DROP TABLE IF EXISTS t_ds_relation_user_alertgroup; -CREATE TABLE t_ds_relation_user_alertgroup ( - id int NOT NULL, - alertgroup_id int DEFAULT NULL, - user_id int DEFAULT NULL, - create_time timestamp DEFAULT NULL, - update_time timestamp DEFAULT NULL, - PRIMARY KEY (id) -); - -- -- Table structure for table t_ds_resources -- @@ -747,9 +728,6 @@ ALTER TABLE t_ds_relation_resources_user ALTER COLUMN id SET DEFAULT NEXTVAL('t_ DROP SEQUENCE IF EXISTS t_ds_relation_udfs_user_id_sequence; CREATE SEQUENCE t_ds_relation_udfs_user_id_sequence; ALTER TABLE t_ds_relation_udfs_user ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_relation_udfs_user_id_sequence'); -DROP SEQUENCE IF EXISTS t_ds_relation_user_alertgroup_id_sequence; -CREATE SEQUENCE t_ds_relation_user_alertgroup_id_sequence; -ALTER TABLE t_ds_relation_user_alertgroup ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_relation_user_alertgroup_id_sequence'); DROP SEQUENCE IF EXISTS t_ds_resources_id_sequence; CREATE SEQUENCE t_ds_resources_id_sequence; @@ -776,21 +754,56 @@ ALTER TABLE t_ds_version ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_version_id_se DROP SEQUENCE IF EXISTS t_ds_worker_group_id_sequence; CREATE SEQUENCE t_ds_worker_group_id_sequence; -ALTER TABLE t_ds_worker_group ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_worker_group_id_sequence'); +ALTER TABLE t_ds_worker_group + ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_worker_group_id_sequence'); DROP SEQUENCE IF EXISTS t_ds_worker_server_id_sequence; -CREATE SEQUENCE t_ds_worker_server_id_sequence; -ALTER TABLE t_ds_worker_server ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_worker_server_id_sequence'); +CREATE SEQUENCE t_ds_worker_server_id_sequence; +ALTER TABLE t_ds_worker_server + ALTER COLUMN id SET DEFAULT NEXTVAL('t_ds_worker_server_id_sequence'); -- Records of t_ds_user?user : admin , password : dolphinscheduler123 -INSERT INTO t_ds_user(user_name,user_password,user_type,email,phone,tenant_id,state,create_time,update_time) VALUES ('admin', '7ad2410b2f4c074479a8937a28a22b8f', '0', 'xxx@qq.com', '', '0', 1, '2018-03-27 15:48:50', '2018-10-24 17:40:22'); +INSERT INTO t_ds_user(user_name, user_password, user_type, email, phone, tenant_id, state, create_time, update_time) +VALUES ('admin', '7ad2410b2f4c074479a8937a28a22b8f', '0', 'xxx@qq.com', '', '0', 1, '2018-03-27 15:48:50', + '2018-10-24 17:40:22'); -- Records of t_ds_alertgroup，dolphinscheduler warning group -INSERT INTO t_ds_alertgroup(group_name,group_type,description,create_time,update_time) VALUES ('dolphinscheduler warning group', '0', 'dolphinscheduler warning group','2018-11-29 10:20:39', '2018-11-29 10:20:39'); -INSERT INTO t_ds_relation_user_alertgroup(alertgroup_id,user_id,create_time,update_time) VALUES ( '1', '1', '2018-11-29 10:22:33', '2018-11-29 10:22:33'); +INSERT INTO t_ds_alertgroup(id,alert_instance_ids, create_user_id, group_name, description, create_time, update_time) +VALUES (1, '1,2','dolphinscheduler warning group', 'dolphinscheduler warning group', '2018-11-29 10:20:39', + '2018-11-29 10:20:39'); -- Records of t_ds_queue,default queue name : default -INSERT INTO t_ds_queue(queue_name,queue,create_time,update_time) VALUES ('default', 'default','2018-11-29 10:22:33', '2018-11-29 10:22:33'); +INSERT INTO t_ds_queue(queue_name, queue, create_time, update_time) +VALUES ('default', 'default', '2018-11-29 10:22:33', '2018-11-29 10:22:33'); -- Records of t_ds_queue,default queue name : default -INSERT INTO t_ds_version(version) VALUES ('1.3.0'); \ No newline at end of file +INSERT INTO t_ds_version(version) VALUES ('1.3.0'); + +-- +-- Table structure for table t_ds_plugin_define +-- +DROP TABLE IF EXISTS t_ds_plugin_define; +CREATE TABLE t_ds_plugin_define ( + id serial NOT NULL, + plugin_name varchar(100) NOT NULL, + plugin_type varchar(100) NOT NULL, + plugin_params text NULL, + create_time timestamp NULL, + update_time timestamp NULL, + CONSTRAINT t_ds_plugin_define_pk PRIMARY KEY (id), + CONSTRAINT t_ds_plugin_define_un UNIQUE (plugin_name, plugin_type) +); + +-- +-- Table structure for table t_ds_alert_plugin_instance +-- +DROP TABLE IF EXISTS t_ds_alert_plugin_instance; +CREATE TABLE t_ds_alert_plugin_instance ( + id serial NOT NULL, + plugin_define_id int4 NOT NULL, + plugin_instance_params text NULL, + create_time timestamp NULL, + update_time timestamp NULL, + instance_name varchar(200) NULL, + CONSTRAINT t_ds_alert_plugin_instance_pk PRIMARY KEY (id) +); \ No newline at end of file diff --git a/sql/dolphinscheduler_mysql.sql b/sql/dolphinscheduler_mysql.sql index 1429a00530..890e00344b 100644 --- a/sql/dolphinscheduler_mysql.sql +++ b/sql/dolphinscheduler_mysql.sql @@ -279,14 +279,10 @@ DROP TABLE IF EXISTS `t_ds_alert`; CREATE TABLE `t_ds_alert` ( `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'key', `title` varchar(64) DEFAULT NULL COMMENT 'title', - `show_type` tinyint(4) DEFAULT NULL COMMENT 'send email type,0:TABLE,1:TEXT', `content` text COMMENT 'Message content (can be email, can be SMS. Mail is stored in JSON map, and SMS is string)', - `alert_type` tinyint(4) DEFAULT NULL COMMENT '0:email,1:sms', `alert_status` tinyint(4) DEFAULT '0' COMMENT '0:wait running,1:success,2:failed', `log` text COMMENT 'log', `alertgroup_id` int(11) DEFAULT NULL COMMENT 'alert group id', - `receivers` text COMMENT 'receivers', - `receivers_cc` text COMMENT 'cc', `create_time` datetime DEFAULT NULL COMMENT 'create time', `update_time` datetime DEFAULT NULL COMMENT 'update time', PRIMARY KEY (`id`) @@ -300,14 +296,15 @@ CREATE TABLE `t_ds_alert` ( -- Table structure for t_ds_alertgroup -- ---------------------------- DROP TABLE IF EXISTS `t_ds_alertgroup`; -CREATE TABLE `t_ds_alertgroup` ( - `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'key', - `group_name` varchar(255) DEFAULT NULL COMMENT 'group name', - `group_type` tinyint(4) DEFAULT NULL COMMENT 'Group type (message 0, SMS 1...)', - `description` varchar(255) DEFAULT NULL, - `create_time` datetime DEFAULT NULL COMMENT 'create time', - `update_time` datetime DEFAULT NULL COMMENT 'update time', - PRIMARY KEY (`id`) +CREATE TABLE `t_ds_alertgroup`( + `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'key', + `alert_instance_ids` varchar (255) DEFAULT NULL COMMENT 'alert instance ids', + `create_user_id` int(11) DEFAULT NULL COMMENT 'create user id', + `group_name` varchar(255) DEFAULT NULL COMMENT 'group name', + `description` varchar(255) DEFAULT NULL, + `create_time` datetime DEFAULT NULL COMMENT 'create time', + `update_time` datetime DEFAULT NULL COMMENT 'update time', + PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; -- ---------------------------- @@ -406,8 +403,7 @@ CREATE TABLE `t_ds_process_definition` ( `flag` tinyint(4) DEFAULT NULL COMMENT '0 not available, 1 available', `locations` text COMMENT 'Node location information', `connects` text COMMENT 'Node connection information', - `receivers` text COMMENT 'receivers', - `receivers_cc` text COMMENT 'cc', + `warning_group_id` int(11) DEFAULT NULL COMMENT 'alert group id', `create_time` datetime DEFAULT NULL COMMENT 'create time', `timeout` int(11) DEFAULT '0' COMMENT 'time out', `tenant_id` int(11) NOT NULL DEFAULT '-1' COMMENT 'tenant id', @@ -436,8 +432,7 @@ CREATE TABLE `t_ds_process_definition_version` ( `global_params` text COMMENT 'global parameters', `locations` text COMMENT 'Node location information', `connects` text COMMENT 'Node connection information', - `receivers` text COMMENT 'receivers', - `receivers_cc` text COMMENT 'cc', + `warning_group_id` int(11) DEFAULT NULL COMMENT 'alert group id', `create_time` datetime DEFAULT NULL COMMENT 'create time', `timeout` int(11) DEFAULT '0' COMMENT 'time out', `resource_ids` varchar(255) DEFAULT NULL COMMENT 'resource ids', @@ -620,27 +615,6 @@ CREATE TABLE `t_ds_relation_udfs_user` ( PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; --- ---------------------------- --- Records of t_ds_relation_udfs_user --- ---------------------------- - --- ---------------------------- --- Table structure for t_ds_relation_user_alertgroup --- ---------------------------- -DROP TABLE IF EXISTS `t_ds_relation_user_alertgroup`; -CREATE TABLE `t_ds_relation_user_alertgroup` ( - `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'key', - `alertgroup_id` int(11) DEFAULT NULL COMMENT 'alert group id', - `user_id` int(11) DEFAULT NULL COMMENT 'user id', - `create_time` datetime DEFAULT NULL COMMENT 'create time', - `update_time` datetime DEFAULT NULL COMMENT 'update time', - PRIMARY KEY (`id`) -) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; - --- ---------------------------- --- Records of t_ds_relation_user_alertgroup --- ---------------------------- - -- ---------------------------- -- Table structure for t_ds_resources -- ---------------------------- @@ -830,20 +804,51 @@ CREATE TABLE `t_ds_version` ( -- ---------------------------- -- Records of t_ds_version -- ---------------------------- -INSERT INTO `t_ds_version` VALUES ('1', '1.3.0'); +INSERT INTO `t_ds_version` +VALUES ('1', '1.3.0'); -- ---------------------------- -- Records of t_ds_alertgroup -- ---------------------------- -INSERT INTO `t_ds_alertgroup` VALUES ('1', 'default admin warning group', '0', 'default admin warning group', '2018-11-29 10:20:39', '2018-11-29 10:20:39'); +INSERT INTO `t_ds_alertgroup` +VALUES (1,"1,2", 1, 'default admin warning group', 'default admin warning group', '2018-11-29 10:20:39', + '2018-11-29 10:20:39'); -- ---------------------------- --- Records of t_ds_relation_user_alertgroup +-- Records of t_ds_user -- ---------------------------- -INSERT INTO `t_ds_relation_user_alertgroup` VALUES ('1', '1', '1', '2018-11-29 10:22:33', '2018-11-29 10:22:33'); +INSERT INTO `t_ds_user` +VALUES ('1', 'admin', '7ad2410b2f4c074479a8937a28a22b8f', '0', 'xxx@qq.com', '', '0', '2018-03-27 15:48:50', + '2018-10-24 17:40:22', null, 1); -- ---------------------------- --- Records of t_ds_user +-- Table structure for t_ds_plugin_define +-- ---------------------------- +SET +sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY','')); +DROP TABLE IF EXISTS `t_ds_plugin_define`; +CREATE TABLE `t_ds_plugin_define` ( + `id` int NOT NULL AUTO_INCREMENT, + `plugin_name` varchar(100) NOT NULL COMMENT 'the name of plugin eg: email', + `plugin_type` varchar(100) NOT NULL COMMENT 'plugin type . alert=alert plugin, job=job plugin', + `plugin_params` text COMMENT 'plugin params', + `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + UNIQUE KEY `t_ds_plugin_define_UN` (`plugin_name`,`plugin_type`) +) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8; + -- ---------------------------- -INSERT INTO `t_ds_user` VALUES ('1', 'admin', '7ad2410b2f4c074479a8937a28a22b8f', '0', 'xxx@qq.com', '', '0', '2018-03-27 15:48:50', '2018-10-24 17:40:22', null, 1); +-- Table structure for t_ds_alert_plugin_instance +-- ---------------------------- +DROP TABLE IF EXISTS `t_ds_alert_plugin_instance`; +CREATE TABLE `t_ds_alert_plugin_instance` ( + `id` int NOT NULL AUTO_INCREMENT, + `plugin_define_id` int NOT NULL, + `plugin_instance_params` text COMMENT 'plugin instance params. Also contain the params value which user input in web ui.', + `create_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP, + `update_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + `instance_name` varchar(200) DEFAULT NULL COMMENT 'alert instance name', + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; \ No newline at end of file diff --git a/sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql b/sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql new file mode 100644 index 0000000000..60b354fdf7 --- /dev/null +++ b/sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. +*/ + +SET sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY','')); +DROP TABLE IF EXISTS `t_ds_plugin_define`; +CREATE TABLE `t_ds_plugin_define` ( + `id` int NOT NULL AUTO_INCREMENT, + `plugin_name` varchar(100) NOT NULL COMMENT 'the name of plugin eg: email', + `plugin_type` varchar(100) NOT NULL COMMENT 'plugin type . alert=alert plugin, job=job plugin', + `plugin_params` text COMMENT 'plugin params', + `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + UNIQUE KEY `t_ds_plugin_define_UN` (`plugin_name`,`plugin_type`) +) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8; + +DROP TABLE IF EXISTS `t_ds_alert_plugin_instance`; +CREATE TABLE `t_ds_alert_plugin_instance` ( + `id` int NOT NULL AUTO_INCREMENT, + `plugin_define_id` int NOT NULL, + `plugin_instance_params` text COMMENT 'plugin instance params. Also contain the params value which user input in web ui.', + `create_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP, + `update_time` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + `instance_name` varchar(200) DEFAULT NULL COMMENT 'alert instance name', + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; + +ALTER TABLE t_ds_process_definition + ADD COLUMN `warning_group_id` int(11) DEFAULT NULL COMMENT 'alert group id' AFTER `connects`; + +ALTER TABLE t_ds_process_definition_version + ADD COLUMN `warning_group_id` int(11) DEFAULT NULL COMMENT 'alert group id' AFTER `connects`; + +ALTER TABLE t_ds_alertgroup + ADD COLUMN `alert_instance_ids` varchar (255) DEFAULT NULL COMMENT 'alert instance ids' AFTER `id`, + ADD COLUMN `create_user_id` int(11) DEFAULT NULL COMMENT 'create user id' AFTER `alert_instance_ids`; + + +-- ---------------------------- +-- These columns will not be used in the new version,if you determine that the historical data is useless, you can delete it using the sql below +-- ---------------------------- +/* +ALTER TABLE t_ds_process_definition DROP receivers, DROP receivers_cc; + +ALTER TABLE t_ds_process_definition_version DROP receivers, DROP receivers_cc; + +ALTER TABLE t_ds_alert DROP show_type,DROP alert_type,DROP receivers,DROP receivers_cc; + +ALTER TABLE t_ds_alertgroup DROP group_type; + +DROP TABLE IF EXISTS t_ds_relation_user_alertgroup; +*/ + + + diff --git a/sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_dml.sql b/sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_dml.sql new file mode 100644 index 0000000000..57ed4d97e8 --- /dev/null +++ b/sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_dml.sql @@ -0,0 +1,18 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. +*/ +SET sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY','')); +SET FOREIGN_KEY_CHECKS=0; diff --git a/sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_ddl.sql b/sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_ddl.sql new file mode 100644 index 0000000000..d7fa51d840 --- /dev/null +++ b/sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_ddl.sql @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. +*/ +DROP TABLE IF EXISTS t_ds_plugin_define; +CREATE TABLE t_ds_plugin_define ( + id serial NOT NULL, + plugin_name varchar(100) NOT NULL, + plugin_type varchar(100) NOT NULL, + plugin_params text NULL, + create_time timestamp NULL, + update_time timestamp NULL, + CONSTRAINT t_ds_plugin_define_pk PRIMARY KEY (id), + CONSTRAINT t_ds_plugin_define_un UNIQUE (plugin_name, plugin_type) +); + + +DROP TABLE IF EXISTS t_ds_alert_plugin_instance; +CREATE TABLE t_ds_alert_plugin_instance ( + id serial NOT NULL, + plugin_define_id int4 NOT NULL, + plugin_instance_params text NULL, + create_time timestamp NULL, + update_time timestamp NULL, + instance_name varchar(200) NULL, + CONSTRAINT t_ds_alert_plugin_instance_pk PRIMARY KEY (id) +); + +ALTER TABLE t_ds_process_definition + ADD COLUMN `warning_group_id` int4 DEFAULT NULL COMMENT 'alert group id' AFTER `connects`; + +ALTER TABLE t_ds_process_definition_version + ADD COLUMN `warning_group_id` int4 DEFAULT NULL COMMENT 'alert group id' AFTER `connects`; + +ALTER TABLE t_ds_alertgroup + ADD COLUMN `alert_instance_ids` int4 DEFAULT NULL COMMENT 'alert instance ids' AFTER `id`; + ADD COLUMN `create_user_id` varchar(255) DEFAULT NULL COMMENT 'create user id' AFTER `alert_instance_ids`, + + +-- ---------------------------- +-- These columns will not be used in the new version,if you determine that the historical data is useless, you can delete it using the sql below +-- ---------------------------- +/* +ALTER TABLE t_ds_process_definition DROP COLUMN "receivers", DROP COLUMN "receivers_cc"; + +ALTER TABLE t_ds_process_definition_version DROP COLUMN "receivers", DROP COLUMN "receivers_cc"; + +ALTER TABLE t_ds_alert DROP COLUMN "show_type",DROP COLUMN "alert_type",DROP COLUMN "receivers",DROP COLUMN "receivers_cc"; + +ALTER TABLE t_ds_alertgroup DROP COLUMN "group_type"; + +DROP TABLE IF EXISTS t_ds_relation_user_alertgroup; +*/ diff --git a/sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_dml.sql b/sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_dml.sql new file mode 100644 index 0000000000..4a14f326b9 --- /dev/null +++ b/sql/upgrade/1.4.0_schema/postgresql/dolphinscheduler_dml.sql @@ -0,0 +1,16 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. +*/ diff --git a/style/checkstyle.xml b/style/checkstyle.xml index d7283abafa..675d82133e 100644 --- a/style/checkstyle.xml +++ b/style/checkstyle.xml @@ -6,9 +6,7 @@ The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. @@ -197,7 +195,7 @@ @@ -284,4 +282,4 @@ - + \ No newline at end of file diff --git a/style/intellij-java-code-style.xml b/style/intellij-java-code-style.xml index 42286025b0..80dc27d5f4 100644 --- a/style/intellij-java-code-style.xml +++ b/style/intellij-java-code-style.xml @@ -6,9 +6,7 @@ The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. diff --git a/tools/dependencies/known-dependencies.txt b/tools/dependencies/known-dependencies.txt index ec055ee856..ebec45992f 100755 --- a/tools/dependencies/known-dependencies.txt +++ b/tools/dependencies/known-dependencies.txt @@ -1,5 +1,5 @@ HikariCP-3.2.0.jar -activation-1.1.jar +animal-sniffer-annotations-1.14.jar aopalliance-1.0.jar apache-el-8.5.54.jar apacheds-i18n-2.0.0-M15.jar @@ -13,6 +13,7 @@ avro-1.7.4.jar aws-java-sdk-1.7.4.jar bonecp-0.8.0.RELEASE.jar byte-buddy-1.9.16.jar +checker-compat-qual-2.0.0.jar classmate-1.4.0.jar clickhouse-jdbc-0.1.52.jar commons-cli-1.2.jar @@ -25,7 +26,6 @@ commons-configuration-1.10.jar commons-daemon-1.0.13.jar commons-beanutils-1.7.0.jar commons-dbcp-1.4.jar -commons-email-1.5.jar commons-httpclient-3.0.1.jar commons-io-2.4.jar commons-lang-2.6.jar @@ -41,9 +41,11 @@ datanucleus-api-jdo-4.2.1.jar datanucleus-core-4.1.6.jar datanucleus-rdbms-4.1.7.jar derby-10.14.2.0.jar +error_prone_annotations-2.1.3.jar druid-1.1.22.jar gson-2.8.6.jar -guava-20.0.jar +guava-24.1-jre.jar +guava-retrying-2.0.0.jar guice-3.0.jar guice-servlet-3.0.jar h2-1.4.200.jar @@ -76,6 +78,7 @@ htrace-core-3.1.0-incubating.jar httpclient-4.4.1.jar httpcore-4.4.1.jar httpmime-4.5.12.jar +j2objc-annotations-1.1.jar jackson-annotations-2.9.10.jar jackson-core-2.9.10.jar jackson-core-asl-1.9.13.jar @@ -93,7 +96,6 @@ javax.activation-api-1.2.0.jar javax.annotation-api-1.3.2.jar javax.inject-1.jar javax.jdo-3.2.0-m3.jar -javax.mail-1.6.2.jar javax.servlet-api-3.1.0.jar javolution-5.5.1.jar jaxb-api-2.3.1.jar @@ -158,6 +160,7 @@ paranamer-2.3.jar parquet-hadoop-bundle-1.8.1.jar poi-3.17.jar postgresql-42.1.4.jar +presto-jdbc-0.238.1.jar protobuf-java-2.5.0.jar quartz-2.3.0.jar quartz-jobs-2.3.0.jar @@ -205,5 +208,3 @@ xml-apis-1.4.01.jar xmlenc-0.52.jar xz-1.0.jar zookeeper-3.4.14.jar -guava-retrying-2.0.0.jar -presto-jdbc-0.238.1.jar