From 920140028bd649e8bec6a88f2c471002896d4cf8 Mon Sep 17 00:00:00 2001 From: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com> Date: Sun, 29 Aug 2021 21:43:38 +0800 Subject: [PATCH] [Feature][JsonSplit-api] update of processInstance (#6061) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [BUG-#5678][Registry]fix registry init node miss (#5686) * [Improvement][UI] Update the update time after the user information is successfully modified (#5684) * improve edit the userinfo success, but the updatetime is not the latest. * Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691) Co-authored-by: shenglm * [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603) * add globalParams new plan with varPool * add unit test * add python task varPoolParams Co-authored-by: wangxj * Issue robot translation judgment changed to Chinese (#5694) Co-authored-by: chenxingchun <438044805@qq.com> * the update function should use post instead of get (#5703) * enhance form verify (#5696) * checkState only supports %s not {} (#5711) * [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697) * update * fix the codestyle error * fix the compile error * support rollback * [Fix-5699][UI] Fix update user error in user information (#5700) * [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676) * bug fix the auto generated spi service can't be recongized * include a new method * [Improvement-5622][project management] Modify the title (#5723) * [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715) * add a new init method. * [Fix#5758] There are some problems in the api documentation that need to be improved (#5759) * add the necessary parameters * openapi improve * fix code style error * [FIX-#5721][master-server] Global params parameter missing (#5757) Co-authored-by: wangxj * [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move` doesn't work. (#5739) * Update relatedItems.vue * Update relatedItems.vue * [Improvement#5741][Worker] Improve task process status log (#5776) * [Improvement-5773][server] need to support two parameters related to task (#5774) * add some new parameter for task * restore official properties * improve imports * modify a variable's name Co-authored-by: jiang hua * [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE * [Improvement][Worker] Task log may be lost #5775 (#5783) * [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789) * [Imporvement #5725][CheckStyle] upgrade checkstyle file Upgrade checkstyle.xml to support checkstyle version 8.24+ * change ci checkstyle version * [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected. (#5796) * improve timestamp format make the startime in the log of httptask to be easier to read. * fix bad code smell and update the note. * [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797) ·the list of workflow instances is sorted by start time and end time ·This closes #5621 * fix (#5803) Co-authored-by: shuangbofu * fix: Remove duplicate "registryClient.close" method calls (#5805) Co-authored-by: wen-hemin * [Improvement][SPI] support load single plugin (#5794) change load operation of 'registry.plugin.dir' * [Improvement][Api Module] refactor registry client, remove spring annotation (#5814) * fix: refactor registry client, remove spring annotation * fix UT * fix UT * fix checkstyle * fix UT * fix UT * fix UT * fix: Rename RegistryCenterUtils method name Co-authored-by: wen-hemin * [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735) * [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727) * enhance the validation in UI * enchance form verifaction * simplify disable condition * fix: Remove unused class (#5833) Co-authored-by: wen-hemin * [fix-5737] [Bug][Datasource] datsource other param check error (#5835) Co-authored-by: wanggang * [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes * [Fix-5825][BUG][WEB] the resource tree in the process definition of latest dev branch can't display correctly (#5826) * resoures-shows-error * fix codestyle error * add license header for new js * fix codesmell * [Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867) * provide two system parameters to support the rest of type of tasks * provide two system parameters to support the rest of type of tasks * improve test conversion * [Improvement][Fix-5769][UI]When we try to delete the existing dag, the console in web browser would shows exception (#5770) * fix bug * cache the this variable * Avoid self name * fix code style compile error * [Fix-5781][UT] Fix test coverage in sonar (#5817) * build(UT): make jacoco running in offline-instrumentation issue: #5781 * build(UT): remove the jacoco agent dependency in microbench issue: #5781 * [Fix-5808][Server] When we try to transfer data using datax between different types of data sources, the worker will exit with ClassCastException (#5809) * bug fix * fix bug * simplify the code format * add a new parameter to make it easier to understand. * [Fix-5830][Improvement][UI] Improve the selection style in dag edit dialog (#5829) * improve the selection style * update another file * remove unnecessary css part. * [Fix-5904][upgrade]fix dev branch upgrade mysql sql script error (#5821) * fix dev branch upgrade mysql sql script error. * Update naming convention. * [Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832) * fix: refactor api utils class, remove spring annotation. * fix: Optimization comments Co-authored-by: wen-hemin * correct the wrong annotion from zk queue implemented to java priority blocking queue (#5906) Co-authored-by: ywang46 * Add a Gitter chat badge to README.md (#5883) * Add Gitter badge * Update README.md Co-authored-by: David * ci: improve maven connection in CI builds (#5924) issue: #5921 * [Improvement][Master]fix typo (#5934) ·fix typo in MasterBaseTaskExecThread * [Fix-5886][server] Enhanced scheduler delete check (#5936) * Add:Name verification remove the first and last spaces. * Update: wrong word: 'WAITTING' ->'WAITING' * Add: Strengthen verification Co-authored-by: Squid <2824638304@qq.com> * [Improvement-5880][api] Optimized data structure of pagination query API results (#5895) * [5880][refactor]Optimized data structure of pagination query API results - refactor PageInfo and delete returnDataListPaging in API - modify the related Controller and Service and the corresponding Test * Merge branch 'dev' of github.com:apache/dolphinscheduler into dev  Conflicts:  dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java Co-authored-by: 蔡泽华 * [IMPROVEMENT]fix mysql comment error (#5959) * [Improvement][Api]fix typo (#5960) * [Imporvement #5621][job instance] start-time and end-time (#5621) ·the list of workflow instances is sorted by start time and end time ·This closes #5621 * [FIX-5975]queryLastRunningProcess sql in ProcessInstanceMapper.xml (#5980) * [NEW FEATURE][FIX-4385] compensation task add the ability to configure parallelism (#5912) * update * web improved * improve the ui * add the ability to configure the parallelism * update variables * enhance the ut and add necessary note * fix code style * fix code style issue * ensure the complation task in parallel mode can run the right numbers of tasks. * [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952) * [Bug][WorkerServer] SqlTask NullPointerException #5549 * [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428 * [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428 * [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428 * [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428 * [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428 * [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428 * [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428 * fix the readme typing issue (#5998) * Fix unchecked type conversions * Use indentation level reported by checkstyle * Reorganize CI workflows to fasten the wasted time and resources (#6011) * Add standalone server module to make it easier to develop (#6022) * fix ut * update of processInstance * fix ut Co-authored-by: Kirs Co-authored-by: kyoty Co-authored-by: ji04xiaogang Co-authored-by: shenglm Co-authored-by: wangxj3 <857234426@qq.com> Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com> Co-authored-by: chenxingchun <438044805@qq.com> Co-authored-by: Shiwen Cheng Co-authored-by: Jianchao Wang Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com> Co-authored-by: Hua Jiang Co-authored-by: jiang hua Co-authored-by: Wenjun Ruan <861923274@qq.com> Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com> Co-authored-by: 傅双波 <786183073@qq.com> Co-authored-by: shuangbofu Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com> Co-authored-by: wen-hemin Co-authored-by: geosmart Co-authored-by: wanggang Co-authored-by: AzureCN Co-authored-by: 深刻 Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com> Co-authored-by: Yao WANG Co-authored-by: ywang46 Co-authored-by: The Gitter Badger Co-authored-by: David Co-authored-by: Squidyu <1297554122@qq.com> Co-authored-by: Squid <2824638304@qq.com> Co-authored-by: soreak <60459867+soreak@users.noreply.github.com> Co-authored-by: 蔡泽华 Co-authored-by: yimaixinchen Co-authored-by: atai-555 <74188560+atai-555@users.noreply.github.com> Co-authored-by: didiaode18 <563646039@qq.com> Co-authored-by: Roy Co-authored-by: lyxell Co-authored-by: Wenjun Ruan Co-authored-by: kezhenxu94 Co-authored-by: JinyLeeChina <297062848@qq.com> --- .../api/controller/ProcessInstanceController.java | 9 +++------ .../api/service/ProcessInstanceService.java | 2 -- .../api/service/impl/ProcessInstanceServiceImpl.java | 3 +-- .../api/service/ProcessInstanceServiceTest.java | 10 +++++----- .../org/apache/dolphinscheduler/common/enums/Flag.java | 3 +-- .../apache/dolphinscheduler/common/enums/Priority.java | 2 +- .../dao/mapper/TaskDefinitionMapper.xml | 2 +- 7 files changed, 12 insertions(+), 19 deletions(-) diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessInstanceController.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessInstanceController.java index 089eb03cb5..f46e7d8ffc 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessInstanceController.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessInstanceController.java @@ -165,7 +165,6 @@ public class ProcessInstanceController extends BaseController { * @param processInstanceId process instance id * @param scheduleTime schedule time * @param syncDefine sync define - * @param flag flag * @param locations locations * @param tenantCode tenantCode * @return update result code @@ -179,8 +178,7 @@ public class ProcessInstanceController extends BaseController { @ApiImplicitParam(name = "globalParams", value = "PROCESS_GLOBAL_PARAMS", type = "String"), @ApiImplicitParam(name = "locations", value = "PROCESS_INSTANCE_LOCATIONS", type = "String"), @ApiImplicitParam(name = "timeout", value = "PROCESS_TIMEOUT", type = "String"), - @ApiImplicitParam(name = "tenantCode", value = "TENANT_CODE", type = "Int", example = "0"), - @ApiImplicitParam(name = "flag", value = "RECOVERY_PROCESS_INSTANCE_FLAG", type = "Flag") + @ApiImplicitParam(name = "tenantCode", value = "TENANT_CODE", type = "Int", example = "0") }) @PostMapping(value = "/update") @ResponseStatus(HttpStatus.OK) @@ -195,10 +193,9 @@ public class ProcessInstanceController extends BaseController { @RequestParam(value = "globalParams", required = false, defaultValue = "[]") String globalParams, @RequestParam(value = "locations", required = false) String locations, @RequestParam(value = "timeout", required = false, defaultValue = "0") int timeout, - @RequestParam(value = "tenantCode", required = true) String tenantCode, - @RequestParam(value = "flag", required = false) Flag flag) { + @RequestParam(value = "tenantCode", required = true) String tenantCode) { Map result = processInstanceService.updateProcessInstance(loginUser, projectCode, processInstanceId, - taskRelationJson, scheduleTime, syncDefine, flag, globalParams, locations, timeout, tenantCode); + taskRelationJson, scheduleTime, syncDefine, globalParams, locations, timeout, tenantCode); return returnDataList(result); } diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java index cd4cfd5365..0190d177ba 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java @@ -119,7 +119,6 @@ public interface ProcessInstanceService { * @param processInstanceId process instance id * @param scheduleTime schedule time * @param syncDefine sync define - * @param flag flag * @param globalParams global params * @param locations locations for nodes * @param timeout timeout @@ -132,7 +131,6 @@ public interface ProcessInstanceService { String taskRelationJson, String scheduleTime, Boolean syncDefine, - Flag flag, String globalParams, String locations, int timeout, diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java index 842b6b71d4..9a6fa3e18c 100644 --- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java +++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java @@ -404,7 +404,6 @@ public class ProcessInstanceServiceImpl extends BaseServiceImpl implements Proce * @param processInstanceId process instance id * @param scheduleTime schedule time * @param syncDefine sync define - * @param flag flag * @param globalParams global params * @param locations locations for nodes * @param timeout timeout @@ -414,7 +413,7 @@ public class ProcessInstanceServiceImpl extends BaseServiceImpl implements Proce @Transactional @Override public Map updateProcessInstance(User loginUser, long projectCode, Integer processInstanceId, String taskRelationJson, - String scheduleTime, Boolean syncDefine, Flag flag, String globalParams, + String scheduleTime, Boolean syncDefine, String globalParams, String locations, int timeout, String tenantCode) { Project project = projectMapper.queryByCode(projectCode); //check user access for project diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java index cbca3b683a..820d5fba98 100644 --- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java +++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java @@ -372,7 +372,7 @@ public class ProcessInstanceServiceTest { when(projectMapper.queryByCode(projectCode)).thenReturn(project); when(projectService.checkProjectAndAuth(loginUser, project, projectCode)).thenReturn(result); Map proejctAuthFailRes = processInstanceService.updateProcessInstance(loginUser, projectCode, 1, - shellJson, "2020-02-21 00:00:00", true, Flag.YES, "", "", 0, ""); + shellJson, "2020-02-21 00:00:00", true, "", "", 0, ""); Assert.assertEquals(Status.PROJECT_NOT_FOUNT, proejctAuthFailRes.get(Constants.STATUS)); //process instance null @@ -382,7 +382,7 @@ public class ProcessInstanceServiceTest { when(projectService.checkProjectAndAuth(loginUser, project, projectCode)).thenReturn(result); when(processService.findProcessInstanceDetailById(1)).thenReturn(null); Map processInstanceNullRes = processInstanceService.updateProcessInstance(loginUser, projectCode, 1, - shellJson, "2020-02-21 00:00:00", true, Flag.YES, "", "", 0, ""); + shellJson, "2020-02-21 00:00:00", true, "", "", 0, ""); Assert.assertEquals(Status.PROCESS_INSTANCE_NOT_EXIST, processInstanceNullRes.get(Constants.STATUS)); //process instance not finish @@ -390,7 +390,7 @@ public class ProcessInstanceServiceTest { processInstance.setState(ExecutionStatus.RUNNING_EXECUTION); putMsg(result, Status.SUCCESS, projectCode); Map processInstanceNotFinishRes = processInstanceService.updateProcessInstance(loginUser, projectCode, 1, - shellJson, "2020-02-21 00:00:00", true, Flag.YES, "", "", 0, ""); + shellJson, "2020-02-21 00:00:00", true, "", "", 0, ""); Assert.assertEquals(Status.PROCESS_INSTANCE_STATE_OPERATION_ERROR, processInstanceNotFinishRes.get(Constants.STATUS)); //process instance finish @@ -410,7 +410,7 @@ public class ProcessInstanceServiceTest { when(processDefinitionService.checkProcessNodeList(shellJson)).thenReturn(result); putMsg(result, Status.SUCCESS, projectCode); Map processInstanceFinishRes = processInstanceService.updateProcessInstance(loginUser, projectCode, 1, - shellJson, "2020-02-21 00:00:00", true, Flag.YES, "", "", 0, "root"); + shellJson, "2020-02-21 00:00:00", true, "", "", 0, "root"); Assert.assertEquals(Status.UPDATE_PROCESS_DEFINITION_ERROR, processInstanceFinishRes.get(Constants.STATUS)); //success @@ -418,7 +418,7 @@ public class ProcessInstanceServiceTest { putMsg(result, Status.SUCCESS, projectCode); Map successRes = processInstanceService.updateProcessInstance(loginUser, projectCode, 1, - shellJson, "2020-02-21 00:00:00", false, Flag.YES, "", "", 0, "root"); + shellJson, "2020-02-21 00:00:00", false, "", "", 0, "root"); Assert.assertEquals(Status.SUCCESS, successRes.get(Constants.STATUS)); } diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Flag.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Flag.java index 622e9d17d4..fa69f2978d 100644 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Flag.java +++ b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Flag.java @@ -34,8 +34,7 @@ public enum Flag { NO(0, "no"), YES(1, "yes"); - - Flag(int code, String descp){ + Flag(int code, String descp) { this.code = code; this.descp = descp; } diff --git a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Priority.java b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Priority.java index bdd7128eac..53ca3f9c74 100644 --- a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Priority.java +++ b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/Priority.java @@ -35,7 +35,7 @@ public enum Priority { LOW(3, "low"), LOWEST(4, "lowest"); - Priority(int code, String descp){ + Priority(int code, String descp) { this.code = code; this.descp = descp; } diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml index 6e14de911b..61f55430d0 100644 --- a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml +++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml @@ -86,7 +86,7 @@ #{taskDefinition.delayTime},#{taskDefinition.resourceIds},#{taskDefinition.createTime},#{taskDefinition.updateTime}) - select td.id, td.code, td.name, td.version, td.description, td.project_code, td.user_id, td.task_type, td.task_params, td.flag, td.task_priority, td.worker_group, td.fail_retry_times, td.fail_retry_interval, td.timeout_flag, td.timeout_notify_strategy, td.timeout, td.delay_time, td.resource_ids, td.create_time, td.update_time, u.user_name,p.name as project_name