Browse Source

[Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent (#14824)

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- delete useless output

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- add log
- run spotless check

---------

Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
3.2.1-prepare
sgw 1 year ago committed by GitHub
parent
commit
76bbcbeb30
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 7
      docs/docs/en/guide/task/dependent.md
  2. 8
      docs/docs/zh/guide/task/dependent.md
  3. BIN
      docs/img/tasks/demo/dependent_task01.png
  4. BIN
      docs/img/tasks/demo/dependent_task02.png
  5. BIN
      docs/img/tasks/demo/dependent_task03.png
  6. 3
      dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/constants/Constants.java
  7. 19
      dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapper.java
  8. 7
      dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/TaskDefinitionDao.java
  9. 22
      dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/TaskInstanceDao.java
  10. 5
      dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/TaskDefinitionDaoImpl.java
  11. 15
      dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/TaskInstanceDaoImpl.java
  12. 35
      dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapper.xml
  13. 6
      dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/dependent/DependentAsyncTaskExecuteFunction.java
  14. 138
      dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java
  15. 4
      dolphinscheduler-ui/src/locales/en_US/project.ts
  16. 3
      dolphinscheduler-ui/src/locales/zh_CN/project.ts
  17. 42
      dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-dependent.ts
  18. 2
      dolphinscheduler-ui/src/views/projects/task/components/node/types.ts

7
docs/docs/en/guide/task/dependent.md

@ -27,11 +27,14 @@ Dependent nodes are **dependency check nodes**. For example, process A depends o
The Dependent node provides a logical judgment function, which can detect the execution of the dependent node according to the logic. The Dependent node provides a logical judgment function, which can detect the execution of the dependent node according to the logic.
For example, process A is a weekly task, processes B and C are daily tasks, and task A requires tasks B and C to be successfully executed every day of the last week. Two dependency modes are supported, including workflow-dependent and task-dependent. The task-dependent mode is divided into two cases: depend on all tasks in the workflow and depend on a single task.
The workflow-dependent mode checks the status of the dependent workflow; the all-task-dependent mode checks the status of all tasks in the workflow; and the single-task-dependent mode checks the status of the dependent task.
For example, process A is a weekly task, processes B and C are daily tasks, and task A requires tasks B and C to be successfully executed last week.
![dependent_task01](../../../../img/tasks/demo/dependent_task01.png) ![dependent_task01](../../../../img/tasks/demo/dependent_task01.png)
And another example is that process A is a weekly report task, processes B and C are daily tasks, and task A requires tasks B or C to be successfully executed every day of the last week: And another example is that process A is a weekly report task, processes B and C are daily tasks, and task A requires tasks B or C to be successfully executed last week:
![dependent_task02](../../../../img/tasks/demo/dependent_task02.png) ![dependent_task02](../../../../img/tasks/demo/dependent_task02.png)

8
docs/docs/zh/guide/task/dependent.md

@ -27,11 +27,15 @@ Dependent 节点,就是**依赖检查节点**。比如 A 流程依赖昨天的
Dependent 节点提供了逻辑判断功能,可以按照逻辑来检测所依赖节点的执行情况。 Dependent 节点提供了逻辑判断功能,可以按照逻辑来检测所依赖节点的执行情况。
例如,A 流程为周报任务,B、C 流程为天任务,A 任务需要 B、C 任务在上周的每一天都执行成功,如图示: 支持两种依赖模式,包括依赖于工作流和依赖于任务。依赖于任务的模式分依赖工作流中的所有任务和依赖单个任务两种情况。
依赖工作流的模式会检查所依赖的工作流的状态;依赖所有任务的模式会检查工作流中所有任务的状态;
依赖单个任务的模式会检查所依赖的任务的状态。
例如,A 流程为周报任务,B、C 流程为天任务,A 任务需要 B、C 任务在上周执行成功,如图示:
![dependent_task01](../../../../img/tasks/demo/dependent_task01.png) ![dependent_task01](../../../../img/tasks/demo/dependent_task01.png)
例如,A 流程为周报任务,B、C 流程为天任务,A 任务需要 B 或 C 任务在上周的每一天都执行成功,如图示: 例如,A 流程为周报任务,B、C 流程为天任务,A 任务需要 B 或 C 任务在上周执行成功,如图示:
![dependent_task02](../../../../img/tasks/demo/dependent_task02.png) ![dependent_task02](../../../../img/tasks/demo/dependent_task02.png)

BIN
docs/img/tasks/demo/dependent_task01.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 156 KiB

After

Width:  |  Height:  |  Size: 141 KiB

BIN
docs/img/tasks/demo/dependent_task02.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 156 KiB

After

Width:  |  Height:  |  Size: 137 KiB

BIN
docs/img/tasks/demo/dependent_task03.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 161 KiB

After

Width:  |  Height:  |  Size: 148 KiB

3
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/constants/Constants.java

@ -488,7 +488,8 @@ public final class Constants {
public static final String ALIAS = "alias"; public static final String ALIAS = "alias";
public static final String CONTENT = "content"; public static final String CONTENT = "content";
public static final String DEPENDENT_SPLIT = ":||"; public static final String DEPENDENT_SPLIT = ":||";
public static final long DEPENDENT_ALL_TASK_CODE = 0; public static final long DEPENDENT_ALL_TASK_CODE = -1;
public static final long DEPENDENT_WORKFLOW_CODE = 0;
/** /**
* preview schedule execute count * preview schedule execute count

19
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapper.java

@ -161,4 +161,23 @@ public interface TaskInstanceMapper extends BaseMapper<TaskInstance> {
void deleteByWorkflowInstanceId(@Param("workflowInstanceId") int workflowInstanceId); void deleteByWorkflowInstanceId(@Param("workflowInstanceId") int workflowInstanceId);
List<TaskInstance> findByWorkflowInstanceId(@Param("workflowInstanceId") Integer workflowInstanceId); List<TaskInstance> findByWorkflowInstanceId(@Param("workflowInstanceId") Integer workflowInstanceId);
/**
* find last task instance list in the date interval
*
* @param taskCodes taskCodes
* @param startTime startTime
* @param endTime endTime
* @param testFlag testFlag
* @return task instance list
*/
List<TaskInstance> findLastTaskInstances(@Param("taskCodes") Set<Long> taskCodes,
@Param("startTime") Date startTime,
@Param("endTime") Date endTime,
@Param("testFlag") int testFlag);
TaskInstance findLastTaskInstance(@Param("taskCode") long depTaskCode,
@Param("startTime") Date startTime,
@Param("endTime") Date endTime,
@Param("testFlag") int testFlag);
} }

7
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/TaskDefinitionDao.java

@ -49,4 +49,11 @@ public interface TaskDefinitionDao extends IDao<TaskDefinition> {
void deleteByTaskDefinitionCodes(Set<Long> needToDeleteTaskDefinitionCodes); void deleteByTaskDefinitionCodes(Set<Long> needToDeleteTaskDefinitionCodes);
List<TaskDefinition> queryByCodes(Collection<Long> taskDefinitionCodes); List<TaskDefinition> queryByCodes(Collection<Long> taskDefinitionCodes);
/**
* Query task definition by code
* @param taskCode task code
* @return task definition
*/
TaskDefinition queryByCode(long taskCode);
} }

22
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/TaskInstanceDao.java

@ -19,8 +19,10 @@ package org.apache.dolphinscheduler.dao.repository;
import org.apache.dolphinscheduler.dao.entity.ProcessInstance; import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
import org.apache.dolphinscheduler.dao.entity.TaskInstance; import org.apache.dolphinscheduler.dao.entity.TaskInstance;
import org.apache.dolphinscheduler.plugin.task.api.model.DateInterval;
import java.util.List; import java.util.List;
import java.util.Set;
/** /**
* Task Instance DAO * Task Instance DAO
@ -85,4 +87,24 @@ public interface TaskInstanceDao extends IDao<TaskInstance> {
void deleteByWorkflowInstanceId(int workflowInstanceId); void deleteByWorkflowInstanceId(int workflowInstanceId);
List<TaskInstance> queryByWorkflowInstanceId(Integer processInstanceId); List<TaskInstance> queryByWorkflowInstanceId(Integer processInstanceId);
/**
* find last task instance list corresponding to taskCodes in the date interval
*
* @param taskCodes taskCodes
* @param dateInterval dateInterval
* @param testFlag test flag
* @return task instance list
*/
List<TaskInstance> queryLastTaskInstanceListIntervalByTaskCodes(Set<Long> taskCodes, DateInterval dateInterval,
int testFlag);
/**
* find last task instance corresponding to taskCode in the date interval
* @param depTaskCode taskCode
* @param dateInterval dateInterval
* @param testFlag test flag
* @return task instance
*/
TaskInstance queryLastTaskInstanceIntervalByTaskCode(long depTaskCode, DateInterval dateInterval, int testFlag);
} }

5
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/TaskDefinitionDaoImpl.java

@ -114,4 +114,9 @@ public class TaskDefinitionDaoImpl extends BaseDao<TaskDefinition, TaskDefinitio
return mybatisMapper.queryByCodeList(taskDefinitionCodes); return mybatisMapper.queryByCodeList(taskDefinitionCodes);
} }
@Override
public TaskDefinition queryByCode(long taskCode) {
return mybatisMapper.queryByCode(taskCode);
}
} }

15
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/TaskInstanceDaoImpl.java

@ -27,11 +27,13 @@ import org.apache.dolphinscheduler.dao.mapper.TaskInstanceMapper;
import org.apache.dolphinscheduler.dao.repository.BaseDao; import org.apache.dolphinscheduler.dao.repository.BaseDao;
import org.apache.dolphinscheduler.dao.repository.TaskInstanceDao; import org.apache.dolphinscheduler.dao.repository.TaskInstanceDao;
import org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus; import org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus;
import org.apache.dolphinscheduler.plugin.task.api.model.DateInterval;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import java.util.Date; import java.util.Date;
import java.util.List; import java.util.List;
import java.util.Set;
import lombok.NonNull; import lombok.NonNull;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@ -171,4 +173,17 @@ public class TaskInstanceDaoImpl extends BaseDao<TaskInstance, TaskInstanceMappe
return mybatisMapper.findByWorkflowInstanceId(workflowInstanceId); return mybatisMapper.findByWorkflowInstanceId(workflowInstanceId);
} }
@Override
public List<TaskInstance> queryLastTaskInstanceListIntervalByTaskCodes(Set<Long> taskCodes,
DateInterval dateInterval, int testFlag) {
return mybatisMapper.findLastTaskInstances(taskCodes, dateInterval.getStartTime(), dateInterval.getEndTime(),
testFlag);
}
@Override
public TaskInstance queryLastTaskInstanceIntervalByTaskCode(long depTaskCode, DateInterval dateInterval,
int testFlag) {
return mybatisMapper.findLastTaskInstance(depTaskCode, dateInterval.getStartTime(), dateInterval.getEndTime(),
testFlag);
}
} }

35
dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapper.xml

@ -334,7 +334,40 @@
where instance.process_instance_id = #{processInstanceId} where instance.process_instance_id = #{processInstanceId}
and que.status = #{status} and que.status = #{status}
</select> </select>
<select id="findLastTaskInstances" resultType="org.apache.dolphinscheduler.dao.entity.TaskInstance">
select
<include refid="baseSqlV2">
<property name="alias" value="instance"/>
</include>
from t_ds_task_instance instance
join (
select task_code, max(end_time) as max_end_time
from t_ds_task_instance
where 1=1 and test_flag = #{testFlag}
<if test="taskCodes != null and taskCodes.size() != 0">
and task_code in
<foreach collection="taskCodes" index="index" item="i" open="(" separator="," close=")">
#{i}
</foreach>
</if>
<if test="startTime!=null and endTime != null">
and start_time <![CDATA[ >= ]]> #{startTime} and start_time <![CDATA[ <= ]]> #{endTime}
</if>
group by task_code
) t_max
on instance.task_code = t_max.task_code and instance.end_time = t_max.max_end_time
</select>
<select id="findLastTaskInstance" resultType="org.apache.dolphinscheduler.dao.entity.TaskInstance">
select
<include refid="baseSql"/>
from t_ds_task_instance
where task_code = #{taskCode}
<if test="startTime!=null and endTime != null">
and start_time <![CDATA[ >= ]]> #{startTime} and start_time <![CDATA[ <= ]]> #{endTime}
</if>
order by end_time desc limit 1
</select>
<delete id="deleteByWorkflowInstanceId"> <delete id="deleteByWorkflowInstanceId">
delete delete
from t_ds_task_instance from t_ds_task_instance

6
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/dependent/DependentAsyncTaskExecuteFunction.java

@ -157,6 +157,12 @@ public class DependentAsyncTaskExecuteFunction implements AsyncTaskExecuteFuncti
log.info("WorkflowName: {}", processDefinition.getName()); log.info("WorkflowName: {}", processDefinition.getName());
log.info("TaskName: {}", "ALL"); log.info("TaskName: {}", "ALL");
log.info("DependentKey: {}", dependentItem.getKey()); log.info("DependentKey: {}", dependentItem.getKey());
} else if (dependentItem.getDepTaskCode() == Constants.DEPENDENT_WORKFLOW_CODE) {
log.info("Add dependent task:");
log.info("DependentRelation: {}", dependentTaskModel.getRelation());
log.info("ProjectName: {}", project.getName());
log.info("WorkflowName: {}", processDefinition.getName());
log.info("DependentKey: {}", dependentItem.getKey());
} else { } else {
TaskDefinition taskDefinition = taskDefinitionMap.get(dependentItem.getDepTaskCode()); TaskDefinition taskDefinition = taskDefinitionMap.get(dependentItem.getDepTaskCode());
if (taskDefinition == null) { if (taskDefinition == null) {

138
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/utils/DependentExecute.java

@ -20,9 +20,16 @@ package org.apache.dolphinscheduler.server.master.utils;
import static org.apache.dolphinscheduler.plugin.task.api.parameters.DependentParameters.DependentFailurePolicyEnum.DEPENDENT_FAILURE_WAITING; import static org.apache.dolphinscheduler.plugin.task.api.parameters.DependentParameters.DependentFailurePolicyEnum.DEPENDENT_FAILURE_WAITING;
import org.apache.dolphinscheduler.common.constants.Constants; import org.apache.dolphinscheduler.common.constants.Constants;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.common.enums.TaskExecuteType;
import org.apache.dolphinscheduler.dao.entity.ProcessInstance; import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
import org.apache.dolphinscheduler.dao.entity.ProcessTaskRelation;
import org.apache.dolphinscheduler.dao.entity.TaskDefinition;
import org.apache.dolphinscheduler.dao.entity.TaskDefinitionLog;
import org.apache.dolphinscheduler.dao.entity.TaskInstance; import org.apache.dolphinscheduler.dao.entity.TaskInstance;
import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao; import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
import org.apache.dolphinscheduler.dao.repository.TaskDefinitionDao;
import org.apache.dolphinscheduler.dao.repository.TaskDefinitionLogDao;
import org.apache.dolphinscheduler.dao.repository.TaskInstanceDao; import org.apache.dolphinscheduler.dao.repository.TaskInstanceDao;
import org.apache.dolphinscheduler.plugin.task.api.enums.DependResult; import org.apache.dolphinscheduler.plugin.task.api.enums.DependResult;
import org.apache.dolphinscheduler.plugin.task.api.enums.DependentRelation; import org.apache.dolphinscheduler.plugin.task.api.enums.DependentRelation;
@ -32,6 +39,7 @@ import org.apache.dolphinscheduler.plugin.task.api.model.DependentItem;
import org.apache.dolphinscheduler.plugin.task.api.parameters.DependentParameters; import org.apache.dolphinscheduler.plugin.task.api.parameters.DependentParameters;
import org.apache.dolphinscheduler.plugin.task.api.utils.DependentUtils; import org.apache.dolphinscheduler.plugin.task.api.utils.DependentUtils;
import org.apache.dolphinscheduler.service.bean.SpringApplicationContext; import org.apache.dolphinscheduler.service.bean.SpringApplicationContext;
import org.apache.dolphinscheduler.service.process.ProcessService;
import java.time.Duration; import java.time.Duration;
import java.time.Instant; import java.time.Instant;
@ -40,6 +48,7 @@ import java.util.Date;
import java.util.HashMap; import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.stream.Collectors;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@ -72,6 +81,22 @@ public class DependentExecute {
*/ */
private Map<String, DependResult> dependResultMap = new HashMap<>(); private Map<String, DependResult> dependResultMap = new HashMap<>();
/**
* process service
*/
private final ProcessService processService = SpringApplicationContext.getBean(ProcessService.class);
/**
* task definition log dao
*/
private final TaskDefinitionLogDao taskDefinitionLogDao =
SpringApplicationContext.getBean(TaskDefinitionLogDao.class);
/**
* task definition dao
*/
private final TaskDefinitionDao taskDefinitionDao = SpringApplicationContext.getBean(TaskDefinitionDao.class);
/** /**
* constructor * constructor
* *
@ -118,10 +143,13 @@ public class DependentExecute {
return DependResult.WAITING; return DependResult.WAITING;
} }
// need to check workflow for updates, so get all task and check the task state // need to check workflow for updates, so get all task and check the task state
if (dependentItem.getDepTaskCode() == Constants.DEPENDENT_ALL_TASK_CODE) { if (dependentItem.getDepTaskCode() == Constants.DEPENDENT_WORKFLOW_CODE) {
result = dependResultByProcessInstance(processInstance); result = dependResultByProcessInstance(processInstance);
} else if (dependentItem.getDepTaskCode() == Constants.DEPENDENT_ALL_TASK_CODE) {
result = dependResultByAllTaskOfProcessInstance(processInstance, dateInterval, testFlag);
} else { } else {
result = getDependTaskResult(dependentItem.getDepTaskCode(), processInstance, testFlag); result = dependResultBySingleTaskInstance(processInstance, dependentItem.getDepTaskCode(), dateInterval,
testFlag);
} }
if (result != DependResult.SUCCESS) { if (result != DependResult.SUCCESS) {
break; break;
@ -131,7 +159,7 @@ public class DependentExecute {
} }
/** /**
* depend type = depend_all * depend type = depend_work_flow
* *
* @return * @return
*/ */
@ -142,6 +170,59 @@ public class DependentExecute {
if (processInstance.getState().isSuccess()) { if (processInstance.getState().isSuccess()) {
return DependResult.SUCCESS; return DependResult.SUCCESS;
} }
log.warn(
"The dependent workflow did not execute successfully, so return depend failed. processCode: {}, processName: {}",
processInstance.getProcessDefinitionCode(), processInstance.getName());
return DependResult.FAILED;
}
/**
* depend type = depend_all
*
* @return
*/
private DependResult dependResultByAllTaskOfProcessInstance(ProcessInstance processInstance,
DateInterval dateInterval, int testFlag) {
if (!processInstance.getState().isFinished()) {
log.info("Wait for the dependent workflow to complete, processCode: {}, processInstanceId: {}.",
processInstance.getProcessDefinitionCode(), processInstance.getId());
return DependResult.WAITING;
}
if (processInstance.getState().isSuccess()) {
List<ProcessTaskRelation> processTaskRelations =
processService.findRelationByCode(processInstance.getProcessDefinitionCode(),
processInstance.getProcessDefinitionVersion());
List<TaskDefinitionLog> taskDefinitionLogs =
taskDefinitionLogDao.queryTaskDefineLogList(processTaskRelations);
Map<Long, String> taskDefinitionCodeMap =
taskDefinitionLogs.stream().filter(taskDefinitionLog -> taskDefinitionLog.getFlag() == Flag.YES)
.collect(Collectors.toMap(TaskDefinitionLog::getCode, TaskDefinitionLog::getName));
List<TaskInstance> taskInstanceList =
taskInstanceDao.queryLastTaskInstanceListIntervalByTaskCodes(taskDefinitionCodeMap.keySet(),
dateInterval, testFlag);
Map<Long, TaskExecutionStatus> taskExecutionStatusMap =
taskInstanceList.stream()
.filter(taskInstance -> taskInstance.getTaskExecuteType() != TaskExecuteType.STREAM)
.collect(Collectors.toMap(TaskInstance::getTaskCode, TaskInstance::getState));
for (Long taskCode : taskDefinitionCodeMap.keySet()) {
if (!taskExecutionStatusMap.containsKey(taskCode)) {
log.warn(
"The task of the workflow is not being executed, taskCode: {}, processInstanceId: {}, processName: {}.",
taskCode, processInstance.getProcessDefinitionCode(), processInstance.getName());
return DependResult.FAILED;
} else {
if (!taskExecutionStatusMap.get(taskCode).isSuccess()) {
log.warn(
"The task of the workflow is not being executed successfully, taskCode: {}, processInstanceId: {}, processName: {}.",
taskCode, processInstance.getProcessDefinitionCode(), processInstance.getName());
return DependResult.FAILED;
}
}
}
return DependResult.SUCCESS;
}
return DependResult.FAILED; return DependResult.FAILED;
} }
@ -180,6 +261,54 @@ public class DependentExecute {
return result; return result;
} }
/**
* depend type = depend_task
*
* @param processInstance last process instance in the date interval
* @param depTaskCode the dependent task code
* @param dateInterval date interval
* @param testFlag test flag
* @return depend result
*/
private DependResult dependResultBySingleTaskInstance(ProcessInstance processInstance, long depTaskCode,
DateInterval dateInterval, int testFlag) {
TaskInstance taskInstance =
taskInstanceDao.queryLastTaskInstanceIntervalByTaskCode(depTaskCode, dateInterval, testFlag);
if (taskInstance == null) {
TaskDefinition taskDefinition = taskDefinitionDao.queryByCode(depTaskCode);
if (taskDefinition == null) {
log.error("The dependent task definition can not be find, so return depend failed, taskCode: {}",
depTaskCode);
return DependResult.FAILED;
}
if (taskDefinition.getFlag() == Flag.NO) {
log.info(
"The dependent task is a forbidden task, so return depend success. Task code: {}, task name: {}",
taskDefinition.getCode(), taskDefinition.getName());
return DependResult.SUCCESS;
}
if (!processInstance.getState().isFinished()) {
log.info("Wait for the dependent workflow to complete, processCode: {}, processInstanceId: {}.",
processInstance.getProcessDefinitionCode(), processInstance.getId());
return DependResult.WAITING;
}
return DependResult.FAILED;
} else {
if (TaskExecuteType.STREAM == taskInstance.getTaskExecuteType()) {
log.info(
"The dependent task is a streaming task, so return depend success. Task code: {}, task name: {}.",
taskInstance.getTaskCode(), taskInstance.getName());
return DependResult.SUCCESS;
}
return getDependResultByState(taskInstance.getState());
}
}
/** /**
* find the last one process instance that : * find the last one process instance that :
* 1. manual run and finish between the interval * 1. manual run and finish between the interval
@ -221,6 +350,9 @@ public class DependentExecute {
} else if (state.isSuccess()) { } else if (state.isSuccess()) {
return DependResult.SUCCESS; return DependResult.SUCCESS;
} else { } else {
log.warn(
"The dependent task were not executed successfully, so return depend failed. Task code: {}, task name: {}.",
taskInstance.getTaskCode(), taskInstance.getName());
return DependResult.FAILED; return DependResult.FAILED;
} }
} }

4
dolphinscheduler-ui/src/locales/en_US/project.ts

@ -876,6 +876,10 @@ export default {
child_node_instance: 'child node instance', child_node_instance: 'child node instance',
yarn_queue: 'Yarn Queue', yarn_queue: 'Yarn Queue',
yarn_queue_tips: 'Please input yarn queue(optional)', yarn_queue_tips: 'Please input yarn queue(optional)',
dependent_type: 'Dependency Type',
dependent_on_workflow: 'Dependent on workflow',
dependent_on_task: 'Dependent on task',
}, },
menu: { menu: {
fav: 'Favorites', fav: 'Favorites',

3
dolphinscheduler-ui/src/locales/zh_CN/project.ts

@ -850,6 +850,9 @@ export default {
child_node_instance: '子节点实例', child_node_instance: '子节点实例',
yarn_queue: 'Yarn队列', yarn_queue: 'Yarn队列',
yarn_queue_tips: '请输入Yarn队列(选填)', yarn_queue_tips: '请输入Yarn队列(选填)',
dependent_type: '依赖类型',
dependent_on_workflow: '依赖于工作流',
dependent_on_task: '依赖于任务',
}, },
menu: { menu: {
fav: '收藏组件', fav: '收藏组件',

42
dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-dependent.ts

@ -69,6 +69,17 @@ export function useDependent(model: { [field: string]: any }): IJsonItem[] {
} }
const selectOptions = ref([] as IDependTaskOptions[]) const selectOptions = ref([] as IDependTaskOptions[])
const DependentTypeOptions = [
{
value: 'DEPENDENT_ON_WORKFLOW',
label: t('project.node.dependent_on_workflow')
},
{
value: 'DEPENDENT_ON_TASK',
label: t('project.node.dependent_on_task')
}
]
const CYCLE_LIST = [ const CYCLE_LIST = [
{ {
value: 'month', value: 'month',
@ -229,7 +240,7 @@ export function useDependent(model: { [field: string]: any }): IJsonItem[] {
filterLabel: item.name filterLabel: item.name
})) }))
taskList.unshift({ taskList.unshift({
value: 0, value: -1,
label: 'ALL' label: 'ALL'
}) })
taskCache[processCode] = taskList taskCache[processCode] = taskList
@ -266,6 +277,13 @@ export function useDependent(model: { [field: string]: any }): IJsonItem[] {
item.dependItemList?.forEach( item.dependItemList?.forEach(
async (dependItem: IDependentItem, itemIndex: number) => { async (dependItem: IDependentItem, itemIndex: number) => {
itemListOptions.value[itemIndex] = {} itemListOptions.value[itemIndex] = {}
if (!dependItem.dependentType) {
if (dependItem.depTaskCode == 0)
dependItem.dependentType = 'DEPENDENT_ON_WORKFLOW'
else
dependItem.dependentType = 'DEPENDENT_ON_TASK'
}
if (dependItem.projectCode) { if (dependItem.projectCode) {
itemListOptions.value[itemIndex].definitionCodeOptions = itemListOptions.value[itemIndex].definitionCodeOptions =
await getProcessList(dependItem.projectCode) await getProcessList(dependItem.projectCode)
@ -298,6 +316,21 @@ export function useDependent(model: { [field: string]: any }): IJsonItem[] {
field: 'dependItemList', field: 'dependItemList',
span: 18, span: 18,
children: [ children: [
(j = 0) => ({
type: 'select',
field: 'dependentType',
name: t('project.node.dependent_type'),
span: 24,
props: {
onUpdateValue: (dependentType: string) => {
const item = model.dependTaskList[i].dependItemList[j]
if (item.definitionCode)
item.depTaskCode = dependentType === 'DEPENDENT_ON_WORKFLOW' ? 0 : -1
}
},
options: DependentTypeOptions,
value: 'DEPENDENT_ON_WORKFLOW'
}),
(j = 0) => ({ (j = 0) => ({
type: 'select', type: 'select',
field: 'projectCode', field: 'projectCode',
@ -353,7 +386,7 @@ export function useDependent(model: { [field: string]: any }): IJsonItem[] {
const item = model.dependTaskList[i].dependItemList[j] const item = model.dependTaskList[i].dependItemList[j]
selectOptions.value[i].dependItemList[j].depTaskCodeOptions = selectOptions.value[i].dependItemList[j].depTaskCodeOptions =
await getTaskList(item.projectCode, processCode) await getTaskList(item.projectCode, processCode)
item.depTaskCode = 0 item.depTaskCode = item.dependentType === 'DEPENDENT_ON_WORKFLOW' ? 0 : -1
} }
}, },
options: options:
@ -373,7 +406,10 @@ export function useDependent(model: { [field: string]: any }): IJsonItem[] {
(j = 0) => ({ (j = 0) => ({
type: 'select', type: 'select',
field: 'depTaskCode', field: 'depTaskCode',
span: 24, span: computed(() => {
const item = model.dependTaskList[i].dependItemList[j]
return item.dependentType === 'DEPENDENT_ON_WORKFLOW' ? 0 : 24
}),
name: t('project.node.task_name'), name: t('project.node.task_name'),
props: { props: {
filterable: true, filterable: true,

2
dolphinscheduler-ui/src/views/projects/task/components/node/types.ts

@ -83,6 +83,7 @@ interface IResponseJsonItem extends Omit<IJsonItemParams, 'type'> {
} }
interface IDependentItemOptions { interface IDependentItemOptions {
dependentTypeOptions?: IOption[]
definitionCodeOptions?: IOption[] definitionCodeOptions?: IOption[]
depTaskCodeOptions?: IOption[] depTaskCodeOptions?: IOption[]
dateOptions?: IOption[] dateOptions?: IOption[]
@ -99,6 +100,7 @@ interface IDependentItem {
definitionCode?: number definitionCode?: number
cycle?: 'month' | 'week' | 'day' | 'hour' cycle?: 'month' | 'week' | 'day' | 'hour'
dateValue?: string dateValue?: string
dependentType?: 'DEPENDENT_ON_WORKFLOW' | 'DEPENDENT_ON_TASK'
} }
interface IDependTask { interface IDependTask {

Loading…
Cancel
Save