diff --git a/docs/docs/en/guide/parameter/context.md b/docs/docs/en/guide/parameter/context.md index 1a48c211ef..362f0fa607 100644 --- a/docs/docs/en/guide/parameter/context.md +++ b/docs/docs/en/guide/parameter/context.md @@ -78,7 +78,7 @@ The result of Node_mysql is as follows: ![context-log03](../../../../img/new_ui/dev/parameter/context_log03.png) -Even though output is assigned a value of 1 in Node_A's script, the log still shows a value of 100. But according to the principle from [parameter priority](priority.md): `Local Parameter > Parameter Context > Global Parameter`, the output value in Node_B is 1. It proves that the output parameter is passed in the workflow with reference to the expected value, and the query operation is completed using this value in Node_mysql. +Even though output is assigned a value of 1 in Node_A's script, the log still shows a value of 100. But according to the principle from [parameter priority](priority.md): `Startup Parameter > Local Parameter > Parameter Context > Global Parameter`, the output value in Node_B is 1. It proves that the output parameter is passed in the workflow with reference to the expected value, and the query operation is completed using this value in Node_mysql. But the output value 66 only shows in the Node_A, the reason is that the direction of value is selected as IN, and only when the direction is OUT will it be defined as a variable output. diff --git a/docs/docs/en/guide/parameter/priority.md b/docs/docs/en/guide/parameter/priority.md index 0586c9f385..95e36bf258 100644 --- a/docs/docs/en/guide/parameter/priority.md +++ b/docs/docs/en/guide/parameter/priority.md @@ -3,12 +3,13 @@ DolphinScheduler has three parameter types: * [Global Parameter](global.md): parameters defined at the workflow define page. +* [Startup Parameter](startup-parameter.md): parameters defined at the workflow launch page. * [Parameter Context](context.md): parameters passed by upstream task nodes. * [Local Parameter](local.md): parameters belong to its node, which is the parameters defined by the user in [Custom Parameters]. The user can define part of the parameters when creating workflow definitions. -As there are multiple sources of the parameter value, it will raise parameter priority issues when the parameter name is the same. The priority of DolphinScheduler parameters from high to low is: `Local Parameter > Parameter Context > Global Parameter`. +As there are multiple sources of the parameter value, it will raise parameter priority issues when the parameter name is the same. The priority of DolphinScheduler parameters from high to low is: `Startup Parameter > Local Parameter > Parameter Context > Global Parameter`. In the case of upstream tasks can pass parameters to the downstream, there may be multiple tasks upstream that pass the same parameter name: diff --git a/docs/docs/en/guide/parameter/startup-parameter.md b/docs/docs/en/guide/parameter/startup-parameter.md new file mode 100644 index 0000000000..bf3acc2034 --- /dev/null +++ b/docs/docs/en/guide/parameter/startup-parameter.md @@ -0,0 +1,43 @@ +# Startup Parameter + +## Scope + +Parameters are valid for all task nodes of the entire workflow. It can be configured on the task launch page. + +## Usage + +Usage of startup parameters is: at the task launch page, click the '+' below the 'Startup Parameter' and fill in the key and value to save. The workflow will add them into global parameters. + +## Example + +This example shows how to use startup parameters to print different date. + +### Create a Shell task + +Create a shell task and enter `echo ${dt}` in the script content. In this case, dt is the global parameter we need to declare. As shown below: + +![startup-parameter01](../../../../img/new_ui/dev/parameter/startup_parameter01.png) + +### Save the workflow and set startup parameters in task launch page + +Set startup parameter as follows: + +![startup-parameter02](../../../../img/new_ui/dev/parameter/startup_parameter02.png) + +> Note: The dt parameter defined here can be referenced by the local parameters of any other node. + +### In task instance view execution result + +On the task instance page, you can check the log to verify the execution result of the task and determine whether the parameters are valid. + +![startup-parameter03](../../../../img/new_ui/dev/parameter/startup_parameter03.png) + +### Set different startup parameter and run again + +![startup-parameter04](../../../../img/new_ui/dev/parameter/startup_parameter04.png) + +### In task instance view execution result + +You can check the log to verify whether the shell task output different dates. + +![startup-parameter05](../../../../img/new_ui/dev/parameter/startup_parameter05.png) diff --git a/docs/docs/zh/guide/parameter/context.md b/docs/docs/zh/guide/parameter/context.md index a80044b1b3..f2879f7412 100644 --- a/docs/docs/zh/guide/parameter/context.md +++ b/docs/docs/zh/guide/parameter/context.md @@ -78,7 +78,7 @@ Node_mysql 运行结果如下： ![context-log03](../../../../img/new_ui/dev/parameter/context_log03.png) -虽然在 Node_A 的脚本中为 output 赋值为 1，但日志中显示的值仍然为 100。但根据[参数优先级](priority.md)的原则：`本地参数 > 上游任务传递的参数 > 全局参数`，在 Node_B 中输出的值为 1。则证明 output 参数参照预期的值在该工作流中传递，并在 Node_mysql 中使用该值完成查询操作。 +虽然在 Node_A 的脚本中为 output 赋值为 1，但日志中显示的值仍然为 100。但根据[参数优先级](priority.md)的原则：`启动参数 > 本地参数 > 上游任务传递的参数 > 全局参数`，在 Node_B 中输出的值为 1。则证明 output 参数参照预期的值在该工作流中传递，并在 Node_mysql 中使用该值完成查询操作。 但是 value 的值却只有在 Node_A 中输出为 66，其原因为 value 的方向选择为 IN，只有当方向为 OUT 时才会被定义为变量输出。 diff --git a/docs/docs/zh/guide/parameter/priority.md b/docs/docs/zh/guide/parameter/priority.md index a9d779b2a5..8a10197003 100644 --- a/docs/docs/zh/guide/parameter/priority.md +++ b/docs/docs/zh/guide/parameter/priority.md @@ -3,10 +3,11 @@ DolphinScheduler 中所涉及的参数值的定义可能来自三种类型： * [全局参数](global.md)：在工作流保存页面定义时定义的变量 +* [启动参数](startup-parameter.md)：在工作流启动页面定义的变量 * [上游任务传递的参数](context.md)：上游任务传递过来的参数 * [本地参数](local.md)：节点的自有变量，用户在“自定义参数”定义的变量，并且用户可以在工作流定义时定义该部分变量的值 -因为参数的值存在多个来源，当参数名相同时，就需要会存在参数优先级的问题。DolphinScheduler 参数的优先级从高到低为：`本地参数 > 上游任务传递的参数 > 全局参数` +因为参数的值存在多个来源，当参数名相同时，就需要会存在参数优先级的问题。DolphinScheduler 参数的优先级从高到低为：`启动参数 > 本地参数 > 上游任务传递的参数 > 全局参数` 在上游任务传递的参数中，由于上游可能存在多个任务向下游传递参数，当上游传递的参数名称相同时： diff --git a/docs/docs/zh/guide/parameter/startup-parameter.md b/docs/docs/zh/guide/parameter/startup-parameter.md new file mode 100644 index 0000000000..7da65657a7 --- /dev/null +++ b/docs/docs/zh/guide/parameter/startup-parameter.md @@ -0,0 +1,43 @@ +# 启动参数 + +## 作用域 + +启动参数是针对**整个工作流**的所有任务节点都有效的参数，在工作流启动页面配置。 + +## 使用方式 + +启动参数配置方式如下：在启动前参数设置界面，点击“启动参数“下面的加号，填写对应的参数名称和对应的值，点击确定，工作流会将启动参数加入全局参数中。 + +## 任务样例 + +本样例展示了如何使用启动参数，打印输出不同天的日期。 + +### 创建 Shell 任务 + +创建一个 Shell 任务，并在脚本内容中输入 `echo ${dt}`。此时 dt 则为我们需要声明的启动参数。如下图所示： + +![startup-parameter01](../../../../img/new_ui/dev/parameter/startup_parameter01.png) + +### 保存工作流，上线运行并设置启动参数 + +启动参数配置如下图所示： + +![startup-parameter02](../../../../img/new_ui/dev/parameter/startup_parameter02.png) + +> 注：这里定义的 dt 参数可以被其它任一节点的局部参数引用。 + +### 任务实例查看执行结果 + +进入任务实例页面，可以通过查看日志，验证任务的执行结果，判断参数是否有效。 + +![startup-parameter03](../../../../img/new_ui/dev/parameter/startup_parameter03.png) + +### 修改启动参数，再次执行 + +![startup-parameter04](../../../../img/new_ui/dev/parameter/startup_parameter04.png) + +### 任务实例查看执行结果 + +验证任务执行结果，判断Shell任务是否输出了不同日期。 + +![startup-parameter05](../../../../img/new_ui/dev/parameter/startup_parameter05.png) diff --git a/docs/img/new_ui/dev/parameter/startup_parameter01.png b/docs/img/new_ui/dev/parameter/startup_parameter01.png new file mode 100644 index 0000000000..191834d5a2 Binary files /dev/null and b/docs/img/new_ui/dev/parameter/startup_parameter01.png differ diff --git a/docs/img/new_ui/dev/parameter/startup_parameter02.png b/docs/img/new_ui/dev/parameter/startup_parameter02.png new file mode 100644 index 0000000000..cbeb96b658 Binary files /dev/null and b/docs/img/new_ui/dev/parameter/startup_parameter02.png differ diff --git a/docs/img/new_ui/dev/parameter/startup_parameter03.png b/docs/img/new_ui/dev/parameter/startup_parameter03.png new file mode 100644 index 0000000000..3ba2ceda70 Binary files /dev/null and b/docs/img/new_ui/dev/parameter/startup_parameter03.png differ diff --git a/docs/img/new_ui/dev/parameter/startup_parameter04.png b/docs/img/new_ui/dev/parameter/startup_parameter04.png new file mode 100644 index 0000000000..e32afb4b31 Binary files /dev/null and b/docs/img/new_ui/dev/parameter/startup_parameter04.png differ diff --git a/docs/img/new_ui/dev/parameter/startup_parameter05.png b/docs/img/new_ui/dev/parameter/startup_parameter05.png new file mode 100644 index 0000000000..9d663892d2 Binary files /dev/null and b/docs/img/new_ui/dev/parameter/startup_parameter05.png differ diff --git a/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java index f3e8bf6714..8878fc3c2e 100644 --- a/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java +++ b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParams.java @@ -103,13 +103,11 @@ public class CuringGlobalParams implements CuringParamsService { Map resolveMap = new HashMap<>(); for (Map.Entry entry : entries) { String val = entry.getValue(); - if (val.startsWith(Constants.FUNCTION_START_WITH)) { - String str = ""; + if (val.contains(Constants.FUNCTION_START_WITH)) { + String str = val; // whether external scaling calculation is required if (timeFunctionNeedExpand(val)) { str = timeFunctionExtension(processInstanceId, timezone, val); - } else { - str = convertParameterPlaceholders(val, allParamMap); } resolveMap.put(entry.getKey(), str); } @@ -174,14 +172,16 @@ public class CuringGlobalParams implements CuringParamsService { if (MapUtils.isNotEmpty(localParams)) { globalParams.putAll(localParams); } - + if (MapUtils.isNotEmpty(cmdParam)) { + globalParams.putAll(ParamUtils.getUserDefParamsMap(cmdParam)); + } Iterator> iter = globalParams.entrySet().iterator(); while (iter.hasNext()) { Map.Entry en = iter.next(); Property property = en.getValue(); if (StringUtils.isNotEmpty(property.getValue()) - && property.getValue().startsWith(Constants.FUNCTION_START_WITH)) { + && property.getValue().contains(Constants.FUNCTION_START_WITH)) { /** * local parameter refers to global parameter with the same name * note: the global parameters of the process instance here are solidified parameters, @@ -191,8 +191,6 @@ public class CuringGlobalParams implements CuringParamsService { // whether external scaling calculation is required if (timeFunctionNeedExpand(val)) { val = timeFunctionExtension(taskInstance.getProcessInstanceId(), timeZone, val); - } else { - val = convertParameterPlaceholders(val, params); } property.setValue(val); } diff --git a/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParamsServiceTest.java b/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParamsServiceTest.java index 464e1ea8d9..00476da32e 100644 --- a/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParamsServiceTest.java +++ b/dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/expand/CuringGlobalParamsServiceTest.java @@ -156,6 +156,6 @@ public class CuringGlobalParamsServiceTest { String result6 = dolphinSchedulerCuringGlobalParams.curingGlobalParams(1, globalParamMap, globalParamList, CommandType.START_CURRENT_TASK_PROCESS, scheduleTime, null); - Assertions.assertTrue(result6.contains("20191220")); + Assertions.assertEquals(result6, JSONUtils.toJsonString(globalParamList)); } }