Browse Source

[Feature-16269][dinky-task]Supports dinky tasks to accept Prepare Params passed within a task or from a previous task (#15947)

* fix dinky unresolve params

* Supplement the Dinky documentation
dev
gaoyan 4 months ago committed by GitHub
parent
commit
82b501aa8e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
  1. 11
      docs/docs/en/guide/task/dinky.md
  2. 11
      docs/docs/zh/guide/task/dinky.md
  3. BIN
      docs/img/tasks/demo/dinky_task_id.png
  4. 8
      dolphinscheduler-task-plugin/dolphinscheduler-task-dinky/src/main/java/org/apache/dolphinscheduler/plugin/task/dinky/DinkyTask.java

11
docs/docs/en/guide/task/dinky.md

@ -17,11 +17,12 @@ it will call `Dinky API` to trigger dinky task. Click [here](http://www.dlink.to
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters. - Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** | | **Parameter** | **Description** |
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Dinky Address | The url for a dinky server. | | Dinky Address | The URL for the Dinky service, e.g., http://localhost:8888. |
| Dinky Task ID | The unique task id for a dinky task. | | Dinky Task ID | The unique task id for a dinky task. |
| Online Task | Specify whether the current dinky job is online. If yes, the submitted job can only be submitted successfully when it is published and there is no corresponding Flink job instance running. | | Online Task | Specify whether the current dinky job is online. If yes, the submitted job can only be submitted successfully when it is published and there is no corresponding Flink job instance running. |
| Custom Parameters | Starting from Dinky 1.0, support for passing custom parameters is available. Currently, only `IN` type inputs are supported, with no support for `OUT` type outputs. Supports the `${param}` syntax for retrieving global or local dynamic parameters. |
## Task Example ## Task Example

11
docs/docs/zh/guide/task/dinky.md

@ -17,11 +17,12 @@
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。 - 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** | | **任务参数** | **描述** |
|-------------|---------------------------------------------------------------------| |-------------|----------------------------------------------------------------------------|
| Dinky 地址 | Dinky 服务的 url。 | | Dinky 地址 | Dinky 服务的 url,例如:`http://localhost:8888`。 |
| Dinky 任务 ID | Dinky 作业对应的唯一ID。 | | Dinky 任务 ID | Dinky 作业对应的唯一ID。 |
| 上线作业 | 指定当前 Dinky 作业是否上线,如果是,则该被提交的作业只能处于已发布且当前无对应的 Flink Job 实例在运行才可提交成功。 | | 上线作业 | 指定当前 Dinky 作业是否上线,如果是,则该被提交的作业只能处于已发布且当前无对应的 Flink Job 实例在运行才可提交成功。 |
| 自定义参数 | 从Dinky 1.0开始支持传递自定义参数,目前仅支持`IN`类型输入,不支持`OUT`类型输出。支持`${param}`方式获取全局或局部动态参数 |
## Task Example ## Task Example

BIN
docs/img/tasks/demo/dinky_task_id.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 216 KiB

After

Width:  |  Height:  |  Size: 357 KiB

8
dolphinscheduler-task-plugin/dolphinscheduler-task-dinky/src/main/java/org/apache/dolphinscheduler/plugin/task/dinky/DinkyTask.java

@ -27,6 +27,8 @@ import org.apache.dolphinscheduler.plugin.task.api.TaskException;
import org.apache.dolphinscheduler.plugin.task.api.TaskExecutionContext; import org.apache.dolphinscheduler.plugin.task.api.TaskExecutionContext;
import org.apache.dolphinscheduler.plugin.task.api.model.Property; import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters; import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
import org.apache.dolphinscheduler.plugin.task.api.parser.PlaceholderUtils;
import org.apache.dolphinscheduler.plugin.task.api.utils.ParameterUtils;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpResponse; import org.apache.http.HttpResponse;
@ -343,11 +345,15 @@ public class DinkyTask extends AbstractRemoteTask {
} }
} }
List<Property> localParams = this.dinkyParameters.getLocalParams(); List<Property> localParams = this.dinkyParameters.getLocalParams();
Map<String, Property> prepareParamsMap = taskExecutionContext.getPrepareParamsMap();
if (localParams == null || localParams.isEmpty()) { if (localParams == null || localParams.isEmpty()) {
return variables; return variables;
} }
Map<String, String> convertMap = ParameterUtils.convert(prepareParamsMap);
for (Property property : localParams) { for (Property property : localParams) {
variables.put(property.getProp(), property.getValue()); String propertyValue = property.getValue();
String value = PlaceholderUtils.replacePlaceholders(propertyValue, convertMap, true);
variables.put(property.getProp(), value);
} }
return variables; return variables;
} }

Loading…
Cancel
Save