Kerwin
2 years ago
committed by
GitHub
25 changed files with 861 additions and 179 deletions
@ -0,0 +1,82 @@
|
||||
# Apache SeaTunnel |
||||
|
||||
## Overview |
||||
|
||||
`SeaTunnel` task type for creating and executing `SeaTunnel` tasks. When the worker executes this task, it will parse the config file through the `start-seatunnel-spark.sh` or `start-seatunnel-flink.sh` command. |
||||
Click [here](https://seatunnel.apache.org/) for more information about `Apache SeaTunnel`. |
||||
|
||||
## Create Task |
||||
|
||||
- Click Project Management -> Project Name -> Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page. |
||||
- Drag the <img src="../../../../img/tasks/icons/seatunnel.png" width="15"/> from the toolbar to the drawing board. |
||||
|
||||
## Task Parameter |
||||
|
||||
- Node name: The node name in a workflow definition is unique. |
||||
- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch. |
||||
- Descriptive information: describe the function of the node. |
||||
- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle. |
||||
- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution. |
||||
- Environment Name: Configure the environment name in which to run the script. |
||||
- Number of failed retry attempts: The number of times the task failed to be resubmitted. |
||||
- Failed retry interval: The time, in cents, interval for resubmitting the task after a failed task. |
||||
- Cpu quota: Assign the specified CPU time quota to the task executed. Takes a percentage value. Default -1 means unlimited. For example, the full CPU load of one core is 100%,and that of 16 cores is 1600%. This function is controlled by [task.resource.limit.state](../../architecture/configuration.md) |
||||
- Max memory：Assign the specified max memory to the task executed. Exceeding this limit will trigger oom to be killed and will not automatically retry. Takes an MB value. Default -1 means unlimited. This function is controlled by [task.resource.limit.state](../../architecture/configuration.md) |
||||
- Delayed execution time: The time, in cents, that a task is delayed in execution. |
||||
- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail. |
||||
- Engine: Supports FLINK and SPARK |
||||
- FLINK |
||||
- Run model: supports `run` and `run-application` modes |
||||
- Option parameters: used to add the parameters of the Flink engine, such as `-m yarn-cluster -ynm seatunnel` |
||||
- SPARK |
||||
- Deployment mode: specify the deployment mode, `cluster` `client` `local` |
||||
- Master: Specify the `Master` model, `yarn` `local` `spark` `mesos`, where `spark` and `mesos` need to specify the `Master` service address, for example: 127.0.0.1:7077 |
||||
> Click [here](https://seatunnel.apache.org/docs/2.1.2/command/usage) for more information on the usage of `Apache SeaTunnel command` |
||||
- Custom Configuration: Supports custom configuration or select configuration file from Resource Center |
||||
> Click [here](https://seatunnel.apache.org/docs/2.1.2/concept/config) for more information about `Apache SeaTunnel config` file |
||||
- Script: Customize configuration information on the task node, including four parts: `env` `source` `transform` `sink` |
||||
- Resource file: The configuration file of the resource center can be referenced in the task node, and only one configuration file can be referenced. |
||||
- Predecessor task: Selecting a predecessor task for the current task will set the selected predecessor task as upstream of the current task. |
||||
|
||||
## Task Example |
||||
|
||||
This sample demonstrates using the Flink engine to read data from a Fake source and print to the console. |
||||
|
||||
### Configuring the SeaTunnel environment in DolphinScheduler |
||||
|
||||
If you want to use the SeaTunnel task type in the production environment, you need to configure the required environment first. The configuration file is as follows: `/dolphinscheduler/conf/env/dolphinscheduler_env.sh`. |
||||
|
||||
 |
||||
|
||||
### Configuring SeaTunnel Task Node |
||||
|
||||
According to the above parameter description, configure the required content. |
||||
|
||||
 |
||||
|
||||
### Config example |
||||
|
||||
```Config |
||||
|
||||
env { |
||||
execution.parallelism = 1 |
||||
} |
||||
|
||||
source { |
||||
FakeSource { |
||||
result_table_name = "fake" |
||||
field_name = "name,age" |
||||
} |
||||
} |
||||
|
||||
transform { |
||||
sql { |
||||
sql = "select name,age from fake" |
||||
} |
||||
} |
||||
|
||||
sink { |
||||
ConsoleSink {} |
||||
} |
||||
|
||||
``` |
||||
@ -0,0 +1,82 @@
|
||||
# Apache SeaTunnel |
||||
|
||||
## 综述 |
||||
|
||||
`SeaTunnel` 任务类型，用于创建并执行 `SeaTunnel` 类型任务。worker 执行该任务的时候，会通过 `start-seatunnel-spark.sh` 或 `start-seatunnel-flink.sh` 命令解析 config 文件。 |
||||
点击 [这里](https://seatunnel.apache.org/) 获取更多关于 `Apache SeaTunnel` 的信息。 |
||||
|
||||
## 创建任务 |
||||
|
||||
- 点击项目管理 -> 项目名称 -> 工作流定义，点击“创建工作流”按钮，进入 DAG 编辑页面； |
||||
- 拖动工具栏的<img src="../../../../img/tasks/icons/seatunnel.png" width="15"/> 任务节点到画板中。 |
||||
|
||||
## 任务参数 |
||||
|
||||
- 节点名称：设置任务节点的名称。一个工作流定义中的节点名称是唯一的。 |
||||
- 运行标志：标识这个结点是否能正常调度，如果不需要执行，可以打开禁止执行开关。 |
||||
- 描述：描述该节点的功能。 |
||||
- 任务优先级：worker 线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。 |
||||
- Worker 分组：任务分配给 worker 组的机器执行，选择 Default ，会随机选择一台 worker 机执行。 |
||||
- 环境名称：配置运行脚本的环境。 |
||||
- 失败重试次数：任务失败重新提交的次数。 |
||||
- 失败重试间隔：任务失败重新提交任务的时间间隔，以分为单位。 |
||||
- Cpu 配额: 为执行的任务分配指定的CPU时间配额，单位百分比，默认-1代表不限制，例如1个核心的CPU满载是100%，16个核心的是1600%。这个功能由 [task.resource.limit.state](../../architecture/configuration.md) 控制 |
||||
- 最大内存：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 [task.resource.limit.state](../../architecture/configuration.md) 控制 |
||||
- 延时执行时间：任务延迟执行的时间，以分为单位。 |
||||
- 超时警告：勾选超时警告、超时失败，当任务超过“超时时长”后，会发送告警邮件并且任务执行失败。 |
||||
- 引擎：支持 FLINK 和 SPARK |
||||
- FLINK |
||||
- 运行模型：支持 `run` 和 `run-application` 两种模式 |
||||
- 选项参数：用于添加 Flink 引擎本身参数，例如 `-m yarn-cluster -ynm seatunnel` |
||||
- SPARK |
||||
- 部署方式：指定部署模式，`cluster` `client` `local` |
||||
- Master：指定 `Master` 模型，`yarn` `local` `spark` `mesos`，其中 `spark` 和 `mesos` 需要指定 `Master` 服务地址，例如：127.0.0.1:7077 |
||||
> 点击 [这里](https://seatunnel.apache.org/docs/2.1.2/command/usage) 获取更多关于`Apache SeaTunnel command` 使用的信息 |
||||
- 自定义配置：支持自定义配置或从资源中心选择配置文件 |
||||
> 点击 [这里](https://seatunnel.apache.org/docs/2.1.2/concept/config) 获取更多关于`Apache SeaTunnel config` 文件介绍 |
||||
- 脚本：在任务节点那自定义配置信息，包括四部分：`env` `source` `transform` `sink` |
||||
- 资源文件：在任务节点引用资源中心的配置文件，只可以引用一个配置文件。 |
||||
- 前置任务：选择当前任务的前置任务，会将被选择的前置任务设置为当前任务的上游。 |
||||
|
||||
## 任务样例 |
||||
|
||||
该样例演示为使用 Flink 引擎从 Fake 源读取数据打印到控制台。 |
||||
|
||||
### 在 DolphinScheduler 中配置 SeaTunnel 环境 |
||||
|
||||
若生产环境中要是使用到 SeaTunnel 任务类型，则需要先配置好所需的环境，配置文件如下：`/dolphinscheduler/conf/env/dolphinscheduler_env.sh`。 |
||||
|
||||
 |
||||
|
||||
### 配置 SeaTunnel 任务节点 |
||||
|
||||
根据上述参数说明，配置所需的内容即可。 |
||||
|
||||
 |
||||
|
||||
### Config 样例 |
||||
|
||||
```Config |
||||
|
||||
env { |
||||
execution.parallelism = 1 |
||||
} |
||||
|
||||
source { |
||||
FakeSource { |
||||
result_table_name = "fake" |
||||
field_name = "name,age" |
||||
} |
||||
} |
||||
|
||||
transform { |
||||
sql { |
||||
sql = "select name,age from fake" |
||||
} |
||||
} |
||||
|
||||
sink { |
||||
ConsoleSink {} |
||||
} |
||||
|
||||
``` |
||||
|
After Width: | Height: | Size: 155 KiB |
|
After Width: | Height: | Size: 203 KiB |
|
After Width: | Height: | Size: 5.3 KiB |
@ -0,0 +1,31 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel; |
||||
|
||||
public class Constants { |
||||
|
||||
private Constants() { |
||||
throw new IllegalStateException("Utility class"); |
||||
} |
||||
|
||||
public static final String CONFIG_OPTIONS = "--config"; |
||||
public static final String DEPLOY_MODE_OPTIONS = "--deploy-mode"; |
||||
public static final String MASTER_OPTIONS = "--master"; |
||||
public static final String QUEUE_OPTIONS = "--queue"; |
||||
|
||||
} |
||||
@ -0,0 +1,34 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel; |
||||
|
||||
public enum DeployModeEnum { |
||||
cluster("cluster"), |
||||
client("client"), |
||||
local("client"); |
||||
|
||||
private String command; |
||||
|
||||
DeployModeEnum(String command) { |
||||
this.command = command; |
||||
} |
||||
|
||||
public String getCommand() { |
||||
return command; |
||||
} |
||||
} |
||||
@ -0,0 +1,34 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel; |
||||
|
||||
public enum EngineEnum { |
||||
|
||||
FLINK("${SEATUNNEL_HOME}/bin/start-seatunnel-flink.sh"), |
||||
SPARK("${SEATUNNEL_HOME}/bin/start-seatunnel-spark.sh"); |
||||
|
||||
private String command; |
||||
|
||||
EngineEnum(String command) { |
||||
this.command = command; |
||||
} |
||||
|
||||
public String getCommand() { |
||||
return command; |
||||
} |
||||
} |
||||
@ -0,0 +1,58 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel.flink; |
||||
|
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.SeatunnelParameters; |
||||
|
||||
public class SeatunnelFlinkParameters extends SeatunnelParameters { |
||||
|
||||
private RunModeEnum runMode; |
||||
|
||||
private String others; |
||||
|
||||
public static enum RunModeEnum { |
||||
RUN("--run-mode run"), |
||||
RUN_APPLICATION("--run-mode run-application"); |
||||
|
||||
private String command; |
||||
|
||||
RunModeEnum(String command) { |
||||
this.command = command; |
||||
} |
||||
|
||||
public String getCommand() { |
||||
return command; |
||||
} |
||||
} |
||||
|
||||
public RunModeEnum getRunMode() { |
||||
return runMode; |
||||
} |
||||
|
||||
public void setRunMode(RunModeEnum runMode) { |
||||
this.runMode = runMode; |
||||
} |
||||
|
||||
public String getOthers() { |
||||
return others; |
||||
} |
||||
|
||||
public void setOthers(String others) { |
||||
this.others = others; |
||||
} |
||||
} |
||||
@ -0,0 +1,53 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel.flink; |
||||
|
||||
import org.apache.dolphinscheduler.plugin.task.api.TaskExecutionContext; |
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.SeatunnelTask; |
||||
import org.apache.dolphinscheduler.spi.utils.JSONUtils; |
||||
|
||||
import org.apache.commons.lang3.StringUtils; |
||||
|
||||
import java.util.List; |
||||
import java.util.Objects; |
||||
|
||||
public class SeatunnelFlinkTask extends SeatunnelTask { |
||||
|
||||
private SeatunnelFlinkParameters seatunnelParameters; |
||||
public SeatunnelFlinkTask(TaskExecutionContext taskExecutionContext) { |
||||
super(taskExecutionContext); |
||||
} |
||||
|
||||
@Override |
||||
public void init() { |
||||
seatunnelParameters = JSONUtils.parseObject(taskExecutionContext.getTaskParams(), SeatunnelFlinkParameters.class); |
||||
setSeatunnelParameters(seatunnelParameters); |
||||
super.init(); |
||||
} |
||||
|
||||
@Override |
||||
public List<String> buildOptions() throws Exception { |
||||
List<String> args = super.buildOptions(); |
||||
args.add(Objects.isNull(seatunnelParameters.getRunMode()) ? SeatunnelFlinkParameters.RunModeEnum.RUN.getCommand() : seatunnelParameters.getRunMode().getCommand()); |
||||
if (StringUtils.isNotBlank(seatunnelParameters.getOthers())) { |
||||
args.add(seatunnelParameters.getOthers()); |
||||
} |
||||
return args; |
||||
} |
||||
|
||||
} |
||||
@ -0,0 +1,90 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel.spark; |
||||
|
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.DeployModeEnum; |
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.SeatunnelParameters; |
||||
|
||||
import org.apache.commons.lang3.StringUtils; |
||||
|
||||
import java.util.Objects; |
||||
|
||||
public class SeatunnelSparkParameters extends SeatunnelParameters { |
||||
|
||||
private DeployModeEnum deployMode; |
||||
private MasterTypeEnum master; |
||||
private String masterUrl; |
||||
private String queue; |
||||
|
||||
@Override |
||||
public boolean checkParameters() { |
||||
return super.checkParameters() |
||||
&& Objects.nonNull(deployMode) |
||||
&& (DeployModeEnum.local != deployMode && Objects.nonNull(master)) |
||||
&& (DeployModeEnum.local != deployMode && (MasterTypeEnum.SPARK == master || MasterTypeEnum.MESOS == master) && StringUtils.isNotBlank(masterUrl)); |
||||
} |
||||
|
||||
public static enum MasterTypeEnum { |
||||
YARN("yarn"), |
||||
LOCAL("local"), |
||||
SPARK("spark://"), |
||||
MESOS("mesos://"); |
||||
|
||||
private String command; |
||||
|
||||
MasterTypeEnum(String command) { |
||||
this.command = command; |
||||
} |
||||
|
||||
public String getCommand() { |
||||
return command; |
||||
} |
||||
} |
||||
|
||||
public DeployModeEnum getDeployMode() { |
||||
return deployMode; |
||||
} |
||||
|
||||
public void setDeployMode(DeployModeEnum deployMode) { |
||||
this.deployMode = deployMode; |
||||
} |
||||
|
||||
public MasterTypeEnum getMaster() { |
||||
return master; |
||||
} |
||||
|
||||
public void setMaster(MasterTypeEnum master) { |
||||
this.master = master; |
||||
} |
||||
|
||||
public String getMasterUrl() { |
||||
return masterUrl; |
||||
} |
||||
|
||||
public void setMasterUrl(String masterUrl) { |
||||
this.masterUrl = masterUrl; |
||||
} |
||||
|
||||
public String getQueue() { |
||||
return queue; |
||||
} |
||||
|
||||
public void setQueue(String queue) { |
||||
this.queue = queue; |
||||
} |
||||
} |
||||
@ -0,0 +1,73 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.plugin.task.seatunnel.spark; |
||||
|
||||
import static org.apache.dolphinscheduler.plugin.task.seatunnel.Constants.DEPLOY_MODE_OPTIONS; |
||||
import static org.apache.dolphinscheduler.plugin.task.seatunnel.Constants.MASTER_OPTIONS; |
||||
import static org.apache.dolphinscheduler.plugin.task.seatunnel.Constants.QUEUE_OPTIONS; |
||||
|
||||
import org.apache.dolphinscheduler.plugin.task.api.TaskExecutionContext; |
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.DeployModeEnum; |
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.SeatunnelTask; |
||||
import org.apache.dolphinscheduler.plugin.task.seatunnel.spark.SeatunnelSparkParameters.MasterTypeEnum; |
||||
import org.apache.dolphinscheduler.spi.utils.JSONUtils; |
||||
|
||||
import org.apache.commons.lang3.StringUtils; |
||||
|
||||
import java.util.List; |
||||
|
||||
public class SeatunnelSparkTask extends SeatunnelTask { |
||||
|
||||
private SeatunnelSparkParameters seatunnelParameters; |
||||
/** |
||||
* constructor |
||||
* |
||||
* @param taskExecutionContext taskExecutionContext |
||||
*/ |
||||
public SeatunnelSparkTask(TaskExecutionContext taskExecutionContext) { |
||||
super(taskExecutionContext); |
||||
} |
||||
|
||||
@Override |
||||
public void init() { |
||||
seatunnelParameters = JSONUtils.parseObject(taskExecutionContext.getTaskParams(), SeatunnelSparkParameters.class); |
||||
setSeatunnelParameters(seatunnelParameters); |
||||
super.init(); |
||||
} |
||||
|
||||
@Override |
||||
public List<String> buildOptions() throws Exception { |
||||
List<String> args = super.buildOptions(); |
||||
args.add(DEPLOY_MODE_OPTIONS); |
||||
args.add(seatunnelParameters.getDeployMode().getCommand()); |
||||
|
||||
MasterTypeEnum master = DeployModeEnum.local == seatunnelParameters.getDeployMode() ? MasterTypeEnum.LOCAL : seatunnelParameters.getMaster(); |
||||
|
||||
args.add(MASTER_OPTIONS); |
||||
args.add(master.getCommand()); |
||||
if (MasterTypeEnum.SPARK.equals(master) || MasterTypeEnum.MESOS.equals(master)) { |
||||
args.add(seatunnelParameters.getMasterUrl()); |
||||
} |
||||
|
||||
if (StringUtils.isNotBlank(seatunnelParameters.getQueue())) { |
||||
args.add(QUEUE_OPTIONS); |
||||
args.add(seatunnelParameters.getQueue()); |
||||
} |
||||
return args; |
||||
} |
||||
} |
||||
Loading…
Reference in new issue