diff --git a/docs/docs/en/guide/installation/cluster.md b/docs/docs/en/guide/installation/cluster.md index 0014a9c7f1..ce5b60baa2 100644 --- a/docs/docs/en/guide/installation/cluster.md +++ b/docs/docs/en/guide/installation/cluster.md @@ -14,21 +14,7 @@ Configure all the configurations refer to [pseudo-cluster deployment](pseudo-clu ### Modify Configuration -This step differs quite a lot from [pseudo-cluster deployment](pseudo-cluster.md), because the deployment script transfers the required resources for installation to each deployment machine by using `scp`. So we only need to modify the configuration of the machine that runs `install.sh` script and configurations will dispatch to cluster by `scp`. The configuration file is under the path `bin/env/install_env.sh`, here we only need to modify section **INSTALL MACHINE**, **DolphinScheduler ENV, Database, Registry Server** and keep other sections the same as [pseudo-cluster deployment](pseudo-cluster.md), the following describes the parameters that must be modified: - -```shell -# --------------------------------------------------------- -# INSTALL MACHINE -# --------------------------------------------------------- -# Using IP or machine hostname for the server going to deploy master, worker, API server, the IP of the server -# If you using a hostname, make sure machines could connect each other by hostname -# As below, the hostname of the machine deploying DolphinScheduler is ds1, ds2, ds3, ds4, ds5, where ds1, ds2 install the master server, ds3, ds4, and ds5 installs worker server, the alert server is installed in ds4, and the API server is installed in ds5 -ips="ds1,ds2,ds3,ds4,ds5" -masters="ds1,ds2" -workers="ds3:default,ds4:default,ds5:default" -alertServer="ds4" -apiServers="ds5" -``` +This step differs quite a lot from [pseudo-cluster deployment](pseudo-cluster.md), please use `scp` or other methods to distribute the configuration files to each machine, then modify the configuration files. ## Start and Login DolphinScheduler diff --git a/docs/docs/en/guide/installation/pseudo-cluster.md b/docs/docs/en/guide/installation/pseudo-cluster.md index 64f28022e4..f0dc100d84 100644 --- a/docs/docs/en/guide/installation/pseudo-cluster.md +++ b/docs/docs/en/guide/installation/pseudo-cluster.md @@ -71,31 +71,7 @@ Go to the ZooKeeper installation directory, copy configure file `zoo_sample.cfg` ## Modify Configuration After completing the preparation of the basic environment, you need to modify the configuration file according to the -environment you used. Change the environment configurations via `export =`. The configuration files are located in directory `bin/env` as `install_env.sh` and `dolphinscheduler_env.sh`. - -### Modify `install_env.sh` - -File `install_env.sh` describes which machines will be installed DolphinScheduler and what server will be installed on -each machine. You could find this file in the path `bin/env/install_env.sh` and the detail of the configuration as below. - -```shell -# --------------------------------------------------------- -# INSTALL MACHINE -# --------------------------------------------------------- -# Due to the master, worker, and API server being deployed on a single node, the IP of the server is the machine IP or localhost -ips="localhost" -sshPort="22" -masters="localhost" -workers="localhost:default" -alertServer="localhost" -apiServers="localhost" - -# DolphinScheduler installation path, it will auto-create if not exists -installPath=~/dolphinscheduler - -# Deploy user, use the user you create in section **Configure machine SSH password-free login** -deployUser="dolphinscheduler" -``` +environment you used. Change the environment configurations via `export =`. The configuration files are located in directory `bin/env` as `dolphinscheduler_env.sh`. ### Modify `dolphinscheduler_env.sh` @@ -146,11 +122,7 @@ Follow the instructions in [datasource-setting](../howto/datasource-setting.md) ## Start DolphinScheduler -Use **deployment user** you created above, running the following command to complete the deployment, and the server log will be stored in the logs folder. - -```shell -bash ./bin/install.sh -``` +Use **deployment user** you created above, running the command to complete the deployment, and the server log will be stored in the logs folder. > **_Note:_** For the first time deployment, there maybe occur five times of `sh: bin/dolphinscheduler-daemon.sh: No such file or directory` in the terminal, > this is non-important information that you can ignore. diff --git a/docs/docs/zh/guide/installation/cluster.md b/docs/docs/zh/guide/installation/cluster.md index 97c1792787..c266a671e0 100644 --- a/docs/docs/zh/guide/installation/cluster.md +++ b/docs/docs/zh/guide/installation/cluster.md @@ -14,21 +14,7 @@ ### 修改相关配置 -这个是与[伪集群部署](pseudo-cluster.md)差异较大的一步，因为部署脚本会通过 `scp` 的方式将安装需要的资源传输到各个机器上，所以这一步我们仅需要修改运行`install.sh`脚本的所在机器的配置即可。配置文件在路径在`bin/env/install_env.sh`下，此处我们仅需修改**INSTALL MACHINE**，**DolphinScheduler ENV、Database、Registry Server**与伪集群部署保持一致，下面对必须修改参数进行说明 - -```shell -# --------------------------------------------------------- -# INSTALL MACHINE -# --------------------------------------------------------- -# 需要配置master、worker、API server，所在服务器的IP均为机器IP或者localhost -# 如果是配置hostname的话，需要保证机器间可以通过hostname相互链接 -# 如下图所示，部署 DolphinScheduler 机器的 hostname 为 ds1,ds2,ds3,ds4,ds5，其中 ds1,ds2 安装 master 服务，ds3,ds4,ds5安装 worker 服务，alert server安装在ds4中，api server 安装在ds5中 -ips="ds1,ds2,ds3,ds4,ds5" -masters="ds1,ds2" -workers="ds3:default,ds4:default,ds5:default" -alertServer="ds4" -apiServers="ds5" -``` +这个是与[伪集群部署](pseudo-cluster.md)差异较大的一步，请使用 scp 等方式将配置文件分发到各台机器上，然后修改配置文件 ## 启动 DolphinScheduler && 登录 DolphinScheduler && 启停服务 diff --git a/docs/docs/zh/guide/installation/pseudo-cluster.md b/docs/docs/zh/guide/installation/pseudo-cluster.md index a199167e04..8e1c133c07 100644 --- a/docs/docs/zh/guide/installation/pseudo-cluster.md +++ b/docs/docs/zh/guide/installation/pseudo-cluster.md @@ -70,30 +70,7 @@ chmod 600 ~/.ssh/authorized_keys ## 修改相关配置 -完成基础环境的准备后，需要根据你的机器环境修改配置文件。配置文件可以在目录 `bin/env` 中找到，他们分别是 并命名为 `install_env.sh` 和 `dolphinscheduler_env.sh`。 - -### 修改 `install_env.sh` 文件 - -文件 `install_env.sh` 描述了哪些机器将被安装 DolphinScheduler 以及每台机器对应安装哪些服务。您可以在路径 `bin/env/install_env.sh` 中找到此文件，可通过以下方式更改 env 变量,export =，配置详情如下。 - -```shell -# --------------------------------------------------------- -# INSTALL MACHINE -# --------------------------------------------------------- -# Due to the master, worker, and API server being deployed on a single node, the IP of the server is the machine IP or localhost -ips="localhost" -sshPort="22" -masters="localhost" -workers="localhost:default" -alertServer="localhost" -apiServers="localhost" - -# DolphinScheduler installation path, it will auto-create if not exists -installPath=~/dolphinscheduler - -# Deploy user, use the user you create in section **Configure machine SSH password-free login** -deployUser="dolphinscheduler" -``` +完成基础环境的准备后，需要根据你的机器环境修改配置文件。配置文件可以在目录 `bin/env/dolphinscheduler_env.sh` 中找到。 ### 修改 `dolphinscheduler_env.sh` 文件 @@ -141,11 +118,7 @@ export PATH=$HADOOP_HOME/bin:$SPARK_HOME/bin:$PYTHON_LAUNCHER:$JAVA_HOME/bin:$HI ## 启动 DolphinScheduler -使用上面创建的**部署用户**运行以下命令完成部署，部署后的运行日志将存放在 logs 文件夹内 - -```shell -bash ./bin/install.sh -``` +使用上面创建的**部署用户**运行命令完成部署，部署后的运行日志将存放在 logs 文件夹内 > **_注意:_** 第一次部署的话，可能出现 5 次`sh: bin/dolphinscheduler-daemon.sh: No such file or directory`相关信息，此为非重要信息直接忽略即可 diff --git a/script/env/install_env.sh b/script/env/install_env.sh deleted file mode 100644 index 8de1c78637..0000000000 --- a/script/env/install_env.sh +++ /dev/null @@ -1,63 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# --------------------------------------------------------- -# INSTALL MACHINE -# --------------------------------------------------------- -# A comma separated list of machine hostname or IP would be installed DolphinScheduler, -# including master, worker, api, alert. If you want to deploy in pseudo-distributed -# mode, just write a pseudo-distributed hostname -# Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5" -ips=${ips:-"ds1,ds2,ds3,ds4,ds5"} - -# Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine -# modify it if you use different ssh port -sshPort=${sshPort:-"22"} - -# A comma separated list of machine hostname or IP would be installed Master server, it -# must be a subset of configuration `ips`. -# Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2" -masters=${masters:-"ds1,ds2"} - -# A comma separated list of machine : or :.All hostname or IP must be a -# subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts -# Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default" -workers=${workers:-"ds1:default,ds2:default,ds3:default,ds4:default,ds5:default"} - -# A comma separated list of machine hostname or IP would be installed Alert server, it -# must be a subset of configuration `ips`. -# Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3" -alertServer=${alertServer:-"ds3"} - -# A comma separated list of machine hostname or IP would be installed API server, it -# must be a subset of configuration `ips`. -# Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1" -apiServers=${apiServers:-"ds1"} - -# The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists. -# Do not set this configuration same as the current path (pwd). Do not add quotes to it if you using related path. -installPath=${installPath:-"/tmp/dolphinscheduler"} - -# The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh` -# script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs -# to be created by this user -deployUser=${deployUser:-"dolphinscheduler"} - -# The root of zookeeper, for now DolphinScheduler default registry server is zookeeper. -# It will delete ${zkRoot} in the zookeeper when you run install.sh, so please keep it same as registry.zookeeper.namespace in yml files. -# Similarly, if you want to modify the value, please modify registry.zookeeper.namespace in yml files as well. -zkRoot=${zkRoot:-"/dolphinscheduler"} diff --git a/script/install.sh b/script/install.sh deleted file mode 100755 index d36f90a3f3..0000000000 --- a/script/install.sh +++ /dev/null @@ -1,58 +0,0 @@ -#!/bin/bash -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -workDir=`dirname $0` -workDir=`cd ${workDir};pwd` -baseDir=`cd ${workDir}/..;pwd` - -source ${workDir}/env/install_env.sh -source ${workDir}/env/dolphinscheduler_env.sh - -echo "1.create directory" - -# If install Path equal to "/" or related path is "/" or is empty, will cause directory "/bin" be overwrite or file adding, -# so we should check its value. Here use command `realpath` to get the related path, and it will skip if your shell env -# without command `realpath`. -if [ ${baseDir} = $installPath ]; then - echo "Fatal: The installPath can not be same as the current path: ${installPath}" - exit 1 -elif [ ! -d $installPath ];then - sudo mkdir -p $installPath - sudo chown -R $deployUser:$deployUser $installPath -elif [[ -z "${installPath// }" || "${installPath// }" == "/" || ( $(command -v realpath) && $(realpath -s "${installPath}") == "/" ) ]]; then - echo "Parameter installPath can not be empty, use in root path or related path of root path, currently use ${installPath}" - exit 1 -fi - -echo "2.scp resources" -bash ${workDir}/scp-hosts.sh -if [ $? -eq 0 ];then - echo 'scp copy completed' -else - echo 'scp copy failed to exit' - exit 1 -fi - -echo "3.stop server" -bash ${workDir}/stop-all.sh - -echo "4.delete zk node" -bash ${workDir}/remove-zk-node.sh $zkRoot - -echo "5.startup" -bash ${workDir}/start-all.sh