github
/
DolphinScheduler
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git
1
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

[doc] Update kubernetes' FAQ (#10221)

3.1.0-release
旺阳 3 years ago committed by GitHub
parent
6cebdfdf50
commit
c12848a4ff
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 79 additions and 101 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 89
      docs/docs/en/guide/installation/kubernetes.md
  2. 89
      docs/docs/zh/guide/installation/kubernetes.md

89
docs/docs/en/guide/installation/kubernetes.md
Unescape View File

@ -186,24 +186,29 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names
> Because of the commercial license, we cannot directly use the driver of MySQL. > Because of the commercial license, we cannot directly use the driver of MySQL.
> >
> If you want to use MySQL, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions: > If you want to use MySQL, you can build a new image based on the `apache/dolphinscheduler-<service>` image follow the following instructions:
>
> Since version 3.0.0, dolphinscheduler has been microserviced and the change of metadata storage requires replacing all services with MySQL driver, which including dolphinscheduler-tools, dolphinscheduler-master, dolphinscheduler-worker, dolphinscheduler-api, dolphinscheduler-alert-server
1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar). 1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar).
2. Create a new `Dockerfile` to add MySQL driver: 2. Create a new `Dockerfile` to add MySQL driver:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:<version>
# For example
# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools:<version>
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
``` ```
3. Build a new docker image including MySQL driver: 3. Build a new docker image including MySQL driver:
``` ```
docker build -t apache/dolphinscheduler:mysql-driver . docker build -t apache/dolphinscheduler-<service>:mysql-driver .
``` ```
4. Push the docker image `apache/dolphinscheduler:mysql-driver` to a docker registry. 4. Push the docker image `apache/dolphinscheduler-<service>:mysql-driver` to a docker registry.
5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`. 5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`.
@ -214,7 +219,6 @@ docker build -t apache/dolphinscheduler:mysql-driver .
```yaml ```yaml
externalDatabase: externalDatabase:
type: "mysql" type: "mysql"
driver: "com.mysql.jdbc.Driver"
host: "localhost" host: "localhost"
port: "3306" port: "3306"
username: "root" username: "root"
@ -225,70 +229,53 @@ externalDatabase:
8. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**). 8. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**).
### How to Support MySQL Datasource in `Datasource manage`? ### How to Support MySQL or Oracle Datasource in `Datasource manage`?
> Because of the commercial license, we cannot directly use the driver of MySQL. > Because of the commercial license, we cannot directly use the driver of MySQL or Oracle.
>
> If you want to add MySQL or Oracle datasource, you can build a new image based on the `apache/dolphinscheduler-<service>` image follow the following instructions:
> >
> If you want to add MySQL datasource, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions: > You need to change the two service images including dolphinscheduler-worker, dolphinscheduler-api.
1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar). 1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar).
or download the Oracle driver [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (such as `ojdbc8-19.9.0.0.jar`)
2. Create a new `Dockerfile` to add MySQL driver: 2. Create a new `Dockerfile` to add MySQL or Oracle driver:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:<version>
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib # For example
``` # FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
3. Build a new docker image including MySQL driver:
```
docker build -t apache/dolphinscheduler:mysql-driver .
```
4. Push the docker image `apache/dolphinscheduler:mysql-driver` to a docker registry.
5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`. # If you want to support MySQL Datasource
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**).
7. Add a MySQL datasource in `Datasource manage`.
### How to Support Oracle Datasource in `Datasource manage`?
> Because of the commercial license, we cannot directly use the driver of Oracle.
>
> If you want to add Oracle datasource, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions:
1. Download the Oracle driver [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (such as `ojdbc8-19.9.0.0.jar`)
2. Create a new `Dockerfile` to add Oracle driver:
``` # If you want to support Oracle Datasource
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version>
COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib
``` ```
3. Build a new docker image including Oracle driver: 3. Build a new docker image including MySQL or Oracle driver:
``` ```
docker build -t apache/dolphinscheduler:oracle-driver . docker build -t apache/dolphinscheduler-<service>:new-driver .
``` ```
4. Push the docker image `apache/dolphinscheduler:oracle-driver` to a docker registry. 4. Push the docker image `apache/dolphinscheduler-<service>:new-driver` to a docker registry.
5. Modify image `repository` and update `tag` to `oracle-driver` in `values.yaml`. 5. Modify image `repository` and update `tag` to `new-driver` in `values.yaml`.
6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**). 6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**).
7. Add an Oracle datasource in `Datasource manage`. 7. Add a MySQL or Oracle datasource in `Datasource manage`.
### How to Support Python 2 pip and Custom requirements.txt? ### How to Support Python 2 pip and Custom requirements.txt?
> Just change the image of the dolphinscheduler-worker service.
1. Create a new `Dockerfile` to install pip: 1. Create a new `Dockerfile` to install pip:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
COPY requirements.txt /tmp COPY requirements.txt /tmp
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y --no-install-recommends python-pip && \ apt-get install -y --no-install-recommends python-pip && \
@ -299,16 +286,16 @@ RUN apt-get update && \
The command will install the default **pip 18.1**. If you upgrade the pip, just add the following command. The command will install the default **pip 18.1**. If you upgrade the pip, just add the following command.
``` ```
pip install --no-cache-dir -U pip && \ pip install --no-cache-dir -U pip && \
``` ```
2. Build a new docker image including pip: 2. Build a new docker image including pip:
``` ```
docker build -t apache/dolphinscheduler:pip . docker build -t apache/dolphinscheduler-worker:pip .
``` ```
3. Push the docker image `apache/dolphinscheduler:pip` to a docker registry. 3. Push the docker image `apache/dolphinscheduler-worker:pip` to a docker registry.
4. Modify image `repository` and update `tag` to `pip` in `values.yaml`. 4. Modify image `repository` and update `tag` to `pip` in `values.yaml`.
@ -318,10 +305,12 @@ docker build -t apache/dolphinscheduler:pip .
### How to Support Python 3? ### How to Support Python 3?
> Just change the image of the dolphinscheduler-worker service.
1. Create a new `Dockerfile` to install Python 3: 1. Create a new `Dockerfile` to install Python 3:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y --no-install-recommends python3 && \ apt-get install -y --no-install-recommends python3 && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
@ -330,16 +319,16 @@ RUN apt-get update && \
The command will install the default **Python 3.7.3**. If you also want to install **pip3**, just replace `python3` with `python3-pip` like: The command will install the default **Python 3.7.3**. If you also want to install **pip3**, just replace `python3` with `python3-pip` like:
``` ```
apt-get install -y --no-install-recommends python3-pip && \ apt-get install -y --no-install-recommends python3-pip && \
``` ```
2. Build a new docker image including Python 3: 2. Build a new docker image including Python 3:
``` ```
docker build -t apache/dolphinscheduler:python3 . docker build -t apache/dolphinscheduler-worker:python3 .
``` ```
3. Push the docker image `apache/dolphinscheduler:python3` to a docker registry. 3. Push the docker image `apache/dolphinscheduler-worker:python3` to a docker registry.
4. Modify image `repository` and update `tag` to `python3` in `values.yaml`. 4. Modify image `repository` and update `tag` to `python3` in `values.yaml`.

89
docs/docs/zh/guide/installation/kubernetes.md
Unescape View File

@ -187,24 +187,29 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names
> 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包. > 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包.
> >
> 如果你要使用 MySQL, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建. > 如果你要使用 MySQL, 你可以基于官方镜像 `apache/dolphinscheduler-<service>` 进行构建.
>
> 从3.0.0版本起，dolphinscheduler已经微服务化，更改元数据存储需要对把所有的服务都替换为 MySQL 驱动包，包括 dolphinscheduler-tools, dolphinscheduler-master, dolphinscheduler-worker, dolphinscheduler-api, dolphinscheduler-alert-server .
1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar) 1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar)
2. 创建一个新的 `Dockerfile`，用于添加 MySQL 的驱动包: 2. 创建一个新的 `Dockerfile`，用于添加 MySQL 的驱动包:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:<version>
# For example
# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools:<version>
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
``` ```
3. 构建一个包含 MySQL 驱动包的新镜像: 3. 构建一个包含 MySQL 驱动包的新镜像:
``` ```
docker build -t apache/dolphinscheduler:mysql-driver . docker build -t apache/dolphinscheduler-<service>:mysql-driver .
``` ```
4. 推送 docker 镜像 `apache/dolphinscheduler:mysql-driver` 到一个 docker registry 中 4. 推送 docker 镜像 `apache/dolphinscheduler-<service>:mysql-driver` 到一个 docker registry 中
5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver` 5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver`
@ -215,7 +220,6 @@ docker build -t apache/dolphinscheduler:mysql-driver .
```yaml ```yaml
externalDatabase: externalDatabase:
type: "mysql" type: "mysql"
driver: "com.mysql.jdbc.Driver"
host: "localhost" host: "localhost"
port: "3306" port: "3306"
username: "root" username: "root"
@ -226,70 +230,53 @@ externalDatabase:
8. 部署 dolphinscheduler (详见**安装 dolphinscheduler**) 8. 部署 dolphinscheduler (详见**安装 dolphinscheduler**)
### 如何在数据源中心支持 MySQL 数据源？ ### 如何在数据源中心支持 MySQL 或者 Oracle 数据源？
> 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包. > 由于商业许可证的原因，我们不能直接使用 MySQL 或者 Oracle 的驱动包.
>
> 如果你要添加 MySQL 或者 Oracle, 你可以基于官方镜像 `apache/dolphinscheduler-<service>` 进行构建.
> >
> 如果你要添加 MySQL 数据源, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建. > 需要更改 dolphinscheduler-worker, dolphinscheduler-api 两个服务的镜像.
1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar) 1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar)
或者 Oracle 驱动包 [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (例如 `ojdbc8-19.9.0.0.jar`)
2. 创建一个新的 `Dockerfile`，用于添加 MySQL 驱动包: 3. 创建一个新的 `Dockerfile`，用于添加 MySQL 或者 Oracle 驱动包:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:<version>
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib # For example
``` # FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
3. 构建一个包含 MySQL 驱动包的新镜像: # If you want to support MySQL Datasource
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
```
docker build -t apache/dolphinscheduler:mysql-driver .
```
4. 推送 docker 镜像 `apache/dolphinscheduler:mysql-driver` 到一个 docker registry 中
5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver`
6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**)
7. 在数据源中心添加一个 MySQL 数据源
### 如何在数据源中心支持 Oracle 数据源？
> 由于商业许可证的原因，我们不能直接使用 Oracle 的驱动包.
>
> 如果你要添加 Oracle 数据源, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建.
1. 下载 Oracle 驱动包 [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (例如 `ojdbc8-19.9.0.0.jar`)
2. 创建一个新的 `Dockerfile`，用于添加 Oracle 驱动包:
``` # If you want to support Oracle Datasource
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version>
COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib
``` ```
3. 构建一个包含 Oracle 驱动包的新镜像: 3. 构建一个包含 MySQL 或者 Oracle 驱动包的新镜像:
``` ```
docker build -t apache/dolphinscheduler:oracle-driver . docker build -t apache/dolphinscheduler-<service>:new-driver .
``` ```
4. 推送 docker 镜像 `apache/dolphinscheduler:oracle-driver` 到一个 docker registry 中 4. 推送 docker 镜像 `apache/dolphinscheduler-<service>:new-driver` 到一个 docker registry 中
5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `oracle-driver` 5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `new-driver`
6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**) 6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**)
7. 在数据源中心添加一个 Oracle 数据源 7. 在数据源中心添加一个 MySQL 或者 Oracle 数据源
### 如何支持 Python 2 pip 以及自定义 requirements.txt？ ### 如何支持 Python 2 pip 以及自定义 requirements.txt？
> 只需要更改 dolphinscheduler-worker 服务的镜像.
1. 创建一个新的 `Dockerfile`，用于安装 pip: 1. 创建一个新的 `Dockerfile`，用于安装 pip:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
COPY requirements.txt /tmp COPY requirements.txt /tmp
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y --no-install-recommends python-pip && \ apt-get install -y --no-install-recommends python-pip && \
@ -300,16 +287,16 @@ RUN apt-get update && \
这个命令会安装默认的 **pip 18.1**. 如果你想升级 pip, 只需添加一行 这个命令会安装默认的 **pip 18.1**. 如果你想升级 pip, 只需添加一行
``` ```
pip install --no-cache-dir -U pip && \ pip install --no-cache-dir -U pip && \
``` ```
2. 构建一个包含 pip 的新镜像: 2. 构建一个包含 pip 的新镜像:
``` ```
docker build -t apache/dolphinscheduler:pip . docker build -t apache/dolphinscheduler-worker:pip .
``` ```
3. 推送 docker 镜像 `apache/dolphinscheduler:pip` 到一个 docker registry 中 3. 推送 docker 镜像 `apache/dolphinscheduler-worker:pip` 到一个 docker registry 中
4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `pip` 4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `pip`
@ -319,10 +306,12 @@ docker build -t apache/dolphinscheduler:pip .
### 如何支持 Python 3？ ### 如何支持 Python 3？
> 只需要更改 dolphinscheduler-worker 服务的镜像.
1. 创建一个新的 `Dockerfile`，用于安装 Python 3: 1. 创建一个新的 `Dockerfile`，用于安装 Python 3:
``` ```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:<version> FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y --no-install-recommends python3 && \ apt-get install -y --no-install-recommends python3 && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
@ -331,16 +320,16 @@ RUN apt-get update && \
这个命令会安装默认的 **Python 3.7.3**. 如果你也想安装 **pip3**, 将 `python3` 替换为 `python3-pip` 即可 这个命令会安装默认的 **Python 3.7.3**. 如果你也想安装 **pip3**, 将 `python3` 替换为 `python3-pip` 即可
``` ```
apt-get install -y --no-install-recommends python3-pip && \ apt-get install -y --no-install-recommends python3-pip && \
``` ```
2. 构建一个包含 Python 3 的新镜像: 2. 构建一个包含 Python 3 的新镜像:
``` ```
docker build -t apache/dolphinscheduler:python3 . docker build -t apache/dolphinscheduler-worker:python3 .
``` ```
3. 推送 docker 镜像 `apache/dolphinscheduler:python3` 到一个 docker registry 中 3. 推送 docker 镜像 `apache/dolphinscheduler-worker:python3` 到一个 docker registry 中
4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `python3` 4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `python3`

Write Preview
Loading…
Cancel
Save