diff --git a/docs/docs/en/guide/installation/kubernetes.md b/docs/docs/en/guide/installation/kubernetes.md index 1a4d6ce33b..32c7466a14 100644 --- a/docs/docs/en/guide/installation/kubernetes.md +++ b/docs/docs/en/guide/installation/kubernetes.md @@ -186,24 +186,29 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names > Because of the commercial license, we cannot directly use the driver of MySQL. > -> If you want to use MySQL, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions: +> If you want to use MySQL, you can build a new image based on the `apache/dolphinscheduler-` image follow the following instructions: +> +> Since version 3.0.0, dolphinscheduler has been microserviced and the change of metadata storage requires replacing all services with MySQL driver, which including dolphinscheduler-tools, dolphinscheduler-master, dolphinscheduler-worker, dolphinscheduler-api, dolphinscheduler-alert-server 1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar). 2. Create a new `Dockerfile` to add MySQL driver: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-: +# For example +# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools: + COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib ``` 3. Build a new docker image including MySQL driver: ``` -docker build -t apache/dolphinscheduler:mysql-driver . +docker build -t apache/dolphinscheduler-:mysql-driver . ``` -4. Push the docker image `apache/dolphinscheduler:mysql-driver` to a docker registry. +4. Push the docker image `apache/dolphinscheduler-:mysql-driver` to a docker registry. 5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`. @@ -214,7 +219,6 @@ docker build -t apache/dolphinscheduler:mysql-driver . ```yaml externalDatabase: type: "mysql" - driver: "com.mysql.jdbc.Driver" host: "localhost" port: "3306" username: "root" @@ -225,70 +229,53 @@ externalDatabase: 8. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**). -### How to Support MySQL Datasource in `Datasource manage`? +### How to Support MySQL or Oracle Datasource in `Datasource manage`? -> Because of the commercial license, we cannot directly use the driver of MySQL. +> Because of the commercial license, we cannot directly use the driver of MySQL or Oracle. > -> If you want to add MySQL datasource, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions: +> If you want to add MySQL or Oracle datasource, you can build a new image based on the `apache/dolphinscheduler-` image follow the following instructions: +> +> You need to change the two service images including dolphinscheduler-worker, dolphinscheduler-api. 1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar). +or download the Oracle driver [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (such as `ojdbc8-19.9.0.0.jar`) -2. Create a new `Dockerfile` to add MySQL driver: +2. Create a new `Dockerfile` to add MySQL or Oracle driver: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: -COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib -``` +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-: +# For example +# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker: -3. Build a new docker image including MySQL driver: - -``` -docker build -t apache/dolphinscheduler:mysql-driver . -``` - -4. Push the docker image `apache/dolphinscheduler:mysql-driver` to a docker registry. - -5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`. - -6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**). - -7. Add a MySQL datasource in `Datasource manage`. - -### How to Support Oracle Datasource in `Datasource manage`? - -> Because of the commercial license, we cannot directly use the driver of Oracle. -> -> If you want to add Oracle datasource, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions: - -1. Download the Oracle driver [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (such as `ojdbc8-19.9.0.0.jar`) - -2. Create a new `Dockerfile` to add Oracle driver: +# If you want to support MySQL Datasource +COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib -``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +# If you want to support Oracle Datasource COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib ``` -3. Build a new docker image including Oracle driver: +3. Build a new docker image including MySQL or Oracle driver: ``` -docker build -t apache/dolphinscheduler:oracle-driver . +docker build -t apache/dolphinscheduler-:new-driver . ``` -4. Push the docker image `apache/dolphinscheduler:oracle-driver` to a docker registry. +4. Push the docker image `apache/dolphinscheduler-:new-driver` to a docker registry. -5. Modify image `repository` and update `tag` to `oracle-driver` in `values.yaml`. +5. Modify image `repository` and update `tag` to `new-driver` in `values.yaml`. 6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**). -7. Add an Oracle datasource in `Datasource manage`. +7. Add a MySQL or Oracle datasource in `Datasource manage`. ### How to Support Python 2 pip and Custom requirements.txt? +> Just change the image of the dolphinscheduler-worker service. + 1. Create a new `Dockerfile` to install pip: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker: COPY requirements.txt /tmp RUN apt-get update && \ apt-get install -y --no-install-recommends python-pip && \ @@ -299,16 +286,16 @@ RUN apt-get update && \ The command will install the default **pip 18.1**. If you upgrade the pip, just add the following command. ``` - pip install --no-cache-dir -U pip && \ +pip install --no-cache-dir -U pip && \ ``` 2. Build a new docker image including pip: ``` -docker build -t apache/dolphinscheduler:pip . +docker build -t apache/dolphinscheduler-worker:pip . ``` -3. Push the docker image `apache/dolphinscheduler:pip` to a docker registry. +3. Push the docker image `apache/dolphinscheduler-worker:pip` to a docker registry. 4. Modify image `repository` and update `tag` to `pip` in `values.yaml`. @@ -318,10 +305,12 @@ docker build -t apache/dolphinscheduler:pip . ### How to Support Python 3? +> Just change the image of the dolphinscheduler-worker service. + 1. Create a new `Dockerfile` to install Python 3: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker: RUN apt-get update && \ apt-get install -y --no-install-recommends python3 && \ rm -rf /var/lib/apt/lists/* @@ -330,16 +319,16 @@ RUN apt-get update && \ The command will install the default **Python 3.7.3**. If you also want to install **pip3**, just replace `python3` with `python3-pip` like: ``` - apt-get install -y --no-install-recommends python3-pip && \ +apt-get install -y --no-install-recommends python3-pip && \ ``` 2. Build a new docker image including Python 3: ``` -docker build -t apache/dolphinscheduler:python3 . +docker build -t apache/dolphinscheduler-worker:python3 . ``` -3. Push the docker image `apache/dolphinscheduler:python3` to a docker registry. +3. Push the docker image `apache/dolphinscheduler-worker:python3` to a docker registry. 4. Modify image `repository` and update `tag` to `python3` in `values.yaml`. diff --git a/docs/docs/zh/guide/installation/kubernetes.md b/docs/docs/zh/guide/installation/kubernetes.md index 9b3e44b875..525eae05ce 100644 --- a/docs/docs/zh/guide/installation/kubernetes.md +++ b/docs/docs/zh/guide/installation/kubernetes.md @@ -187,24 +187,29 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names > 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包. > -> 如果你要使用 MySQL, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建. +> 如果你要使用 MySQL, 你可以基于官方镜像 `apache/dolphinscheduler-` 进行构建. +> +> 从3.0.0版本起，dolphinscheduler已经微服务化，更改元数据存储需要对把所有的服务都替换为 MySQL 驱动包，包括 dolphinscheduler-tools, dolphinscheduler-master, dolphinscheduler-worker, dolphinscheduler-api, dolphinscheduler-alert-server . 1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar) 2. 创建一个新的 `Dockerfile`，用于添加 MySQL 的驱动包: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-: +# For example +# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools: + COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib ``` 3. 构建一个包含 MySQL 驱动包的新镜像: ``` -docker build -t apache/dolphinscheduler:mysql-driver . +docker build -t apache/dolphinscheduler-:mysql-driver . ``` -4. 推送 docker 镜像 `apache/dolphinscheduler:mysql-driver` 到一个 docker registry 中 +4. 推送 docker 镜像 `apache/dolphinscheduler-:mysql-driver` 到一个 docker registry 中 5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver` @@ -215,7 +220,6 @@ docker build -t apache/dolphinscheduler:mysql-driver . ```yaml externalDatabase: type: "mysql" - driver: "com.mysql.jdbc.Driver" host: "localhost" port: "3306" username: "root" @@ -226,70 +230,53 @@ externalDatabase: 8. 部署 dolphinscheduler (详见**安装 dolphinscheduler**) -### 如何在数据源中心支持 MySQL 数据源？ +### 如何在数据源中心支持 MySQL 或者 Oracle 数据源？ -> 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包. +> 由于商业许可证的原因，我们不能直接使用 MySQL 或者 Oracle 的驱动包. > -> 如果你要添加 MySQL 数据源, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建. - -1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar) - -2. 创建一个新的 `Dockerfile`，用于添加 MySQL 驱动包: +> 如果你要添加 MySQL 或者 Oracle, 你可以基于官方镜像 `apache/dolphinscheduler-` 进行构建. +> +> 需要更改 dolphinscheduler-worker, dolphinscheduler-api 两个服务的镜像. -``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: -COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib -``` +1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar) +或者 Oracle 驱动包 [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (例如 `ojdbc8-19.9.0.0.jar`) -3. 构建一个包含 MySQL 驱动包的新镜像: +3. 创建一个新的 `Dockerfile`，用于添加 MySQL 或者 Oracle 驱动包: ``` -docker build -t apache/dolphinscheduler:mysql-driver . -``` - -4. 推送 docker 镜像 `apache/dolphinscheduler:mysql-driver` 到一个 docker registry 中 - -5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver` - -6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**) - -7. 在数据源中心添加一个 MySQL 数据源 - -### 如何在数据源中心支持 Oracle 数据源？ - -> 由于商业许可证的原因，我们不能直接使用 Oracle 的驱动包. -> -> 如果你要添加 Oracle 数据源, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建. - -1. 下载 Oracle 驱动包 [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (例如 `ojdbc8-19.9.0.0.jar`) +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-: +# For example +# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker: -2. 创建一个新的 `Dockerfile`，用于添加 Oracle 驱动包: +# If you want to support MySQL Datasource +COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib -``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +# If you want to support Oracle Datasource COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib ``` -3. 构建一个包含 Oracle 驱动包的新镜像: +3. 构建一个包含 MySQL 或者 Oracle 驱动包的新镜像: ``` -docker build -t apache/dolphinscheduler:oracle-driver . +docker build -t apache/dolphinscheduler-:new-driver . ``` -4. 推送 docker 镜像 `apache/dolphinscheduler:oracle-driver` 到一个 docker registry 中 +4. 推送 docker 镜像 `apache/dolphinscheduler-:new-driver` 到一个 docker registry 中 -5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `oracle-driver` +5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `new-driver` 6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**) -7. 在数据源中心添加一个 Oracle 数据源 +7. 在数据源中心添加一个 MySQL 或者 Oracle 数据源 ### 如何支持 Python 2 pip 以及自定义 requirements.txt？ +> 只需要更改 dolphinscheduler-worker 服务的镜像. + 1. 创建一个新的 `Dockerfile`，用于安装 pip: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker: COPY requirements.txt /tmp RUN apt-get update && \ apt-get install -y --no-install-recommends python-pip && \ @@ -300,16 +287,16 @@ RUN apt-get update && \ 这个命令会安装默认的 **pip 18.1**. 如果你想升级 pip, 只需添加一行 ``` - pip install --no-cache-dir -U pip && \ +pip install --no-cache-dir -U pip && \ ``` 2. 构建一个包含 pip 的新镜像: ``` -docker build -t apache/dolphinscheduler:pip . +docker build -t apache/dolphinscheduler-worker:pip . ``` -3. 推送 docker 镜像 `apache/dolphinscheduler:pip` 到一个 docker registry 中 +3. 推送 docker 镜像 `apache/dolphinscheduler-worker:pip` 到一个 docker registry 中 4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `pip` @@ -319,10 +306,12 @@ docker build -t apache/dolphinscheduler:pip . ### 如何支持 Python 3？ +> 只需要更改 dolphinscheduler-worker 服务的镜像. + 1. 创建一个新的 `Dockerfile`，用于安装 Python 3: ``` -FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler: +FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker: RUN apt-get update && \ apt-get install -y --no-install-recommends python3 && \ rm -rf /var/lib/apt/lists/* @@ -331,16 +320,16 @@ RUN apt-get update && \ 这个命令会安装默认的 **Python 3.7.3**. 如果你也想安装 **pip3**, 将 `python3` 替换为 `python3-pip` 即可 ``` - apt-get install -y --no-install-recommends python3-pip && \ +apt-get install -y --no-install-recommends python3-pip && \ ``` 2. 构建一个包含 Python 3 的新镜像: ``` -docker build -t apache/dolphinscheduler:python3 . +docker build -t apache/dolphinscheduler-worker:python3 . ``` -3. 推送 docker 镜像 `apache/dolphinscheduler:python3` 到一个 docker registry 中 +3. 推送 docker 镜像 `apache/dolphinscheduler-worker:python3` 到一个 docker registry 中 4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `python3`