github
/
DolphinScheduler
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git
1
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

[Improvement][Docker] update outdated readme and fix PSQLException (#4652)

Shiwen Cheng 4 years ago committed by GitHub
parent
50c268fa8b
commit
1e0ff5fb90
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 100 additions and 128 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 12
      docker/build/Dockerfile
  2. 45
      docker/build/README.md
  3. 45
      docker/build/README_zh_CN.md
  4. 4
      docker/build/conf/dolphinscheduler/common.properties.tpl
  5. 2
      docker/build/conf/dolphinscheduler/logback/logback-alert.xml
  6. 4
      docker/build/startup-init-conf.sh
  7. 28
      docker/build/startup.sh
  8. 59
      docker/docker-swarm/docker-compose.yml
  9. 29
      docker/docker-swarm/docker-stack.yml

12
docker/build/Dockerfile
Unescape View File

@ -39,12 +39,12 @@ RUN ln -s /opt/apache-dolphinscheduler-incubating-${VERSION}-dolphinscheduler-bi
ENV DOLPHINSCHEDULER_HOME /opt/dolphinscheduler ENV DOLPHINSCHEDULER_HOME /opt/dolphinscheduler
# 3. add configuration and modify permissions and set soft links # 3. add configuration and modify permissions and set soft links
ADD ./checkpoint.sh /root/checkpoint.sh COPY ./checkpoint.sh /root/checkpoint.sh
ADD ./startup-init-conf.sh /root/startup-init-conf.sh COPY ./startup-init-conf.sh /root/startup-init-conf.sh
ADD ./startup.sh /root/startup.sh COPY ./startup.sh /root/startup.sh
ADD ./conf/dolphinscheduler/*.tpl /opt/dolphinscheduler/conf/ COPY ./conf/dolphinscheduler/*.tpl /opt/dolphinscheduler/conf/
ADD ./conf/dolphinscheduler/logback/* /opt/dolphinscheduler/conf/ COPY ./conf/dolphinscheduler/logback/* /opt/dolphinscheduler/conf/
ADD conf/dolphinscheduler/env/dolphinscheduler_env.sh /opt/dolphinscheduler/conf/env/ COPY ./conf/dolphinscheduler/env/dolphinscheduler_env.sh /opt/dolphinscheduler/conf/env/
RUN dos2unix /root/checkpoint.sh && \ RUN dos2unix /root/checkpoint.sh && \
dos2unix /root/startup-init-conf.sh && \ dos2unix /root/startup-init-conf.sh && \
dos2unix /root/startup.sh && \ dos2unix /root/startup.sh && \

45
docker/build/README.md
Unescape View File

@ -13,39 +13,31 @@ Official Website: https://dolphinscheduler.apache.org
## How to use this docker image ## How to use this docker image
#### You can start a dolphinscheduler instance #### You can start a dolphinscheduler by docker-compose (recommended)
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker-compose -f ./docker/docker-swarm/docker-compose.yml up -d
-e DATABASE_USERNAME=test -e DATABASE_PASSWORD=test -e DATABASE_DATABASE=dolphinscheduler \
-p 12345:12345 \
dolphinscheduler all
``` ```
The default postgres user `root`, postgres password `root` and database `dolphinscheduler` are created in the `startup.sh`. The default **postgres** user `root`, postgres password `root` and database `dolphinscheduler` are created in the `docker-compose.yml`.
The default **zookeeper** is created in the `docker-compose.yml`.
The default zookeeper is created in the `startup.sh`. Access the Web UI：http://192.168.xx.xx:12345
#### Or via Environment Variables **`DATABASE_HOST`** **`DATABASE_PORT`** **`DATABASE_DATABASE`** **`ZOOKEEPER_QUORUM`** #### Or via Environment Variables **`DATABASE_HOST`** **`DATABASE_PORT`** **`DATABASE_DATABASE`** **`ZOOKEEPER_QUORUM`**
You can specify **existing postgres service**. Example: You can specify **existing postgres and zookeeper service**. Example:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-p 12345:12345 \ -p 12345:12345 \
dolphinscheduler all dolphinscheduler all
``` ```
You can specify **existing zookeeper service**. Example: Access the Web UI：http://192.168.xx.xx:12345
```
$ docker run -dit --name dolphinscheduler \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181"
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" -e DATABASE_DATABASE="dolphinscheduler" \
-p 12345:12345 \
dolphinscheduler all
```
#### Or start a standalone dolphinscheduler server #### Or start a standalone dolphinscheduler server
@ -54,8 +46,8 @@ You can start a standalone dolphinscheduler server.
* Start a **master server**, For example: * Start a **master server**, For example:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-master \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" -e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
dolphinscheduler master-server dolphinscheduler master-server
@ -64,17 +56,16 @@ dolphinscheduler master-server
* Start a **worker server**, For example: * Start a **worker server**, For example:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-worker \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" -e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
dolphinscheduler worker-server dolphinscheduler worker-server
``` ```
* Start a **api server**, For example: * Start a **api server**, For example:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-api \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-p 12345:12345 \ -p 12345:12345 \
@ -84,7 +75,7 @@ dolphinscheduler api-server
* Start a **alert server**, For example: * Start a **alert server**, For example:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-alert \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
dolphinscheduler alert-server dolphinscheduler alert-server
@ -106,7 +97,7 @@ $ sh ./docker/build/hooks/build
In Windows, Example: In Windows, Example:
```bat ```bat
c:\incubator-dolphinscheduler>.\docker\build\hooks\build.bat C:\incubator-dolphinscheduler>.\docker\build\hooks\build.bat
``` ```
Please read `./docker/build/hooks/build` `./docker/build/hooks/build.bat` script files if you don't understand Please read `./docker/build/hooks/build` `./docker/build/hooks/build.bat` script files if you don't understand

45
docker/build/README_zh_CN.md
Unescape View File

@ -13,39 +13,31 @@ Official Website: https://dolphinscheduler.apache.org
## 如何使用docker镜像 ## 如何使用docker镜像
#### 你可以运行一个dolphinscheduler实例 #### **以 docker-compose 的方式启动dolphinscheduler(推荐)**
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker-compose -f ./docker/docker-swarm/docker-compose.yml up -d
-e DATABASE_USERNAME=test -e DATABASE_PASSWORD=test -e DATABASE_DATABASE=dolphinscheduler \
-p 12345:12345 \
dolphinscheduler all
``` ```
在`startup.sh`脚本中，默认的创建`Postgres`的用户、密码和数据库，默认值分别为：`root`、`root`、`dolphinscheduler`。 在`docker-compose.yml`文件中，默认的创建`Postgres`的用户、密码和数据库，默认值分别为：`root`、`root`、`dolphinscheduler`。
同时，默认的`Zookeeper`也会在`docker-compose.yml`文件中被创建。
同时，默认的`Zookeeper`也会在`startup.sh`脚本中被创建。 访问前端界面：http://192.168.xx.xx:12345
#### 或者通过环境变量 **`DATABASE_HOST`** **`DATABASE_PORT`** **`ZOOKEEPER_QUORUM`** 使用已存在的服务 #### 或者通过环境变量 **`DATABASE_HOST`** **`DATABASE_PORT`** **`ZOOKEEPER_QUORUM`** 使用已存在的服务
你可以指定一个已经存在的 **`Postgres`** 服务. 如下: 你可以指定已经存在的 **`Postgres`** 和 **`Zookeeper`** 服务. 如下:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-p 12345:12345 \ -p 12345:12345 \
dolphinscheduler all dolphinscheduler all
``` ```
你也可以指定一个已经存在的 **Zookeeper** 服务. 如下: 访问前端界面：http://192.168.xx.xx:12345
```
$ docker run -dit --name dolphinscheduler \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181"
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" -e DATABASE_DATABASE="dolphinscheduler" \
-p 12345:12345 \
dolphinscheduler all
```
#### 或者运行dolphinscheduler中的部分服务 #### 或者运行dolphinscheduler中的部分服务
@ -54,8 +46,8 @@ dolphinscheduler all
* 启动一个 **master server**, 如下: * 启动一个 **master server**, 如下:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-master \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" -e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
dolphinscheduler master-server dolphinscheduler master-server
@ -64,17 +56,16 @@ dolphinscheduler master-server
* 启动一个 **worker server**, 如下: * 启动一个 **worker server**, 如下:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-worker \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" -e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
dolphinscheduler worker-server dolphinscheduler worker-server
``` ```
* 启动一个 **api server**, 如下: * 启动一个 **api server**, 如下:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-api \
-e ZOOKEEPER_QUORUM="l92.168.x.x:2181" \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-p 12345:12345 \ -p 12345:12345 \
@ -84,7 +75,7 @@ dolphinscheduler api-server
* 启动一个 **alert server**, 如下: * 启动一个 **alert server**, 如下:
``` ```
$ docker run -dit --name dolphinscheduler \ $ docker run -d --name dolphinscheduler-alert \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \ -e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \ -e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
dolphinscheduler alert-server dolphinscheduler alert-server
@ -106,7 +97,7 @@ $ sh ./docker/build/hooks/build
Windows系统, 如下: Windows系统, 如下:
```bat ```bat
c:\incubator-dolphinscheduler>.\docker\build\hooks\build.bat C:\incubator-dolphinscheduler>.\docker\build\hooks\build.bat
``` ```
如果你不理解这些脚本 `./docker/build/hooks/build` `./docker/build/hooks/build.bat`，请阅读里面的内容。 如果你不理解这些脚本 `./docker/build/hooks/build` `./docker/build/hooks/build.bat`，请阅读里面的内容。

4
docker/build/conf/dolphinscheduler/common.properties.tpl
Unescape View File

@ -25,7 +25,7 @@ dolphinscheduler.env.path=${DOLPHINSCHEDULER_ENV_PATH}
data.basedir.path=${DOLPHINSCHEDULER_DATA_BASEDIR_PATH} data.basedir.path=${DOLPHINSCHEDULER_DATA_BASEDIR_PATH}
# resource upload startup type : HDFS,S3,NONE # resource upload startup type : HDFS,S3,NONE
resource.storage.type=${DOLPHINSCHEDULER_RESOURCE_STORAGE_TYPE} resource.storage.type=${RESOURCE_STORAGE_TYPE}
#============================================================================ #============================================================================
# HDFS # HDFS
@ -58,7 +58,7 @@ kerberos.expire.time=7
# S3 # S3
#============================================================================ #============================================================================
# if resource.storage.type=S3，the value like: s3a://dolphinscheduler ; if resource.storage.type=HDFS, When namenode HA is enabled, you need to copy core-site.xml and hdfs-site.xml to conf dir # if resource.storage.type=S3，the value like: s3a://dolphinscheduler ; if resource.storage.type=HDFS, When namenode HA is enabled, you need to copy core-site.xml and hdfs-site.xml to conf dir
fs.defaultFS=${DOLPHINSCHEDULER_FS_DEFAULTFS} fs.defaultFS=${FS_DEFAULT_FS}
# if resource.storage.type=S3，s3 endpoint # if resource.storage.type=S3，s3 endpoint
fs.s3a.endpoint=${FS_S3A_ENDPOINT} fs.s3a.endpoint=${FS_S3A_ENDPOINT}

2
docker/build/conf/dolphinscheduler/logback/logback-alert.xml
Unescape View File

@ -37,7 +37,7 @@
</appender> </appender>
<root level="INFO"> <root level="INFO">
<appender-ref ref="APILOGFILE"/> <appender-ref ref="ALERTLOGFILE"/>
</root> </root>
</configuration> </configuration>

4
docker/build/startup-init-conf.sh
Unescape View File

@ -38,9 +38,9 @@ export DATABASE_PARAMS=${DATABASE_PARAMS:-"characterEncoding=utf8"}
#============================================================================ #============================================================================
export DOLPHINSCHEDULER_ENV_PATH=${DOLPHINSCHEDULER_ENV_PATH:-"/opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh"} export DOLPHINSCHEDULER_ENV_PATH=${DOLPHINSCHEDULER_ENV_PATH:-"/opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh"}
export DOLPHINSCHEDULER_DATA_BASEDIR_PATH=${DOLPHINSCHEDULER_DATA_BASEDIR_PATH:-"/tmp/dolphinscheduler"} export DOLPHINSCHEDULER_DATA_BASEDIR_PATH=${DOLPHINSCHEDULER_DATA_BASEDIR_PATH:-"/tmp/dolphinscheduler"}
export DOLPHINSCHEDULER_RESOURCE_STORAGE_TYPE=${DOLPHINSCHEDULER_RESOURCE_STORAGE_TYPE:-"HDFS"} export RESOURCE_STORAGE_TYPE=${RESOURCE_STORAGE_TYPE:-"HDFS"}
export RESOURCE_UPLOAD_PATH=${RESOURCE_UPLOAD_PATH:-"/ds"} export RESOURCE_UPLOAD_PATH=${RESOURCE_UPLOAD_PATH:-"/ds"}
export DOLPHINSCHEDULER_FS_DEFAULTFS=${DOLPHINSCHEDULER_FS_DEFAULTFS:-"file:///data/dolphinscheduler"} export FS_DEFAULT_FS=${FS_DEFAULT_FS:-"file:///"}
export FS_S3A_ENDPOINT=${FS_S3A_ENDPOINT:-"s3.xxx.amazonaws.com"} export FS_S3A_ENDPOINT=${FS_S3A_ENDPOINT:-"s3.xxx.amazonaws.com"}
export FS_S3A_ACCESS_KEY=${FS_S3A_ACCESS_KEY:-"xxxxxxx"} export FS_S3A_ACCESS_KEY=${FS_S3A_ACCESS_KEY:-"xxxxxxx"}
export FS_S3A_SECRET_KEY=${FS_S3A_SECRET_KEY:-"xxxxxxx"} export FS_S3A_SECRET_KEY=${FS_S3A_SECRET_KEY:-"xxxxxxx"}

28
docker/build/startup.sh
Unescape View File

@ -22,8 +22,8 @@ DOLPHINSCHEDULER_BIN=${DOLPHINSCHEDULER_HOME}/bin
DOLPHINSCHEDULER_SCRIPT=${DOLPHINSCHEDULER_HOME}/script DOLPHINSCHEDULER_SCRIPT=${DOLPHINSCHEDULER_HOME}/script
DOLPHINSCHEDULER_LOGS=${DOLPHINSCHEDULER_HOME}/logs DOLPHINSCHEDULER_LOGS=${DOLPHINSCHEDULER_HOME}/logs
# start database # wait database
initDatabase() { waitDatabase() {
echo "test ${DATABASE_TYPE} service" echo "test ${DATABASE_TYPE} service"
while ! nc -z ${DATABASE_HOST} ${DATABASE_PORT}; do while ! nc -z ${DATABASE_HOST} ${DATABASE_PORT}; do
counter=$((counter+1)) counter=$((counter+1))
@ -49,13 +49,16 @@ initDatabase() {
exit 1 exit 1
fi fi
fi fi
}
# init database
initDatabase() {
echo "import sql data" echo "import sql data"
${DOLPHINSCHEDULER_SCRIPT}/create-dolphinscheduler.sh ${DOLPHINSCHEDULER_SCRIPT}/create-dolphinscheduler.sh
} }
# start zk # wait zk
initZK() { waitZK() {
echo "connect remote zookeeper" echo "connect remote zookeeper"
echo "${ZOOKEEPER_QUORUM}" | awk -F ',' 'BEGIN{ i=1 }{ while( i <= NF ){ print $i; i++ } }' | while read line; do echo "${ZOOKEEPER_QUORUM}" | awk -F ',' 'BEGIN{ i=1 }{ while( i <= NF ){ print $i; i++ } }' | while read line; do
while ! nc -z ${line%:*} ${line#*:}; do while ! nc -z ${line%:*} ${line#*:}; do
@ -122,7 +125,8 @@ source /root/startup-init-conf.sh
case "$1" in case "$1" in
(all) (all)
initZK waitZK
waitDatabase
initDatabase initDatabase
initMasterServer initMasterServer
initWorkerServer initWorkerServer
@ -132,26 +136,26 @@ case "$1" in
LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-api-server.log LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-api-server.log
;; ;;
(master-server) (master-server)
initZK waitZK
initDatabase waitDatabase
initMasterServer initMasterServer
LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-master.log LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-master.log
;; ;;
(worker-server) (worker-server)
initZK waitZK
initDatabase
initWorkerServer initWorkerServer
initLoggerServer initLoggerServer
LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-worker.log LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-worker.log
;; ;;
(api-server) (api-server)
initZK waitZK
waitDatabase
initDatabase initDatabase
initApiServer initApiServer
LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-api-server.log LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-api-server.log
;; ;;
(alert-server) (alert-server)
initDatabase waitDatabase
initAlertServer initAlertServer
LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-alert.log LOGFILE=${DOLPHINSCHEDULER_LOGS}/dolphinscheduler-alert.log
;; ;;
@ -166,7 +170,7 @@ case "$1" in
esac esac
# init directories and log files # init directories and log files
mkdir -p ${DOLPHINSCHEDULER_LOGS} && mkdir -p /var/log/nginx/ && cat /dev/null >> ${LOGFILE} mkdir -p ${DOLPHINSCHEDULER_LOGS} && cat /dev/null >> ${LOGFILE}
echo "tail begin" echo "tail begin"
exec bash -c "tail -n 1 -f ${LOGFILE}" exec bash -c "tail -n 1 -f ${LOGFILE}"

59
docker/docker-swarm/docker-compose.yml
Unescape View File

@ -28,10 +28,10 @@ services:
POSTGRESQL_USERNAME: root POSTGRESQL_USERNAME: root
POSTGRESQL_PASSWORD: root POSTGRESQL_PASSWORD: root
POSTGRESQL_DATABASE: dolphinscheduler POSTGRESQL_DATABASE: dolphinscheduler
POSTGRESQL_REPLICATION_PASSWORD: dolphinscheduler@123
volumes: volumes:
- dolphinscheduler-postgresql:/bitnami/postgresql - dolphinscheduler-postgresql:/bitnami/postgresql
- dolphinscheduler-postgresql-initdb:/docker-entrypoint-initdb.d - dolphinscheduler-postgresql-initdb:/docker-entrypoint-initdb.d
restart: unless-stopped
networks: networks:
- dolphinscheduler - dolphinscheduler
@ -46,13 +46,14 @@ services:
ZOO_4LW_COMMANDS_WHITELIST: srvr,ruok,wchs,cons ZOO_4LW_COMMANDS_WHITELIST: srvr,ruok,wchs,cons
volumes: volumes:
- dolphinscheduler-zookeeper:/bitnami/zookeeper - dolphinscheduler-zookeeper:/bitnami/zookeeper
restart: unless-stopped
networks: networks:
- dolphinscheduler - dolphinscheduler
dolphinscheduler-api: dolphinscheduler-api:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
container_name: dolphinscheduler-api container_name: dolphinscheduler-api
command: ["api-server"] command: api-server
ports: ports:
- 12345:12345 - 12345:12345
environment: environment:
@ -63,9 +64,9 @@ services:
DATABASE_PASSWORD: root DATABASE_PASSWORD: root
DATABASE_DATABASE: dolphinscheduler DATABASE_DATABASE: dolphinscheduler
ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181 ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181
ZOOKEEPER_ROOT: /dolphinscheduler RESOURCE_STORAGE_TYPE: HDFS
DOLPHINSCHEDULER_RESOURCE_STORAGE_TYPE: "HDFS" RESOURCE_UPLOAD_PATH: /dolphinscheduler
DOLPHINSCHEDULER_FS_DEFAULTFS: "file:///data/dolphinscheduler" FS_DEFAULT_FS: file:///
healthcheck: healthcheck:
test: ["CMD", "/root/checkpoint.sh", "ApiApplicationServer"] test: ["CMD", "/root/checkpoint.sh", "ApiApplicationServer"]
interval: 30s interval: 30s
@ -77,14 +78,15 @@ services:
- dolphinscheduler-zookeeper - dolphinscheduler-zookeeper
volumes: volumes:
- dolphinscheduler-logs:/opt/dolphinscheduler/logs - dolphinscheduler-logs:/opt/dolphinscheduler/logs
- dolphinscheduler-resource-storage-local:/dolphinscheduler - dolphinscheduler-resource-local:/dolphinscheduler
restart: unless-stopped
networks: networks:
- dolphinscheduler - dolphinscheduler
dolphinscheduler-alert: dolphinscheduler-alert:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
container_name: dolphinscheduler-alert container_name: dolphinscheduler-alert
command: ["alert-server"] command: alert-server
environment: environment:
TZ: Asia/Shanghai TZ: Asia/Shanghai
XLS_FILE_PATH: "/tmp/xls" XLS_FILE_PATH: "/tmp/xls"
@ -116,13 +118,14 @@ services:
- dolphinscheduler-postgresql - dolphinscheduler-postgresql
volumes: volumes:
- dolphinscheduler-logs:/opt/dolphinscheduler/logs - dolphinscheduler-logs:/opt/dolphinscheduler/logs
restart: unless-stopped
networks: networks:
- dolphinscheduler - dolphinscheduler
dolphinscheduler-master: dolphinscheduler-master:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
container_name: dolphinscheduler-master container_name: dolphinscheduler-master
command: ["master-server"] command: master-server
ports: ports:
- 5678:5678 - 5678:5678
environment: environment:
@ -140,7 +143,6 @@ services:
DATABASE_PASSWORD: root DATABASE_PASSWORD: root
DATABASE_DATABASE: dolphinscheduler DATABASE_DATABASE: dolphinscheduler
ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181 ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181
ZOOKEEPER_ROOT: /dolphinscheduler
healthcheck: healthcheck:
test: ["CMD", "/root/checkpoint.sh", "MasterServer"] test: ["CMD", "/root/checkpoint.sh", "MasterServer"]
interval: 30s interval: 30s
@ -152,13 +154,14 @@ services:
- dolphinscheduler-zookeeper - dolphinscheduler-zookeeper
volumes: volumes:
- dolphinscheduler-logs:/opt/dolphinscheduler/logs - dolphinscheduler-logs:/opt/dolphinscheduler/logs
restart: unless-stopped
networks: networks:
- dolphinscheduler - dolphinscheduler
dolphinscheduler-worker: dolphinscheduler-worker:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
container_name: dolphinscheduler-worker container_name: dolphinscheduler-worker
command: ["worker-server"] command: worker-server
ports: ports:
- 1234:1234 - 1234:1234
- 50051:50051 - 50051:50051
@ -170,16 +173,11 @@ services:
WORKER_MAX_CPULOAD_AVG: "100" WORKER_MAX_CPULOAD_AVG: "100"
WORKER_RESERVED_MEMORY: "0.1" WORKER_RESERVED_MEMORY: "0.1"
WORKER_GROUP: "default" WORKER_GROUP: "default"
DOLPHINSCHEDULER_DATA_BASEDIR_PATH: "/tmp/dolphinscheduler" DOLPHINSCHEDULER_DATA_BASEDIR_PATH: /tmp/dolphinscheduler
DATABASE_HOST: dolphinscheduler-postgresql
DATABASE_PORT: 5432
DATABASE_USERNAME: root
DATABASE_PASSWORD: root
DATABASE_DATABASE: dolphinscheduler
ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181 ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181
ZOOKEEPER_ROOT: /dolphinscheduler RESOURCE_STORAGE_TYPE: HDFS
DOLPHINSCHEDULER_RESOURCE_STORAGE_TYPE: "HDFS" RESOURCE_UPLOAD_PATH: /dolphinscheduler
DOLPHINSCHEDULER_FS_DEFAULTFS: "file:///data/dolphinscheduler" FS_DEFAULT_FS: file:///
healthcheck: healthcheck:
test: ["CMD", "/root/checkpoint.sh", "WorkerServer"] test: ["CMD", "/root/checkpoint.sh", "WorkerServer"]
interval: 30s interval: 30s
@ -190,18 +188,11 @@ services:
- dolphinscheduler-postgresql - dolphinscheduler-postgresql
- dolphinscheduler-zookeeper - dolphinscheduler-zookeeper
volumes: volumes:
- type: bind - ./dolphinscheduler_env.sh:/opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh
source: ./dolphinscheduler_env.sh - dolphinscheduler-worker-data:/tmp/dolphinscheduler
target: /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh - dolphinscheduler-logs:/opt/dolphinscheduler/logs
- type: volume - dolphinscheduler-resource-local:/dolphinscheduler
source: dolphinscheduler-worker-data restart: unless-stopped
target: /tmp/dolphinscheduler
- type: volume
source: dolphinscheduler-logs
target: /opt/dolphinscheduler/logs
- type: volume
source: dolphinscheduler-resource-storage-local
target: /dolphinscheduler
networks: networks:
- dolphinscheduler - dolphinscheduler
@ -215,8 +206,4 @@ volumes:
dolphinscheduler-zookeeper: dolphinscheduler-zookeeper:
dolphinscheduler-worker-data: dolphinscheduler-worker-data:
dolphinscheduler-logs: dolphinscheduler-logs:
dolphinscheduler-resource-storage-local: dolphinscheduler-resource-local:
configs:
dolphinscheduler-worker-task-env:
file: ./dolphinscheduler_env.sh

29
docker/docker-swarm/docker-stack.yml
Unescape View File

@ -27,7 +27,6 @@ services:
POSTGRESQL_USERNAME: root POSTGRESQL_USERNAME: root
POSTGRESQL_PASSWORD: root POSTGRESQL_PASSWORD: root
POSTGRESQL_DATABASE: dolphinscheduler POSTGRESQL_DATABASE: dolphinscheduler
POSTGRESQL_REPLICATION_PASSWORD: dolphinscheduler@123
volumes: volumes:
- dolphinscheduler-postgresql:/bitnami/postgresql - dolphinscheduler-postgresql:/bitnami/postgresql
networks: networks:
@ -54,7 +53,7 @@ services:
dolphinscheduler-api: dolphinscheduler-api:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
command: ["api-server"] command: api-server
ports: ports:
- 12345:12345 - 12345:12345
environment: environment:
@ -65,7 +64,9 @@ services:
DATABASE_PASSWORD: root DATABASE_PASSWORD: root
DATABASE_DATABASE: dolphinscheduler DATABASE_DATABASE: dolphinscheduler
ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181 ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181
ZOOKEEPER_ROOT: /dolphinscheduler RESOURCE_STORAGE_TYPE: HDFS
RESOURCE_UPLOAD_PATH: /dolphinscheduler
FS_DEFAULT_FS: file:///
healthcheck: healthcheck:
test: ["CMD", "/root/checkpoint.sh", "ApiApplicationServer"] test: ["CMD", "/root/checkpoint.sh", "ApiApplicationServer"]
interval: 30s interval: 30s
@ -82,7 +83,7 @@ services:
dolphinscheduler-alert: dolphinscheduler-alert:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
command: ["alert-server"] command: alert-server
environment: environment:
TZ: Asia/Shanghai TZ: Asia/Shanghai
XLS_FILE_PATH: "/tmp/xls" XLS_FILE_PATH: "/tmp/xls"
@ -120,7 +121,7 @@ services:
dolphinscheduler-master: dolphinscheduler-master:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
command: ["master-server"] command: master-server
ports: ports:
- 5678:5678 - 5678:5678
environment: environment:
@ -154,7 +155,7 @@ services:
dolphinscheduler-worker: dolphinscheduler-worker:
image: apache/dolphinscheduler:latest image: apache/dolphinscheduler:latest
command: ["worker-server"] command: worker-server
ports: ports:
- 1234:1234 - 1234:1234
- 50051:50051 - 50051:50051
@ -166,25 +167,23 @@ services:
WORKER_MAX_CPULOAD_AVG: "100" WORKER_MAX_CPULOAD_AVG: "100"
WORKER_RESERVED_MEMORY: "0.1" WORKER_RESERVED_MEMORY: "0.1"
WORKER_GROUP: "default" WORKER_GROUP: "default"
DOLPHINSCHEDULER_DATA_BASEDIR_PATH: "/tmp/dolphinscheduler" DOLPHINSCHEDULER_DATA_BASEDIR_PATH: /tmp/dolphinscheduler
DATABASE_HOST: dolphinscheduler-postgresql
DATABASE_PORT: 5432
DATABASE_USERNAME: root
DATABASE_PASSWORD: root
DATABASE_DATABASE: dolphinscheduler
ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181 ZOOKEEPER_QUORUM: dolphinscheduler-zookeeper:2181
RESOURCE_STORAGE_TYPE: HDFS
RESOURCE_UPLOAD_PATH: /dolphinscheduler
FS_DEFAULT_FS: file:///
healthcheck: healthcheck:
test: ["CMD", "/root/checkpoint.sh", "WorkerServer"] test: ["CMD", "/root/checkpoint.sh", "WorkerServer"]
interval: 30s interval: 30s
timeout: 5s timeout: 5s
retries: 3 retries: 3
start_period: 30s start_period: 30s
volumes:
- dolphinscheduler-worker-data:/tmp/dolphinscheduler
- dolphinscheduler-logs:/opt/dolphinscheduler/logs
configs: configs:
- source: dolphinscheduler-worker-task-env - source: dolphinscheduler-worker-task-env
target: /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh target: /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh
volumes:
- dolphinscheduler-worker-data:/tmp/dolphinscheduler
- dolphinscheduler-logs:/opt/dolphinscheduler/logs
networks: networks:
- dolphinscheduler - dolphinscheduler
deploy: deploy:

Write Preview
Loading…
Cancel
Save