From c76301ab51735a79d6bfd3c00288ce7d5fe60e54 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E9=98=BF=E7=A6=8FChris?= Date: Mon, 5 Sep 2022 10:54:01 +0800 Subject: [PATCH] [Fix-11217] [Doc] add postgresql config in doc: datasource-setting (#11326) * add postgresql config for datasoruce-setting * Update docs/docs/en/guide/howto/datasource-setting.md Co-authored-by: Eric Gao --- .../docs/en/guide/howto/datasource-setting.md | 39 +++++++++++++++++-- .../docs/zh/guide/howto/datasource-setting.md | 37 +++++++++++++++++- .../zh/guide/installation/pseudo-cluster.md | 2 +- 3 files changed, 72 insertions(+), 6 deletions(-) diff --git a/docs/docs/en/guide/howto/datasource-setting.md b/docs/docs/en/guide/howto/datasource-setting.md index 7db7662523..4dddb9a7a2 100644 --- a/docs/docs/en/guide/howto/datasource-setting.md +++ b/docs/docs/en/guide/howto/datasource-setting.md @@ -4,6 +4,9 @@ We here use MySQL as an example to illustrate how to configure an external database: +> NOTE: If you use MySQL, you need to manually download [mysql-connector-java driver][mysql] (8.0.16) and move it to the libs directory of DolphinScheduler +which is `api-server/libs` and `alert-server/libs` and `master-server/libs` and `worker-server/libs`. + * First of all, follow the instructions in [datasource-setting](datasource-setting.md) `Pseudo-Cluster/Cluster Initialize the Database` section to create and initialize database * Set the following environment variables in your terminal or modify the `bin/env/dolphinscheduler_env.sh` with your database username and password for `{user}` and `{password}`: @@ -19,8 +22,10 @@ export SPRING_DATASOURCE_PASSWORD={password} ## Pseudo-Cluster/Cluster Initialize the Database -DolphinScheduler metadata is stored in the relational database. Currently, supports PostgreSQL and MySQL. If you use MySQL, you need to manually download [mysql-connector-java driver][mysql] (8.0.16) and move it to the libs directory of DolphinScheduler -which is `api-server/libs/` and `alert-server/libs` and `master-server/libs` and `worker-server/libs` and `tools/libs`. Let's take MySQL as an example for how to initialize the database: +DolphinScheduler stores metadata in `relational database`. Currently, we support `PostgreSQL` and `MySQL`. Let's walk through how to initialize the database in `MySQL` and `PostgreSQL` : + +> If you use MySQL, you need to manually download [mysql-connector-java driver][mysql] (8.0.16) and move it to the libs directory of DolphinScheduler which is `api-server/libs` and `alert-server/libs` and `master-server/libs` and `worker-server/libs`. + For mysql 5.6 / 5.7 @@ -51,9 +56,27 @@ mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost'; mysql> FLUSH PRIVILEGES; ``` -Then, modify `./bin/env/dolphinscheduler_env.sh` to use mysql, change {user} and {password} to what you set in the previous step. +For PostgreSQL: +```shell +# Use psql-tools to login PostgreSQL +psql +# Create a database +postgres=# CREATE DATABASE dolphinscheduler; +# Replace {user} and {password} with your username and password +postgres=# CREATE USER {user} PASSWORD {password}; +postgres=# ALTER DATABASE dolphinscheduler OWNER TO {user}; +# Logout PostgreSQL +postgres=#\q +# Exec cmd below in terminal, add config to pg_hba.conf and reload PostgreSQL config, replace {ip} to DS cluster ip addresses +echo "host dolphinscheduler {user} {ip} md5" >> $PGDATA/pg_hba.conf +pg_ctl reload +``` + +Then, modify `./bin/env/dolphinscheduler_env.sh`, change {user} and {password} to what you set in the previous step. +For MySQL: ```shell +# for mysql export DATABASE=${DATABASE:-mysql} export SPRING_PROFILES_ACTIVE=${DATABASE} export SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false" @@ -61,6 +84,16 @@ export SPRING_DATASOURCE_USERNAME={user} export SPRING_DATASOURCE_PASSWORD={password} ``` +For PostgreSQL: +```shell +# for postgresql +export DATABASE=${DATABASE:-postgresql} +export SPRING_PROFILES_ACTIVE=${DATABASE} +export SPRING_DATASOURCE_URL="jdbc:postgresql://127.0.0.1:5432/dolphinscheduler" +export SPRING_DATASOURCE_USERNAME={user} +export SPRING_DATASOURCE_PASSWORD={password} +``` + After the above steps done you would create a new database for DolphinScheduler, then run the Shell script to init database: ```shell diff --git a/docs/docs/zh/guide/howto/datasource-setting.md b/docs/docs/zh/guide/howto/datasource-setting.md index 97154659b2..f9a223ee84 100644 --- a/docs/docs/zh/guide/howto/datasource-setting.md +++ b/docs/docs/zh/guide/howto/datasource-setting.md @@ -4,6 +4,8 @@ 我们这里以 MySQL 为例来说明如何配置外部数据库： +> 如果使用 MySQL 需要手动下载 [mysql-connector-java 驱动][mysql] (8.0.16) 并移动到 DolphinScheduler 的每个模块的 libs 目录下，其中包括 `api-server/libs` 和 `alert-server/libs` 和 `master-server/libs` 和 `worker-server/libs`。 + * 首先，参照 [数据源配置](datasource-setting.md) `伪分布式/分布式安装初始化数据库` 创建并初始化数据库 * 在你的命令行或者修改 bin/env/dolphinscheduler_env.sh 设定下列环境变量，将 `{user}` 和 `{password}` 改为你数据库的用户名和密码 @@ -19,8 +21,10 @@ export SPRING_DATASOURCE_PASSWORD={password} ## 伪分布式/分布式安装初始化数据库 -DolphinScheduler 元数据存储在关系型数据库中，目前支持 PostgreSQL 和 MySQL，如果使用 MySQL 则需要手动下载 [mysql-connector-java 驱动][mysql] (8.0.16) 并移动到 DolphinScheduler 的每个模块的 libs 目录下 -其中包括 `api-server/libs/` 和 `alert-server/libs` 和 `master-server/libs` 和 `worker-server/libs` 和 `tools/libs`。下面以 MySQL 为例，说明如何初始化数据库 +DolphinScheduler 元数据存储在关系型数据库中，目前支持 PostgreSQL 和 MySQL。下面分别介绍如何使用 MySQL 和 PostgresQL 初始化数据库。 + +> 如果使用 MySQL 需要手动下载 [mysql-connector-java 驱动][mysql] (8.0.16) 并移动到 DolphinScheduler 的每个模块的 libs 目录下，其中包括 `api-server/libs` 和 `alert-server/libs` 和 `master-server/libs` 和 `worker-server/libs`。 + 对于mysql 5.6 / 5.7： @@ -51,9 +55,28 @@ mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost'; mysql> FLUSH PRIVILEGES; ``` +对于 PostgreSQL： + +```shell +# 采用命令行工具登陆 PostgreSQL +psql +# 创建数据库 +postgres=# CREATE DATABASE dolphinscheduler; +# 修改 {user} 和 {password} 为你希望的用户名和密码 +postgres=# CREATE USER {user} PASSWORD {password}; +postgres=# ALTER DATABASE dolphinscheduler OWNER TO {user}; +# 退出 PostgreSQL +postgres=#\q +# 在终端执行如下命令，向配置文件新增登陆权限，并重载 PostgreSQL 配置，替换 {ip} 为对应的 DS 集群服务器 IP 地址段 +echo "host dolphinscheduler {user} {ip} md5" >> $PGDATA/pg_hba.conf +pg_ctl reload +``` + 然后修改`./bin/env/dolphinscheduler_env.sh`，将username和password改成你在上一步中设置的用户名{user}和密码{password} +对于 MySQL： ```shell +# for mysql export DATABASE=${DATABASE:-mysql} export SPRING_PROFILES_ACTIVE=${DATABASE} export SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false" @@ -61,6 +84,16 @@ export SPRING_DATASOURCE_USERNAME={user} export SPRING_DATASOURCE_PASSWORD={password} ``` +对于 PostgreSQL： +```shell +# for postgresql +export DATABASE=${DATABASE:-postgresql} +export SPRING_PROFILES_ACTIVE=${DATABASE} +export SPRING_DATASOURCE_URL="jdbc:postgresql://127.0.0.1:5432/dolphinscheduler" +export SPRING_DATASOURCE_USERNAME={user} +export SPRING_DATASOURCE_PASSWORD={password} +``` + 完成上述步骤后，您已经为 DolphinScheduler 创建一个新数据库，现在你可以通过快速的 Shell 脚本来初始化数据库 ```shell diff --git a/docs/docs/zh/guide/installation/pseudo-cluster.md b/docs/docs/zh/guide/installation/pseudo-cluster.md index 6fb20f6304..3856f884c7 100644 --- a/docs/docs/zh/guide/installation/pseudo-cluster.md +++ b/docs/docs/zh/guide/installation/pseudo-cluster.md @@ -150,7 +150,7 @@ export PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME/bin: bash ./bin/install.sh ``` -> **_注意:_** 第一次部署的话，可能出现 5 次`sh: bin/dolphinscheduler-daemon.sh: No such file or directory`相关信息，次为非重要信息直接忽略即可 +> **_注意:_** 第一次部署的话，可能出现 5 次`sh: bin/dolphinscheduler-daemon.sh: No such file or directory`相关信息，此为非重要信息直接忽略即可 ## 登录 DolphinScheduler