Remove sensitive configs from dolphinscheduler_env.sh (#11917)

docs/docs/en/architecture/configuration.md

@@ -339,28 +339,12 @@ The default configuration is as follows:
 ### dolphinscheduler_env.sh [load environment variables configs]
 
 When using shell to commit tasks, DolphinScheduler will export environment variables from `bin/env/dolphinscheduler_env.sh`. The
-mainly configuration including `JAVA_HOME`, mata database, registry center, and task configuration.
+mainly configuration including `JAVA_HOME` and other environment paths.
 
 ```bash
 # JAVA_HOME, will use it to start DolphinScheduler server
 export JAVA_HOME=${JAVA_HOME:-/opt/soft/java}
 
-# Database related configuration, set database type, username and password
-export DATABASE=${DATABASE:-postgresql}
-export SPRING_PROFILES_ACTIVE=${DATABASE}
-export SPRING_DATASOURCE_URL
-export SPRING_DATASOURCE_USERNAME
-export SPRING_DATASOURCE_PASSWORD
-
-# DolphinScheduler server related configuration
-export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none}
-export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC}
-export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10}
-
-# Registry center configuration, determines the type and link of the registry center
-export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper}
-export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}
-
 # Tasks related configurations, need to change the configuration if you use the related tasks.
 export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop}
 export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/opt/soft/hadoop/etc/hadoop}

docs/docs/en/guide/howto/datasource-setting.md

@@ -8,11 +8,12 @@ We here use MySQL as an example to illustrate how to configure an external datab
 which is `api-server/libs` and `alert-server/libs` and `master-server/libs` and `worker-server/libs`.
 
 * First of all, follow the instructions in [datasource-setting](datasource-setting.md) `Pseudo-Cluster/Cluster Initialize the Database` section to create and initialize database
-* Set the following environment variables in your terminal or modify the `bin/env/dolphinscheduler_env.sh` with your database username and password for `{user}` and `{password}`:
+* Set the following environment variables in your terminal with your database address, username and password for `{address}`, `{user}` and `{password}`:
 
 ```shell
 export DATABASE=mysql
 export SPRING_PROFILES_ACTIVE=${DATABASE}
+export SPRING_DATASOURCE_URL="jdbc:mysql://{address}/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
 export SPRING_DATASOURCE_USERNAME={user}
 export SPRING_DATASOURCE_PASSWORD={password}
 ```
@@ -54,9 +55,9 @@ mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%';
 mysql> CREATE USER '{user}'@'localhost' IDENTIFIED BY '{password}';
 mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost';
 mysql> FLUSH PRIVILEGES;
-``` 
+```
 
-For PostgreSQL: 
+For PostgreSQL:
 ```shell
 # Use psql-tools to login PostgreSQL
 psql
@@ -72,9 +73,10 @@ echo "host    dolphinscheduler   {user}    {ip}     md5" >> $PGDATA/pg_hba.conf
 pg_ctl reload
 ```
 
-Then, modify `./bin/env/dolphinscheduler_env.sh`, change {user} and {password} to what you set in the previous step.
+Then, set the database configurations by exporting the following environment variables, change {user} and {password} to what you set in the previous step.
 
 For MySQL:
+
 ```shell
 # for mysql
 export DATABASE=${DATABASE:-mysql}
@@ -85,6 +87,7 @@ export SPRING_DATASOURCE_PASSWORD={password}
 ```
 
 For PostgreSQL:
+
 ```shell
 # for postgresql
 export DATABASE=${DATABASE:-postgresql}

docs/docs/en/guide/upgrade/upgrade.md

@@ -23,10 +23,10 @@ Stop all services of dolphinscheduler according to your deployment method. If yo
 
 ### Upgrade Database
 
-Change configuration in `./bin/env/dolphinscheduler_env.sh` ({user} and {password} are changed to your database username and password), and then run the upgrade script.
+Set the following environment variables ({user} and {password} are changed to your database username and password), and then run the upgrade script.
 
 Using MySQL as an example, change the value if you use other databases. Please manually download the [mysql-connector-java driver jar](https://downloads.MySQL.com/archives/c-j/)
-jar package and add it to the `./tools/libs` directory, then change `./bin/ env/dolphinscheduler_env.sh` file
+jar package and add it to the `./tools/libs` directory, then export the following environment variables
 
     ```shell
     export DATABASE=${DATABASE:-mysql}
@@ -45,7 +45,7 @@ Execute database upgrade script: `sh ./tools/bin/upgrade-schema.sh`
 - If you deploy with Pseudo-Cluster deployment, change it according to [Pseudo-Cluster](../installation/pseudo-cluster.md) section "Modify Configuration".
 - If you deploy with Cluster deployment, change it according to [Cluster](../installation/cluster.md) section "Modify Configuration".
 
-And them run command `sh ./bin/start-all.sh` to start all services. 
+And them run command `sh ./bin/start-all.sh` to start all services.
 
 ## Notice
 
@@ -54,7 +54,7 @@ And them run command `sh ./bin/start-all.sh` to start all services.
 The architecture of worker group is different between version before version 1.3.1 until version 2.0.0
 
 - Before version 1.3.1(include itself) worker group can be created through UI interface.
-- Since version 1.3.1 and before version 2.0.0, worker group can be created by modifying the worker configuration. 
+- Since version 1.3.1 and before version 2.0.0, worker group can be created by modifying the worker configuration.
 
 #### How Can I Do When I Upgrade from 1.3.1 to version before 2.0.0
 
@@ -78,13 +78,13 @@ Assume bellow are the machine worker service to be deployed:
 To keep worker group config consistent with the previous version, we need to modify workers configuration as below:
 
 ```shell
-#worker service is deployed on which machine, and also specify which worker group this worker belongs to. 
+#worker service is deployed on which machine, and also specify which worker group this worker belongs to.
 workers="ds1:service1,ds2:service2,ds3:service2"
 ```
 
 #### The Worker Group has Been Enhanced in Version 1.3.2
 
-Workers in 1.3.1 can only belong to one worker group, but after version 1.3.2 and before version 2.0.0 worker support more than one worker group. 
+Workers in 1.3.1 can only belong to one worker group, but after version 1.3.2 and before version 2.0.0 worker support more than one worker group.
 
 ```sh
 workers="ds1:service1,ds1:service2"

docs/docs/zh/architecture/configuration.md

@@ -147,7 +147,7 @@ export DOLPHINSCHEDULER_OPTS="
 |spring.datasource.hikari.leak-detection-threshold|0|连接泄露检测阈值|
 |spring.datasource.hikari.initialization-fail-timeout|1|连接池初始化失败timeout|
 
-DolphinScheduler同样可以通过`bin/env/dolphinscheduler_env.sh`进行数据库连接相关的配置。
+DolphinScheduler同样可以通过设置环境变量进行数据库连接相关的配置, 将以上小写字母转成大写并把`.`换成`_`作为环境变量名, 设置值即可。
 
 
 ## Zookeeper相关配置
@@ -322,28 +322,12 @@ common.properties配置文件目前主要是配置hadoop/s3/yarn相关的配置
 
 ## dolphinscheduler_env.sh [环境变量配置]
 
-通过类似shell方式提交任务的的时候，会加载该配置文件中的环境变量到主机中。涉及到的 `JAVA_HOME`、元数据库、注册中心和任务类型配置，其中任务类型主要有: Shell任务、Python任务、Spark任务、Flink任务、Datax任务等等。
+通过类似shell方式提交任务的的时候，会加载该配置文件中的环境变量到主机中。涉及到的 `JAVA_HOME` 任务类型的环境配置，其中任务类型主要有: Shell任务、Python任务、Spark任务、Flink任务、Datax任务等等。
 
 ```bash
 # JAVA_HOME, will use it to start DolphinScheduler server
 export JAVA_HOME=${JAVA_HOME:-/opt/soft/java}
 
-# Database related configuration, set database type, username and password
-export DATABASE=${DATABASE:-postgresql}
-export SPRING_PROFILES_ACTIVE=${DATABASE}
-export SPRING_DATASOURCE_URL
-export SPRING_DATASOURCE_USERNAME
-export SPRING_DATASOURCE_PASSWORD
-
-# DolphinScheduler server related configuration
-export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none}
-export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC}
-export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10}
-
-# Registry center configuration, determines the type and link of the registry center
-export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper}
-export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}
-
 # Tasks related configurations, need to change the configuration if you use the related tasks.
 export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop}
 export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/opt/soft/hadoop/etc/hadoop}

docs/docs/zh/guide/howto/datasource-setting.md

@@ -7,11 +7,12 @@
 > 如果使用 MySQL 需要手动下载 [mysql-connector-java 驱动][mysql] (8.0.16) 并移动到 DolphinScheduler 的每个模块的 libs 目录下，其中包括 `api-server/libs` 和 `alert-server/libs` 和 `master-server/libs` 和 `worker-server/libs`。
 
 * 首先，参照 [数据源配置](datasource-setting.md) `伪分布式/分布式安装初始化数据库` 创建并初始化数据库
-* 在你的命令行或者修改 bin/env/dolphinscheduler_env.sh 设定下列环境变量，将 `{user}` 和 `{password}` 改为你数据库的用户名和密码
+* 在你的命令行设定下列环境变量，将 `{address}`, `{user}` 和 `{password}` 改为你数据库的地址, 用户名和密码
 
 ```shell
 export DATABASE=mysql
 export SPRING_PROFILES_ACTIVE=${DATABASE}
+export SPRING_DATASOURCE_URL="jdbc:mysql://{address}/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
 export SPRING_DATASOURCE_USERNAME={user}
 export SPRING_DATASOURCE_PASSWORD={password}
 ```
@@ -72,9 +73,10 @@ echo "host    dolphinscheduler   {user}    {ip}     md5" >> $PGDATA/pg_hba.conf
 pg_ctl reload
 ```
 
-然后修改`./bin/env/dolphinscheduler_env.sh`，将username和password改成你在上一步中设置的用户名{user}和密码{password}
+然后设置以下环境变量，将username和password改成你在上一步中设置的用户名{user}和密码{password}
 
 对于 MySQL：
+
 ```shell
 # for mysql
 export DATABASE=${DATABASE:-mysql}
@@ -82,9 +84,10 @@ export SPRING_PROFILES_ACTIVE=${DATABASE}
 export SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
 export SPRING_DATASOURCE_USERNAME={user}
 export SPRING_DATASOURCE_PASSWORD={password}
-```  
+```
 
 对于 PostgreSQL：
+
 ```shell
 # for postgresql
 export DATABASE=${DATABASE:-postgresql}

docs/docs/zh/guide/howto/general-setting.md

@@ -16,7 +16,7 @@ DolphinScheduler 支持时区设置。
 
 服务时区
 
-使用脚本 `bin/dolphinshceduler_daemon.sh`启动服务， 服务的默认时区为UTC， 可以在`bin/env/dolphinscheduler_env.sh`中进行修改， 如`export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-Asia/Shanghai}`。<br>
+使用脚本 `bin/dolphinshceduler_daemon.sh`启动服务， 服务的默认时区为UTC， 可以在 `application.yaml` 文件中进行修改，或通过环境变量修改, 如`export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-Asia/Shanghai}`。<br>
 IDEA 启动服务默认时区为本地时区，可以加jvm参数如`-Duser.timezone=UTC`来修改时区。 时区选择详见[List of tz database time zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
 
 用户时区

docs/docs/zh/guide/installation/pseudo-cluster.md

@@ -100,8 +100,6 @@ deployUser="dolphinscheduler"
 
 * DolphinScheduler 的数据库配置，详细配置方法见[初始化数据库](#初始化数据库)
 * 一些任务类型外部依赖路径或库文件，如 `JAVA_HOME` 和 `SPARK_HOME`都是在这里定义的
-* 注册中心`zookeeper`
-* 服务端相关配置，比如缓存，时区设置等
 
 如果您不使用某些任务类型，您可以忽略任务外部依赖项，但您必须根据您的环境更改 `JAVA_HOME`、注册中心和数据库相关配置。
 

docs/docs/zh/guide/start/docker.md

@@ -2,7 +2,7 @@
 
 本教程使用三种不同的方式通过 Docker 完成 DolphinScheduler 的部署，如果你想要快速体验，推荐使用 standalone-server 镜像，
 如果你想要体验比较完成的服务，推荐使用 docker-compose 启动服务。如果你已经有自己的数据库或者 Zookeeper 服务
-你想要沿用这些基础服务，你可以参考沿用已有的 PostgreSQL 和 ZooKeeper 服务完成部署。 
+你想要沿用这些基础服务，你可以参考沿用已有的 PostgreSQL 和 ZooKeeper 服务完成部署。
 
 ## 前置条件
 
@@ -56,7 +56,7 @@ $ docker-compose --profile schema up -d
 $ docker-compose --profile all up -d
 ```
 
-> 提醒：通过 docker-compose 启动服务时，除了会启动 DolphinScheduler 对应的服务外，还会启动必要依赖服务，如数据库 PostgreSQL(用户 
+> 提醒：通过 docker-compose 启动服务时，除了会启动 DolphinScheduler 对应的服务外，还会启动必要依赖服务，如数据库 PostgreSQL(用户
 > `root`, 密码 `root`, 数据库 `dolphinscheduler`) 和 服务发现 ZooKeeper。
 
 ### 沿用已有的 PostgreSQL 和 ZooKeeper 服务
@@ -126,4 +126,4 @@ $ docker run -d --name dolphinscheduler-alert-server \
 ## 环境变量
 
 可以通过环境变量来修改 Docker 运行的配置，我们在沿用已有的 PostgreSQL 和 ZooKeeper 服务中就通过环境变量修改了 Docker 的数据库配置和
-注册中心配置，关于全部的配置环境可以查看[全部的配置文件](https://github.com/apache/dolphinscheduler/blob/<version>/script/env/dolphinscheduler_env.sh) 了解 <!-- markdown-link-check-disable-line -->
+注册中心配置，关于全部的配置环境可以查看对应组件的 application.yaml 文件了解 <!-- markdown-link-check-disable-line -->

docs/docs/zh/guide/upgrade/upgrade.md

@@ -22,10 +22,10 @@
 
 ### 数据库升级
 
-修改 `./bin/env/dolphinscheduler_env.sh` 中的如下配置（{user}和{password}改成你数据库的用户名和密码），然后运行升级脚本。
+设置相关环境变量（{user}和{password}改成你数据库的用户名和密码），然后运行升级脚本。
 
 下面以 MySQL 为例，别的数据库仅需要修改成对应的配置即可。请先手动下载 [mysql-connector-java 驱动 jar](https://downloads.MySQL.com/archives/c-j/)
-jar 包 并添加到 `./tools/libs` 目录下，修改 `./bin/env/dolphinscheduler_env.sh` 文件
+jar 包 并添加到 `./tools/libs` 目录下，设置以下环境变量
 
     ```shell
     export DATABASE=${DATABASE:-mysql}

script/env/dolphinscheduler_env.sh

@@ -18,21 +18,8 @@
 # JAVA_HOME, will use it to start DolphinScheduler server
 export JAVA_HOME=${JAVA_HOME:-/opt/soft/java}
 
-# Database related configuration, set database type, username and password
-export DATABASE=${DATABASE:-postgresql}
-export SPRING_PROFILES_ACTIVE=${DATABASE}
-export SPRING_DATASOURCE_URL
-export SPRING_DATASOURCE_USERNAME
-export SPRING_DATASOURCE_PASSWORD
-
-# DolphinScheduler server related configuration
-export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none}
-export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC}
-export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10}
-
-# Registry center configuration, determines the type and link of the registry center
-export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper}
-export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}
+# Never put sensitive config such as database password here in your production environment,
+# this file will be sourced everytime a new task is executed.
 
 # Tasks related configurations, need to change the configuration if you use the related tasks.
 export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop}