github
/
DolphinScheduler
mirror of https://gitee.com/dolphinscheduler/DolphinScheduler.git
1
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

[doc] Correct docs of development-environment-setup (#9995) 

(cherry picked from commit f666c64946)
3.0.0/version-upgrade
chuxing 2 years ago committed by Jiajie Zhong
parent
25538aaf67
commit
ea284526f9
2 changed files with 37 additions and 55 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 41
      docs/docs/en/development/development-environment-setup.md
  2. 41
      docs/docs/zh/development/development-environment-setup.md

41
docs/docs/en/development/development-environment-setup.md
Unescape View File

@ -5,10 +5,9 @@
Before setting up the DolphinScheduler development environment, please make sure you have installed the software as below: Before setting up the DolphinScheduler development environment, please make sure you have installed the software as below:
* [Git](https://git-scm.com/downloads): DolphinScheduler version control system * [Git](https://git-scm.com/downloads): DolphinScheduler version control system
* [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html): DolphinScheduler backend language * [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html): DolphinScheduler backend language, have to use version after JDK 1.8
* [Maven](http://maven.apache.org/download.cgi): Java Package Management System * [Maven](http://maven.apache.org/download.cgi): Java Package Management System
* [Node](https://nodejs.org/en/download): DolphinScheduler frontend * [Node](https://nodejs.org/en/download): DolphinScheduler frontend, have to use version after Node 12.20.2
language
### Clone Git Repository ### Clone Git Repository
@ -19,13 +18,11 @@ mkdir dolphinscheduler
cd dolphinscheduler cd dolphinscheduler
git clone git@github.com:apache/dolphinscheduler.git git clone git@github.com:apache/dolphinscheduler.git
``` ```
### compile source code
i. If you use MySQL database, pay attention to modify pom.xml in the root project, and change the scope of the mysql-connector-java dependency to compile. ### compile source code
ii. Run `mvn clean install -Prelease -Dmaven.test.skip=true` ii. Run `mvn clean install -Prelease -Dmaven.test.skip=true`
## Notice ## Notice
There are two ways to configure the DolphinScheduler development environment, standalone mode and normal mode There are two ways to configure the DolphinScheduler development environment, standalone mode and normal mode
@ -52,7 +49,8 @@ Find the class `org.apache.dolphinscheduler.server.StandaloneServer` in Intellij
### Start frontend server ### Start frontend server
Install frontend dependencies and run it Install frontend dependencies and run it.
> Note: You can see more detail about the frontend setting in [frontend development](./frontend-development.md).
```shell ```shell
cd dolphinscheduler-ui cd dolphinscheduler-ui
@ -94,36 +92,29 @@ Following steps will guide how to start the DolphinScheduler backend service
##### Backend Start Prepare ##### Backend Start Prepare
* Open project: Use IDE open the project, here we use Intellij IDEA as an example, after opening it will take a while for Intellij IDEA to complete the dependent download * Open project: Use IDE open the project, here we use Intellij IDEA as an example, after opening it will take a while for Intellij IDEA to complete the dependent download
* Plugin installation(**Only required for 2.0 or later**)
* Registry plug-in configuration, take Zookeeper as an example (registry.properties)
dolphinscheduler-service/src/main/resources/registry.properties
```registry.properties
registry.plugin.name=zookeeper
registry.servers=127.0.0.1:2181
```
* File change * File change
* If you use MySQL as your metadata database, you need to modify `dolphinscheduler/pom.xml` and change the `scope` of the `mysql-connector-java` dependency to `compile`. This step is not necessary to use PostgreSQL * If you use MySQL as your metadata database, you need to modify `dolphinscheduler/pom.xml` and change the `scope` of the `mysql-connector-java` dependency to `compile`. This step is not necessary to use PostgreSQL
* Modify database configuration, modify the database configuration in the `dolphinscheduler-dao/src/main/resources/application-mysql.yaml` * Modify database configuration, modify the database configuration in the `dolphinscheduler-master/src/main/resources/application.yaml`
* Modify database configuration, modify the database configuration in the `dolphinscheduler-worker/src/main/resources/application.yaml`
* Modify database configuration, modify the database configuration in the `dolphinscheduler-api/src/main/resources/application.yaml`
We here use MySQL with database, username, password named dolphinscheduler as an example We here use MySQL with database, username, password named dolphinscheduler as an example
```application-mysql.yaml ```application.yaml
spring: spring:
datasource: datasource:
driver-class-name: com.mysql.jdbc.Driver driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8 url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
username: ds_user username: dolphinscheduler
password: dolphinscheduler password: dolphinscheduler
``` ```
* Log level: add a line `<appender-ref ref="STDOUT"/>` to the following configuration to enable the log to be displayed on the command line * Log level: add a line `<appender-ref ref="STDOUT"/>` to the following configuration to enable the log to be displayed on the command line
`dolphinscheduler-server/src/main/resources/logback-worker.xml` `dolphinscheduler-master/src/main/resources/logback-spring.xml`
`dolphinscheduler-worker/src/main/resources/logback-spring.xml`
`dolphinscheduler-server/src/main/resources/logback-master.xml` `dolphinscheduler-api/src/main/resources/logback-spring.xml`
`dolphinscheduler-api/src/main/resources/logback-api.xml`
here we add the result after modify as below: here we add the result after modify as below:
@ -141,9 +132,9 @@ Following steps will guide how to start the DolphinScheduler backend service
There are three services that need to be started, including MasterServer, WorkerServer, ApiApplicationServer. There are three services that need to be started, including MasterServer, WorkerServer, ApiApplicationServer.
* MasterServer：Execute function `main` in the class `org.apache.dolphinscheduler.server.master.MasterServer` by Intellij IDEA, with the configuration *VM Options* `-Dlogging.config=classpath:logback-master.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql` * MasterServer：Execute function `main` in the class `org.apache.dolphinscheduler.server.master.MasterServer` by Intellij IDEA, with the configuration *VM Options* `-Dlogging.config=classpath:logback-spring.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql`
* WorkerServer：Execute function `main` in the class `org.apache.dolphinscheduler.server.worker.WorkerServer` by Intellij IDEA, with the configuration *VM Options* `-Dlogging.config=classpath:logback-worker.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql` * WorkerServer：Execute function `main` in the class `org.apache.dolphinscheduler.server.worker.WorkerServer` by Intellij IDEA, with the configuration *VM Options* `-Dlogging.config=classpath:logback-spring.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql`
* ApiApplicationServer：Execute function `main` in the class `org.apache.dolphinscheduler.api.ApiApplicationServer` by Intellij IDEA, with the configuration *VM Options* `-Dlogging.config=classpath:logback-api.xml -Dspring.profiles.active=api,mysql`. After it started, you could find Open API documentation in http://localhost:12345/dolphinscheduler/doc.html * ApiApplicationServer：Execute function `main` in the class `org.apache.dolphinscheduler.api.ApiApplicationServer` by Intellij IDEA, with the configuration *VM Options* `-Dlogging.config=classpath:logback-spring.xml -Dspring.profiles.active=api,mysql`. After it started, you could find Open API documentation in http://localhost:12345/dolphinscheduler/doc.html
> The `mysql` in the VM Options `-Dspring.profiles.active=mysql` means specified configuration file > The `mysql` in the VM Options `-Dspring.profiles.active=mysql` means specified configuration file

41
docs/docs/zh/development/development-environment-setup.md
Unescape View File

@ -5,9 +5,9 @@
在搭建 DolphinScheduler 开发环境之前请确保你已经安装一下软件 在搭建 DolphinScheduler 开发环境之前请确保你已经安装一下软件
* [Git](https://git-scm.com/downloads): 版本控制系统 * [Git](https://git-scm.com/downloads): 版本控制系统
* [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html): 后端开发 * [JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html): 后端开发，必须使用JDK1.8及以后的版本
* [Maven](http://maven.apache.org/download.cgi): Java包管理系统 * [Maven](http://maven.apache.org/download.cgi): Java包管理系统
* [Node](https://nodejs.org/en/download): 前端开发 * [Node](https://nodejs.org/en/download): 前端开发，必须使用Node12.20.2及以后的版本
### 克隆代码库 ### 克隆代码库
@ -18,11 +18,10 @@ mkdir dolphinscheduler
cd dolphinscheduler cd dolphinscheduler
git clone git@github.com:apache/dolphinscheduler.git git clone git@github.com:apache/dolphinscheduler.git
``` ```
### 编译源码
* 如果使用MySQL数据库，请注意修改pom.xml， 添加 ` mysql-connector-java ` 依赖。
* 运行 `mvn clean install -Prelease -Dmaven.test.skip=true`
### 编译源码
* 运行 `mvn clean install -Prelease -Dmaven.test.skip=true`
## 开发者须知 ## 开发者须知
@ -50,6 +49,7 @@ DolphinScheduler 开发环境配置有两个方式，分别是standalone模式
### 启动前端 ### 启动前端
安装前端依赖并运行前端组件 安装前端依赖并运行前端组件
> 注意：你可以在[frontend development](./frontend-development.md)里查看更多前端的相关配置
```shell ```shell
cd dolphinscheduler-ui cd dolphinscheduler-ui
@ -91,36 +91,27 @@ DolphinScheduler 的元数据存储在关系型数据库中，目前支持的关
* 打开项目：使用开发工具打开项目，这里以 Intellij IDEA 为例，打开后需要一段时间，让 Intellij IDEA 完成以依赖的下载 * 打开项目：使用开发工具打开项目，这里以 Intellij IDEA 为例，打开后需要一段时间，让 Intellij IDEA 完成以依赖的下载
* 插件的配置（**仅 2.0 及以后的版本需要**）：
* 注册中心插件配置, 以Zookeeper 为例 (registry.properties)
dolphinscheduler-service/src/main/resources/registry.properties
```registry.properties
registry.plugin.name=zookeeper
registry.servers=127.0.0.1:2181
```
* 必要的修改 * 必要的修改
* 如果使用 MySQL 作为元数据库，需要先修改 `dolphinscheduler/pom.xml`，将 `mysql-connector-java` 依赖的 `scope` 改为 `compile`，使用 PostgreSQL 则不需要 * 如果使用 MySQL 作为元数据库，需要先修改 `dolphinscheduler/pom.xml`，将 `mysql-connector-java` 依赖的 `scope` 改为 `compile`，使用 PostgreSQL 则不需要
* 修改数据库配置，修改 `dolphinscheduler-dao/src/main/resources/application-mysql.yaml` 文件中的数据库配置 * 修改 Master 数据库配置，修改 `dolphinscheduler-master/src/main/resources/application.yaml` 文件中的数据库配置
* 修改 Worker 数据库配置，修改 `dolphinscheduler-worker/src/main/resources/application.yaml` 文件中的数据库配置
* 修改 Api 数据库配置，修改 `dolphinscheduler-api/src/main/resources/application.yaml` 文件中的数据库配置
本样例以 MySQL 为例，其中数据库名为 dolphinscheduler，账户名密码均为 dolphinscheduler 本样例以 MySQL 为例，其中数据库名为 dolphinscheduler，账户名密码均为 dolphinscheduler
```application-mysql.yaml ```application.yaml
spring: spring:
datasource: datasource:
driver-class-name: com.mysql.jdbc.Driver driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8 url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
username: ds_user username: dolphinscheduler
password: dolphinscheduler password: dolphinscheduler
``` ```
* 修改日志级别：为以下配置增加一行内容 `<appender-ref ref="STDOUT"/>` 使日志能在命令行中显示 * 修改日志级别：为以下配置增加一行内容 `<appender-ref ref="STDOUT"/>` 使日志能在命令行中显示
`dolphinscheduler-server/src/main/resources/logback-worker.xml` `dolphinscheduler-master/src/main/resources/logback-spring.xml`
`dolphinscheduler-worker/src/main/resources/logback-spring.xml`
`dolphinscheduler-server/src/main/resources/logback-master.xml` `dolphinscheduler-api/src/main/resources/logback-spring.xml`
`dolphinscheduler-api/src/main/resources/logback-api.xml`
修改后的结果如下： 修改后的结果如下：
@ -136,9 +127,9 @@ DolphinScheduler 的元数据存储在关系型数据库中，目前支持的关
我们需要启动三个服务，包括 MasterServer，WorkerServer，ApiApplicationServer 我们需要启动三个服务，包括 MasterServer，WorkerServer，ApiApplicationServer
* MasterServer：在 Intellij IDEA 中执行 `org.apache.dolphinscheduler.server.master.MasterServer` 中的 `main` 方法，并配置 *VM Options* `-Dlogging.config=classpath:logback-master.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql` * MasterServer：在 Intellij IDEA 中执行 `org.apache.dolphinscheduler.server.master.MasterServer` 中的 `main` 方法，并配置 *VM Options* `-Dlogging.config=classpath:logback-spring.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql`
* WorkerServer：在 Intellij IDEA 中执行 `org.apache.dolphinscheduler.server.worker.WorkerServer` 中的 `main` 方法，并配置 *VM Options* `-Dlogging.config=classpath:logback-worker.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql` * WorkerServer：在 Intellij IDEA 中执行 `org.apache.dolphinscheduler.server.worker.WorkerServer` 中的 `main` 方法，并配置 *VM Options* `-Dlogging.config=classpath:logback-spring.xml -Ddruid.mysql.usePingMethod=false -Dspring.profiles.active=mysql`
* ApiApplicationServer：在 Intellij IDEA 中执行 `org.apache.dolphinscheduler.api.ApiApplicationServer` 中的 `main` 方法，并配置 *VM Options* `-Dlogging.config=classpath:logback-api.xml -Dspring.profiles.active=api,mysql`。启动完成可以浏览 Open API 文档，地址为 http://localhost:12345/dolphinscheduler/doc.html * ApiApplicationServer：在 Intellij IDEA 中执行 `org.apache.dolphinscheduler.api.ApiApplicationServer` 中的 `main` 方法，并配置 *VM Options* `-Dlogging.config=classpath:logback-spring.xml -Dspring.profiles.active=api,mysql`。启动完成可以浏览 Open API 文档，地址为 http://localhost:12345/dolphinscheduler/doc.html
> VM Options `-Dspring.profiles.active=mysql` 中 `mysql` 表示指定的配置文件 > VM Options `-Dspring.profiles.active=mysql` 中 `mysql` 表示指定的配置文件

Write Preview
Loading…
Cancel
Save