DolphinScheduler/ambari_plugin

Instructions for using the Dolphin Scheduler's Ambari plug-in

Note

1. This document is intended for users with a basic understanding of Ambari
2. This document is a description of adding the Dolphin Scheduler service to the installed Ambari service
3. This document is based on version 2.5.2 of Ambari

Installation preparation

1. Prepare the RPM packages

   • It is generated by executing the command mvn -U clean install -Prpmbuild -Dmaven.test.skip=true -X in the project root directory (In the directory: dolphinscheduler-dist/target/rpm/apache-dolphinscheduler/RPMS/noarch )

2. Create an installation for DS，who have read and write access to the installation directory (/opt/soft)

3. Install with rpm package

   • Manual installation (recommended）：

     • Copy the prepared RPM packages to each node of the cluster.
     • Execute with DS installation user: rpm -ivh apache-dolphinscheduler-xxx.noarch.rpm
     • Mysql-connector-java packaged using the default POM file will not be included.
     • The RPM package was packaged in the project with the installation path of /opt/soft. If you use mysql as the database, you need add it manually.

   • Automatic installation with ambari

     • Each node of the cluster needs to configure the local yum source
     • Copy the prepared RPM packages to each node local yum source

4. Copy plug-in directory

   • copy directory ambari_plugin/common-services/DOLPHIN to ambari-server/resources/common-services/
   • copy directory ambari_plugin/statcks/DOLPHIN to ambari-server/resources/stacks/HDP/2.6/services/--stack version is selected based on the actual situation

5. Initializes the database information

   -- Create the database for the Dolphin Scheduler：dolphinscheduler
   CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE
   utf8_general_ci;
   
   -- Initialize the user and password for the dolphinscheduler database and assign permissions
   -- Replace the {user} in the SQL statement below with the user of the dolphinscheduler database
   GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%' IDENTIFIED BY '{password}';
   GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost' IDENTIFIED BY
   '{password}';
   flush privileges;
   

Ambari Install Dolphin Scheduler

• NOTE: You have to install zookeeper first

1. Install Dolphin Scheduler on ambari web interface

   

2. Select the nodes for the Dolphin Scheduler's Master installation

   

3. Configure the Dolphin Scheduler's nodes for Worker, Api, Logger, Alert installation

   

4. Set the installation users of the Dolphin Scheduler service (created in step 1) and the user groups they belong to

   

5. System Env Optimization will export some system environment config. Modify according to actual situation

   

6. Configure the database information (same as in the initialization database in step 1)

   

7. Configure additional information if needed

   

   

8. Perform the next steps as normal

   

9. The interface after successful installation

   

---

Add components to the node through Ambari -- for example, add a DS Worker

NOTE: DS Logger is the installation dependent component of DS Worker in Dolphin's Ambari installation (need to add installation first; Prevent the Job log on the corresponding Worker from being checked)

1. Locate the component node to add -- for example, node ark3

   

2. Add components -- the drop-down list is all addable

   

3. Confirm component addition

   

4. After adding DS Worker and DS Logger components

   

5. Start the component

   

Remove the component from the node with Ambari

1. Stop the component in the corresponding node

   

2. Remove components