wangxj3
1 month ago
committed by
GitHub
5 changed files with 183 additions and 0 deletions
@ -0,0 +1,86 @@ |
|||||||
|
# Apache DolphinScheduler Security Model |
||||||
|
|
||||||
|
This document is mainly used to describe the scope of work, responsibilities and key functions of users with different roles. By introducing user permissions and functions, it helps users understand the precautions and rules in the deployment, use, operation and maintenance stages. Developers can use this document to understand the boundaries between security vulnerabilities and normal functions. |
||||||
|
|
||||||
|
## Apache DolphinScheduler Workflow |
||||||
|
|
||||||
|
From understanding to using Apache DolphinScheduler, users generally go through the following stages. |
||||||
|
|
||||||
|
1. System deployment, configure the operating environment |
||||||
|
|
||||||
|
2. Create system users and configure corresponding resources |
||||||
|
|
||||||
|
3. Create workflow definitions and configure task operations |
||||||
|
|
||||||
|
4. System operation and maintenance |
||||||
|
|
||||||
|
Whether the user uses a single node, pseudo cluster, or cluster deployment (server or cloud deployment), the use of the system will go through the above four steps. For the above four steps, the following three types of users are generally involved. |
||||||
|
|
||||||
|
## User Type |
||||||
|
|
||||||
|
### 1. Service deployment personnel |
||||||
|
|
||||||
|
Service deployment personnel need to have the authority to operate the server. Service deployment personnel need to understand the way related tasks are run to ensure the server security boundaries and environmental requirements. |
||||||
|
(1). For multi-tenant operation scenarios, server deployment users need to have the permissions to create and switch users. |
||||||
|
(2). Apache DolphinScheduler can run user-defined scripts and codes. Users can execute any command or code on the machine through node configuration. Service deployment personnel need to confirm the permissions of the service startup user, protect some sensitive files through permissions, and clarify the boundaries of the deployment user's operation permissions. |
||||||
|
(3). The server will perform data source connection operations and execute user-defined SQL statements. The platform will not limit the type of SQL executed by the user. The permission for SQL execution is related to the user permissions for creating the data source. |
||||||
|
(4). Server deployment personnel need to ensure the network and interaction security requirements between all worker servers in the worker group required by the business and the resources required for task operation. |
||||||
|
(5). For worker local task types (such as datax), permissions to call corresponding services are required. |
||||||
|
(6). The resource center provided by Apache DolphinScheduler can be directly connected to the local file system. In a cluster deployment environment, other server files can be mounted to the API server through shared files to achieve file access. Here, service deployment personnel need to ensure that the files contained in the mounted file directory allow system users to operate and trust the operation behavior of the operating user. |
||||||
|
(7).Apache DolphinScheduler supports k8s task types. The k8s cluster is provided by the operation and maintenance. The operation and maintenance needs to ensure the security of k8s services and prevent security issues such as pod escape. |
||||||
|
|
||||||
|
### 2. System administrator |
||||||
|
|
||||||
|
The system administrator has all the operation permissions of Apache DolphinScheduler. In actual use, the scope of use of the administrator user should be ensured, and the administrator user should be highly trusted not to abuse this function. |
||||||
|
(1). The administrator user can operate queue management, tenant management, user management, alarm group management, worker group management, token management and other functions. The administrator user can operate all configurations, including important information such as sensitive credentials required to connect to resources. It is necessary to ensure that the person using the administrator user can operate the corresponding resources. At the same time, the administrator user can authorize operations on resources, data sources, projects, etc. in the user management module. The administrator user needs to clearly authorize the user to have all usage permissions for the corresponding resources. |
||||||
|
(2). System administrators have all the operational permissions that ordinary users have |
||||||
|
|
||||||
|
### 3. Ordinary system users |
||||||
|
|
||||||
|
Ordinary users of Apache DolphinScheduler are defined as actual workflow development and operation users. Of course, they also need to maintain some resources required during the workflow development process. These users should be highly trusted not to abuse this function. |
||||||
|
|
||||||
|
(1). Users can create workflows and tasks. For a list of supported task types, see [Task List]. Tasks will be executed in workers. Users can customize any command and code to run in a specified worker group. Be sure to pay attention to any command and code here. Users can execute all tasks under the task types supported by Apache DolphinScheduler, including shell, sql, and jump to other servers to execute shell scripts. At the same time, logs will be generated during the task running process. Users can view and download task running logs through the UI page. |
||||||
|
|
||||||
|
(2). Users can create data source connections, modify and delete authorized connections including corresponding configurations, especially operations on sensitive credentials of authorized connections. These operations may have a certain impact on the resources themselves or the system. Data sources include many types. For details, please see [Data Source List] on the official website. |
||||||
|
|
||||||
|
### 4. Unlogged in users |
||||||
|
|
||||||
|
Apache DolphinScheduler does not allow unlogged in users to access the system. The users mentioned below do not include this type of users. |
||||||
|
|
||||||
|
In addition to the core workflow development and operation, the normal use of the platform also requires the configuration and management of the corresponding environment and resources. |
||||||
|
|
||||||
|
## Data source management |
||||||
|
|
||||||
|
All users can operate data source management, and administrator users can operate after authorizing ordinary users. The corresponding permissions for data source task operation are provided by the data source connection, and the connection configuration should try to control the task execution permissions. In the data source configuration, users can customize the connection parameters and they will take effect on all tasks using the data source. |
||||||
|
|
||||||
|
## Resource Center |
||||||
|
|
||||||
|
The resource center can configure local, distributed file storage, cloud object storage and other methods. When the resource center needs to be used to create or upload related files, all files and resources will be stored in the distributed file system HDFS or remote object storage. At the same time, users can modify the content of authorized files. In this process, it is necessary to trust that users will not damage files and will not cause other security risks. |
||||||
|
|
||||||
|
## Alarm Management |
||||||
|
|
||||||
|
The list of supported alarm methods can be viewed in [Alarm] on the official website. All users can configure authorized alarm channels to their respective processes. Users can modify alarm configurations containing sensitive credentials. Alarm configurations will be applied to alarms of rules such as workflow timeouts and results. It is necessary to trust that the configuration of user alarms and the sending of alarm information will not affect the alarm channel and the person receiving the alarm. |
||||||
|
|
||||||
|
## Authentication method |
||||||
|
|
||||||
|
Apache DolphinScheduler supports four authentication methods: login with your own account and password, LDAP, SSO login through Casdoor, and login through Oauth2 authorization, and the oauth2 authorization login method can be used with other authentication methods. It is necessary to highly trust that users who log in in any way will not abuse the corresponding permissions and functions. |
||||||
|
|
||||||
|
## Security Center |
||||||
|
|
||||||
|
Administrator users can configure resources such as queues, tenants, users, alarm groups, worker groups, tokens, k8s clusters, k8s namespaces, etc. It is necessary to trust that the user's permission allocation, use, and maintenance of resources will not affect the platform and service itself. |
||||||
|
|
||||||
|
### Examples of Mistaken Security Vulnerabilities |
||||||
|
|
||||||
|
The following are some erroneous vulnerabilities raised by users and developers in the past. |
||||||
|
|
||||||
|
1. Using the insecure settings of plug-ins to attack or perform other operations |
||||||
|
When a user uses a plug-in, some parameters are set to insecure configurations, and then the system is attacked through the configuration. This problem does not belong to a security vulnerability. This type of plug-in includes but is not limited to data sources, tasks, etc. The user's setting of parameters is an active behavior, and the authorization has trusted the user's parameter configuration operations. When setting the corresponding parameters or a certain configuration, the user believes that the configuration user has fully understood the configured functions and the risks brought about, so this type of problem does not belong to a vulnerability. For example, when using MySQL driver to connect to Doris, {"aaa":"dsf&allowLoadLocalInfile=true#"} is added to the JDBC connection parameters. This configuration may send local sensitive files to the server. In this process, the user adds configurations as needed, and all operations of the user are trusted. |
||||||
|
2. Use the security configuration during deployment to access the system for attack or other operations |
||||||
|
When deploying the system, the deployment user should follow the operation of the official website to modify the sensitive configuration. The configuration belongs to the service sensitive information, and its importance and security level are equivalent to the service database connection and other information. When other users obtain sensitive configurations through any means, the platform considers the user to be a normal authorized user and fully trusts all operations of the authorized user. For example, the user obtains auth-token data from the configuration file, authenticates and creates a user through the configuration, and uses the created user to operate the system. In this process, since the user obtains the authentication information of the platform, all operations of the user are considered to be trusted. |
||||||
|
3. Intermediate files generated by the execution platform during task execution |
||||||
|
In the process of running tasks in Apache DolphinScheduler, some intermediate files are generated. This file mainly encapsulates the environment and parameters required for the task to run. The file is related to the task and is stored in the same node as the running task. Running these files is no different from running the same task in the same node by other users. During the deployment and permission allocation process, the corresponding worker or other resources are allocated to the corresponding user. This operation means that the user is fully trusted for all operations on the service node, including task running and reading and modifying the resources with permissions in the server, and of course, the files generated by the platform. Therefore, this type of problem does not belong to a vulnerability. For example, the remote shell task will produce an intermediate file in the server. The user knows the file information and operates the file through the shell node. In this process, the user has the permission of the node, and all operations of the user on the node are trusted. |
||||||
|
4. Authorized users enter scripts through the page input box to attack or other operations |
||||||
|
There are multiple input boxes in Apache DolphinScheduler, allowing users to customize configurations as needed. As an open source task scheduling system, Apache DolphinScheduler requires administrators to fully trust all authorized operations of the target user in the process of deployment, authorization, and other security-related processes. If the user's behavior of adding and modifying configurations through pages or calling interfaces is within the scope of permissions, then the behavior of attacking or other operations in this way does not belong to security vulnerabilities. |
||||||
|
5. Attack or other operations by modifying the image or providing an unsafe image to run |
||||||
|
Apache DolphinScheduler itself and task operations both support k8s clusters. Before the service or task runs, the user needs to ensure the image's functions and configured parameters, and trust all operations during the service and task running process. Therefore, modifying tasks or parameters by any means before the image runs to attack or complete other operations does not constitute a security vulnerability. |
||||||
|
|
||||||
@ -0,0 +1,83 @@ |
|||||||
|
# Apache DolphinScheduler安全模型 |
||||||
|
|
||||||
|
本文档主要用于描述不同角色用户的工作范围、职责和重点功能，通过对用户权限和功能的介绍辅助用户在了解部署、使用、运维等阶段的注意事项和规则。针对开发者可以通过本文档了解安全漏洞与正常功能的边界。 |
||||||
|
|
||||||
|
## Apache DolphinScheduler工作流程 |
||||||
|
|
||||||
|
从了解到使用Apache DolphinScheduler，用户一般会经过以下几个阶段。 |
||||||
|
|
||||||
|
1.系统部署，配置运行环境 |
||||||
|
|
||||||
|
2.创建系统用户，配置对应资源 |
||||||
|
|
||||||
|
3.创建工作流定义，配置任务运行 |
||||||
|
|
||||||
|
4.系统运维 |
||||||
|
|
||||||
|
无论用户使用的是单节点、伪集群、还是集群部署（服务器或云部署），对于系统的使用都会经过以上四个步骤。针对上述四个步骤，一般会涉及到以下三种类型的用户。 |
||||||
|
|
||||||
|
## 用户类型 |
||||||
|
|
||||||
|
### 1.服务部署人员 |
||||||
|
|
||||||
|
服务部署人员需要拥有操作服务器的权限，服务部署人员需要了解相关任务运行的方式，已确保服务器安全边界和环境要求。 |
||||||
|
(1).对于多租户运行场景，服务器部署用户需要具备创建用户和切换用户权限。 |
||||||
|
(2).Apache DolphinScheduler中可以运行用户自定的脚本和代码，用户可以通过节点配置运行机器上执行任意命令或代码，服务部署人员需要确认服务启动用户的权限，通过权限保护一些敏感文件，明确部署用户的操作权限边界。 |
||||||
|
(3).服务器会进行数据源连接操作，执行用户自定义的sql语句，平台不会限制用户执行sql的类型，sql执行的权限与创建创建数据源的用户权限有关。 |
||||||
|
(4).服务器部署人员需要确保业务所需的worker分组中所有的worker服务器与任务运行所需资源之间的网络和交互安全要求。 |
||||||
|
(5).对于worker本地任务类型（datax等）需要具备调用对应服务的权限。 |
||||||
|
(6).Apache DolphinScheduler提供的资源中心可以直接对接本地文件系统，在集群部署环境下，可以通过共享文件方式挂载其他服务器文件到api服务器上实现文件访问，此处服务部署人员需要确保挂载文件目录中包含的文件允许系统用户操作，并且信任操作用户的操作行为。 |
||||||
|
(7).Apache DolphinScheduler 支持k8s任务类型，k8s集群由运维提供，运维需要保证k8s的服务安全问题，防止出现pod逃逸等安全问题。 |
||||||
|
|
||||||
|
### 2.系统管理员 |
||||||
|
|
||||||
|
系统管理员拥有Apache DolphinScheduler所有的操作权限，在实际使用中应该确保管理员用户的使用范围，并高度信任管理员用户不会滥用此功能。 |
||||||
|
(1).管理员用户可以操作队列管理、租户管理、用户管理、告警组管理、worker分组管理、令牌管理等功能，管理员用户可以操作所有的配置，其中包括连接资源所需的敏感凭证等重要信息，要确保使用管理员用户的人员可以操作对应资源。同时管理员用户在用户管理模块可以对资源、数据源、项目等授权操作，管理员用户需要明确授权后用户将对对应资源拥有所有使用权限。 |
||||||
|
(2).系统管理员用户拥有普通用户所有的操作权限 |
||||||
|
|
||||||
|
### 3.系统普通用户 |
||||||
|
|
||||||
|
Apache DolphinScheduler的普通用户定义为实际的工作流开发和运营用户，当然也需要维护在工作流开发过程中所需要的部分资源。应该高度信任这些用户不会滥用此功能。 |
||||||
|
(1).用户可以创建工作流和任务，任务支撑类型列表查看【任务列表】，任务会在worker中执行，用户可以自定义任意命令和代码在指定的worker分组中运行，一定要注意这里的任意命令和代码，用户可以执行Apache DolphinScheduler支持任务类型下的所有任务，包括shell、sql、和跳转至其他服务器执行shell脚本等任务。同时任务运行过程会产生日志，用户可以通过UI页面查看、下载任务运行日志。 |
||||||
|
(2).用户可以创建数据源连接，可以修改、删除有权限的连接包括对应的配置，尤其是对有权限连接的敏感凭据的操作，这些操作可能会对资源本身或系统造成一定的影响，数据源包括众多种类，具体可以查看官网中【数据源列表】。 |
||||||
|
|
||||||
|
### 4.未登录用户 |
||||||
|
|
||||||
|
Apache DolphinScheduler不允许未登录用户访问系统。下文中提到的用户不包含该类型用户。 |
||||||
|
|
||||||
|
平台正常使用除核心的工作流开发、运行外，还需要配置、管理对应的环境和资源。 |
||||||
|
|
||||||
|
## 数据源管理 |
||||||
|
|
||||||
|
所有用户均可操作数据源管理，管理员用户可对普通用户进行授权后进行操作。数据源任务运行的对应权限由数据源连接提供，连接配置应尽量控制任务执行权限。数据源配置中，用户可以自定义连接参数，并对所有使用该数据源的任务生效。 |
||||||
|
|
||||||
|
## 资源中心 |
||||||
|
|
||||||
|
资源中心可以配置本地、分布式文件存储、云对象存储等多种方式，当需要使用资源中心进行相关文件的创建或者上传操作时，所有的文件和资源都会被存储在分布式文件系统HDFS或者远端的对象存储。同时用户可修改有权限文件的内容，在这个过程中需要信任用户不会损坏文件，不会造成其他安全风险。 |
||||||
|
|
||||||
|
## 告警管理 |
||||||
|
|
||||||
|
告警方式支持列表可查看官网中【告警】，所有用户可将有权限的告警通道配置到各自的流程中。用户可以修改包含敏感凭据的告警配置。告警配置会应用在工作流运行超时、结果等规则的告警中，需要信任用户告警的配置和告警信息的发送不会对告警通道和接受告警人员产生影响。 |
||||||
|
|
||||||
|
## 认证方式 |
||||||
|
|
||||||
|
Apache DolphinScheduler支持四种认证方式，自身账号密码登录，LDAP, 通过Casdoor实现的SSO登录和通过Oauth2授权登录，并且oauth2授权登录方式可以和其他认证方式同时使用。需要高度信任用户以任何方式登录的用户都不会滥用对应权限和功能。 |
||||||
|
|
||||||
|
## 安全中心 |
||||||
|
|
||||||
|
管理员用户可以配置队列、租户、用户、告警组、worker分组、令牌、k8s集群、k8s命名空间等资源，需要信用用户对资源的权限分配、使用、维护，都不会对平台和服务本身产生影响。 |
||||||
|
|
||||||
|
### 误认为是安全漏洞的示例 |
||||||
|
|
||||||
|
以下是以往使用者和开发者提出的一些错误的漏洞情况。 |
||||||
|
|
||||||
|
1.利用插件的不安全设置进行攻击或其他操作 |
||||||
|
用户使用某个插件时，将某些参数设置成不安全的配置，进而通过该配置进行系统攻击，这种问题不属于安全漏洞。这种插件包括但不限于数据源、任务等，用户对于参数的设置属于主动行为，授权事已信任用户的参数配置操作，用户在设置对应的参数或某个配置时认为该配置用户已经完全了解所配置的功能和所带来的风险，所以该类型的问题不属于漏洞。例如在使用mysql驱动链接doris时，在jdbc的连接参数中增加了{"aaa":"dsf&allowLoadLocalInfile=true#"}，该配置可能会将本地的敏感文件发送至服务器，在该过程中，用户按需增加了配置，则信任用户的所有操作。 |
||||||
|
2.利用部署时的安全配置访问系统进行攻击或其他操作 |
||||||
|
部署用户在系统部署时应该按照官网的操作，将敏感配置进行修改，该配置属于服务敏感信息，其重要和安全级别等同与服务数据库连接等信息。当其他用户无论通过任何途径获取到敏感配置后，平台变认为该用户为正常授权用户，完全信任授权用户的所有操作。例如用户通过配置文件中获取了auth-token数据，通过该配置进行鉴权并且创建用户，利用创建的用户对系统进行操作，该过程中既然用户获得了平台的鉴权信息，则认为信任用户的所有操作。 |
||||||
|
3.执行平台在任务执行过程中产生的中间文件 |
||||||
|
在Apache DolphinScheduler运行任务的过程中会产生部分中间文件，该文件主要是封装任务运行所需的环境和参数等信息，该文件与任务相关，并且与运行的任务存储在同一个节点中，运行这些文件与其他用户在同一个节点中运行同一个任务没有差异。在部署和权限的分配过程中，将对应的worker或其他资源分配给对应的用户，这个操作意味着完全信任该用户对该服务节点的所有操作，包括任务运行和读取、修改服务器中有权限的资源，当然也包括平台所生成的文件。所以该类型问题不属于漏洞。例如remote shell任务会在服务器中生产一个中间文件，用户得知了文件信息，通过shell节点操作该文件，在该过程中，用户具有该节点的权限，则信任用户对该节点的所有操作。 |
||||||
|
4.已授权用户通过页面输入框输入脚本进行攻击或其他操作 |
||||||
|
在Apache DolphinScheduler有多个输入框，允许用户按需进行自定义配置，Apache DolphinScheduler作为开源任务调度系统，管理员用户在部署、授权等涉及安全的过程中需要完全信任目标用户的所有授权范围内的操作，用户通过页面或调用接口增加、修改配置的行为如果属于权限范围内的操作，则通过该方式进行攻击或其他操作的行为不属于安全漏洞。 |
||||||
|
5.通过修改镜像或提供不安全的镜像运行进行攻击或其他操作 |
||||||
|
Apache DolphinScheduler的本身和任务运行均支持k8s集群，在服务或任务运行之前，用户需要确保镜像的功能和所配置参数，信任服务、任务运行过程中的所有操作。所以在镜像运行之前通过任何途径将任务或参数进行修改进行攻击或其他完成其他操作的行为不属于安全漏洞。 |
||||||
Loading…
Reference in new issue