diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index eceda6a97a..f92f0076e9 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -39,9 +39,9 @@ /dolphinscheduler-tools/ @caishunfeng @SbloodyS @zhongjiajie @EricGao888 /script/ @caishunfeng @SbloodyS @zhongjiajie @EricGao888 /dolphinscheduler-ui/ @songjianet @Amy0104 -/docs/ @zhongjiajie @Tianqi-Dotes @EricGao888 +/docs/ @zhongjiajie @EricGao888 /licenses/ @kezhenxu94 @zhongjiajie -/images/ @zhongjiajie @Tianqi-Dotes @EricGao888 +/images/ @zhongjiajie @EricGao888 /style/ @caishunfeng # All pom files diff --git a/docs/configs/docsdev.js b/docs/configs/docsdev.js index 1196850b5e..88e0850cc3 100644 --- a/docs/configs/docsdev.js +++ b/docs/configs/docsdev.js @@ -521,15 +521,15 @@ export default { children: [ { title: 'Open API', - link: '/en-us/docs/dev/user_doc/guide/open-api.html', + link: '/en-us/docs/dev/user_doc/guide/api/open-api.html', }, { title: 'PyDolphinScheduler', - link: '/python/main/index.html', + link: '/en-us/docs/dev/user_doc/guide/api/pydolphinscheduler.html', }, { title: 'Health Check', - link: '/en-us/docs/dev/user_doc/guide/healthcheck.html', + link: '/en-us/docs/dev/user_doc/guide/api/healthcheck.html', }, ], }, @@ -1197,15 +1197,15 @@ export default { children: [ { title: 'API调用', - link: '/zh-cn/docs/dev/user_doc/guide/open-api.html', + link: '/zh-cn/docs/dev/user_doc/guide/api/open-api.html', }, { title: 'PyDolphinScheduler', - link: '/python/main/index.html', + link: '/zh-cn/docs/dev/user_doc/guide/api/pydolphinscheduler.html', }, { title: '健康检查', - link: '/zh-cn/docs/dev/user_doc/guide/healthcheck.html', + link: '/zh-cn/docs/dev/user_doc/guide/api/healthcheck.html', }, ], }, diff --git a/docs/docs/en/guide/healthcheck.md b/docs/docs/en/guide/api/healthcheck.md similarity index 100% rename from docs/docs/en/guide/healthcheck.md rename to docs/docs/en/guide/api/healthcheck.md diff --git a/docs/docs/en/guide/open-api.md b/docs/docs/en/guide/api/open-api.md similarity index 72% rename from docs/docs/en/guide/open-api.md rename to docs/docs/en/guide/api/open-api.md index 5bb0962b46..231a314dd5 100644 --- a/docs/docs/en/guide/open-api.md +++ b/docs/docs/en/guide/api/open-api.md @@ -10,11 +10,11 @@ Generally, projects and processes are created through pages, but considering the 1. Log in to the scheduling system, click "Security", then click "Token manage" on the left, and click "Create token" to create a token. -![create-token](../../../img/new_ui/dev/security/create-token.png) +![create-token](../../../../img/new_ui/dev/security/create-token.png) 2. Select the "Expiration time" (Token validity time), select "User" (choose the specified user to perform the API operation), click "Generate token", copy the `Token` string, and click "Submit". -![token-expiration](../../../img/new_ui/dev/open-api/token_expiration.png) +![token-expiration](../../../../img/new_ui/dev/open-api/token_expiration.png) ### Examples @@ -24,7 +24,7 @@ Generally, projects and processes are created through pages, but considering the > Address：http://{API server ip}:12345/dolphinscheduler/swagger-ui/index.html?language=en_US&lang=en -![api-doc](../../../img/new_ui/dev/open-api/api_doc.png) +![api-doc](../../../../img/new_ui/dev/open-api/api_doc.png) 2. select a test API, the API selected for this test is `queryAllProjectList` @@ -36,7 +36,7 @@ Generally, projects and processes are created through pages, but considering the token: The Token just generated ``` -![api-test](../../../img/new_ui/dev/open-api/api_test.png) +![api-test](../../../../img/new_ui/dev/open-api/api_test.png) #### Create a Project @@ -44,15 +44,15 @@ This demonstrates how to use the calling api to create the corresponding project By consulting the api documentation, configure the KEY as Accept and VALUE as the parameter of application/json in the headers of Postman. -![create-project01](../../../img/new_ui/dev/open-api/create_project01.png) +![create-project01](../../../../img/new_ui/dev/open-api/create_project01.png) And then configure the required projectName and description parameters in Body. -![create-project02](../../../img/new_ui/dev/open-api/create_project02.png) +![create-project02](../../../../img/new_ui/dev/open-api/create_project02.png) Check the post request result. -![create-project03](../../../img/new_ui/dev/open-api/create_project03.png) +![create-project03](../../../../img/new_ui/dev/open-api/create_project03.png) The returned `msg` information is "success", indicating that we have successfully created the project through API. @@ -60,6 +60,6 @@ If you are interested in the source code of creating a project, please continue ### Appendix： The Source Code of Creating a Project -![api-source01](../../../img/new_ui/dev/open-api/api_source01.png) +![api-source01](../../../../img/new_ui/dev/open-api/api_source01.png) -![api-source02](../../../img/new_ui/dev/open-api/api_source02.png) +![api-source02](../../../../img/new_ui/dev/open-api/api_source02.png) diff --git a/docs/docs/en/guide/api/pydolphinscheduler.md b/docs/docs/en/guide/api/pydolphinscheduler.md new file mode 100644 index 0000000000..a3690bdabd --- /dev/null +++ b/docs/docs/en/guide/api/pydolphinscheduler.md @@ -0,0 +1,5 @@ +# PyDolphinScheduler + +PyDolphinScheduler is python API for Apache DolphinScheduler, which allow you definition your workflow by python code, aka workflow-as-codes. + +For more information, please refer to [PyDolphinScheduler](https://dolphinscheduler.apache.org/python/main/) diff --git a/docs/docs/en/guide/parameter/file-parameter.md b/docs/docs/en/guide/parameter/file-parameter.md index 4ba9b30424..e399778eff 100644 --- a/docs/docs/en/guide/parameter/file-parameter.md +++ b/docs/docs/en/guide/parameter/file-parameter.md @@ -51,7 +51,7 @@ The configuration in the figure below indicates that the task gets the file iden - The file transfer between upstream and downstream tasks is based on the resource center as a transfer, and the data is saved in the `DATA_TRANSFER` directory of the resource center. Therefore, **the resource center function must be enabled**, please refer to [Resource Center Configuration Details](../resource/configuration.md) for details, otherwise the file parameter function cannot be used. - The file naming rule is `DATA_TRANSFER/DATE/ProcessDefineCode/ProcessDefineVersion_ProcessInstanceID/TaskName_TaskInstanceID_FileName` - If the transferred file data is a folder, it will be packaged into a compressed file with a suffix of `.zip` and uploaded. The downstream task will unzip and save it in the corresponding directory after receiving it -- If you need to delete the file data, you can delete the corresponding folder in the `DATA_TRANSFER` directory of the resource center. If you delete the date subdirectory directly, all the file data under that date will be deleted. You can also use the [Open API interface](../open-api.md) (`resources/data-transfer`) to delete the corresponding file data (delete data N days ago). +- If you need to delete the file data, you can delete the corresponding folder in the `DATA_TRANSFER` directory of the resource center. If you delete the date subdirectory directly, all the file data under that date will be deleted. You can also use the [Open API interface](../api/open-api.md) (`resources/data-transfer`) to delete the corresponding file data (delete data N days ago). - If there is a task chain task1->task2->tas3, then the downstream task task3 can also get the file data of task1 - Support one-to-many transmission and many-to-one transmission - If you frequently transfer a large number of files, it is obvious that the system IO performance will be affected by the amount of transferred data diff --git a/docs/docs/zh/guide/healthcheck.md b/docs/docs/zh/guide/api/healthcheck.md similarity index 100% rename from docs/docs/zh/guide/healthcheck.md rename to docs/docs/zh/guide/api/healthcheck.md diff --git a/docs/docs/zh/guide/open-api.md b/docs/docs/zh/guide/api/open-api.md similarity index 69% rename from docs/docs/zh/guide/open-api.md rename to docs/docs/zh/guide/api/open-api.md index c41010452e..47c2d0122e 100644 --- a/docs/docs/zh/guide/open-api.md +++ b/docs/docs/zh/guide/api/open-api.md @@ -10,11 +10,11 @@ 1. 登录调度系统，点击 "安全中心"，再点击左侧的 "令牌管理"，点击 "令牌管理" 创建令牌。 -![create-token](../../../img/new_ui/dev/security/create-token.png) +![create-token](../../../../img/new_ui/dev/security/create-token.png) 2. 选择 "失效时间" (Token 有效期)，选择 "用户" (以指定的用户执行接口操作)，点击 "生成令牌" ，拷贝 Token 字符串，然后点击 "提交" 。 -![token-expiration](../../../img/new_ui/dev/open-api/token_expiration.png) +![token-expiration](../../../../img/new_ui/dev/open-api/token_expiration.png) ### 使用案例 @@ -24,7 +24,7 @@ > 地址：http://{api server ip}:12345/dolphinscheduler/swagger-ui/index.html?language=zh_CN&lang=cn -![api-doc](../../../img/new_ui/dev/open-api/api_doc.png) +![api-doc](../../../../img/new_ui/dev/open-api/api_doc.png) 2. 选一个测试的接口，本次测试选取的接口是：查询所有项目 @@ -36,7 +36,7 @@ token: 刚刚生成的 Token ``` -![api-test](../../../img/new_ui/dev/open-api/api_test.png) +![api-test](../../../../img/new_ui/dev/open-api/api_test.png) #### 创建项目 @@ -44,15 +44,15 @@ 通过查阅 api 文档，在 Postman 的 Headers 中配置 KEY 为 Accept，VALUE 为 application/json 的参数。 -![create-project01](../../../img/new_ui/dev/open-api/create_project01.png) +![create-project01](../../../../img/new_ui/dev/open-api/create_project01.png) 然后再 Body 中配置所需的 projectName 和 description 参数。 -![create-project02](../../../img/new_ui/dev/open-api/create_project02.png) +![create-project02](../../../../img/new_ui/dev/open-api/create_project02.png) 检查 post 请求结果。 -![create-project03](../../../img/new_ui/dev/open-api/create_project03.png) +![create-project03](../../../../img/new_ui/dev/open-api/create_project03.png) 返回 msg 信息为 "success"，说明我们已经成功通过 API 的方式创建了项目。 @@ -60,6 +60,6 @@ ### 附:创建项目源码 -![api-source01](../../../img/new_ui/dev/open-api/api_source01.png) +![api-source01](../../../../img/new_ui/dev/open-api/api_source01.png) -![api-source02](../../../img/new_ui/dev/open-api/api_source02.png) +![api-source02](../../../../img/new_ui/dev/open-api/api_source02.png) diff --git a/docs/docs/zh/guide/api/pydolphinscheduler.md b/docs/docs/zh/guide/api/pydolphinscheduler.md new file mode 100644 index 0000000000..0fab37fe2c --- /dev/null +++ b/docs/docs/zh/guide/api/pydolphinscheduler.md @@ -0,0 +1,6 @@ +# PyDolphinScheduler + +PyDolphinScheduler 是 Apache DolphinScheduler 的 Python API，它允许您通过 Python 代码定义您的工作流，也称为 workflow-as-codes。 + +更多信息请参考[PyDolphinScheduler](https://dolphinscheduler.apache.org/python/main/) + diff --git a/docs/docs/zh/guide/parameter/file-parameter.md b/docs/docs/zh/guide/parameter/file-parameter.md index 6ddfcbf44e..7e5970b9f2 100644 --- a/docs/docs/zh/guide/parameter/file-parameter.md +++ b/docs/docs/zh/guide/parameter/file-parameter.md @@ -50,7 +50,7 @@ - 上下游任务间的文件传递基于资源中心作为中转，数据保存在资源中心`DATA_TRANSFER`的目录下, 因此**必须开启资源中心功能**，详情请参考[资源中心配置详情](../resource/configuration.md), 否则无法使用文件参数功能。 - 文件命名规则为 `DATA_TRANSFER/日期/工作流Code/工作流版本_工作流实例ID/任务名称_任务实例ID_文件名` - 若传输的文件数据为文件夹，则会打包成后缀为`.zip`的压缩文件再上传，下游任务接到后会解压并保存在对应目录 -- 若需要删除文件数据，可以在资源中心的`DATA_TRANSFER`目录下删除对应文件夹即可, 如直接按照日期子目录删除，会删除该日期下所有的文件数据. 也可以使用`resources/data-transfer`[Open API 接口](../open-api.md)（删除N天前的数据）删除对应文件数据。 +- 若需要删除文件数据，可以在资源中心的`DATA_TRANSFER`目录下删除对应文件夹即可, 如直接按照日期子目录删除，会删除该日期下所有的文件数据. 也可以使用`resources/data-transfer`[Open API 接口](../api/open-api.md)（删除N天前的数据）删除对应文件数据。 - 如果存在任务链 task1->task2->tas3, 则最下游任务task3也能获取task1的文件数据 - 支持一对多传输以及多对一传输 - 如果频繁大量传输文件，毫无疑问会因传输的数据量影响到系统IO性能