From 8ec2f792488ce528c9bc4340bc0b0cc4a5743f9d Mon Sep 17 00:00:00 2001 From: sgw <56352018+guowei-su@users.noreply.github.com> Date: Wed, 17 Aug 2022 21:57:02 +0800 Subject: [PATCH] [DS-11481][docs] Add log specification document to contribution guidelines. (#11484) --- docs/configs/docsdev.js | 8 +++ docs/docs/en/contribute/log-specification.md | 52 ++++++++++++++++++++ docs/docs/zh/contribute/log-specification.md | 50 +++++++++++++++++++ 3 files changed, 110 insertions(+) create mode 100644 docs/docs/en/contribute/log-specification.md create mode 100644 docs/docs/zh/contribute/log-specification.md diff --git a/docs/configs/docsdev.js b/docs/configs/docsdev.js index d195e15fd1..fcb1eead67 100644 --- a/docs/configs/docsdev.js +++ b/docs/configs/docsdev.js @@ -567,6 +567,10 @@ export default { title: 'API Automation Test', link: '/en-us/docs/dev/user_doc/contribute/api-test.html', }, + { + title: 'Log Specification', + link: '/en-us/docs/dev/user_doc/contribute/log-specification.html', + }, ], }, { @@ -1155,6 +1159,10 @@ export default { title: 'API 自动化测试', link: '/zh-cn/docs/dev/user_doc/contribute/api-test.html', }, + { + title: '日志规范', + link: '/zh-cn/docs/dev/user_doc/contribute/log-specification.html', + }, ], }, { diff --git a/docs/docs/en/contribute/log-specification.md b/docs/docs/en/contribute/log-specification.md new file mode 100644 index 0000000000..69692495e1 --- /dev/null +++ b/docs/docs/en/contribute/log-specification.md @@ -0,0 +1,52 @@ +# Log specification + +## Preface + +Logs are used to track and record various actions during the development and operation of a system. Standardized log printing can help users or developers to quickly understand the system operation status and locate problems. + +Apache DolphinScheduler uses the Logback logging framework to print logs according to four levels: DEBUG, WARN, INFO, and ERROR, with a priority of DEBUG < INFO < WARN < ERROR. + +## Specifications + +### Log level specification + +Different levels of logs play different roles in the business process, and failure to use reasonable log levels for printing can cause great difficulties for system operations and maintenance. + +- DEBUG level is used in the development and testing process to output debugging information. Developers should print parameter information, process details, and result information during debugging as much as possible using this level to facilitate locating and analyzing problems during the development and testing phases. In addition, it is prohibited to use this level to print logs in the production environment. +- INFO level is used to record information during system operation. The logs printed using this level should be able to reflect the behavior of the system, such as status changes of workflows, tasks, etc. +- WARN level is used to warn of problems that will occur during operation. For example, the checksum of API module parameters, etc. +- ERROR level is used to record some unpredictable errors and exceptions that will affect the system process. For example, errors and exceptions that cause workflows and tasks to fail to complete properly. + +### Log content specification + +The content of the logs determines whether the logs can completely restore the system behavior or state. + +- DEBUG-level logs record debugging information during the development process, and appear at critical programs that need to be debugged, covering detailed site information, parameters, results, etc. + +- INFO-level logs need to record the status information or operation information of the current program calls, and play the role of describing the system operation process. Therefore, the logs at this level need to appear at the critical point of the system operation, and their contents need to cover the description of the critical point, parameters and results. For example, when a workflow instance is scheduled, the status change of each key link is printed. + +- WARN-level logs record information about tolerable errors that occur in current program calls that do not affect the normal operation of the system or functionality, but the content of this level log also needs to cover detailed site descriptions, parameters, and results. For example, when the API module interface parameter verification fails, the description of the verification failure and the parameters are recorded. + +- ERROR-level logs record information about intolerable errors that occur in the current program call, which may cause the system or function to fail to operate normally. Therefore, the logs at this level need to record the error description, site parameters, error results, etc. in detail to ensure that the problem and the cause can be quickly located based on the log. In addition, when handling exceptions, if you are sure you want to print the stack information, use the following format: + + ```java + logger.error("description of current error, parameter is {}", parameter, e); + ``` + +### Log format specification + +The logs of Master module and Worker module are printed using the following format. + +```xml +[%level] %date{yyyy-MM-dd HH:mm:ss.SSS Z} %logger{96}:[%line] - [WorkflowInstance-%X{workflowInstanceId:-0}][TaskInstance-%X{taskInstanceId:-0}] - %msg%n +``` + +That is, the workflow instance ID and task instance ID are injected in the printed logs using MDC, so the developer needs to get the IDs and inject them before printing the logs related to the workflow instance and task instance in these two modules; after the printing is finished, the related IDs need to be removed. + +## Cautions + +- Disable the use of standard output to print logs. Standard output can greatly affect system performance. +- The use of printStackTrace() is prohibited for exception handling. This method prints the exception stack information to the standard error output. +- Branch printing of logs is prohibited. The contents of the logs need to be associated with the relevant information in the log format, and printing them in separate lines will cause the contents of the logs to not match the time and other information, and cause the logs to be mixed in a large number of log environments, which will make log retrieval more difficult. +- The use of the "+" operator for splicing log content is prohibited. Use placeholders for formatting logs for printing to improve memory usage efficiency. +- When the log content includes object instances, you need to make sure to override the toString() method to prevent printing meaningless hashcode. \ No newline at end of file diff --git a/docs/docs/zh/contribute/log-specification.md b/docs/docs/zh/contribute/log-specification.md new file mode 100644 index 0000000000..4ca68a71e3 --- /dev/null +++ b/docs/docs/zh/contribute/log-specification.md @@ -0,0 +1,50 @@ +# 日志规范 + +## 前言 + +日志被用来追踪、记录系统开发、运行期间的各种行为。规范的日志打印可以帮助用户或开发者快速了解系统运行状态、定位问题。 + +Apache DolphinScheduler使用Logback日志框架，按照DEBUG、WARN、INFO、ERROR四个级别对日志进行打印，其优先级为DEBUG < INFO < WARN < ERROR。 + +## 规范 + +### 日志级别规范 + +不同级别的日志在业务过程中起着不同的作用，如果不能使用合理的日志级别进行打印，会对系统运维带来很大的困难。 + +- DEBUG级别在开发、测试过程中使用，用于输出调试信息。开发者尽可能将调试过程中的参数信息，过程细节，结果信息使用该级别打印，方便在开发、测试阶段定位、分析问题。此外，禁止在生产环境中使用该级别打印日志。 +- INFO级别用于记录系统运行期间的信息。使用该级别打印的日志应能够反映系统的行为，比如工作流、任务的状态变化等。 +- WARN级别用于对运行过程中将会出现的问题进行警告。比如API模块参数的校验等。 +- ERROR级别用于记录一些不可预知的错误、异常，这些错误、异常会影响系统流程。比如导致工作流、任务无法正常完成的错误、异常。 + +### 日志内容规范 + +日志内容是否规范决定着日志能否完整地还原系统行为或状态。 + +- DEBUG级别的日志记录开发过程中的调试信息。DEBUG级别的日志出现在需要调试的关键程序处，其内容涵盖细致的现场信息，参数，结果等。 + +- INFO级别的日志需要记录当前程序调用的状态信息或运行信息，起到描述系统运行过程的作用。所以该级别的日志需要在系统运行的关键环节出现，其内容需涵盖关键环节描述，参数以及结果等。比如工作流实例被调度时，打印每一个关键环节的状态变化。 + +- WARN级别的日志记录当前程序调用中发生的可容忍错误的信息，该错误不会影响系统或功能的正常运行，但是该级别的日志内容也需要涵盖详细地环节描述，参数以及结果等信息。比如API模块接口参数校验失败时，记录校验失败的描述，参数。 + +- ERROR级别的日志记录当前程序调用中发生的不可容忍错误的信息，该错误会导致系统或功能无法正常运行。所以该级别的日志需要详细地记录错误描述，现场参数，错误结果等，保证能够根据日志快速定位到问题以及原因。此外，处理异常时如果确定要打印堆栈信息，使用如下格式： + + ```java + logger.error("description of current error, parameter is {}", parameter, e); + ``` + +### 日志格式规范 + +Master模块和Worker模块的日志打印使用如下格式。即在打印的日志中使用MDC注入工作流实例ID和任务实例ID，因此开发者在打印这两个模块中与工作流实例和任务实例有关的日志前，需要获取ID并注入；在打印完成后，需要移除相关ID。 + +```xml +[%level] %date{yyyy-MM-dd HH:mm:ss.SSS Z} %logger{96}:[%line] - [WorkflowInstance-%X{workflowInstanceId:-0}][TaskInstance-%X{taskInstanceId:-0}] - %msg%n +``` + +## 注意事项 + +- 禁止使用标准输出打印日志。标准输出会极大影响系统性能。 +- 异常处理时禁止使用printStackTrace()。该方法会将异常堆栈信息打印到标准错误输出中。 +- 禁止分行打印日志。日志的内容需要与日志格式中的相关信息关联，如果分行打印会导致日志内容与时间等信息匹配不上，并且在大量日志环境下导致日志混合，会加大日志检索难度。 +- 禁止使用"+"运算符对日志内容进行拼接。使用占位符进行日志格式化打印，提高内存使用效率。 +- 日志内容中包括对象实例时，需要确保重写toString()方法，防止打印无意义的hashcode。 \ No newline at end of file