Browse Source
* Add mater metrics
* fix UT
* Add url to mysql profile
* Add worker metrics
* Update grafana config
* Add system metrics doc
* Add process failover counter
* Add metrics image
* Change jpg to png
* Add command insert metrics
* Fix UT
* Revert UT
(cherry picked from commit e21d7b1551)
3.0.0/version-upgrade
Wenjun Ruan
2 years ago
40 changed files with 4466 additions and 573 deletions
@ -0,0 +1,154 @@
|
||||
# Introduction |
||||
|
||||
Apache DolphinScheduler has export some metrics to monitor the system. We use micrometer for the exporter facade, and |
||||
the default exporter is prometheus, more exporter is coming soon. |
||||
|
||||
## Quick Start |
||||
|
||||
You can add the following config in master/worker/alert/api's yaml file to open the metrics exporter. |
||||
|
||||
```yaml |
||||
metrics: |
||||
enabled: true |
||||
``` |
||||
|
||||
Once you open the metrics exporter, you can access the metrics by the url: `http://ip:port/actuator/prometheus` |
||||
|
||||
The exporter port is the `server.port` defined in application.yaml, e.g: master: `server.port: 5679`, worker: `server.port: 1235`, alert: `server.port: 50053`, api: `server.port: 12345`. |
||||
|
||||
For example, you can get the master metrics by `curl http://localhost:5679/actuator/prometheus` |
||||
|
||||
We have prepared the out-of-the-box Grafana configuration for you, you can find the Grafana dashboard |
||||
at `dolphinscheduler-meter/resources/grafana`, you can directly import these dashboards to grafana. |
||||
|
||||
If you want to try at docker, you can use the following command to start the prometheus with grafana: |
||||
|
||||
```shell |
||||
cd dolphinscheduler-meter/src/main/resources/grafana-demo |
||||
docker compose up |
||||
``` |
||||
|
||||
Then you can access the grafana by the url: `http://localhost/3001` |
||||
|
||||
 |
||||
 |
||||
 |
||||
|
||||
## Master Metrics |
||||
|
||||
Master metrics are exported by the DolphinScheduler master server. |
||||
|
||||
### System Metrics |
||||
|
||||
* dolphinscheduler_master_overload_count: Indicates the number of times the master has been overloaded. |
||||
* dolphinscheduler_master_consume_command_count: Indicates the number of commands has consumed. |
||||
|
||||
### Process Metrics |
||||
|
||||
* dolphinscheduler_create_command_count: Indicates the number of command has been inserted. |
||||
* dolphinscheduler_process_instance_submit_count: Indicates the number of process has been submitted. |
||||
* dolphinscheduler_process_instance_running_gauge: Indicates the number of process are running now. |
||||
* dolphinscheduler_process_instance_timeout_count: Indicates the number of process has been timeout. |
||||
* dolphinscheduler_process_instance_finish_count: Indicates the number of process has been finished, include success or |
||||
failure. |
||||
* dolphinscheduler_process_instance_success_count: Indicates the number of process has been successful. |
||||
* dolphinscheduler_process_instance_stop_count: Indicates the number of process has been stopped. |
||||
* dolphinscheduler_process_instance_failover_count: Indicates the number of process has been failed over. |
||||
|
||||
### Task Metrics |
||||
|
||||
* dolphinscheduler_task_timeout_count: Indicates the number of tasks has been timeout. |
||||
* dolphinscheduler_task_finish_count: Indicates the number of tasks has been finished, include success or failure. |
||||
* dolphinscheduler_task_success_count: Indicates the number of tasks has been successful. |
||||
* dolphinscheduler_task_timeout_count: Indicates the number of tasks has been timeout. |
||||
* dolphinscheduler_task_retry_count: Indicates the number of tasks has been retry. |
||||
* dolphinscheduler_task_failover_count: Indicates the number of tasks has been failover. |
||||
* dolphinscheduler_task_dispatch_count: Indicates the number of tasks has been dispatched to worker. |
||||
* dolphinscheduler_task_dispatch_failed_count: Indicates the number of tasks dispatched failed, if dispatched failed |
||||
will retry. |
||||
* dolphinscheduler_task_dispatch_error_count: Indicates the number of tasks dispatched error, if dispatched error, means |
||||
there are exception occur. |
||||
|
||||
## Worker Metrics |
||||
|
||||
Worker metrics are exported by the DolphinScheduler worker server. |
||||
|
||||
### System Metrics |
||||
|
||||
* dolphinscheduler_worker_overload_count: Indicates the number of times the worker has been overloaded. |
||||
* dolphinscheduler_worker_submit_queue_is_full_count: Indicates the number of times the worker's submit queue has been |
||||
full. |
||||
|
||||
### Task Metrics |
||||
|
||||
* dolphinscheduler_task_execute_count: Indicates the number of times a task has been executed, it contains a tag - |
||||
`task_type`. |
||||
* dolphinscheduler_task_execution_count: Indicates the total number of task has been executed. |
||||
* dolphinscheduler_task_execution_timer: Indicates the time spent executing tasks. |
||||
|
||||
## Default System Metrics |
||||
|
||||
In each server, there are some default metrics related to the system instance. |
||||
|
||||
### Database Metrics |
||||
|
||||
* hikaricp_connections_creation_seconds_max: Connection creation time max. |
||||
* hikaricp_connections_creation_seconds_count: Connection creation time count. |
||||
* hikaricp_connections_creation_seconds_sum: Connection creation time sum. |
||||
* hikaricp_connections_acquire_seconds_max: Connection acquire time max. |
||||
* hikaricp_connections_acquire_seconds_count: Connection acquire time count. |
||||
* hikaricp_connections_acquire_seconds_sum: Connection acquire time sum. |
||||
* hikaricp_connections_usage_seconds_max: Connection usage max. |
||||
* hikaricp_connections_usage_seconds_count: Connection usage time count. |
||||
* hikaricp_connections_usage_seconds_sum: Connection usage time sum. |
||||
* hikaricp_connections_max: Max connections. |
||||
* hikaricp_connections_min Min connections |
||||
* hikaricp_connections_active: Active connections. |
||||
* hikaricp_connections_idle: Idle connections. |
||||
* hikaricp_connections_pending: Pending connections. |
||||
* hikaricp_connections_timeout_total: Timeout connections. |
||||
* hikaricp_connections: Total connections |
||||
* jdbc_connections_max: Maximum number of active connections that can be allocated at the same time. |
||||
* jdbc_connections_min: Minimum number of idle connections in the pool. |
||||
* jdbc_connections_idle: Number of established but idle connections. |
||||
* jdbc_connections_active: Current number of active connections that have been allocated from the data source. |
||||
|
||||
### JVM Metrics |
||||
|
||||
* jvm_buffer_total_capacity_bytes: An estimate of the total capacity of the buffers in this pool. |
||||
* jvm_buffer_count_buffers: An estimate of the number of buffers in the pool. |
||||
* jvm_buffer_memory_used_bytes: An estimate of the memory that the Java virtual machine is using for this buffer pool. |
||||
* jvm_memory_committed_bytes: The amount of memory in bytes that is committed for the Java virtual machine to use. |
||||
* jvm_memory_max_bytes: The maximum amount of memory in bytes that can be used for memory management. |
||||
* jvm_memory_used_bytes: The amount of used memory. |
||||
* jvm_threads_peak_threads: The peak live thread count since the Java virtual machine started or peak was reset. |
||||
* jvm_threads_states_threads: The current number of threads having NEW state. |
||||
* jvm_gc_memory_allocated_bytes_total: Incremented for an increase in the size of the (young) heap memory pool after one GC to before the next. |
||||
* jvm_gc_max_data_size_bytes: Max size of long-lived heap memory pool. |
||||
* jvm_gc_pause_seconds_count: Time spent count in GC pause. |
||||
* jvm_gc_pause_seconds_sum: Time spent sum in GC pause. |
||||
* jvm_gc_pause_seconds_max: Time spent max in GC pause. |
||||
* jvm_gc_live_data_size_bytes: Size of long-lived heap memory pool after reclamation. |
||||
* jvm_gc_memory_promoted_bytes_total: Count of positive increases in the size of the old generation memory pool before GC to after GC. |
||||
* jvm_classes_loaded_classes: The number of classes that are currently loaded in the Java virtual machine. |
||||
* jvm_threads_live_threads: The current number of live threads including both daemon and non-daemon threads. |
||||
* jvm_threads_daemon_threads: The current number of live daemon threads. |
||||
* jvm_classes_unloaded_classes_total: The total number of classes unloaded since the Java virtual machine has started execution. |
||||
* process_cpu_usage: The "recent cpu usage" for the Java Virtual Machine process. |
||||
* process_start_time_seconds: Start time of the process since unix epoch. |
||||
* process_uptime_seconds: The uptime of the Java virtual machine. |
||||
|
||||
|
||||
## Other Metrics |
||||
* jetty_threads_config_max: The maximum number of threads in the pool. |
||||
* jetty_threads_config_min: The minimum number of threads in the pool. |
||||
* jetty_threads_current: The total number of threads in the pool. |
||||
* jetty_threads_idle: The number of idle threads in the pool. |
||||
* jetty_threads_busy: The number of busy threads in the pool. |
||||
* jetty_threads_jobs: Number of jobs queued waiting for a thread. |
||||
* process_files_max_files: The maximum file descriptor count. |
||||
* process_files_open_files: The open file descriptor count. |
||||
* system_cpu_usage: The "recent cpu usage" for the whole system. |
||||
* system_cpu_count: The number of processors available to the Java virtual machine. |
||||
* system_load_average_1m: The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time. |
||||
* logback_events_total: Number of level events that made it to the logs |
||||
|
After Width: | Height: | Size: 329 KiB |
|
After Width: | Height: | Size: 457 KiB |
|
After Width: | Height: | Size: 394 KiB |
@ -0,0 +1,53 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.server.master.metrics; |
||||
|
||||
import io.micrometer.core.instrument.Counter; |
||||
import io.micrometer.core.instrument.Metrics; |
||||
|
||||
public final class MasterServerMetrics { |
||||
|
||||
private MasterServerMetrics() { |
||||
throw new UnsupportedOperationException("Utility class"); |
||||
} |
||||
|
||||
/** |
||||
* Used to measure the master server is overload. |
||||
*/ |
||||
private static final Counter MASTER_OVERLOAD_COUNTER = |
||||
Counter.builder("dolphinscheduler_master_overload_count") |
||||
.description("Master server overload count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
/** |
||||
* Used to measure the number of process command consumed by master. |
||||
*/ |
||||
private static final Counter MASTER_CONSUME_COMMAND_COUNTER = |
||||
Counter.builder("dolphinscheduler_master_consume_command_count") |
||||
.description("Master server consume command count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
public static void incMasterOverload() { |
||||
MASTER_OVERLOAD_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incMasterConsumeCommand(int commandCount) { |
||||
MASTER_CONSUME_COMMAND_COUNTER.increment(commandCount); |
||||
} |
||||
|
||||
} |
||||
@ -0,0 +1,101 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.server.master.metrics; |
||||
|
||||
import java.util.function.Supplier; |
||||
|
||||
import io.micrometer.core.instrument.Counter; |
||||
import io.micrometer.core.instrument.Gauge; |
||||
import io.micrometer.core.instrument.Metrics; |
||||
|
||||
public final class ProcessInstanceMetrics { |
||||
|
||||
private ProcessInstanceMetrics() { |
||||
throw new UnsupportedOperationException("Utility class"); |
||||
} |
||||
|
||||
private static final Counter PROCESS_INSTANCE_SUBMIT_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_submit_count") |
||||
.description("Process instance submit total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter PROCESS_INSTANCE_TIMEOUT_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_timeout_count") |
||||
.description("Process instance timeout total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter PROCESS_INSTANCE_FINISH_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_finish_count") |
||||
.description("Process instance finish total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter PROCESS_INSTANCE_SUCCESS_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_success_count") |
||||
.description("Process instance success total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter PROCESS_INSTANCE_FAILURE_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_failure_count") |
||||
.description("Process instance failure total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter PROCESS_INSTANCE_STOP_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_stop_count") |
||||
.description("Process instance stop total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter PROCESS_INSTANCE_FAILOVER_COUNTER = |
||||
Counter.builder("dolphinscheduler_process_instance_failover_count") |
||||
.description("Process instance failover total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
public static synchronized void registerProcessInstanceRunningGauge(Supplier<Number> function) { |
||||
Gauge.builder("dolphinscheduler_process_instance_running_gauge", function) |
||||
.description("The current running process instance count") |
||||
.register(Metrics.globalRegistry); |
||||
} |
||||
|
||||
public static void incProcessInstanceSubmit() { |
||||
PROCESS_INSTANCE_SUBMIT_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incProcessInstanceTimeout() { |
||||
PROCESS_INSTANCE_TIMEOUT_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incProcessInstanceFinish() { |
||||
PROCESS_INSTANCE_FINISH_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incProcessInstanceSuccess() { |
||||
PROCESS_INSTANCE_SUCCESS_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incProcessInstanceFailure() { |
||||
PROCESS_INSTANCE_FAILURE_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incProcessInstanceStop() { |
||||
PROCESS_INSTANCE_STOP_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incProcessInstanceFailover() { |
||||
PROCESS_INSTANCE_FAILOVER_COUNTER.increment(); |
||||
} |
||||
|
||||
} |
||||
@ -0,0 +1,137 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.server.master.metrics; |
||||
|
||||
import java.util.function.Supplier; |
||||
|
||||
import io.micrometer.core.instrument.Counter; |
||||
import io.micrometer.core.instrument.Gauge; |
||||
import io.micrometer.core.instrument.Metrics; |
||||
|
||||
|
||||
public final class TaskMetrics { |
||||
private TaskMetrics() { |
||||
throw new UnsupportedOperationException("Utility class"); |
||||
} |
||||
|
||||
private static final Counter TASK_SUBMIT_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_submit_count") |
||||
.description("Task submit total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_FINISH_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_finish_count") |
||||
.description("Task finish total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_SUCCESS_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_success_count") |
||||
.description("Task success total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_FAILURE_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_failure_count") |
||||
.description("Task failure total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_TIMEOUT_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_timeout_count") |
||||
.description("Task timeout total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_RETRY_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_retry_count") |
||||
.description("Task retry total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_STOP_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_stop_count") |
||||
.description("Task stop total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_FAILOVER_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_failover_count") |
||||
.description("Task failover total count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_DISPATCH_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_dispatch_count") |
||||
.description("Task dispatch count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_DISPATCHER_FAILED = |
||||
Counter.builder("dolphinscheduler_task_dispatch_failed_count") |
||||
.description("Task dispatch failed count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter TASK_DISPATCH_ERROR = |
||||
Counter.builder("dolphinscheduler_task_dispatch_error_count") |
||||
.description("Task dispatch error") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
public static void incTaskSubmit() { |
||||
TASK_SUBMIT_COUNTER.increment(); |
||||
} |
||||
|
||||
public synchronized static void registerTaskRunning(Supplier<Number> consumer) { |
||||
Gauge.builder("dolphinscheduler_task_running_gauge", consumer) |
||||
.description("Task running count") |
||||
.register(Metrics.globalRegistry); |
||||
} |
||||
|
||||
public static void incTaskFinish() { |
||||
TASK_FINISH_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskSuccess() { |
||||
TASK_SUCCESS_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskFailure() { |
||||
TASK_FAILURE_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskTimeout() { |
||||
TASK_TIMEOUT_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskRetry() { |
||||
TASK_RETRY_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskStop() { |
||||
TASK_STOP_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskFailover() { |
||||
TASK_FAILOVER_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incTaskDispatchFailed(int failedCount) { |
||||
TASK_DISPATCHER_FAILED.increment(failedCount); |
||||
} |
||||
|
||||
public static void incTaskDispatchError() { |
||||
TASK_DISPATCH_ERROR.increment(); |
||||
} |
||||
|
||||
public static void incTaskDispatch() { |
||||
TASK_DISPATCH_COUNTER.increment(); |
||||
} |
||||
|
||||
} |
||||
@ -1,60 +0,0 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.server.master.registry; |
||||
|
||||
import org.apache.dolphinscheduler.dao.AlertDao; |
||||
import org.apache.dolphinscheduler.dao.mapper.WorkerGroupMapper; |
||||
import org.apache.dolphinscheduler.service.registry.RegistryClient; |
||||
|
||||
import org.junit.Before; |
||||
import org.junit.Test; |
||||
import org.junit.runner.RunWith; |
||||
import org.mockito.Mock; |
||||
import org.powermock.api.mockito.PowerMockito; |
||||
import org.powermock.core.classloader.annotations.PowerMockIgnore; |
||||
import org.powermock.core.classloader.annotations.PrepareForTest; |
||||
import org.powermock.modules.junit4.PowerMockRunner; |
||||
|
||||
/** |
||||
* server node manager test |
||||
*/ |
||||
@RunWith(PowerMockRunner.class) |
||||
@PrepareForTest({ RegistryClient.class }) |
||||
@PowerMockIgnore({"javax.management.*"}) |
||||
public class ServerNodeManagerTest { |
||||
|
||||
private ServerNodeManager serverNodeManager; |
||||
|
||||
@Mock |
||||
private WorkerGroupMapper workerGroupMapper; |
||||
|
||||
@Mock |
||||
private AlertDao alertDao; |
||||
|
||||
@Before |
||||
public void before() { |
||||
PowerMockito.suppress(PowerMockito.constructor(RegistryClient.class)); |
||||
serverNodeManager = PowerMockito.mock(ServerNodeManager.class); |
||||
} |
||||
|
||||
@Test |
||||
public void test(){ |
||||
//serverNodeManager.getWorkerGroupNodes()
|
||||
} |
||||
|
||||
} |
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,58 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.server.worker.metrics; |
||||
|
||||
import org.apache.dolphinscheduler.plugin.task.api.TaskChannelFactory; |
||||
|
||||
import java.util.HashMap; |
||||
import java.util.Map; |
||||
import java.util.ServiceLoader; |
||||
|
||||
import io.micrometer.core.instrument.Counter; |
||||
import io.micrometer.core.instrument.Metrics; |
||||
|
||||
public final class TaskMetrics { |
||||
|
||||
private TaskMetrics() { |
||||
throw new UnsupportedOperationException("Utility class"); |
||||
} |
||||
|
||||
private static Map<String, Counter> TASK_TYPE_EXECUTE_COUNTER = new HashMap<>(); |
||||
private static final Counter UNKNOWN_TASK_EXECUTE_COUNTER = |
||||
Counter.builder("dolphinscheduler_task_execute_count") |
||||
.tag("task_type", "unknown") |
||||
.description("task execute counter") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
static { |
||||
for (TaskChannelFactory taskChannelFactory : ServiceLoader.load(TaskChannelFactory.class)) { |
||||
TASK_TYPE_EXECUTE_COUNTER.put( |
||||
taskChannelFactory.getName(), |
||||
Counter.builder("dolphinscheduler_task_execute_count") |
||||
.tag("task_type", taskChannelFactory.getName()) |
||||
.description("task execute counter") |
||||
.register(Metrics.globalRegistry) |
||||
); |
||||
} |
||||
} |
||||
|
||||
public static void incrTaskTypeExecuteCount(String taskType) { |
||||
TASK_TYPE_EXECUTE_COUNTER.getOrDefault(taskType, UNKNOWN_TASK_EXECUTE_COUNTER).increment(); |
||||
} |
||||
|
||||
} |
||||
@ -0,0 +1,56 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
|
||||
package org.apache.dolphinscheduler.server.worker.metrics; |
||||
|
||||
import java.util.function.Supplier; |
||||
|
||||
import io.micrometer.core.instrument.Counter; |
||||
import io.micrometer.core.instrument.Gauge; |
||||
import io.micrometer.core.instrument.Metrics; |
||||
|
||||
public final class WorkerServerMetrics { |
||||
|
||||
public WorkerServerMetrics() { |
||||
throw new UnsupportedOperationException("Utility class"); |
||||
} |
||||
|
||||
private static final Counter WORKER_OVERLOAD_COUNTER = |
||||
Counter.builder("dolphinscheduler_worker_overload_count") |
||||
.description("worker load count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
private static final Counter WORKER_SUBMIT_QUEUE_IS_FULL_COUNTER = |
||||
Counter.builder("dolphinscheduler_worker_submit_queue_is_full_count") |
||||
.description("worker task submit queue is full count") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
public static void incWorkerOverloadCount() { |
||||
WORKER_OVERLOAD_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void incWorkerSubmitQueueIsFullCount() { |
||||
WORKER_SUBMIT_QUEUE_IS_FULL_COUNTER.increment(); |
||||
} |
||||
|
||||
public static void registerWorkerRunningTaskGauge(Supplier<Number> supplier) { |
||||
Gauge.builder("dolphinscheduler_worker_running_task_gauge", supplier) |
||||
.description("worker running task gauge") |
||||
.register(Metrics.globalRegistry); |
||||
|
||||
} |
||||
} |
||||
Loading…
Reference in new issue