CPU

High CPU Utilization

By design, we expect thatDot Streaming Graph to utilize as many resources as it is allotted. High CPU usage alone, without additional symptoms, is desirable and reflects good utilization of the system.

Low CPU Utilization

thatDot Streaming Graph is designed to use backpressure to limit incoming data from producers. If any aspect of the graph is overwhelmed in the cluster (e.g. the data storage layer is slow), Streaming Graph will backpressure incoming data to ensure all work can complete. This mechanism provides data safety by not dropping incoming records, and system resilience by not sending more data than the graph can handle. To understand if the ingest is backpressuring, view the shared.valve.ingest.{ingest-name}.metric to see if the value is increasing. A higher value means more backpressure.

If the ingest stream parallelism setting is too low, the system may not take advantage of all the parallel processing it is capable of. This setting determines how many items from the ingest source will be causing their effects in parallel. Too low of a parallelism setting will cause the CPU to be underutilized.

Variable CPU Utilization Between Members

If some thatDot Streaming Graph cluster members have high CPU utilization and others have low CPU utilization, it may or may not be a concern. The variation indicates an imbalance in the workload among members. Depending on the use case, that might be expected.

Even if it is not expected, it can be a temporary occurrence; for instance when a cluster member is replaced by a hot spare and the newly active member is warming up its data cache, causing some queries to be backpressured on other members.

If the condition persists and is unexpected, it can be caused by data modeling choices; for instance when a node in the graph is particularly hot, or has become a supernode. See the section on supernode mitigation.