How Do I Avoid Getting Fooled by 60-Second CPU Graphs?

When managing cloud infrastructure, particularly compute resources, one of the most common pitfalls is relying on coarse-grained CPU metrics that mask the true workload characteristics. A 60-second CPU utilization graph might look stable and low, but it often hides brief but critical CPU spikes that cause latency, throttling, or over-provisioning.

In this post, I’ll share the key lessons learned from 12 years of cloud infrastructure and SRE experience spanning AWS, Azure, and Google Cloud. I’ll dig into why metric granularity, observation windows, and percentile calculations matter, especially for always-on small services. I’ll also walk through how tools like AWS Compute Optimizer and Azure Advisor can help you avoid wasting money or experiencing unexpected performance hiccups.

Why 60-Second CPU Graphs Deceive You

Cloud providers typically expose CPU utilization metrics aggregated into one-minute data points by default. Visualizing CPU in 1-minute averages smooths out any short bursts of Visit this site consumption below the sampling granularity. Although this may look like your workload is using 10% CPU consistently, the reality could be brief spikes to 80% CPU for a few seconds, masking critical bottlenecks or over-provisioning.

Let’s call out some of the deceptive effects of 60-second CPU datapoints:

Leveling effect: Short CPU spikes and dips get averaged out into a flat line, hiding usage patterns important for capacity planning.
Ignored bursty workloads: Queues, event-driven workers, and always-on small services are often bursty by design. Averaging dilutes those bursts into low averages that mislead decision-making.
Faulty instance selection: Relying on average CPU can lead to downgrading instance types or families that are undersized for peak load, risking saturation or throttling.
Cost waste: Conversely, overestimating steady-state need without looking at peak usage patterns can lead to unnecessary over-provisioning.

Real-World Scenario: Always-On Small Services

Always-on services—such as lightweight HTTP APIs, internal dashboards, or heartbeat emitters—are classic cases where 60-second sampling works against you. These services usually spend most of their time idle but may have short CPU bursts to process requests or background jobs. Averaging out CPU hides these spikes and might make you think the service only needs a fraction of the CPU it occasionally consumes in bursts.

Understanding How CPU Sharing Varies Across Cloud Providers

Another layer of complexity: the definition of "shared CPU" isn’t uniform across cloud providers. Naively assuming “shared CPU” means poor performance or unreliable uptime is misleading.

Provider Shared vCPU Concept Typical CPU Burst Behavior Impact on Performance AWS T-series burstable instances with CPU credits Can accumulate CPU credits during idle periods and burst above baseline Good performance for bursty workloads but can throttle if credits exhausted Azure B-series burstable VMs with CPU credits CPU credits model similar to AWS allowing short bursts Reliable for moderate workloads, throttling possible on sustained peak Google Cloud Shared-core instances using fractional vCPUs Shared physical cores with potential for burst above baseline Sufficient for light workloads but capacity contention possible

Understanding these nuances helps you interpret CPU metrics correctly in context. For instance, a 5% average CPU on a burstable AWS T3 instance could mean you’re using almost 100% CPU in short bursts periodically.

Measure Peaks Correctly: Observation Window and Granularity Matter

How you aggregate and observe CPU metrics determines whether you see the truth about your workload.

Sampling interval: Pick a sampling interval small enough to catch sub-minute bursts. Five seconds or less is ideal.
Observation window: Choose windows that cover relevant workload patterns—e.g., 5-15 minutes for short bursts instead of 1 hour averages.
Percentiles vs averages: Always analyze the 95th, 99th percentile CPU utilization during the observation window, not just averages.
Spike duration tracking: Notice how long CPU spikes last. Very brief spikes may not impact performance but sustained spikes indicate saturation.

What P95 and P99 Look Like Before Switching Instances

Before you consider switching VM or instance types for cost savings or performance reasons, ask yourself: what do the P95 and P99 CPU usage percentiles look like over the sampling window? If your P99 hits 80-90% CPU frequently for a few https://dibz.me/blog/what-should-i-measure-besides-cpu-for-a-shared-cpu-migration-1253 seconds, moving to a smaller CPU count instance might backfire.

This approach helps avoid basing decisions on average CPU alone—a metric that inherently hides the spikes that matter most.

Using AWS Compute Optimizer and Azure Advisor Effectively

Both AWS Compute Optimizer and Azure Advisor provide insightful recommendations on right-sizing and optimizing workloads, but relying solely on their default suggestions can lead to overconfidence if you don’t validate input metrics closely.

AWS Compute Optimizer

Leverages metrics from Amazon CloudWatch, but defaults to 5-minute granularity for EC2 instances.
Produces recommendations based on average CPU usage; less accurate for bursty workloads with brief spikes.
Supports enhanced monitoring (1-minute) but still can miss sub-minute spikes due to metric aggregation.
To get better recommendations, enable detailed monitoring and supplement with custom CloudWatch metrics or agent-based monitoring for finer granularity.

Azure Advisor

Uses aggregated metrics across multiple time windows but primarily relies on 1-minute sampling in Azure Monitor.
Offers VM right-sizing and cost-saving recommendations that might overlook short CPU bursts.
Integration with Azure Monitor allows fine-tuning queries for percentiles and spike detection to validate suggestions.

Best Practices to Avoid Cost Waste and Performance Issues

Here are my field-tested guidelines for avoiding getting fooled by 60-second CPU graphs:

Enable high granularity monitoring: Always collect 5-second or better CPU metrics via cloud agents or CloudWatch custom metrics where possible.
Analyze percentiles, not just averages: Use P95 and P99 CPU utilization on relevant time windows instead of mean CPU.
Track spike duration: Filter CPU spikes by their length to avoid reacting to short, harmless bursts.
Understand your CPU concept: Know if your instance uses burstable CPUs, shared cores, or dedicated cores and how that affects metrics.
Keep the bigger picture in mind: Complement CPU usage with metrics on latency, queue depths, and error rates to correlate performance impact.
Write rollback criteria: Before experimenting with instance right-sizing or SKU changes, define clear thresholds to revert if latency or errors rise.
Don’t trust cloud optimizer tools blindly: Use them as a starting point but validate with your own high-resolution metrics and business context.

Conclusion

60-second CPU graphs can lull you into a false sense of security or spur misguided cost-saving efforts that cause more harm than good. Always remember to:

Scrutinize metric granularity and sampling intervals
Rely on percentile distributions rather than averages
Understand cloud provider CPU sharing nuances
Incorporate spike duration to contextualize CPU peaks
Augment cloud optimizer recommendations with your own analysis

By shifting focus from coarse averages to detailed percentiles and shorter observation windows, you’ll uncover hidden CPU behaviors that directly impact performance and cost. This quantitative foundation empowers better right-sizing decisions, optimized budgets, and resilient distributed systems.

Don’t get fooled by a nice flat 60-second CPU graph—dig into the fine-grained data and measure to understand before you optimize.