simonsnewchat.rivetgarden.com
←Back

What Measurement Window Should I Use Before Resizing a VM?

When managing cloud infrastructure, one of the most common and impactful tasks is right-sizing virtual machines (VMs). However, the decision to resize a VM is often wrapped around vague averages and insufficient observation windows, leading to wasted spend or degraded performance. This article dives deep into how to select the right observation window before resizing a VM, drawing on insights from tools like AWS Compute Optimizer and Azure Advisor.

We’ll cover why small always-on services can hide cloud waste, the nuances of shared CPU definitions across cloud providers, and why measuring peaks—and not averages—with the right metrics and timeframes is finops cloud waste playbook essential. Key themes include understanding release cycle spikes and month-end job bursts, utilizing percentiles like P95 and P99, and factoring in spike duration before making sizing decisions.

Why Observation Window Matters in VM Resizing

The observation window is the period you use to collect performance and utilization data to inform your resizing decisions. Choosing too short or too narrow a window means missing important peaks or dips in resource demand, while too long or overly broad windows may dilute meaningful spikes into harmless averages.

Traditional resizing often relies on:

  • Average CPU utilization over a day or week
  • Requests per second or throughput averages
  • Cost estimations based on “typical” usage

These approaches are flawed for two main reasons:

  1. Average CPU Utilization hides Peaks: Average CPU can mask short burst workloads or periodic jobs that require temporarily higher capacity.
  2. Missed Context on Workload Patterns: Enterprises often have cyclical workloads—release cycles, month-end processing, backups—that create temporary demand spikes which need to be observed and planned for.

Avoiding Cloud Waste in Always-On Small Services

One subtle source of wasted cloud spend is always-on small services that operate comfortably below nominal utilization most of the time. Because these services run 24/7, it’s tempting to pick the smallest VM instance and call it a day. But sporadic spikes in usage can cause latency or failures that don’t show up in averages, leading to overprovisioning with larger VM sizes “just in case.”

Measuring with the right observation window and metrics can help detect these spike patterns, enabling either auto-scaling or right-sized VMs that meet peak demand without overspending during idle times.

Cloud Provider Nuances: Shared CPUs and Performance Guarantees

When evaluating CPU utilization, remember that definitions of shared CPU and vCPU—notions frequently tossed around—vary significantly by cloud provider, which can affect how you interpret your observations.

Cloud Provider Shared CPU Definition Implications for Measurement AWS Some burstable instance families (e.g., T3, T4g) share physical cores, with CPU credits accumulated and consumed. CPU credits affect burst behavior; monitoring CPU credit balance alongside CPU utilization is essential. Azure Shared CPU VMs (B-series) accumulate credits and may be throttled; others have dedicated vCPUs. Assess both CPU usage and credit availability; treat vCPU counts with caution as a strict speed indicator. Google Cloud Shared and non-shared vCPU options exist; shared cores can be limited by hypervisor scheduling. Performance can be inconsistent; observe P95/P99 latencies in addition to CPU usage.

Understanding these nuances prevents naive resizing decisions based purely on average CPU percentages or vCPU counts. Because a vCPU is not a linear performance guarantee, measuring with respect to real workload latency and tail percentiles remains critical.

Picking the Right Observation Window

So, how do you pick the right observation window? The objective is to capture enough data to reliably observe peak utilization moments without drowning in irrelevant detail.

  • Look Beyond Average CPU: Focus on P95, P99 CPU usage or latency percentiles instead of averages. These help catch spikes that cause user-facing slowdowns or queue bottlenecks.
  • Consider Spike Duration: Short, sharp spikes that complete in seconds may not need resizing—burstable instances or autoscaling can handle them. But persistent spikes over minutes or hours will require larger VM sizes.
  • Match Measurement Window to Workload Cycles: Align observation windows with known operational cycles:
    • Release cycle spikes: For example, if marathon build jobs run every 2 weeks and last 6 hours, monitor that full period.
    • Month-end jobs: Heavy batch or report generation often causes sustained spike across days; observe at least the full month-end window.
  • Use Sliding Windows and Compare: Combining shorter windows (e.g., hourly) aggregated into longer horizons (weekly, monthly) reveals hidden patterns.
  • Cross-Check with Cost Tools: Correlate findings from AWS Compute Optimizer or Azure Advisor with raw metrics to balance cost and performance.

Example Measurement Windows

Workload Type Suggested Observation Window Focus Metrics Always-on small service 1 week sliding window, capturing daily peak periods P95 CPU, latency spikes, CPU credit usage (if burstable) Release cycle builds and integration tests 2 weeks covering build cycle + 1 day buffer Peak CPU, spike duration, latency P99 during build windows Month-end batch jobs 5 days spanning month-end + prior preparation window CPU and memory utilization, job completion time, queue depth

How AWS Compute Optimizer and Azure Advisor Help

Both AWS Compute Optimizer and Azure Advisor offer automated recommendations for resizing VMs by analyzing historical usage data. However, to understand the validity of these recommendations, you must understand the observation window and metrics they've used.

AWS Compute Optimizer

Compute Optimizer analyzes up to 14 days of detailed utilization data to provide recommendations. It uses a mix of metrics including average CPU utilization, memory, network I/O, and considers burstable credit balances. However, relying on a 14-day window might miss longer-term spike patterns such as monthly batch jobs or quarterly release spikes.

To improve sizing decisions using Compute Optimizer:

  • Correlate recommendations with application deployment and workload schedules—know when cyclical spikes happen.
  • Complement recommendations with your own long-term metric analysis via CloudWatch or third-party monitoring.
  • Check P95/P99 latency and CPU utilization within the 14-day period rather than only averages.

Azure Advisor

Azure Advisor collects similar telemetry but also looks at VM performance against Azure Service Level Agreements. Its recommendation windows can vary but often default to 7 days of aggregated usage data. Like AWS, this can miss important cyclical patterns unless you configure longer custom observation periods using Azure Monitor logs.

When using Azure Advisor:

  • Review peak CPU and memory utilization timelines around known batch job executions.
  • Supplement short-term Advisor recommendations with longer-term trends via Azure Monitor or Log Analytics.
  • Validate how shared CPU credit balances on Burstable VMs affect observed utilization.

Putting It All Together: A Practical Approach

  1. Identify critical workload cycles. Document when and how long the known spikes occur — build cycles, reports, month-end jobs.
  2. Select multiple observation windows. Use sliding windows which include immediate past days, weeks, and relevant cyclical spike periods.
  3. Collect percentiles, not averages. Focus on P95/P99 CPU and latency metrics, alongside spike duration.
  4. Check cloud provider burst or credit balances. Don’t treat vCPU counts or baseline CPU percentages as direct performance guarantees.
  5. Complement automated recommendations. Use tools like AWS Compute Optimizer or Azure Advisor as starting points, not decision-makers.
  6. Run pilots with clear rollback criteria. Avoid changing instance sizes based on partial or unrepresentative data.

Summary

To make smart VM resizing decisions, you must be intentional about what observation window you use and what performance data you collect within it. Short-term averages can mask important workload bursts that occur during release cycle spikes or month-end jobs. Instead, track percentiles (P95, P99) of CPU and latency, factor in spike durations, and remember the quirks of shared CPU offerings per cloud provider.

Using AWS Compute Optimizer or Azure Advisor is valuable, but augment these tools with custom analysis matching your workload cadence and operational realities. This comprehensive approach helps minimize cloud waste, improves reliability, and ensures your VM sizing is data-driven rather than guesswork.

Before you resize, always ask: What do the P95 and P99 metrics look like during peak windows? This question, along with appropriate observation windows, is the key to right-sizing your cloud infrastructure effectively.