Amazon CloudWatch Explained: Metrics, Logs, Alarms and Dashboards

Monitor AWS like an operator: CloudWatch metrics and namespaces, the CloudWatch agent for memory and disk, Logs and Logs Insights, metric filters, alarms, composite alarms and dashboards.

Amazon CloudWatch is where AWS resources report how they are doing. It collects metrics and logs, raises alarms when something crosses a threshold, and draws dashboards for your team. Almost every operational question ("is the site slow?", "why did the job fail?", "who needs to be paged?") starts in CloudWatch.

Metrics

A metric is a time series of data points, organized by:

  • Namespace: AWS/EC2, AWS/ApplicationELB, AWS/Lambda, or your own like Shop/Checkout.
  • Dimensions: name/value pairs identifying the resource, such as InstanceId=i-0abc or FunctionName=orders.
  • Statistics: Average, Sum, Minimum, Maximum, SampleCount and percentiles like p99.

Many AWS services publish metrics automatically and free. EC2 sends basic metrics every five minutes, or every minute with detailed monitoring. You can publish custom metrics from your code with PutMetricData or, more cheaply at scale, with the Embedded Metric Format in your logs.

Use percentiles for latency. An average of 120 ms can hide a p99 of 3 seconds that real users feel.

What EC2 does not tell you: the CloudWatch agent

EC2's built-in metrics come from the hypervisor, so they cannot see inside the operating system. Memory usage and disk space are not included. Install the CloudWatch agent to collect them, plus log files, and store its configuration in Systems Manager Parameter Store so every instance uses the same setup.

Logs

CloudWatch Logs stores log events in log groups (one per application or function) and log streams (one per instance or container). Lambda, API Gateway, ECS, VPC Flow Logs and many services send logs here.

  • Set a retention period on every log group. The default is "never expire", which quietly grows your bill.
  • Logs Insights queries logs with a purpose-built language. Example: the slowest Lambda invocations in the last hour: 
    filter @type = "REPORT"
| stats max(@duration) as maxMs, avg(@duration) as avgMs by bin(5m)
| sort maxMs desc
  • Structured JSON logs make Insights much more powerful: fields are parsed automatically.
  • Metric filters turn log patterns into metrics, such as counting "level":"ERROR" lines, which you can then alarm on.
  • Subscription filters stream logs in real time to Lambda, Kinesis or Firehose for processing or archiving.

Alarms

An alarm watches a metric (or a math expression over metrics) and changes state between OK, ALARM and INSUFFICIENT_DATA. Key settings:

  • Period and evaluation: for example "3 out of 5 one-minute periods above 80%" filters out brief spikes.
  • Missing data treatment: decide whether no data means breaching, not breaching or ignore. For a heartbeat metric, missing data should alarm.
  • Actions: notify an SNS topic, scale an Auto Scaling group, stop or recover an EC2 instance, or start a Systems Manager automation.

Composite alarms combine several alarms with AND/OR/NOT logic, so you page a human only when "errors are high AND latency is high", reducing alert fatigue. Anomaly detection alarms learn a metric's normal pattern and alarm on deviations, useful when "normal" changes by time of day.

Dashboards

Dashboards put the metrics that matter for a service on one screen: traffic, errors, latency, saturation. Build one per service and one for the overall system, and include alarm status widgets so on-call engineers see what is firing.

Beyond the basics

  • Application Signals and X-Ray for traces across services.
  • Synthetics canaries that test your website or API on a schedule like a real user.
  • Container Insights and Lambda Insights for deeper runtime metrics.

Keeping CloudWatch costs sensible

  • Set log retention and avoid debug logging in production.
  • Be careful with high-cardinality custom metric dimensions (one metric per user ID multiplies cost).
  • Delete unused alarms and dashboards; review the CloudWatch line in Cost Explorer monthly.

Practice it

Try CloudWatch Alarm to Automatically Stop Unused EC2, CloudWatch Logs Insights: JSON Logs and a Saved Query, CloudWatch Agent: Memory, Disk and Log Collection and Composite Alarms to Cut Alert Noise. Then investigate The Alarm Fired but Nobody Was Paged.

FAQ

Why is my alarm stuck in INSUFFICIENT_DATA?

The metric is not being published for that dimension combination, the period is shorter than the metric's resolution, or the resource is stopped.

Can CloudWatch monitor on-premises servers?

Yes. The CloudWatch agent runs on on-premises Linux and Windows servers with appropriate credentials.

Browse all AWS tutorials · Hands-on AWS labs