AWS Unified CloudWatch Agent: Metrics, Logs, and Installation – SAP-C02 Study Guide

Purpose of This Lesson

The Unified CloudWatch Agent extends monitoring beyond the default metrics available from AWS services and EC2. It runs directly on EC2 instances or on-premises servers to collect detailed operating-system metrics, application metrics, and log files.

For the exam, distinguish between metrics available automatically from EC2 and metrics that require an installed agent.

Key Concepts

What the Unified CloudWatch Agent Collects

The agent can collect:

• Detailed system-level metrics from EC2 instances
• Detailed system-level metrics from on-premises servers
• Application custom metrics using StatsD and CollectD
• Log files from Linux and Windows systems

Depending on the operating system and configuration, collected metrics can include information such as memory usage, disk usage, processes, and swap utilization—data that is generally not included in standard EC2 monitoring.

Where the Agent Runs

The agent must be installed on the server being monitored. Supported environments include:

• Amazon EC2
• On-premises servers
• Linux
• Windows Server
• macOS

The agent is not a remote, agentless monitoring mechanism. If detailed guest operating-system metrics or logs are required, software must run on the host.

Standard EC2 Metrics Versus Agent Metrics

EC2 automatically publishes foundational instance metrics such as CPU utilization, network traffic, and disk activity. These are hypervisor- or instance-level metrics and do not require an operating-system agent.

The Unified CloudWatch Agent is appropriate when the requirement involves data inside the guest operating system, including:

• Memory utilization
• Disk-space utilization
• Detailed disk and process metrics
• Application-generated metrics
• Operating-system and application log files

Installing the agent does not replace standard EC2 metrics; it supplements them.

Installation and Configuration Workflow

A typical deployment sequence is:

1. Create or select the IAM permissions required for CloudWatch metric and log delivery.
2. Install the agent package on the target server.
3. Create or update the agent configuration, defining the metrics, log files, destinations, and collection intervals.
4. Start the agent.
5. Validate that metrics appear in CloudWatch and logs arrive in the configured CloudWatch Logs log groups.

On some AWS-provided operating-system images, the package may already be available, but the configuration and permissions still need to be established.

IAM and Credentials

For an EC2 instance, attach an IAM instance role with permissions for the agent to publish metrics and logs. The AWS managed policy CloudWatchAgentServerPolicy is commonly associated with this use case, although production environments should apply least privilege and use a customer-managed policy when appropriate.

For an on-premises server, the agent cannot use an EC2 instance profile. It needs AWS credentials through a named AWS CLI/shared-credentials profile or another supported credential mechanism. Credentials should be protected and rotated according to organizational policy.

If Systems Manager is used, the server also needs the permissions and connectivity required to operate as a Systems Manager managed node. This is separate from the permissions needed to publish CloudWatch data.

Systems Manager Integration

AWS Systems Manager can help deploy and manage the agent across fleets. It can be used to:

• Run installation commands remotely
• Distribute or update the agent configuration
• Start or stop the agent
• Manage EC2 instances without direct SSH access, when the Systems Manager prerequisites are satisfied

Systems Manager can simplify fleet-wide deployment, but it does not eliminate the need for an agent on each monitored server.

Exam-Relevant Takeaways

• The Unified CloudWatch Agent must be installed on the EC2 instance or on-premises server.
• Use the agent for guest operating-system metrics, application metrics, and log collection.
• Standard EC2 metrics do not provide complete visibility into memory or filesystem utilization.
• StatsD and CollectD can provide application or custom metrics to the agent.
• EC2 uses an instance profile; on-premises servers require another credential source, commonly a named profile.
• The agent configuration determines which metrics and logs are collected.
• Systems Manager is an optional deployment and management mechanism, not a replacement for the CloudWatch agent.
• Separate CloudWatch publishing permissions from Systems Manager managed-node permissions.

Architecture Decision Guide

Requirement	Recommended approach	Important consideration
CPU, network, and basic EC2 instance monitoring	Use native EC2/CloudWatch metrics	No host agent is required for standard metrics
Memory or filesystem utilization from an EC2 instance	Install and configure the Unified CloudWatch Agent	Requires an EC2 IAM role with suitable permissions
Application custom metrics	Use the agent with StatsD or CollectD	Configure the application protocol and metric destination
Centralized application or operating-system logs	Use the agent’s log collection configuration	Define log files and CloudWatch Logs destinations
Monitoring an on-premises Linux or Windows server	Install the agent on the server	Use an appropriate credential profile or credential provider
Deploying the agent to many EC2 instances	Use Systems Manager automation, Run Command, State Manager, or related capabilities	Instances must meet Systems Manager connectivity and IAM prerequisites
Avoiding host software installation	Use native service metrics or a service-specific integration	Agent-level guest metrics and file logs cannot be collected without host-side software

Common Exam Traps

• Assuming EC2 automatically reports memory utilization: Standard EC2 metrics generally do not include guest memory usage. The agent is required.
• Confusing detailed monitoring with agent monitoring: EC2 detailed monitoring increases the frequency of supported EC2 metrics; it does not provide guest memory, filesystem, or application metrics.
• Using an instance role for an on-premises server: On-premises systems do not have EC2 instance profiles. Configure a named profile or another secure credential source.
• Treating Systems Manager as agentless CloudWatch monitoring: Systems Manager can deploy and manage the agent, but the CloudWatch agent still runs on the host.
• Forgetting log permissions: Metric publishing permissions alone may not be sufficient for log delivery. The role or credentials must allow the required CloudWatch Logs actions.
• Ignoring configuration: Installing the package does not automatically collect every useful metric or log file. The agent configuration controls collection.
• Granting broad administrative permissions: Monitoring does not require AdministratorAccess; use the smallest practical CloudWatch and Systems Manager permissions.

Real-World Engineer Notes

• Use a standardized agent configuration and distribute it through Systems Manager for consistent fleet management.
• Store configurations in a controlled location, such as Systems Manager Parameter Store, when appropriate, and restrict who can modify them.
• Scope CloudWatch Logs permissions to approved log groups where practical.
• Monitor agent health and configuration drift; a stopped agent creates a monitoring blind spot.
• Avoid collecting unnecessary high-cardinality or high-frequency metrics because custom metrics and log ingestion can increase cost.
• Confirm network access to AWS endpoints. Private subnets may require suitable VPC endpoints or a NAT path for CloudWatch, CloudWatch Logs, and Systems Manager operations.
• Use log retention policies and subscription filters deliberately so centralized logging does not become an uncontrolled storage expense.

Quick Reference Summary

• Purpose: Collect detailed host metrics, custom application metrics, and logs.
• Location: Runs on EC2 and on-premises servers.
• Operating systems: Linux, Windows Server, and macOS.
• Protocols: StatsD and CollectD for custom metrics.
• EC2 credentials: IAM instance profile.
• On-premises credentials: Named profile or another supported AWS credential mechanism.
• Deployment options: Command line, Systems Manager, or CloudFormation-based automation.
• Configuration: Defines metrics, log files, collection intervals, and destinations.
• Key distinction: Native EC2 metrics are not the same as guest OS metrics collected by the agent.

Flashcards

1. Q: Why install the Unified CloudWatch Agent on an EC2 instance?

A: To collect guest operating-system metrics, application metrics, and log files that native EC2 metrics do not provide.

1. Q: Does the CloudWatch agent run remotely from CloudWatch?

A: No. It must be installed and running on the monitored server.

1. Q: Which custom metric protocols does the agent support in this lesson?

A: StatsD and CollectD.

1. Q: What credential mechanism is normally used by an EC2 instance running the agent?

A: An IAM instance profile attached to the instance.

1. Q: What is a common credential approach for an on-premises server?

A: A named AWS CLI/shared-credentials profile or another supported secure credential provider.

1. Q: Does EC2 detailed monitoring collect memory utilization?

A: No. Detailed monitoring changes the frequency of supported EC2 metrics; the agent is generally needed for memory metrics.

1. Q: What determines which metrics and logs the agent collects?

A: The agent configuration file or configuration document.

1. Q: Is Systems Manager mandatory for the CloudWatch agent?

A: No. It is optional, but useful for fleet deployment and management.

1. Q: What must be present before the agent can publish data?

A: Suitable IAM permissions and network connectivity to the required AWS endpoints.

1. Q: Can the agent collect logs from both Linux and Windows servers?

A: Yes, when installed and configured on supported Linux or Windows systems.

Practice Questions

Question 1

A company needs memory utilization and filesystem usage for a fleet of EC2 instances. Native EC2 metrics are already enabled. What is the most appropriate solution?

A. Enable EC2 detailed monitoring only
B. Install and configure the Unified CloudWatch Agent
C. Enable VPC Flow Logs
D. Create a CloudTrail trail

Correct answer: B

Explanation: Native and detailed EC2 monitoring do not normally provide guest memory and filesystem utilization. The Unified CloudWatch Agent must run on the instances and be configured to publish these metrics.

Question 2

An organization wants to collect application logs from Linux servers in its data center and send them to CloudWatch Logs. Which requirement is essential?

A. Attach an EC2 instance profile to each server
B. Install and configure the Unified CloudWatch Agent on each server
C. Enable enhanced networking on the servers
D. Create a CloudFront distribution

Correct answer: B

Explanation: The agent runs on on-premises servers and can collect configured log files. On-premises systems use an appropriate AWS credential source rather than an EC2 instance profile.

Question 3

A company has hundreds of EC2 instances and wants to deploy a consistent CloudWatch agent configuration without allowing SSH access. Which AWS service is most suitable?

A. AWS Systems Manager
B. Amazon Inspector only
C. AWS CloudTrail
D. Amazon Route 53

Correct answer: A

Explanation: Systems Manager can execute commands and manage configurations across managed instances without requiring direct SSH access, assuming the instances have the required IAM permissions and connectivity.

Question 4

An on-premises server runs the CloudWatch agent. The administrator configured an EC2 IAM role but metrics are not being published. What is the most likely issue?

A. On-premises servers cannot use EC2 instance profiles
B. EC2 detailed monitoring is disabled
C. The server must be in an AWS Availability Zone
D. CloudWatch accepts metrics only from EC2

Correct answer: A

Explanation: Instance profiles are an EC2 credential mechanism. An on-premises server needs a named profile or another supported credential provider with permissions to publish the configured CloudWatch data.

Question 5

A team installed the Unified CloudWatch Agent but sees no application metrics. The application emits metrics using StatsD. What should the team check first?

A. Whether the agent configuration enables the StatsD receiver and defines the metric destination
B. Whether an internet-facing load balancer is deployed
C. Whether the EC2 instance uses a placement group
D. Whether CloudTrail data events are enabled

Correct answer: A

Explanation: Installing the agent alone does not enable every collection source. The configuration must enable and correctly define the StatsD input and the CloudWatch publishing destination.