Study guide

Technical reference and lesson notes

AWS VPC Flow Logs: Configuration, IAM Permissions, Destinations, and Troubleshooting

Purpose of This Lesson

VPC Flow Logs capture information about IP traffic going to and from network interfaces in a VPC. This lesson focuses on creating a flow log, configuring the IAM role required for delivery to Amazon CloudWatch Logs, generating test traffic, validating the resulting log stream, and cleaning up the lab resources.

The workflow is relevant to AWS Certified CloudOps Engineer Associate scenario questions involving network visibility, IAM permissions, logging destinations, and troubleshooting missing flow-log data.

Key Concepts

VPC Flow Logs: Capture information about network traffic associated with network interfaces in a VPC.
Scope: Flow logs can be created at the:
VPC level
Subnet level
Network interface level
Destinations: Flow-log data can be delivered to:
Amazon CloudWatch Logs
Amazon S3
Traffic filter: The configuration can capture all traffic, as used in the lab.
Aggregation interval: The lab leaves the maximum aggregation interval at 10 minutes.
CloudWatch delivery role: When CloudWatch Logs is the destination, the VPC Flow Logs service needs an IAM role that it can assume.
Observed record content: The example log stream contains accepted and rejected traffic, source and destination IP addresses, source and destination ports, and additional traffic information.
Log-stream naming: The demonstrated CloudWatch log stream uses an elastic network interface (ENI) ID.

VPC Flow Logs Configuration and Validation

1. Create the IAM role

For delivery to CloudWatch Logs, create an IAM role with a custom trust policy that allows the VPC Flow Logs service principal to assume it:

{
  "Effect": "Allow",
  "Principal": {
    "Service": "vpc-flow-logs.amazonaws.com"
  },
  "Action": "sts:AssumeRole"
}

The role in the lab is named VPC Flow Logs-CW. The important requirement is not the name but the trust relationship: the VPC Flow Logs service must be allowed to assume the role.

2. Attach CloudWatch Logs permissions

Attach an inline permissions policy to the role. The lab grants these CloudWatch Logs actions:

logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents
logs:DescribeLogGroups
logs:DescribeLogStreams

The policy applies to any resource in the demonstrated configuration. In a production environment, permissions should be scoped appropriately where practical.

3. Generate traffic in the target VPC

The lab launches an EC2 instance in the default VPC and a public subnet, with a public IP automatically assigned and SSH access allowed. Instance Connect is then used to generate traffic, including:

ping google.com
curl amazon.com

This step is useful because a newly created destination may not show data immediately. Flow logs need traffic to record, and delivery can also take time.

4. Create the flow log

From the target VPC, create a flow log with these lab settings:

Scope: The default VPC
Maximum aggregation interval: 10 minutes
Filter: All traffic
Destination: A CloudWatch Logs log group, such as /VPC-logs
IAM role: The role created for CloudWatch delivery

If the trust policy and permissions are correct, the flow log can be created successfully.

5. Validate the output

Select the configured destination to open CloudWatch Logs. Initially, the log group or stream may not exist or may contain no data. After generating traffic and waiting several minutes, a log stream should appear.

In the lab, the stream is named with an ENI ID. The records show traffic outcomes such as ACCEPT and REJECT, along with source and destination addresses and ports.

6. Clean up

After validation, delete the flow log and terminate the temporary EC2 instance. This prevents the lab from continuing to collect logs and avoids leaving unnecessary compute resources running.

Exam- or Assessment-Relevant Takeaways

When CloudWatch Logs is the destination, identify the need for an IAM role that the VPC Flow Logs service can assume.
Distinguish the role’s trust policy from its permissions policy. The trust policy enables role assumption; the permissions policy grants CloudWatch Logs actions.
Remember the three supported configuration scopes demonstrated here: VPC, subnet, and network interface.
A missing log stream immediately after configuration does not necessarily indicate failure. Generate traffic and allow time for delivery.
Flow-log records can show accepted and rejected traffic, IP addresses, and ports, making them useful for investigating network behavior.
Amazon S3 is an alternative destination when the data needs to be retained or analyzed with services such as Amazon Athena.
A successful flow-log configuration does not automatically mean that records will appear instantly. Check traffic generation, IAM permissions, destination settings, and elapsed time.

Tool / Feature Decision Guide

Requirement	Appropriate choice	Reason
Monitor traffic for an entire VPC	Create a flow log at the VPC level	Provides coverage for the VPC scope selected in the configuration
Focus collection on one subnet	Create a flow log at the subnet level	Limits the scope to that subnet
Focus on one network interface	Create a flow log at the network-interface level	Targets traffic associated with a specific ENI
View records through an operational logging service	CloudWatch Logs	The lab uses CloudWatch Logs for log-stream inspection and delivery
Store data for longer-term analysis with SQL queries	Amazon S3, optionally analyzed with Amazon Athena	The lesson identifies S3 and Athena as a useful analysis path
Validate a new flow log quickly	Generate traffic, then wait and inspect the destination	Records require traffic and may take time to appear

Common Traps / Misconceptions

Confusing the trust policy with the permissions policy: Allowing the service to assume a role is different from granting that role permission to create streams and put log events.
Expecting the log group or stream to appear immediately: The destination may initially report that it does not exist. Wait after traffic has been generated.
Testing without generating traffic: A flow log cannot display useful records if the selected resources have no traffic to capture.
Using the wrong VPC: The flow log and test EC2 instance must be associated with the same VPC for this lab workflow.
Assuming only accepted traffic is recorded: The example includes both accepted and rejected records because the filter is set to all traffic.
Forgetting the destination choice: CloudWatch Logs and S3 support different operational workflows. Choose based on whether immediate log inspection or storage and analysis is the priority.
Leaving lab resources active: Delete the flow log and terminate the temporary EC2 instance after testing.

Real-World Engineer / Analyst Notes

Start troubleshooting from the configuration boundary: confirm the flow-log scope, destination, filter, and IAM role.
If CloudWatch Logs is empty, verify that the service can assume the role and that the role includes the required CreateLogGroup, CreateLogStream, PutLogEvents, and describe actions.
Generate a known test event, such as outbound ping or curl traffic from an instance, rather than relying only on incidental background traffic.
Use the ENI identifier in the log-stream name to relate records back to the network interface under investigation.
Use CloudWatch Logs when operational visibility is the main need. Consider S3 when retaining a larger body of flow-log data and querying it with Athena is more useful.
Treat the aggregation interval as a visibility-latency consideration. The lab uses the maximum 10-minute interval, so records should not be expected to appear as an instantaneous packet-by-packet feed.

Quick Reference Summary

Purpose: Capture information about IP traffic to and from VPC network interfaces.
Scopes: VPC, subnet, or network interface.
Destinations: CloudWatch Logs or Amazon S3.
CloudWatch prerequisite: An IAM role trusted by vpc-flow-logs.amazonaws.com.
Required CloudWatch actions in the lab: CreateLogGroup, CreateLogStream, PutLogEvents, DescribeLogGroups, and DescribeLogStreams.
Validation method: Generate traffic, wait for delivery, and inspect the destination log stream.
Example traffic: ping google.com and curl amazon.com.
Example records: Accepted and rejected traffic, source and destination IP addresses, and ports.
Cleanup: Delete the flow log and terminate the test EC2 instance.

Flashcards

Q: A VPC Flow Log must deliver records to CloudWatch Logs. What IAM configuration is required?

A: Create a role whose trust policy allows vpc-flow-logs.amazonaws.com to assume it, then grant the role the required CloudWatch Logs actions.

Q: When would you create a subnet-level flow log instead of a VPC-level flow log?

A: Use a subnet-level flow log when the monitoring requirement is limited to one subnet rather than the entire VPC.

Q: What is the distinction between the flow-log role’s trust policy and permissions policy?

A: The trust policy permits the VPC Flow Logs service to assume the role. The permissions policy allows the assumed role to create log resources and publish events to CloudWatch Logs.

Q: A newly created CloudWatch destination has no log stream. What should you do before concluding that configuration failed?

A: Generate traffic in the selected VPC or interface and wait several minutes for aggregation and delivery before checking the destination again.

Q: Which destination is the better fit when the team wants to run SQL queries over retained flow-log data?

A: Amazon S3 is the better fit in the lesson’s example because the data can be analyzed with Amazon Athena.

Q: What three resource scopes can be used when creating VPC Flow Logs?

A: Flow logs can be created for a VPC, a subnet, or a network interface.

Q: A lab flow log is configured for all traffic. What types of outcomes can appear in the records?

A: The records can include both accepted and rejected traffic, along with source and destination addresses and ports.

Q: Why does the lab launch an EC2 instance before validating the flow log?

A: The instance provides a controlled source of traffic, such as ping and curl, so there is activity for the flow log to capture.

Q: What commands were used to create outbound test traffic in the instance?

A: ping google.com and curl amazon.com were used to generate traffic.

Q: What does an ENI ID in a CloudWatch log-stream name help an operator identify?

A: It helps associate the stream with the elastic network interface whose traffic is being logged.

Q: Which CloudWatch Logs actions are specifically included in the lab role’s permissions?

A: logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, logs:DescribeLogGroups, and logs:DescribeLogStreams.

Q: What is the operational tradeoff between CloudWatch Logs and Amazon S3 in this lesson?

A: CloudWatch Logs is convenient for viewing log streams operationally, while S3 is useful for longer-term retention and analysis with Athena.

Q: What is the trap when testing a flow log immediately after creation with no generated traffic?

A: An empty destination may simply mean there has been no relevant traffic yet or delivery is still pending; it is not by itself proof of an IAM or configuration failure.

Practice Questions

Question 1

An engineer creates a VPC Flow Log with CloudWatch Logs as the destination, but AWS does not allow the flow log to be created. The role has CloudWatch permissions, but its trust policy does not mention the VPC Flow Logs service. What is the most likely issue?

A. The EC2 instance lacks a key pair
B. The role cannot be assumed by VPC Flow Logs
C. The flow-log filter is set to all traffic
D. The aggregation interval is set to 10 minutes

Correct answer: B. The service must be allowed to assume the IAM role through the role’s trust policy. CloudWatch permissions alone are insufficient.

Question 2

A flow log is configured successfully for a VPC, but the CloudWatch destination initially says that no log group or stream exists. The engineer has not generated any traffic in the VPC. What is the best next step?

A. Delete the VPC and recreate it
B. Change the filter from all traffic to rejected traffic
C. Generate test traffic and wait for delivery
D. Add an EC2 key pair to the instance

Correct answer: C. The lesson’s validation process generates traffic with ping and curl, then waits several minutes for the records to appear.

Question 3

A security analyst needs to focus flow-log collection on traffic associated with one specific network interface rather than an entire VPC or subnet. Which scope should be selected?

A. VPC level
B. Subnet level
C. Network interface level
D. S3 level

Correct answer: C. VPC Flow Logs can be created at the network-interface level when the investigation is limited to a particular ENI.

Question 4

An organization wants to retain flow-log data in S3 and perform SQL-style analysis using Amazon Athena. Which destination best matches this requirement?

A. CloudWatch Logs only
B. Amazon S3
C. EC2 instance storage
D. The IAM role itself

Correct answer: B. The lesson identifies S3 as a suitable destination for retained data that can later be analyzed with Athena.

WordPress Metadata

Suggested Slug:
aws-vpc-flow-logs-configuration-iam-destinations

Meta Description:
Learn how to configure AWS VPC Flow Logs at the VPC, subnet, or network-interface level, deliver records to CloudWatch Logs or Amazon S3, and validate IAM and traffic-generation requirements.

Tags:
AWS, VPC, VPC Flow Logs, CloudWatch Logs, Amazon S3, IAM, EC2, Network Monitoring, Athena, SOA-C03