Amazon S3 Event Notifications: SNS, SQS, Lambda, and Permissions – SAP-C02 Study Guide

Purpose of This Lesson

Amazon S3 event notifications allow a bucket to publish information about object-level events to another AWS service. Common destinations are:

• Amazon SNS topics
• Amazon SQS queues
• AWS Lambda functions

This pattern separates object storage from downstream processing. For example, an object upload can trigger an email notification through SNS or invoke a Lambda function that processes the new object.

Key Concepts

S3 event notifications

An event notification is configured on an S3 bucket. When a matching event occurs, S3 sends event data to the configured destination.

Typical object creation events include uploads performed through different S3 API operations, such as:

• PutObject
• PostObject
• CopyObject
• Completion of a multipart upload

The notification payload contains metadata that downstream services can use, including information such as:

• Event source
• AWS Region
• Event timestamp
• Event type or API operation
• Bucket name
• Object key
• Request and principal information
• Source IP address, where available

The object key is particularly important because it identifies the object that downstream processing should inspect.

Prefix and suffix filters

A notification can be restricted using optional filters:

• Prefix: Match keys beginning with a specified value, such as incoming/.
• Suffix: Match keys ending with a specified value, such as .jpg or .pdf.

These filters allow a bucket to route only relevant events. For example, a notification can invoke a Lambda function only when a PDF is uploaded beneath documents/.

An S3 prefix represents part of an object key. It does not require a separately defined folder because S3 uses a flat object namespace.

SNS as a notification destination

Amazon SNS can distribute S3 event messages to subscribers such as email endpoints or other AWS services. A typical workflow is:

1. Create an SNS topic.
2. Create a subscription, such as an email subscription.
3. Confirm the subscription from the message delivered by SNS.
4. Configure the S3 bucket to publish matching events to the topic.
5. Upload an object and verify that the event is delivered.

An email subscription is not active until it has been confirmed.

Destination permissions

S3 must be authorized to send messages to the destination. For an SNS destination, this is normally implemented through a resource-based access policy on the SNS topic.

The policy should allow the S3 service principal to call SNS:Publish against the specific topic. It should also restrict the request to the intended source bucket and, where appropriate, the owning AWS account.

Conceptually, the policy must establish:

• Principal: s3.amazonaws.com
• Action: SNS:Publish
• Resource: The target SNS topic ARN
• Condition: The source is the intended S3 bucket ARN, and optionally the expected account

If the topic policy does not grant this access, S3 may reject the event notification configuration with an error similar to “unable to validate the destination configurations.”

The same principle applies to other destinations: the destination must have a policy or configuration that allows S3 to deliver the event.

Exam-Relevant Takeaways

• S3 event notifications can send bucket event data to SNS, SQS, or Lambda.
• Configure notifications at the S3 bucket level.
• Use prefix and suffix filters to limit which object keys generate notifications.
• An “all object create events” configuration covers multiple creation mechanisms, not only a basic upload.
• S3 needs permission to publish to an SNS topic. Configure the SNS topic’s resource-based policy.
• A destination validation failure commonly indicates missing or incorrect destination permissions.
• For email delivery through SNS, the subscription must be confirmed before messages are received.
• The event payload identifies the bucket and object key, allowing downstream logic to locate and process the object.
• Scope permissions to the exact topic and source bucket rather than granting broad access.

Architecture Decision Guide

Requirement	Appropriate destination	Design consideration
Notify email recipients when an object is created	SNS topic with email subscription	Confirm the subscription and allow S3 to publish to the topic
Fan out an S3 event to multiple subscribers	SNS topic	Use SNS subscription types appropriate to the consumers
Trigger code to inspect or transform an object	Lambda function	Use the event payload to identify the bucket and object key
Deliver events to a queue for asynchronous consumption	SQS queue	A queue helps decouple producers and consumers and supports later processing
Process only images in a specific key path	Lambda, SNS, or SQS with prefix and suffix filters	Configure filters such as prefix incoming/ and suffix .jpg
Resolve destination validation errors	Review destination resource policy	Verify the service principal, action, resource ARN, and source-bucket condition

Common Exam Traps

• Assuming S3 automatically has permission to publish: The notification configuration and the destination policy are separate requirements.
• Confusing the bucket ARN with the topic ARN: The SNS policy resource is the SNS topic ARN; the bucket ARN is typically used in a source condition.
• Forgetting SNS email confirmation: An unconfirmed email subscription cannot receive the notification.
• Treating an S3 folder as a real directory: Prefix filters match object-key strings; they do not operate on a traditional filesystem hierarchy.
• Assuming “object created” means only PutObject: Creation events can result from uploads, POST requests, copies, and completed multipart uploads.
• Granting overly broad permissions: Prefer a policy limited to the required S3 service principal, SNS publish action, exact topic, and intended bucket.
• Ignoring filters: Without a prefix or suffix filter, every event matching the selected event category can generate a notification.

Real-World Engineer Notes

• Give notification configurations descriptive names, especially when a bucket has several rules.
• Test each notification path with a representative object and verify both the event and the downstream action.
• When debugging, check the event type, bucket Region, destination ARN, and resource policy before changing application code.
• Use suffix filters to prevent unrelated file types from invoking processing logic.
• Use prefix filters to separate workflows such as incoming/, processed/, and failed/.
• Keep the downstream consumer focused on the event payload and validate the bucket and key before processing. This reduces the risk of acting on an unexpected event.
• For production workflows, consider how duplicate or repeated delivery would affect processing and make consumers idempotent where necessary.

Quick Reference Summary

• Source: Amazon S3 bucket event
• Destinations: SNS, SQS, Lambda
• Common event category: Object creation
• Useful filters: Object-key prefix and suffix
• SNS requirement: Topic policy allowing S3 to publish
• Email requirement: SNS subscription confirmation
• Payload usefulness: Identifies the event, bucket, object key, and related metadata
• Typical use cases: Notifications, asynchronous processing, file transformation, and event-driven workflows

Flashcards

1. Q: What is an S3 event notification?

A: A bucket configuration that sends information about matching S3 events to SNS, SQS, or Lambda.

1. Q: Which AWS services can directly receive S3 event notifications?

A: Amazon SNS, Amazon SQS, and AWS Lambda.

1. Q: What does a prefix filter match?

A: The beginning of an S3 object key, such as incoming/.

1. Q: What does a suffix filter match?

A: The ending of an object key, such as .pdf.

1. Q: What permission does S3 generally need for an SNS destination?

A: Permission for the S3 service principal to call SNS:Publish on the target topic.

1. Q: Where is the SNS publish permission usually configured?

A: In the SNS topic’s resource-based access policy.

1. Q: What does an SNS email subscriber need to do before receiving notifications?

A: Confirm the subscription using the confirmation message sent by SNS.

1. Q: What commonly causes S3 to report that it cannot validate a destination configuration?

A: Missing, incorrect, or insufficient permissions on the destination.

1. Q: What information does a downstream processor commonly extract from an S3 event?

A: The bucket name and object key, along with event type and metadata.

1. Q: Does an S3 prefix correspond to a physical directory?

A: No. It is a string matched against the object key in S3’s flat namespace.

Practice Questions

Question 1

A company configures an S3 bucket to publish object creation events to an SNS topic. The configuration fails with an error stating that the destination cannot be validated. The topic exists, but its access policy has not been changed. What should the solutions architect do?

A. Enable versioning on the S3 bucket
B. Add an SNS topic policy allowing the S3 service principal to call SNS:Publish for that topic
C. Add an IAM policy allowing the bucket owner to publish to SNS
D. Enable S3 Transfer Acceleration

Correct answer: B

Explanation: S3 must be authorized by the SNS topic’s resource-based policy. The policy should allow s3.amazonaws.com to publish to the specific topic and should preferably restrict the source to the intended bucket.

Question 2

A media-processing Lambda function should run only when JPEG files are uploaded under the incoming/images/ key path. Which configuration meets the requirement?

A. Configure all S3 events with no filters
B. Configure an object-created notification with prefix incoming/images/ and suffix .jpg
C. Configure an SNS email subscription with the same filters
D. Configure an S3 lifecycle rule with a JPEG suffix

Correct answer: B

Explanation: S3 event notifications support prefix and suffix filtering. The object-created event can be limited to keys beginning with incoming/images/ and ending in .jpg.

Question 3

An organization wants to send an email whenever a new object is uploaded to an S3 bucket. Which sequence is required?

A. Create an SNS topic, subscribe an email endpoint, confirm the subscription, and configure S3 to publish to the topic
B. Create an SQS queue and configure the email endpoint as a queue consumer
C. Create a Lambda function and configure the email address as its execution role
D. Configure an S3 lifecycle rule to send email directly

Correct answer: A

Explanation: SNS supports email subscriptions. The recipient must confirm the subscription, and the SNS topic policy must permit S3 to publish before the S3 notification configuration can operate successfully.

Question 4

A downstream application receives an S3 object-created event and must process the uploaded file. Which information in the event is most directly needed to locate the file?

A. The SNS topic display name only
B. The S3 bucket name and object key
C. The AWS account alias only
D. The source IP address only

Correct answer: B

Explanation: The bucket name identifies the S3 container and the object key identifies the specific object. Together they allow the application to retrieve or process the uploaded object.

Question 5

A bucket receives uploads through both normal API uploads and multipart uploads. The architect wants notifications for every object creation. Which event selection is most appropriate?

A. Only PutObject events
B. Only PostObject events
C. All object-created events
D. Delete events only

Correct answer: C

Explanation: An all-object-create configuration covers the relevant creation mechanisms, including standard uploads, POST operations, copies, and completed multipart uploads.