AWS Serverless Services and Event-Driven Architecture – SAP-C02 Study Guide

Purpose of This Lesson

Serverless architecture allows teams to build applications without provisioning or managing servers, operating systems, or application runtimes. AWS manages the underlying infrastructure, capacity, patching, availability, and much of the scaling. The application team focuses on code, service configuration, permissions, and business logic.

This lesson focuses on how serverless services can be connected using event-driven architecture, where an event in one AWS service initiates processing in another service.

Key Concepts

What Serverless Means in AWS

Serverless does not mean that servers do not exist. It means that AWS manages the servers and exposes an abstraction that reduces or eliminates infrastructure administration.

Typical serverless characteristics include:

• No customer-managed EC2 instances or operating systems.
• Automatic capacity provisioning and scaling.
• Built-in high availability at the service level, where supported.
• Usage-based pricing for many services.
• Reduced responsibility for patching and infrastructure maintenance.
• Configuration and application code remain the customer’s responsibility.

Examples include AWS Lambda, Amazon S3, Amazon DynamoDB, Amazon SQS, Amazon SNS, and Amazon EventBridge. These services have different invocation, delivery, scaling, and durability models, so they should not be treated as interchangeable.

Event-Driven Architecture

In an event-driven architecture, a change or action produces an event. That event is delivered to a consumer, which performs the next operation. The producer and consumer can be loosely coupled and can often scale independently.

A representative workflow is:

1. A user uploads an object through a static website hosted on Amazon S3.
2. Amazon S3 emits an object-created event.
3. An S3 event notification invokes an AWS Lambda function or sends the event to Amazon SQS or Amazon SNS.
4. Lambda processes the object, such as validating, transforming, or extracting metadata.
5. The function stores output in another S3 bucket or publishes a message to an SQS queue.
6. A downstream consumer processes the queue message and writes application data to Amazon DynamoDB.
7. An SNS topic can distribute notifications to subscribers such as email endpoints, Lambda functions, SQS queues, or HTTP/S endpoints.

The user experiences one upload operation, while the rest of the workflow proceeds asynchronously.

Amazon S3 Event Notifications

Amazon S3 can generate event notifications for actions such as object creation and deletion. Notifications can be filtered by object key prefix and suffix, which is useful for routing only relevant files to a processor.

Supported notification destinations commonly used in this pattern include:

• AWS Lambda
• Amazon SQS
• Amazon SNS

Important implementation considerations:

• The destination must have a resource policy or permissions allowing Amazon S3 to publish or invoke it.
• S3 event notifications are designed for asynchronous processing and can result in duplicate deliveries.
• Consumers should be idempotent.
• If a Lambda function writes back to the same bucket that triggers it, use a different prefix or bucket for output to avoid recursive invocations.
• For more advanced event routing, filtering, fan-out, and integration across many AWS services, Amazon EventBridge may be a better choice.

AWS Lambda in the Workflow

Lambda runs code in response to events without requiring the customer to manage servers. It can be invoked directly by services such as S3 or asynchronously through an event source such as SQS.

For SQS integration, Lambda uses an event source mapping to poll the queue, retrieve messages in batches, and invoke the function. SQS does not directly push a request to Lambda in the same way that a direct service invocation does.

Lambda design concerns include:

• Execution duration and memory configuration.
• Concurrency and downstream service limits.
• Retry behavior and failure handling.
• IAM permissions for reading source data and writing results.
• VPC networking requirements when accessing private resources.
• Idempotency when events or messages are delivered more than once.

Amazon SQS for Decoupling

Amazon SQS provides durable asynchronous messaging between producers and consumers. A queue absorbs traffic spikes, allows consumers to process work independently, and prevents a temporary downstream failure from immediately failing the producer.

With SQS, the consumer normally polls for messages. When Lambda is used as the consumer, the Lambda event source mapping performs the polling and invokes the function.

Key operational details include:

• Messages remain available until successfully processed or retained until their retention period expires.
• A visibility timeout temporarily hides a message while it is being processed.
• If processing fails or the visibility timeout expires, the message can become visible again.
• A dead-letter queue can isolate messages that repeatedly fail.
• Visibility timeout, batch size, reserved concurrency, and downstream capacity must be designed together.
• Standard queues provide at-least-once delivery and may deliver messages out of order.
• FIFO queues support ordering and deduplication features, subject to FIFO throughput and grouping constraints.

Amazon SNS for Fan-Out and Notifications

Amazon SNS is a publish-subscribe service. A producer publishes to a topic, and SNS delivers the message to multiple subscribed endpoints.

SNS is useful when one event must reach several independent consumers. For example, a single processing result could be sent to a Lambda function, an SQS queue, and an HTTP/S endpoint. Email is appropriate for human notifications, but it is generally not a substitute for a durable application-processing queue.

SNS and SQS are frequently combined:

• SNS provides fan-out.
• Separate SQS queues provide independent buffering and retry behavior for each consumer.
• Consumers can process messages at different rates without blocking one another.

Amazon DynamoDB as a Serverless Data Store

DynamoDB provides a managed NoSQL database that can be used by Lambda to store processing results, metadata, or application state. The design must still address partition-key selection, capacity mode, throttling, conditional writes, and idempotency.

A common pattern is to use an object key or event identifier as an idempotency key and perform a conditional write so that duplicate processing does not create incorrect results.

Architecture Decision Guide

Requirement	Suitable AWS choice	Important decision point
Run code in response to an S3 object event	S3 event notification to Lambda	Configure invocation permissions and make the function idempotent
Buffer work during traffic spikes	Amazon SQS	Tune visibility timeout, retention, batch size, and dead-letter handling
Distribute one event to multiple consumers	Amazon SNS, often with SQS subscriptions	Give each consumer its own queue when independent buffering is required
Route events using advanced filters across AWS services	Amazon EventBridge	Prefer when centralized event buses, rules, schemas, or cross-account routing are needed
Store processed object metadata or results	Amazon DynamoDB	Design keys and write conditions for scale and duplicate protection
Host a static front end	Amazon S3, often behind CloudFront	Do not place application server logic in the static hosting layer
Process a file asynchronously	S3 to SQS or Lambda, depending on workload	Use SQS when buffering, controlled concurrency, or durable retry isolation is important

Exam-Relevant Takeaways

• Serverless shifts infrastructure operations to AWS but does not remove the need for application, IAM, data, and observability design.
• Amazon S3 event notifications can send events to Lambda, SQS, or SNS.
• SQS is primarily a queue for decoupling and buffering; consumers pull messages, directly or through Lambda event source mapping.
• SNS is primarily for publish-subscribe fan-out and notifications.
• Use SNS with separate SQS queues when multiple consumers need independent retry and buffering behavior.
• Event-driven workflows are generally asynchronous, so design for eventual consistency and delayed processing.
• At-least-once delivery means duplicate processing is possible. Idempotent consumers are essential.
• A Lambda function triggered by S3 should avoid writing triggering objects back into the same unfiltered location.
• A dead-letter queue is a failure-isolation mechanism, not a replacement for monitoring and remediation.
• Permissions are part of the architecture: event sources need permission to invoke functions or publish to destinations, and functions need IAM permissions to access downstream services.

Common Exam Traps

• Assuming SQS pushes directly to Lambda: Lambda polls SQS through an event source mapping. This affects concurrency, batching, visibility timeout, and failure behavior.
• Treating SNS as a durable work queue: SNS distributes messages, but SQS is the better choice for durable consumer-specific buffering and retry control.
• Assuming exactly-once processing: Standard SQS and many event integrations can produce duplicate deliveries. Build idempotency into the consumer.
• Ignoring recursive S3 triggers: A function that reads from and writes to the same triggering prefix can repeatedly invoke itself.
• Forgetting destination permissions: A correctly configured event source still fails if the target resource policy or Lambda permission is missing.
• Using synchronous processing for every upload: Asynchronous processing with SQS is often more resilient when workloads are bursty or downstream systems have limited capacity.
• Confusing serverless with no configuration: Serverless still requires capacity modes, concurrency controls, timeouts, retries, encryption, IAM, logging, and alarms.
• Using email as the processing mechanism: Email notifications are suited to human awareness. Machine processing should normally use Lambda, SQS, SNS subscriptions, or EventBridge.

Real-World Engineer Notes

• Define an event contract containing the source, event type, object key, version or ETag, timestamp, and correlation ID.
• Store uploaded objects under an input prefix and write transformed results under a separate output prefix or bucket.
• For large files, pass the S3 bucket and object key through the event rather than copying the entire object into a message.
• Configure Lambda reserved concurrency when a downstream database or external API cannot tolerate unlimited parallel processing.
• Set the SQS visibility timeout longer than the maximum expected Lambda processing time, with margin for retries. Align the Lambda timeout and batch behavior accordingly.
• Use a dead-letter queue or failure destination and alert on its depth. Reprocessing should be deliberate and safe.
• Apply least-privilege IAM policies to Lambda execution roles, S3 notification permissions, SNS topics, and SQS queues.
• Use encryption where required: S3 encryption, SQS/SNS server-side encryption, and customer-managed KMS keys when key-control requirements justify the added complexity.
• Monitor end-to-end latency, Lambda errors and throttles, SQS age of oldest message, queue depth, and DynamoDB throttling.
• Consider Step Functions when the workflow needs explicit orchestration, branching, long-running state, human approval, or clearly visible retry steps.

Quick Reference Summary

• Serverless: AWS manages the infrastructure; you manage configuration, code, permissions, and data design.
• S3 event notification: Emits object events to Lambda, SQS, or SNS.
• Lambda: Executes code in response to events and scales according to the invocation model and concurrency controls.
• SQS: Decouples producers and consumers, buffers bursts, and supports durable asynchronous processing.
• SNS: Publishes one message to multiple subscribers for fan-out and notifications.
• DynamoDB: Managed NoSQL storage for application state and processing results.
• EventBridge: Advanced event routing and integration for broader event-driven architectures.
• Core reliability rule: Assume retries and duplicates; make consumers idempotent.

Flashcards

1. Q: What does serverless mean in AWS?

A: AWS manages the underlying servers, operating systems, capacity, and much of the availability and patching; the customer manages application configuration, permissions, code, and data.

1. Q: Which destinations can Amazon S3 event notifications commonly target?

A: AWS Lambda, Amazon SQS, and Amazon SNS.

1. Q: What is the main purpose of event-driven architecture?

A: To have an event in one component initiate processing in another component, usually with loose coupling and asynchronous execution.

1. Q: Why is Amazon SQS used in a serverless workflow?

A: It decouples producers and consumers, buffers bursts, and allows work to be retried independently.

1. Q: How does Lambda consume messages from SQS?

A: A Lambda event source mapping polls SQS, retrieves messages in batches, and invokes the function.

1. Q: What is the primary use of SNS?

A: Publish-subscribe fan-out, where one published message is delivered to multiple subscribers.

1. Q: Why combine SNS with SQS queues?

A: SNS distributes the event, while separate SQS queues provide each consumer with independent buffering, retry, and processing speed.

1. Q: What delivery assumption should consumers make for standard SQS?

A: At-least-once delivery; duplicate messages and out-of-order delivery are possible.

1. Q: How can an S3-triggered Lambda avoid recursive invocation?

A: Use separate input and output buckets or configure different prefixes and filter the event notification.

1. Q: What is the purpose of an SQS dead-letter queue?

A: To isolate messages that fail repeatedly so they can be inspected and remediated separately.

1. Q: What must be configured for S3 to invoke a Lambda function?

A: The S3 event notification and a Lambda resource-based permission allowing the S3 bucket to invoke the function.

1. Q: What is a key property of a reliable event consumer?

A: Idempotency, so processing the same event more than once does not corrupt state or create unintended duplicate effects.

Practice Questions

Question 1

A company uploads images to an Amazon S3 bucket. Image processing can take several minutes, and uploads may arrive in large bursts. The processing Lambda function writes metadata to DynamoDB. The company wants to prevent lost work and limit the processing rate when DynamoDB is throttled. Which architecture is most appropriate?

A. Configure S3 to invoke Lambda synchronously and write directly to DynamoDB.
B. Configure S3 to publish to an SQS queue, and configure Lambda with an event source mapping to consume the queue.
C. Configure S3 to send an email notification for each object and process the email with Lambda.
D. Configure S3 to publish to an SNS topic with no subscriber and have Lambda poll the topic.

Correct answer: B

SQS provides durable buffering and decouples the upload rate from the processing rate. Lambda can consume messages in batches, while concurrency and visibility timeout can be tuned for DynamoDB capacity. The other options either lack durable work buffering or use an invalid consumption model.

Question 2

A Lambda function is triggered when an object is created in an S3 bucket. The function creates a resized version and stores it in the same bucket. The function begins invoking itself repeatedly. What is the best solution?

A. Increase the Lambda memory allocation.
B. Disable S3 versioning.
C. Store output in a separate bucket or use a non-overlapping output prefix with event filtering.
D. Replace Lambda with an EC2 instance.

Correct answer: C

The function is processing its own output because the output also matches the triggering event. Separate input and output locations or filtered prefixes prevent recursive invocation.

Question 3

An application must send each order event to three independent processing systems. Each system has a different processing rate and must be able to retry failed messages without affecting the other systems. Which design best meets the requirement?

A. Publish directly to one SQS queue and have all consumers compete for messages.
B. Publish to an SNS topic with a separate SQS queue subscribed for each processing system.
C. Send one email notification to all processing systems.
D. Invoke all three Lambda functions synchronously from the producer.

Correct answer: B

SNS provides fan-out, while separate SQS queues isolate each consumer’s backlog, retry behavior, and processing rate. A single queue would distribute messages among consumers rather than deliver every event to all three systems.

Question 4

A standard SQS message causes a Lambda function to update a DynamoDB item. Occasionally, the same message is delivered twice, resulting in duplicate side effects. Which change is most important?

A. Assume SQS guarantees exactly-once delivery.
B. Make the processing idempotent by using a unique event identifier and conditional writes.
C. Remove the queue and use SNS email delivery.
D. Increase the SQS message retention period.

Correct answer: B

Standard SQS uses at-least-once delivery, so duplicate processing is possible. Idempotency keys and DynamoDB conditional writes can ensure that repeated delivery does not apply the operation incorrectly.

Question 5

A team needs to route events from multiple AWS services using content-based rules, maintain a centralized event bus, and support integration across accounts. Which service is the best fit?

A. Amazon EventBridge
B. Amazon S3 static website hosting
C. Amazon DynamoDB
D. Amazon EBS

Correct answer: A

EventBridge is designed for event buses, rule-based event filtering and routing, service integrations, and cross-account event architectures. S3 notifications, SNS, and SQS can support simpler patterns but do not provide the same centralized event-routing model.