AWS Application Integration Services: SQS, SNS, Kinesis, and Step Functions – SAP-C02 Study Guide

Purpose of This Lesson

AWS application integration services connect application components while reducing direct dependencies between processing layers. The most important exam decisions involve choosing between queues, pub/sub notifications, event streaming, workflow orchestration, and managed message brokers.

For SAP-C02 scenarios, focus on the interaction model, delivery behavior, ordering requirements, replay needs, scaling model, and whether an existing messaging application must be migrated without extensive code changes.

Key Concepts

Amazon SQS: Decoupled Message Queuing

Amazon Simple Queue Service (SQS) stores messages until a consumer retrieves and processes them. Producers and consumers do not need to communicate synchronously or be available at the same time.

Typical uses include:

• Buffering work between application tiers
• Absorbing traffic spikes
• Decoupling producers from workers
• Distributing independent tasks across multiple consumers
• Triggering processing by Lambda, Amazon EC2, or container workloads

A consumer generally polls the queue, processes a message, and deletes it after successful processing. If processing fails or the message is not deleted before the visibility timeout expires, the message can become available again for another attempt.

SQS queue types include:

• Standard queues: Very high throughput and at-least-once delivery, but no strict ordering guarantee. Duplicate delivery is possible, so consumers should be idempotent.
• FIFO queues: Ordered processing and deduplication features, subject to FIFO throughput and message-group constraints.

SQS is a queue, not a broadcast mechanism. A message consumed and deleted by one worker is not automatically delivered to every worker. For broadcast delivery, use SNS, often with one SQS queue per subscriber.

Amazon SNS: Push-Based Publish/Subscribe

Amazon Simple Notification Service (SNS) uses a publisher/subscriber model. A publisher sends a message to an SNS topic, and SNS forwards it to subscribed endpoints such as:

• Amazon SQS queues
• AWS Lambda functions
• HTTP or HTTPS endpoints
• Email or SMS destinations
• Other supported AWS integrations

SNS is useful when one event must be delivered to multiple independent consumers. Unlike SQS, subscribers do not normally poll SNS for messages; SNS pushes notifications to them.

SNS does not function as a durable work queue for arbitrary offline consumers. For durable asynchronous processing, subscribe an SQS queue to the topic. This creates a fan-out pattern in which SNS distributes the event and each queue independently buffers and retries delivery.

Amazon Kinesis: Ordered, Replayable Data Streams

Amazon Kinesis is designed for collecting, processing, and analyzing streaming data. It is especially useful for telemetry, logs, clickstreams, IoT sensor data, and other continuously generated records.

Kinesis differs from SQS in several important ways:

• Consumers pull records from the stream.
• Multiple consumer applications can process the same stream independently.
• Related records can be routed to the same shard or partitioning unit.
• Kinesis Data Streams provides ordering within a shard, not global ordering across the entire stream.
• Records remain available for a retention period, allowing consumers to process or replay them later.
• Stream throughput must be provisioned or otherwise managed according to the selected Kinesis capacity mode.

Use Kinesis when the requirement is a durable stream with multiple consumers, ordered processing within a partition, and the ability to replay data. Use SQS when the requirement is distributing individual work items among workers.

AWS Step Functions: Workflow Orchestration

AWS Step Functions coordinates multiple application components into a stateful workflow. A state machine can invoke Lambda functions, AWS service integrations, and other tasks, while applying logic such as:

• Sequential execution
• Parallel branches
• Conditional choices
• Retries and backoff
• Error handling
• Waiting for callbacks or external completion
• Tracking workflow state

Step Functions is appropriate when the central problem is coordinating a business process rather than simply transporting messages. For example, an order workflow might validate payment, reserve inventory, arrange shipping, and notify the customer with explicit handling for failures at each stage.

Amazon SWF: Specialized and Legacy Workflow Use Cases

Amazon Simple Workflow Service (SWF) is an older workflow service. It can still be relevant when a workflow requires specialized execution logic or human-driven and externally controlled steps.

For most new AWS-native workflows, Step Functions is generally the preferred choice. SWF may appear in migration or compatibility scenarios where existing workflow workers and external processes are already designed around SWF concepts.

Amazon MQ: Managed Traditional Message Brokers

Amazon MQ provides managed brokers based on technologies such as Apache ActiveMQ and RabbitMQ. It is intended for applications that already depend on traditional broker protocols, APIs, or messaging semantics.

Choose Amazon MQ when:

• An existing application uses ActiveMQ or RabbitMQ.
• The application relies on industry-standard messaging protocols.
• Rewriting the application to use native SQS or SNS APIs would be costly or risky.
• Migration to AWS is required without changing the application’s messaging model immediately.

For a new cloud-native application, SQS and SNS are usually simpler operational choices. Amazon MQ does not automatically provide the same serverless scaling model as native AWS messaging services.

Architecture Decision Guide

Requirement	Best-fit service or pattern	Why
Buffer work between producers and workers	Amazon SQS	Durable asynchronous queue with consumer polling
Maximum throughput with no strict ordering requirement	SQS Standard	High scalability and at-least-once delivery
Ordered, deduplicated work items	SQS FIFO	FIFO semantics and message deduplication
Send one notification to many subscribers	Amazon SNS	Push-based publish/subscribe distribution
Broadcast events while allowing each consumer to retry independently	SNS to multiple SQS queues	Fan-out plus durable per-subscriber buffering
Continuous telemetry or event-stream processing	Kinesis Data Streams	Streaming ingestion, multiple consumers, and replay
Preserve order for related records	Kinesis with a consistent partition key	Ordering is maintained within a shard or partition
Coordinate multi-step processing with retries and branches	AWS Step Functions	Managed workflow orchestration and state tracking
Run an existing ActiveMQ or RabbitMQ application on AWS	Amazon MQ	Broker compatibility reduces application changes
Human or externally controlled workflow with specialized logic	Amazon SWF	Suitable for certain existing or specialized workflows

Exam-Relevant Takeaways

• Decoupling means application components can operate independently instead of requiring synchronous, direct communication.
• SQS is pull-based. Messages remain available until consumed, deleted, or expired according to queue and message settings.
• SNS is push-based. A topic forwards notifications to its subscribers.
• Kinesis consumers pull records and can independently process the same stream.
• SQS distributes work; Kinesis distributes a stream. This distinction is central to service-selection questions.
• SQS Standard does not guarantee ordering. Use FIFO when strict ordering and deduplication are required.
• Kinesis ordering is scoped to a shard or partition, not the whole stream.
• SNS plus SQS is the standard fan-out pattern when every subscriber needs durable, independently retriable delivery.
• Step Functions coordinates workflows; it is not simply a message queue.
• Amazon MQ is primarily a compatibility and migration choice for existing broker-based applications.
• Confirm current AWS quotas, throughput limits, retention periods, and regional feature availability in the AWS documentation when designing a production system.

Common Exam Traps

• Choosing SNS when subscribers need to process messages later or retry independently. Use SNS with SQS subscriptions instead.
• Choosing SQS when multiple consumers must each receive every event. A single queue distributes messages among workers; it does not broadcast them.
• Assuming SQS Standard preserves order. It does not provide strict ordering.
• Assuming FIFO means unlimited throughput. FIFO queues have throughput and message-group considerations.
• Treating Kinesis as interchangeable with SQS. Kinesis supports stream retention, multiple independent consumers, and replay-oriented workloads.
• Assuming Kinesis provides global ordering. Ordering is limited to records sharing the same shard or partition scope.
• Selecting Step Functions merely because Lambda functions are involved. Step Functions is justified by workflow coordination, state, branching, retries, or long-running process logic.
• Recommending Amazon MQ for a new application without a compatibility requirement. Native SQS and SNS generally reduce broker administration and integration complexity.
• Forgetting idempotency. At-least-once delivery and retries mean consumers should safely handle duplicate processing.

Real-World Engineer Notes

• Design SQS consumers to be idempotent. A common approach is storing a processed-event identifier in a durable datastore before applying a non-repeatable business action.
• Configure visibility timeout based on realistic processing time. If it is too short, another worker may receive the same message while the first worker is still processing it.
• Use dead-letter queues to isolate messages that repeatedly fail and to support operational investigation.
• For SNS-to-SQS fan-out, give each subscriber its own queue so consumer speed, retry behavior, and retention are independent.
• Select a Kinesis partition key carefully. A highly skewed key can concentrate traffic on one shard and create a hot partition.
• Use Step Functions for explicit business process state, but avoid turning every simple asynchronous handoff into a workflow. SQS or EventBridge may be more appropriate for simple event transport, depending on the broader design.
• Amazon MQ still requires broker-oriented operational planning, including connectivity, failover, storage, client compatibility, and capacity management.

Quick Reference Summary

• SQS: Pull-based queue for decoupled work distribution.
• SQS Standard: High throughput, at-least-once delivery, no strict ordering.
• SQS FIFO: Ordered and deduplicated message processing.
• SNS: Push-based topic and notification fan-out.
• SNS + SQS: Durable pub/sub with independent subscriber processing.
• Kinesis: Pull-based streaming data platform with retention, multiple consumers, and shard-level ordering.
• Step Functions: State-machine orchestration for multi-step workflows.
• SWF: Older workflow service for specialized or externally controlled workflows.
• Amazon MQ: Managed ActiveMQ or RabbitMQ compatibility for existing broker-based applications.

Flashcards

1. Q: What is the primary architectural benefit of SQS?

A: It decouples producers and consumers so they do not need synchronous communication.

1. Q: Does an SQS Standard queue guarantee message order?

A: No. Use SQS FIFO when strict ordering is required.

1. Q: What happens to an SQS message after successful processing?

A: The consumer should explicitly delete it from the queue.

1. Q: What delivery model does SNS use?

A: Push-based publish/subscribe delivery.

1. Q: Why combine SNS with SQS?

A: SNS provides fan-out, while each SQS queue provides durable buffering, independent retries, and separate consumer processing.

1. Q: How does Kinesis differ from SQS?

A: Kinesis is a retained data stream that supports multiple independent consumers and replay; SQS primarily distributes work items among consumers.

1. Q: Where does Kinesis preserve ordering?

A: Within a shard or partition, based on the stream’s partitioning model.

1. Q: When should Step Functions be selected?

A: When an application needs stateful orchestration, branching, retries, parallel steps, or coordination across multiple services.

1. Q: When is Amazon MQ preferred over SQS?

A: When migrating an existing ActiveMQ or RabbitMQ application that depends on compatible protocols and APIs.

1. Q: What is a key consumer design requirement for SQS?

A: Idempotent processing because duplicate delivery and retries can occur.

Practice Questions

Question 1

An order-processing system receives unpredictable bursts of orders. The web tier must remain responsive, and worker instances should process orders asynchronously. Each order should be processed by only one worker under normal conditions, and strict ordering is not required. Which service is the best fit?

A. Amazon SNS topic
B. Amazon SQS Standard queue
C. Kinesis Data Streams
D. AWS Step Functions

Correct answer: B. Amazon SQS Standard queue

SQS decouples the web tier from workers and buffers bursts. A Standard queue is appropriate when high throughput is needed and strict ordering is unnecessary. SNS is for fan-out, Kinesis is for retained streams, and Step Functions is for workflow orchestration.

Question 2

A company publishes an inventory-change event. Three independent applications must each receive every event, process it at their own speed, and retry failed processing without affecting the other applications. Which architecture is most appropriate?

A. One SQS queue consumed by all applications
B. SNS topic with one SQS queue subscribed for each application
C. One SQS FIFO queue with three consumers
D. One Lambda function subscribed directly to SNS

Correct answer: B. SNS topic with one SQS queue subscribed for each application

SNS provides fan-out, and separate SQS queues give each application durable, independently retriable delivery. A single SQS queue distributes messages among consumers rather than delivering every message to each consumer.

Question 3

IoT sensors continuously send telemetry. Two analytics applications need to process the same data independently, and a consumer must be able to replay records from earlier in the retention window. Which service is the best choice?

A. Amazon SQS Standard
B. Amazon SNS
C. Kinesis Data Streams
D. Amazon MQ

Correct answer: C. Kinesis Data Streams

Kinesis is designed for retained streaming data, multiple independent consumers, and replay. SQS is primarily a work queue, while SNS is a push notification service without the same stream-consumption model.

Question 4

An enterprise is migrating an application that currently uses RabbitMQ and relies heavily on its existing broker APIs. The business wants to minimize application code changes during the migration. Which service should the architect evaluate first?

A. Amazon SQS
B. Amazon SNS
C. Amazon MQ
D. AWS Step Functions

Correct answer: C. Amazon MQ

Amazon MQ supports managed broker technologies such as RabbitMQ and is intended for compatibility-focused migrations. Moving directly to SQS or SNS would likely require changing the application’s messaging integration.

Question 5

A payment workflow invokes several services in sequence, includes conditional approval logic, retries transient failures, and must retain the current state when a step fails. Which service best addresses the requirement?

A. Amazon SQS
B. Amazon SNS
C. AWS Step Functions
D. Kinesis Data Streams

Correct answer: C. AWS Step Functions

Step Functions provides state-machine orchestration, sequencing, branching, retries, and error handling. SQS, SNS, and Kinesis transport or distribute data but do not by themselves provide this workflow state management.