Study guide
Technical reference and lesson notes
Purpose of This Lesson
Amazon S3 replication and lifecycle rules solve different storage-management problems:
- Replication creates copies of objects in another S3 bucket, either in the same AWS Region or a different Region.
- Lifecycle management automates transitions between S3 storage classes and the expiration of objects or versions.
- Versioning preserves earlier object versions and is a prerequisite for S3 replication.
These capabilities frequently appear together in architecture and operations scenarios involving data protection, disaster recovery, retention, and storage-cost optimization.
Key Concepts
S3 Replication
S3 replication asynchronously copies objects from a source bucket to a destination bucket. The destination can be:
- In the same AWS Region: Same-Region Replication (SRR)
- In a different AWS Region: Cross-Region Replication (CRR)
Replication rules can apply to:
- Every object in the source bucket
- Objects matching a prefix, such as
files/ - Objects with specified tags
- Objects meeting configured object-size filters
Replication is configured on the source bucket, where the rule identifies the destination bucket and the IAM role that Amazon S3 can assume.
Versioning Is Required
Versioning must be enabled on both the source and destination buckets for S3 replication. Versioning also changes how updates and deletes behave:
- Uploading an object with an existing key creates a new version rather than replacing the previous data permanently.
- Deleting an object normally creates a delete marker instead of immediately removing all versions.
- Older versions remain available until they are explicitly deleted or removed by lifecycle rules.
Versioning protects against accidental overwrites and deletions, but it can increase storage consumption.
IAM Role for Replication
Amazon S3 assumes an IAM role to perform replication. The role requires a trust policy that allows the S3 service to call sts:AssumeRole.
The role’s permissions policy must allow the replication operations needed for the source and destination buckets. In a same-account configuration, the role generally needs permissions to read object data and version information from the source and write replicated objects and metadata to the destination.
The bucket ARNs must be scoped correctly:
- The bucket ARN identifies the bucket itself.
- The object ARN normally uses the form
arn:aws:s3:::bucket-name/*.
A policy that grants access only to the bucket ARN but omits the object ARN will not provide access to objects.
Existing Objects and New Objects
A replication rule normally handles objects created or changed after replication is configured. Existing objects are not automatically copied simply because a rule was added. To copy existing data, use an appropriate one-time S3 Batch Operations job or another supported migration process.
Replication is asynchronous, so an object may not appear in the destination immediately after upload. Applications requiring confirmation should monitor replication status or use an appropriate operational workflow rather than assuming instant consistency between buckets.
S3 Lifecycle Rules
Lifecycle rules automate actions based on object age, version state, prefixes, tags, and—where supported by the rule configuration—object-size filters.
Common actions include:
- Transition current object versions to another storage class
- Transition noncurrent versions to another storage class
- Permanently expire current object versions
- Permanently delete noncurrent versions after a retention period
- Remove expired delete markers
A lifecycle rule can apply to all objects or to a filtered subset. Filters are important when different data sets have different retention or storage-class requirements.
Current and Noncurrent Versions
For a versioned bucket, an object key can have one current version and zero or more noncurrent versions.
For example:
report.csvis uploaded, creating version A.- A revised
report.csvis uploaded, creating version B as the current version. - Version A becomes noncurrent.
- Deleting the object key creates a delete marker, while the underlying versions can remain stored.
Lifecycle rules can manage current and noncurrent versions independently. This distinction is essential when a requirement says to retain the latest object for a period but remove old versions sooner or move them to a cheaper storage class.
Storage-Class Transitions
Replication can optionally use a different storage class for the destination copy. Lifecycle rules can also transition objects to a lower-cost class after a defined number of days.
The correct choice depends on access frequency, retrieval requirements, durability and availability needs, minimum storage-duration charges, and operational requirements. A lower storage price does not necessarily mean lower total cost if data is frequently retrieved or deleted before the class’s minimum duration.
Exam-Relevant Takeaways
- Enable versioning on both buckets before configuring S3 replication.
- Replication may be same-Region or cross-Region; the configuration model is similar, but cross-account deployments require additional permissions and bucket-policy considerations.
- Amazon S3 assumes an IAM role to perform replication. Confirm both the trust policy and the permissions policy.
- Scope IAM resources correctly. Object operations generally require an object ARN such as
bucket-name/*. - Replication is asynchronous and should not be treated as an instantaneous copy operation.
- A newly created replication rule does not automatically copy historical objects unless an explicit batch operation or migration process is used.
- Lifecycle rules can target all objects or use prefixes, tags, and size filters.
- Versioned buckets require separate consideration for current versions, noncurrent versions, and delete markers.
- Deleting an object in a versioned bucket usually creates a delete marker; it does not necessarily remove the previous object versions.
- Lifecycle policies can reduce storage costs, but transitions and expiration must align with retention, retrieval, and minimum-storage-duration requirements.
Architecture Decision Guide
| Requirement | Suitable S3 capability | Important considerations |
|---|---|---|
| Keep a copy in another bucket in the same Region | Same-Region Replication | Enable versioning on both buckets; replication is asynchronous |
| Maintain a copy in another Region | Cross-Region Replication | Consider regional isolation, permissions, encryption, and destination design |
| Copy only a subset of objects | Replication rule with prefix, tag, or size filter | Ensure the filter matches the intended object population |
| Copy objects uploaded after a rule is created | Standard replication rule | Existing objects require a separate batch or migration operation |
| Move infrequently accessed data to a lower-cost class | Lifecycle transition | Evaluate access patterns, retrieval fees, and minimum storage duration |
| Retain recent data but remove older revisions | Noncurrent-version lifecycle rule | Define how long noncurrent versions and newer versions should be retained |
| Remove data after a retention period | Expiration lifecycle rule | In a versioned bucket, account for noncurrent versions and delete markers |
| Recover from accidental overwrites | S3 Versioning | Versioning increases storage use and does not replace backups or access controls |
Common Exam Traps
Assuming Replication Copies Existing Data Automatically
Adding a replication rule is not the same as migrating all historical objects. Check whether the scenario requires existing data to be copied. If it does, select S3 Batch Operations or another explicit data-copy process.
Enabling Versioning on Only the Source Bucket
Replication requires versioning on both the source and destination buckets. A destination bucket without versioning is not a valid target for the standard replication configuration.
Confusing a Delete Marker with Permanent Deletion
A delete request against the current version of an object in a versioned bucket commonly creates a delete marker. Older object versions may continue to incur storage charges.
Using Only the Bucket ARN in the IAM Policy
Bucket-level and object-level permissions use different resource ARNs. Actions that read or write object data need object resources, typically ending in /*.
Treating Replication as Synchronous
The object may be available in the source before it appears in the destination. Do not design a workflow that assumes the destination is immediately ready unless the application includes a suitable confirmation mechanism.
Applying Lifecycle Actions to the Wrong Version State
A rule for current versions does not automatically manage noncurrent versions. If old versions must be transitioned or deleted, configure the corresponding noncurrent-version action.
Choosing Storage Class Solely by Price per Gigabyte
Storage class selection must account for retrieval costs, access frequency, availability requirements, minimum storage durations, and the likelihood of early deletion.
Real-World Engineer Notes
- Use clear replication rule identifiers and document the source, destination, filters, and expected retention behavior.
- Test with representative object keys, tags, and sizes. A filter mistake can silently exclude data from replication or lifecycle processing.
- Monitor replication health and investigate objects that do not arrive within the expected operational window.
- Review replicated data and versioned data separately when estimating storage cost. Replication doubles or multiplies stored data, while versioning can retain multiple historical copies.
- If encryption is used, verify that the replication role and destination configuration can access the required encryption keys and write encrypted replicas.
- Define lifecycle rules carefully before applying them to production data. Expiration actions can be difficult or impossible to reverse once objects and versions are permanently deleted.
- For compliance retention, consider whether ordinary lifecycle expiration is appropriate or whether S3 Object Lock and a formal retention policy are required.
Quick Reference Summary
- SRR: Replicates objects to a bucket in the same Region.
- CRR: Replicates objects to a bucket in a different Region.
- Prerequisite: Versioning enabled on source and destination buckets.
- Permissions: S3 assumes an IAM role with a suitable trust policy and replication permissions.
- Historical data: Requires a separate batch or migration operation.
- Replication timing: Asynchronous; destination visibility is not immediate.
- Lifecycle filters: Can use all objects, prefixes, tags, and object-size conditions.
- Lifecycle targets: Current versions, noncurrent versions, and delete markers.
- Versioned delete: Usually creates a delete marker; old versions can remain.
- Cost control: Use lifecycle transitions and expiration, but account for retrieval and minimum-duration charges.
Flashcards
1. What is the difference between SRR and CRR?
Answer: Same-Region Replication copies objects to a bucket in the same AWS Region. Cross-Region Replication copies objects to a bucket in another Region.
2. What bucket configuration is required for standard S3 replication?
Answer: Versioning must be enabled on both the source and destination buckets.
3. Which principal performs S3 replication?
Answer: Amazon S3 assumes an IAM role configured with a trust policy allowing the S3 service to assume it.
4. Does adding a replication rule copy all existing objects?
Answer: Not automatically. Existing objects require a separate S3 Batch Operations job or another explicit copy process.
5. Is S3 replication synchronous?
Answer: No. Replication is asynchronous, so objects can take time to appear in the destination bucket.
6. What happens when an object is deleted from a versioned bucket?
Answer: A delete marker is generally created for the current version, while previous versions remain unless explicitly removed.
7. What is a noncurrent object version?
Answer: It is an older version of an object key that is no longer the current version because a newer version was uploaded or otherwise became current.
8. What can an S3 lifecycle rule do?
Answer: It can transition current or noncurrent versions between storage classes, expire current versions, delete noncurrent versions, and remove expired delete markers.
9. How can a lifecycle rule target only some objects?
Answer: Use filters such as a key prefix, object tags, or configured object-size conditions.
10. Why might an IAM replication policy need both a bucket ARN and an object ARN?
Answer: Bucket-level actions operate on the bucket resource, while object reads and writes operate on object resources, commonly represented by bucket-name/*.
Practice Questions
Question 1
A company configures S3 replication from source-bucket to destination-bucket. New objects are not appearing in the destination. Versioning is enabled on the source, but not on the destination. What is the most likely correction?
A. Enable MFA Delete on the source bucket
B. Enable versioning on the destination bucket
C. Add a lifecycle rule to the source bucket
D. Change the destination storage class to S3 Standard
Correct answer: B
Explanation: S3 replication requires versioning on both the source and destination buckets. MFA Delete, lifecycle rules, and the destination storage class do not resolve this prerequisite problem.
Question 2
An organization creates a replication rule today and expects the 50 TB of objects already in the source bucket to appear in the destination. What should the architect recommend?
A. Wait for the standard replication rule to process all historical objects
B. Disable versioning and recreate the rule
C. Use S3 Batch Operations or another explicit object-copy process for the existing data
D. Add a delete-marker replication rule
Correct answer: C
Explanation: A newly configured replication rule generally handles qualifying objects after configuration. Existing objects require a separate batch or migration operation.
Question 3
An IAM role used by S3 replication has permission on arn:aws:s3:::source-bucket but replication fails when S3 attempts to read object data. What is the most likely issue?
A. The role needs permission on the source objects, such as arn:aws:s3:::source-bucket/*
B. The role must be assumed by EC2 instead of S3
C. The destination must be in the same Availability Zone
D. The source bucket must use S3 One Zone-IA
Correct answer: A
Explanation: The bucket ARN alone does not grant object-level access. Replication needs permissions for the relevant object resources as well as any required bucket-level actions.
Question 4
A versioned bucket must retain the current version of every object for 90 days, move noncurrent versions to a cheaper storage class after 30 days, and delete noncurrent versions after one year. Which design best meets the requirement?
A. Configure only a current-version expiration action at 30 days
B. Configure separate lifecycle actions for current and noncurrent versions
C. Disable versioning after each upload
D. Use replication instead of lifecycle management
Correct answer: B
Explanation: Current and noncurrent versions are managed independently. The policy needs a noncurrent-version transition after 30 days and a noncurrent-version expiration after one year, while preserving current versions according to the stated retention requirement.
Question 5
An application uploads an object to an S3 source bucket and immediately reads from the replication destination, assuming the object is already present. Which design concern should the architect identify?
A. Replication is asynchronous and destination availability may lag
B. S3 replication works only for objects larger than 5 GB
C. Versioning prevents objects from being replicated
D. Lifecycle rules must be enabled before replication can start
Correct answer: A
Explanation: S3 replication is asynchronous. Applications that depend on the destination copy should use an appropriate replication-status or operational confirmation workflow rather than assuming immediate arrival.