Study guide
Technical reference and lesson notes
Purpose of This Lesson
AWS DataSync is a managed service for accelerating and automating data movement between on-premises storage, edge locations, and AWS storage services. It is particularly useful for recurring transfers, migrations, backups, and hybrid workflows where manually copying files would be slow or operationally complex.
The service uses a DataSync agent to connect AWS to supported storage systems outside AWS. DataSync then transfers data through a configured task to an AWS storage destination such as Amazon S3, Amazon EFS, or Amazon FSx.
Key Concepts
Supported source and destination types
DataSync can work with several storage protocols and AWS destinations:
- Network File System (NFS) shares
- Server Message Block (SMB) shares
- Hadoop Distributed File System (HDFS) clusters
- Self-managed object storage with an S3-compatible interface
- AWS Snowcone, using a DataSync agent on the device
- Amazon S3 buckets
- Amazon EFS file systems
- Amazon FSx for Windows File Server
- Amazon FSx for Lustre
- Amazon FSx for OpenZFS
- Amazon FSx for NetApp ONTAP
- Amazon S3 on Outposts
The exact supported combinations depend on the location types involved. For exam questions, first identify the source protocol and then select a compatible DataSync location and destination.
The DataSync agent
For storage outside AWS, DataSync commonly uses an agent that you deploy close to the source system. The agent can run as a virtual machine in an on-premises environment or be installed on a Snowcone device for edge transfers.
The agent provides connectivity between the external storage system and the DataSync service. It does not replace the source storage system; it enables DataSync to discover and read the data that must be transferred.
DataSync locations and tasks
A typical configuration contains:
- A source location, such as an NFS share, SMB share, HDFS cluster, or self-managed object store.
- A destination location, such as an S3 bucket, EFS file system, or FSx file system.
- A task, which defines how and when DataSync transfers the data.
Tasks can be run on demand or according to a schedule. This makes DataSync suitable for one-time migrations as well as continuous or periodic synchronization patterns.
Security and transport
Data transferred by DataSync is encrypted in transit using Transport Layer Security (TLS). Access still requires correct permissions and network connectivity to the source and destination.
Security design should address:
- Access permissions on the source file system or object store
- IAM permissions for the target AWS service
- Network routing between the agent, source storage, and AWS
- Security groups, firewalls, and required ports
- Encryption at rest on the destination where required by the workload
TLS protects data while it moves, but it does not automatically provide encryption at rest for the destination. Configure the target service’s encryption settings separately.
Online transfer versus Snowcone workflows
DataSync is primarily an online transfer service. It can move data over a network connection from on-premises environments into AWS.
Snowcone provides an edge storage and transport option for locations with limited connectivity. A DataSync agent can run on Snowcone, allowing data to be transferred between the device and AWS-supported storage services when connectivity is available. This is different from using Snowcone only as a manually shipped offline data device.
Exam-Relevant Takeaways
- Choose AWS DataSync when the requirement is to automate or accelerate file or object data movement between on-premises storage and AWS.
- DataSync supports recurring transfers through scheduled tasks, not just one-time migration jobs.
- A DataSync agent is used to connect DataSync to many external storage systems, including NFS, SMB, HDFS, and self-managed object storage.
- AWS destinations include Amazon S3, Amazon EFS, and Amazon FSx services.
- DataSync can work with Snowcone and S3 on Outposts.
- Data is encrypted in transit with TLS, but destination encryption at rest must be configured independently.
- DataSync is not a general database replication service. For database migration and ongoing replication, evaluate services such as AWS Database Migration Service.
- DataSync is not the same as AWS Transfer Family, which provides managed SFTP, FTPS, FTP, or AS2 endpoints for exchanging files with external clients.
- DataSync is also distinct from AWS Storage Gateway, which provides hybrid storage interfaces such as file, volume, or tape access rather than serving primarily as a high-speed migration and synchronization service.
Architecture Decision Guide
| Requirement | Strong candidate | Why |
|---|---|---|
| Move files from an on-premises NFS or SMB share to AWS | AWS DataSync | Supports common file protocols and managed transfer tasks |
| Repeatedly synchronize an on-premises file share with AWS storage | AWS DataSync | Tasks can be scheduled and automated |
| Transfer files into Amazon EFS | AWS DataSync | EFS is a supported file-system destination |
| Import data into Amazon FSx | AWS DataSync | Supports FSx for Windows File Server, Lustre, OpenZFS, and NetApp ONTAP |
| Move self-managed S3-compatible object data to AWS | AWS DataSync | Supports self-managed object storage as a location |
| Transfer data from an edge site using Snowcone | DataSync with a Snowcone agent | Supports edge collection and transfer workflows |
| Provide customers with SFTP access to upload files | AWS Transfer Family | Designed for managed file-transfer endpoints |
| Replicate relational database changes with minimal downtime | AWS Database Migration Service | Designed for database migration and change data capture |
| Cache on-premises access to cloud-backed storage | AWS Storage Gateway | Provides hybrid storage protocols and local caching |
| Move very large datasets when network connectivity is inadequate | Snow Family or another offline migration design | Physical shipment may be more practical than online transfer |
Common Exam Traps
- Confusing DataSync with Transfer Family: DataSync moves data between storage locations. Transfer Family exposes managed file-transfer protocols to users or external systems.
- Assuming DataSync only supports S3: DataSync also supports EFS, multiple FSx services, Snowcone, S3 on Outposts, and several on-premises storage types.
- Ignoring the agent: External NFS, SMB, HDFS, and object storage commonly require a DataSync agent with network access to the source.
- Treating TLS as encryption at rest: TLS protects data in transit. S3, EFS, and FSx encryption settings still need to be selected and authorized.
- Using DataSync for database replication: DataSync is intended for file and object data transfer, not transaction-log replication or database change capture.
- Overlooking connectivity: A scheduled task cannot transfer data if the agent cannot reach the source or if AWS networking and firewall rules are incomplete.
- Assuming all storage combinations behave identically: File metadata, permissions, ownership, and object semantics vary by source and destination. Validate task options and destination behavior for the specific migration.
Real-World Engineer Notes
- Place the agent close to the source data to reduce latency and avoid routing storage traffic unnecessarily through constrained links.
- Treat bandwidth as a shared production resource. Schedule transfers during suitable windows and use task bandwidth controls where appropriate.
- Plan the initial bulk copy separately from incremental synchronization. A recurring task can reduce the final cutover window, but it does not eliminate the need to validate application consistency.
- Confirm how file metadata, permissions, symbolic links, timestamps, and deletes should be handled before production migration.
- Monitor task execution, transferred volume, errors, and verification results. A successful network transfer does not by itself prove that the application is ready to cut over.
- Use IAM policies that grant only the required access to the destination bucket or file system.
- For S3 destinations, consider the target prefix, storage class, lifecycle policy, versioning, and encryption requirements before starting the transfer.
- For a constrained or disconnected location, compare DataSync over the available network with Snowcone or another Snow Family workflow. The correct answer depends on data volume, link capacity, transfer window, and operational constraints.
Quick Reference Summary
- Purpose: Managed, accelerated data transfer and synchronization.
- Source examples: NFS, SMB, HDFS, self-managed object storage, Snowcone.
- AWS targets: S3, EFS, FSx, and S3 on Outposts.
- Control model: Configure source and destination locations, then run a DataSync task.
- Automation: On-demand or scheduled transfers.
- Security: TLS in transit; configure encryption at rest separately.
- Best fit: File or object migration, recurring synchronization, backup, and hybrid data movement.
- Not the primary fit: Database replication, customer-facing SFTP, or generic hybrid caching.
Flashcards
- Q: What is AWS DataSync designed to do?
A: Accelerate and automate data movement between on-premises or edge storage and AWS storage services.
- Q: Which common file protocols does DataSync support?
A: NFS and SMB.
- Q: Can DataSync transfer HDFS data?
A: Yes. HDFS can be used as a supported storage location.
- Q: What component commonly connects DataSync to on-premises storage?
A: A DataSync agent deployed in the environment with access to the source storage.
- Q: Which AWS file services can be DataSync destinations?
A: Amazon EFS and supported Amazon FSx services, including FSx for Windows File Server, Lustre, OpenZFS, and NetApp ONTAP.
- Q: Can DataSync transfer data to Amazon S3?
A: Yes. S3 buckets are supported destinations.
- Q: Can DataSync be used for recurring synchronization?
A: Yes. DataSync tasks can be scheduled or run on demand.
- Q: How is DataSync traffic protected in transit?
A: DataSync uses TLS encryption in transit.
- Q: Does TLS automatically encrypt the destination data at rest?
A: No. Destination encryption must be configured through the target service.
- Q: How does Snowcone integrate with DataSync?
A: A DataSync agent can run on Snowcone, enabling data transfer between the device and AWS services.
- Q: Which service is generally better for managed SFTP endpoints?
A: AWS Transfer Family.
- Q: Which service is generally better for ongoing database change replication?
A: AWS Database Migration Service.
Practice Questions
Question 1
A company must migrate several terabytes of data from an on-premises SMB file share to Amazon S3. The company wants an automated solution that can perform an initial transfer and then run periodic incremental transfers. Which solution meets the requirement with the least operational effort?
A. Deploy AWS Transfer Family and configure an SFTP user
B. Deploy a DataSync agent, configure the SMB source and S3 destination, and schedule a DataSync task
C. Configure AWS Database Migration Service with the SMB share as the source
D. Deploy Storage Gateway Volume Gateway and manually export the files
Correct answer: B
Explanation: DataSync supports SMB sources, S3 destinations, and scheduled tasks for repeated transfers. Transfer Family is for exposing managed file-transfer endpoints, while DMS is intended for database migration.
Question 2
An organization needs to copy data from an on-premises NFS share to Amazon EFS. The security team requires encryption while the data crosses the network. Which service and security property should the architect select?
A. AWS DataSync with TLS-protected transfer
B. AWS Transfer Family with FTP
C. Amazon S3 Transfer Acceleration without a DataSync task
D. AWS Database Migration Service with TLS disabled because the source is private
Correct answer: A
Explanation: DataSync supports NFS-to-EFS transfers and encrypts data in transit using TLS. The architect must still configure appropriate encryption at rest for EFS if required.
Question 3
A field location has a Snowcone device containing data. The location has intermittent connectivity, and the organization wants to use DataSync to move the data to AWS when connectivity is available. What should the architect do?
A. Install a DataSync agent on the Snowcone and configure a transfer task
B. Install AWS Storage Gateway Volume Gateway on an S3 bucket
C. Use AWS Transfer Family to expose Snowcone as an SFTP server
D. Configure DMS to replicate the Snowcone file system
Correct answer: A
Explanation: DataSync supports Snowcone through an agent running on the device. This enables managed transfers to supported AWS storage services when the network is available.
Question 4
A company needs to provide external partners with secure file uploads using SFTP. The files will ultimately be stored in an S3 bucket. Which AWS service is the most direct choice?
A. AWS DataSync
B. AWS Transfer Family
C. Amazon FSx for Lustre
D. AWS Migration Hub
Correct answer: B
Explanation: Transfer Family provides managed SFTP and other file-transfer endpoints backed by AWS storage. DataSync is intended to move data between storage locations, not to provide customer-facing SFTP access.
Question 5
A solutions architect proposes AWS DataSync to replicate transaction changes from an on-premises PostgreSQL database to Amazon Aurora with minimal downtime. What is the primary issue with this design?
A. DataSync cannot use TLS
B. DataSync supports only S3 destinations
C. DataSync is designed for file and object transfers, not database change-data capture
D. DataSync cannot run scheduled tasks
Correct answer: C
Explanation: DataSync is not a database replication service. AWS Database Migration Service is the more appropriate service for database migration and ongoing change-data capture scenarios.