Study guide

Technical reference and lesson notes

Purpose of This Lesson

This hands-on lesson demonstrates how to manage Amazon S3 buckets and objects from AWS CloudShell using the AWS CLI. It also shows how to upload an object programmatically with Python and the Boto3 SDK.

The central skill is recognizing the source and destination in each command: local files and folders are referenced with normal filesystem paths, while S3 locations use the s3://bucket-name URI format.

Key Concepts

S3 bucket: The container that stores objects. Bucket names must be globally unique.
S3 object: A file stored in a bucket. The object key identifies its location within the bucket.
AWS CloudShell: A browser-based shell environment suitable for running AWS CLI commands and creating temporary test files.
S3 URI: S3 locations are specified as s3://bucket-name, optionally followed by an object key or prefix.
Source and destination: For copy operations, the first location is the source and the second is the destination.
Recursive operations: The --recursive option allows commands to process all files under a local directory or all objects under an S3 location.
Overwrite behavior: Uploading an object with the same name replaces the existing object. S3 Versioning can preserve older versions when it is enabled.
CLI versus SDK: The AWS CLI is convenient for interactive administration and testing; Boto3 enables S3 operations to be embedded in Python automation.

S3 Object Operations in AWS CloudShell

Creating and inspecting a bucket

Create a bucket with aws s3 mb and an S3 URI:

aws s3 mb s3://my-bucket-unique-name

The bucket name must be changed to a globally unique value. The lecture creates the bucket in the active AWS Region, such as us-east-1.

List buckets in the account with:

aws s3 ls

To list the objects in one bucket, include the bucket URI:

aws s3 ls s3://my-bucket-unique-name

The first command lists buckets; the second lists the contents of a specified bucket. Confusing these two scopes is a common source of errors when validating an operation.

Uploading one file

Create a local test file in CloudShell:

touch local-file.txt

Upload it with aws s3 cp:

aws s3 cp local-file.txt s3://my-bucket-unique-name

The local file is the source, and the bucket is the destination. Listing the bucket afterward confirms that the object exists.

Uploading a directory

Create a directory and sample files:

mkdir my-folder
cd my-folder
touch test-file-1.txt test-file-2.txt test-file-3.txt
cd ..

Upload the directory contents recursively:

aws s3 cp my-folder s3://my-bucket-unique-name --recursive

Without --recursive, the command does not process the directory contents as a group. The option is important whenever the operation targets multiple files below a directory or prefix.

Downloading one object

Copy an object from S3 to the local filesystem. The destination filename can be different from the object’s current name:

aws s3 cp s3://my-bucket-unique-name/local-file.txt downloaded.txt

Here, the S3 object is the source and downloaded.txt is the local destination.

Downloading all objects to a local directory

Create a destination directory and copy the bucket contents recursively:

mkdir local-folder
aws s3 cp s3://my-bucket-unique-name local-folder --recursive

The direction of the copy is reversed compared with an upload: the S3 URI comes first and the local directory comes second.

Updating an object

To update an existing object, upload the revised local file again using the same object name:

aws s3 cp local-file.txt s3://my-bucket-unique-name

The existing object is overwritten. If older copies must be retained, S3 Versioning is the relevant feature; the basic copy command alone does not preserve the previous content.

Deleting objects and buckets

Delete one object with aws s3 rm:

aws s3 rm s3://my-bucket-unique-name/test-file-1.txt

Delete all objects under a bucket recursively:

aws s3 rm s3://my-bucket-unique-name --recursive

After the bucket is empty, remove the bucket itself with aws s3 rb:

aws s3 rb s3://my-bucket-unique-name

rb means remove bucket, while rm removes objects. The bucket must be empty before the bucket-removal command succeeds.

Programmatic Uploads with Python and Boto3

The same type of S3 upload can be automated with Python. The lecture uses Boto3, the AWS SDK for Python, and creates an S3 resource:

import boto3

s3 = boto3.resource("s3")

bucket_name = "my-bucket-unique-name"
object_name = "python-file.txt"
file_path = "python-file.txt"

s3.Bucket(bucket_name).upload_file(file_path, object_name)
print("Successfully uploaded")

The bucket name, object name, and local file path can be defined as variables, making it possible to adapt the script for dynamic workflows. The execution environment must have the required Boto3 package and AWS credentials or an attached identity with permission to access S3. CloudShell commonly already includes much of the required tooling, but scripts should still be tested in their actual runtime environment.

Exam- or Assessment-Relevant Takeaways

Select aws s3 mb to create a bucket and aws s3 rb to remove a bucket.
Use aws s3 ls alone to list buckets; add an S3 URI to list objects in a particular bucket.
Use aws s3 cp for both uploads and downloads. Determine the direction from the order of source and destination arguments.
Add --recursive when copying a directory or processing multiple objects under a bucket or prefix.
aws s3 rm deletes objects, while aws s3 rb deletes the bucket after its contents have been removed.
Re-uploading an object with the same name overwrites it. Versioning is the feature to consider when previous versions must be preserved.
Use the AWS CLI for direct shell operations and Boto3 when S3 behavior must be incorporated into Python code.
Bucket names must be globally unique, so a command using a placeholder bucket name must be adapted before execution.

Tool / Feature Decision Guide

Requirement	Appropriate choice	Decisive detail
Create a bucket from CloudShell	`aws s3 mb`	Uses an S3 URI and requires a unique bucket name.
Inspect all buckets	`aws s3 ls`	No bucket URI is supplied.
Inspect one bucket’s objects	`aws s3 ls s3://bucket-name`	The bucket URI scopes the listing.
Upload one local file	`aws s3 cp local-file s3://bucket-name`	Local path is the source.
Upload a directory	`aws s3 cp directory s3://bucket-name --recursive`	Recursive processing includes files below the directory.
Download one object	`aws s3 cp s3://bucket-name/object local-file`	S3 URI is the source.
Download multiple objects	`aws s3 cp s3://bucket-name local-directory --recursive`	Recursive processing copies the bucket contents to the local destination.
Remove one object	`aws s3 rm s3://bucket-name/object`	Deletes the specified object.
Empty a bucket	`aws s3 rm s3://bucket-name --recursive`	Deletes objects below the bucket location.
Remove an empty bucket	`aws s3 rb s3://bucket-name`	Bucket removal is distinct from object removal.
Automate an upload in Python	Boto3 `upload_file`	Embeds the S3 operation in application or automation code.
Preserve older copies during replacement	S3 Versioning	A normal re-upload overwrites the object.

Common Traps / Misconceptions

Using a plain bucket name instead of an S3 URI: S3 CLI examples use s3://bucket-name, not just the bucket name.
Forgetting global uniqueness: A bucket name that is locally unique may still conflict with a bucket owned by another AWS account.
Reversing cp arguments: The first argument is always the source and the second is the destination. Uploads and downloads therefore look reversed.
Omitting --recursive: A directory or collection of objects requires recursive processing when copying as a group.
Using rm to remove a bucket: rm removes objects. rb removes the bucket, and only after the bucket is empty.
Assuming re-uploading creates a historical copy: Re-uploading the same object name overwrites the object unless versioning is being used to retain versions.
Validating the wrong scope: aws s3 ls confirms bucket visibility, not necessarily that a particular object exists. List the bucket URI to inspect its contents.
Treating CLI and SDK syntax as interchangeable: The CLI uses commands such as aws s3 cp; Python uses Boto3 methods and variables.
Deleting test data casually: Recursive deletion can remove every object under the specified location. Verify the URI before running it.

Real-World Engineer / Analyst Notes

Use CloudShell for quick, authenticated experiments without setting up a local AWS CLI environment.
During troubleshooting, validate each stage: confirm the bucket exists, list the target bucket, perform the copy, and list the bucket again.
Keep source and destination explicit in scripts. Variables for the bucket name, object key, and local path make automation easier to review and modify.
Test destructive commands against a clearly identified development bucket. aws s3 rm ... --recursive is broad by design.
Replacing an object is operationally different from preserving its history. Decide whether the workflow needs overwrite behavior or version retention before automating updates.
For repeatable applications, use Boto3 rather than constructing shell commands inside Python. The lecture’s Boto3 example illustrates the direct SDK upload path.

Quick Reference Summary

# Create and list
aws s3 mb s3://bucket-name
aws s3 ls
aws s3 ls s3://bucket-name

# Upload
aws s3 cp local-file.txt s3://bucket-name
aws s3 cp local-folder s3://bucket-name --recursive

# Download
aws s3 cp s3://bucket-name/object.txt local-file.txt
aws s3 cp s3://bucket-name local-folder --recursive

# Delete
aws s3 rm s3://bucket-name/object.txt
aws s3 rm s3://bucket-name --recursive
aws s3 rb s3://bucket-name

Remember: cp transfers data, ls lists resources, rm removes objects, rb removes a bucket, and --recursive expands an operation to multiple files or objects.

Flashcards

Q: You need to create an S3 bucket from CloudShell. Which command pattern should you use, and what must be true of the bucket name?

A: Use aws s3 mb s3://bucket-name. The bucket name must be globally unique.

Q: What is the difference between aws s3 ls and aws s3 ls s3://bucket-name?

A: The first lists buckets visible in the account; the second lists objects in the specified bucket.

Q: You need to upload one local file to S3. How do you determine the argument order for aws s3 cp?

A: Put the local file first as the source and the S3 URI second as the destination, such as aws s3 cp file.txt s3://bucket-name.

Q: When should --recursive be added to an S3 copy command?

A: Use it when copying a directory or multiple objects under an S3 location. It causes the command to process the contents below the specified path.

Q: A user wants to download an S3 object under a different local filename. Which operation supports this?

A: Use aws s3 cp with the S3 object as the source and the desired local filename as the destination, such as aws s3 cp s3://bucket/object.txt downloaded.txt.

Q: A directory must receive all objects from a bucket. What source, destination, and option should the command use?

A: The bucket URI is the source, the local directory is the destination, and --recursive is required: aws s3 cp s3://bucket-name local-folder --recursive.

Q: What happens when a revised local file is uploaded using the same S3 object name?

A: The existing object is overwritten. S3 Versioning is the relevant feature when older copies must be preserved.

Q: Which command removes one S3 object, and which command removes the bucket itself?

A: aws s3 rm removes an object; aws s3 rb removes the bucket. The bucket must be empty before rb succeeds.

Q: How can you empty a bucket using the AWS CLI?

A: Run aws s3 rm s3://bucket-name --recursive, after verifying that the URI identifies the intended bucket.

Q: You need S3 functionality inside a Python automation program instead of an interactive shell. Which tool should you select?

A: Use Boto3, the AWS SDK for Python. The example creates an S3 resource and calls upload_file.

Q: In a copy command, what is the decisive clue for whether the operation is an upload or download?

A: Inspect the source and destination order. A local path followed by an S3 URI is an upload; an S3 URI followed by a local path is a download.

Q: What is the operational danger of aws s3 rm s3://bucket-name --recursive?

A: It can delete all objects under the specified bucket location. Confirm the bucket URI and environment before execution.

Practice Questions

Question 1

An engineer creates three files in a CloudShell directory and needs to upload all of them to an S3 bucket in one operation. Which command is most appropriate?

A. aws s3 cp my-folder s3://bucket-name

B. aws s3 cp s3://bucket-name my-folder --recursive

C. aws s3 cp my-folder s3://bucket-name --recursive

D. aws s3 rb s3://bucket-name --recursive

Correct answer: C

The local directory is the source, the S3 URI is the destination, and --recursive is required to include the directory contents.

Question 2

A support analyst runs aws s3 ls and sees the bucket, but needs to verify whether test-file-1.txt exists inside it. What should the analyst run next?

A. aws s3 ls s3://bucket-name

B. aws s3 rb s3://bucket-name

C. aws s3 cp bucket-name test-file-1.txt

D. aws s3 rm s3://bucket-name --recursive

Correct answer: A

Adding the S3 URI changes the listing from account-level buckets to the objects in the specified bucket.

Question 3

A revised file must replace an existing S3 object, and the team does not need to preserve the prior copy. Which action matches the lesson?

A. Delete the bucket and recreate it.

B. Upload the revised file again using the same object name.

C. Use aws s3 rb on the object.

D. Download the object before every upload.

Correct answer: B

Uploading again with the same object name overwrites the existing object. Versioning would be considered if the previous version needed to remain available.

Question 4

An engineer has deleted the objects in a test bucket and now wants to remove the bucket. Which command should be used?

A. aws s3 rm s3://bucket-name

B. aws s3 cp s3://bucket-name /tmp --recursive

C. aws s3 rb s3://bucket-name

D. aws s3 ls s3://bucket-name --recursive

Correct answer: C

rb means remove bucket. The bucket must already be empty; rm is used for objects.

Question 5

A Python application needs to upload a local file to S3 as part of an automated workflow. Which approach is supported by the lesson?

A. Import Boto3, create an S3 resource, and call upload_file.

B. Use aws s3 rb inside the Python script to create the object.

C. Use aws s3 ls without specifying a bucket.

D. Use S3 Console browser actions only.