AWS on the road: S3 in Podgorica

Simple Storage Service (S3) is an object storage, one of the most used Amazon service. It allows to create buckets (logical containers) to store objects (files) with name (key), data, version ID and metadata.

S3 is divided into classes:

  • Standard – used for most uses cases, paid only for storage space. It automatically creates multiple copy in all AZs in Region.
  • Infrequent access (IA) – cheaper than Standard version, but you have to pay also for access to objects (per GB). It can be used to store files that are not access frequently, but yo always want fast access to all files. Amazon has another service called Glacier, where price is lower, but access to your files can take up to 5 hours.
  • Reduced Redundancy Storage (RRS) – does not replicate objects as many times as standard Amazon S3 storage. Can be used for noncritical, reproducible data at lower levels of redundancy and lower price.
  • One Zone-Infrequent Access (One Zone IA) – stores the object data in only one AZ, which makes it less expensive than Standard IA storage.
  • Glacier – uses Amazon Glacier service to store the data. Although the objects are stored in Amazon Glacier, these objects are managed in Amazon S3, and you cannot access them directly through Amazon Glacier. It is suitable for archiving data where data access is infrequent. Archived objects are not available for real-time access. You must first restore the objects before you can access them.

Simple S3 class comparison:

S3 classes

Features

Besides classes, in each S3 bucket you can enable following features:

  • Versioning – allows keeping multiple variants of an object in the same bucket. You can use versioning to preserve, retrieve, and restore every version of every object stored in your Amazon S3 bucket. It is enabled on bucket level. When you overwrite object, it actually creates new object with higher version.  When you delete an object, all versions remain in the bucket and S3 inserts a delete marker on it. Performing GET request when the current version is a delete marker returns a 404 Not Found error. You can permanently delete an object by specifying the version you want to delete. Only the owner of an Amazon S3 bucket can permanently delete a version.
  • Encryption – for data in motion (SSL) and data at rest (server side or client side). For server side encryption you can use S3-Managed keys, AWS KMS–managed keys or you can provide your own keys.
  • Transfer acceleration – can be used to accelerate uploading speed (up to 300%) using Amazon Edge Locations. If you need to accelerate downloading speed you can use Amazon CloudFront – content delivery network. To use acceleration you need to use different endpoint: mybucket.s3-accelerate.amazonaws.com.
  • Lifecycle management – allows you set policies to transfer objects to another class or delete them after some period of time. For example you can set that files tagged with archive=yes will be moved to Glacier after 90 days.
  • Cross regional replication – enables automatic, asynchronous copying of objects across buckets in different AWS Regions, also to different accounts.
  • MFA delete – when enabled requires additional authentication for changing the versioning state of your bucket or permanently deleting an object version.

Eventual consistency

Amazon S3 provides read-after-write consistency for PUTS of new objects with one caveat. The caveat is that if you make a HEAD or GET request to find if the object exists before creating the object, Amazon S3 provides eventual consistency for read-after-write.

Amazon S3 offers eventual consistency for overwrite PUTS and DELETES. It means that when you overwrite or delete a file in bucket, S3 might return the prior data until the change is fully propagated.

S3 does not support object locking. If two PUT requests are simultaneously made to the same key, the request with the latest time stamp wins.

Access

All buckets are private by default. You can grant additional access to bucket or files in three ways:

  • Bucket policy (resource-based policy) – policy set per bucket, each object put into bucket inherits that policy rights.
  • Identity based policy – attached to user, group or role.
  • S3 ACL – read/write access set per object. Here you can set public access to your files or set access for other AWS account.

For more information check my previous post about AWS IAM.

Reference Materials

AWS on the road
AWS S3 documentation
IAM Policies and Bucket Policies

Podgorica, Montenegro

Comments 1

Leave a Reply