How to monitor the cost of individual AWS S3 buckets

#aws

When you have multiple S3 buckets, the cost is generally shown as a total for all buckets. To view the cost of individual S3 buckets, you need to use cost allocation S3 bucket tags.

How

Adding S3 Bucket Tags

  1. Go to the S3 bucket page.
  2. Click on the bucket name (e.g., john-bucket-doe).
  3. Navigate to Properties > Tags > click Edit.
  4. In the Key field, add 'S3-Bucket-Name'.
  5. In the Value field, add the bucket name (e.g., john-bucket-doe).
  6. Save the changes.
  7. Repeat steps 1 to 6 for your other buckets.

Setting Up Cost Allocation Tags

  1. Go to the Cost Management Console.
  2. Click on Cost Allocation Tags from the left sidebar.
  3. Under User-defined cost allocation tags, add the 'S3-Bucket-Name' tag and activate it.
  4. If you don’t see any tags, you may need to wait a few days for AWS to process them.
  5. Once activated, it may take a few days for the tag to appear in the Cost Explorer diagram.

Python Script

You can use a Python script to automate your work.

  1. make sure you setup your AWS CLI, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  2. pip install boto3
  3. create tag-s3-buckets.py 
    import boto3
import botocore

def tag_s3_buckets():
  s3 = boto3.client('s3')
  
  try:
    response = s3.list_buckets()
    buckets = response['Buckets']
    for bucket in buckets:
      bucket_name = bucket['Name']
      try:
        # add tag for each s3 bucket
        s3.put_bucket_tagging(
          Bucket=bucket_name,
          Tagging={
            'TagSet': [
              {
                'Key': 'S3-Bucket-Name',
                'Value': bucket_name
              },
            ]
          }
        )
        print(f"Successfully tagged bucket: {bucket_name}")
      except botocore.exceptions.ClientError as e:
        print(f"Error tagging bucket {bucket_name}: {e}")
  except botocore.exceptions.ClientError as e:
        print(f"An error occurred: {e}")

if __name__ == "__main__":
    tag_s3_buckets()
  4. run script

Reference: