How to Delete Large S3 Bucket on Eumetsat Elasticity

Introduction

Due to an openstack-cli limitation, removing S3 buckets with more then 10 000 objects will fail when using the command:

openstack container delete --recursive <<bucket_name>>

showing the following error error:

Conflict (HTTP 409) (Request-ID: tx00000000000001bb5e8e5-006135c488-35bc5d520-dias_default) clean_up DeleteContainer: Conflict (HTTP 409) (Request-ID:)

Recommended solution

To delete a large S3 bucket we can use s3cmd.

In order to acquire access to your Object Storage buckets via s3cmd, first you have to generate your own EC2 credentials with the following tutorial: How to generate and manage EC2 credentials on Eumetsat Elasticity

After that, you have to configure s3cmd as explained in the following article: How to access private object storage using S3cmd or boto3 on Eumetsat Elasticity

After this, you should be able to list and access your Object Storage.

List your buckets with the command:

eouser@vm01:$ s3cmd ls
2022-02-02 22:22  s3://large-bucket

Now you’re able to delete your large bucket with the command presented below, where -r means recursive removal.

s3cmd rb -r s3://large-bucket

The bucket itself and all the files inside will be removed.

WARNING: Bucket is not empty. Removing all the objects from it first. This may take some time...
delete: 's3://large-bucket/example_file.jpg'
delete: 's3://large-bucket/example_file.txt'
delete: 's3://large-bucket/example_file.png'
...
...
...
Bucket 's3://large-bucket/' removed

Your large bucket has been successfully removed and the list of buckets is empty.

eouser@vm01:$ s3cmd ls
eouser@vm01:$