Docs · Object storage

Lifecycle rules: stop paying for what you no longer need.

Logs, temporary uploads, old versions and abandoned uploads pile up, and you pay for every byte of them. A lifecycle rule deletes them on a schedule you set.

A lifecycle configuration is a list of rules on a bucket. Each rule picks objects with a filter and says what to delete, and when. The rules run every hour and delete for good: test a new rule on a prefix first.

The examples use the AWS CLI profile the panel gives you when you create an access key (Object storage → Access keys → New key), so every command carries --profile computeforge --endpoint-url https://s3.computeforge.com. Any S3 client works the same way.

An example

Save this as lifecycle.json:

{
  "Rules": [
    { "ID": "logs-30-days", "Status": "Enabled",
      "Filter": { "Prefix": "logs/" },
      "Expiration": { "Days": 30 } },

    { "ID": "temp-uploads", "Status": "Enabled",
      "Filter": { "Tag": { "Key": "temp", "Value": "true" } },
      "Expiration": { "Days": 1 } },

    { "ID": "old-versions", "Status": "Enabled",
      "Filter": {},
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 30, "NewerNoncurrentVersions": 3 } },

    { "ID": "lone-delete-markers", "Status": "Enabled",
      "Filter": {},
      "Expiration": { "ExpiredObjectDeleteMarker": true } },

    { "ID": "abandoned-uploads", "Status": "Enabled",
      "Filter": {},
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 } }
  ]
}
aws s3api put-bucket-lifecycle-configuration --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json \
  --profile computeforge --endpoint-url https://s3.computeforge.com

get-bucket-lifecycle-configuration reads the rules back. delete-bucket-lifecycle removes them. Putting a configuration replaces the whole list.

What a rule can do

  • Expiration Days or Date: deletes current files that old. In a versioned bucket, the file is kept as an older version behind a delete marker, as in S3.
  • Expiration ExpiredObjectDeleteMarker: removes delete markers that no longer hide any version.
  • NoncurrentVersionExpiration: deletes older versions NoncurrentDays after they were replaced. NewerNoncurrentVersions (1 to 100) keeps that many of the newest older versions of each file, however old.
  • AbortIncompleteMultipartUpload: drops the parts of multipart uploads that were started that many days ago and never finished.

Transitions to other storage classes are not supported. There is one storage class, and a rule with a transition is refused (501 NotImplemented).

Filters

  • "Filter": {}: every object in the bucket. A rule must have a Filter (or the older Prefix).
  • Prefix: keys that start with it.
  • Tag: objects carrying that tag with that exact value.
  • ObjectSizeGreaterThan / ObjectSizeLessThan: by size in bytes.
  • And: several of these at once, e.g. a prefix and two tags; all of them must match.

Filtering by tag

Tag objects when you upload them, or later:

aws s3api put-object --bucket my-bucket --key exports/run-17.csv \
  --body run-17.csv --tagging "temp=true" \
  --profile computeforge --endpoint-url https://s3.computeforge.com

aws s3api put-object-tagging --bucket my-bucket \
  --key exports/run-17.csv \
  --tagging '{"TagSet": [{"Key": "temp", "Value": "true"}]}' \
  --profile computeforge --endpoint-url https://s3.computeforge.com
  • An object can carry up to 10 tags. Keys are 1 to 128 characters, values up to 256, and keys starting with aws: are reserved.
  • Tags belong to a version. An upload without tags replaces the tags, too. A copy keeps the source's tags unless you replace them.
  • A rule's tags are checked at each hourly run, so removing a tag before the run keeps the object.
  • Delete markers have no tags. So ExpiredObjectDeleteMarker and AbortIncompleteMultipartUpload cannot be combined with a tag filter; the bucket refuses such a rule.

When things are deleted

  • As in S3, an object's age is counted from when it was written and rounded up to the next midnight UTC. With Days: 30, it goes in the first hourly run after midnight UTC once 30 days have passed.
  • A Date must be a midnight UTC; objects go in the first run on or after it.
  • Each run deletes up to 10,000 objects or versions per rule; the next run continues where it stopped.
  • Days can be at most 36,500. A bucket can have up to 1,000 rules.
  • What a rule deletes stops counting towards your storage from the next measurement (every 15 minutes).

A read-only key can read a bucket's rules; setting them needs a read-and-write key granted the bucket. Lifecycle rules can only lower a bill: they delete what your rules say, nothing else.

More on object storage