Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 45 ยท Storage and Databases

Cloud Storage Buckets and Objects

Learn Cloud Storage buckets and objects, storage classes, lifecycle rules, versioning and signed URLs, with the gcloud storage commands and a rule engine.

Cloud Storage is Google Cloud's object storage: a place to keep files of any type and size, from a few bytes to terabytes, and fetch them by name over HTTPS. Websites, backups, images, log archives, datasets for analytics and build artifacts all end up here. It is durable, scales without planning and you pay for what you store and transfer.

This lesson explains buckets and objects, the four storage classes, lifecycle rules that move or delete data on a schedule, versioning and signed URLs. The commands are read-only because they need a project; the stand-ins show the logic.

Buckets and objects

A bucket is a container with a globally unique name. An object is a file plus metadata, stored in a bucket under a name. There are no real folders: a name like photos/2026/cat.png is just one string with slashes, and tools display it as a folder tree for convenience.

Terminal
# Create a bucket in one region (the name must be globally unique)
gcloud storage buckets create gs://my-demo-bucket-4821 \
  --location=us-central1 \
  --uniform-bucket-level-access

# Upload, list, download, delete
gcloud storage cp ./cat.png gs://my-demo-bucket-4821/photos/cat.png
gcloud storage ls gs://my-demo-bucket-4821/photos/
gcloud storage cp gs://my-demo-bucket-4821/photos/cat.png ./copy.png
gcloud storage rm gs://my-demo-bucket-4821/photos/cat.png

gcloud storage is the current command group; the older gsutil tool still works and you will see it in many guides, with similar commands (gsutil cp, gsutil ls).

Choose the bucket's location carefully (see Regions, Zones and Latency): a single region, a dual-region, or a multi-region such as US. It cannot be changed later. Objects are immutable: you replace an object by uploading a new one under the same name, not by editing bytes in place.

Uniform bucket-level access means permissions come only from IAM, not from per-object ACLs. It is the recommended setting and makes it easier to reason about who can read what.

Storage classes

Four storage classes

Standard Frequent access. No minimum storage time.
Nearline About once a month. 30-day minimum.
Coldline About once a quarter. 90-day minimum.
Archive Less than once a year. 365-day minimum.

Colder means cheaper to store, pricier to read.

1 / 2

All four classes have the same durability and the same millisecond access time. They differ in price: colder classes cost less to store but more to read, and they have a minimum storage duration. If you delete an object early, you are still charged for the remainder of the minimum. Do not put data you read every day in Archive. Look up current per-GB prices on the pricing page rather than relying on numbers in a tutorial.

You can set a class when uploading, and with Autoclass enabled on a bucket, Cloud Storage moves objects between classes automatically based on how they are used.

Lifecycle rules

A lifecycle rule pairs a condition with an action. Typical conditions are the object's age in days, how many newer versions exist, or whether it is a live version. Typical actions are SetStorageClass and Delete. The configuration is JSON:

JSON
{
  "rule": [
    {
      "action": { "type": "SetStorageClass", "storageClass": "NEARLINE" },
      "condition": { "age": 30 }
    },
    {
      "action": { "type": "SetStorageClass", "storageClass": "COLDLINE" },
      "condition": { "age": 90 }
    },
    {
      "action": { "type": "Delete" },
      "condition": { "age": 365 }
    }
  ]
}
Terminal
gcloud storage buckets update gs://my-demo-bucket-4821 \
  --lifecycle-file=lifecycle.json

Lifecycle rules run in the background, so changes are not applied the instant an object turns 30 days old. They save money on logs and backups that you must keep for a while but rarely read.

A lifecycle rule engine

This stand-in applies the same kind of rules to a list of objects on a given day. It is not how Cloud Storage is implemented. It shows how conditions and actions combine, and the rule that the most advanced matching class wins is a simplification for clarity.

JS Stand-in: lifecycle rules on a pile of objects

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check the last line: an object over 365 days old is deleted, even though it also matches the 30-day and 90-day rules. Order and thresholds matter when you design real rules, so test them on a scratch bucket first.

Versioning, retention and safety

Turn on object versioning and an overwritten or deleted object is kept as a noncurrent version that you can restore:

Terminal
gcloud storage buckets update gs://my-demo-bucket-4821 --versioning
gcloud storage ls --all-versions gs://my-demo-bucket-4821/report.csv

Versions take space, so pair versioning with a lifecycle rule that deletes noncurrent versions after some days. For compliance data you can add a retention policy, which forbids deleting or replacing objects until their retention period ends. A bucket soft delete window also exists as a recovery net for accidental deletions; check the current default for new buckets.

Public buckets are a classic source of data leaks. Keep public access prevention enabled unless the bucket is meant to serve a public website, and never grant allUsers on a bucket that holds anything private. See Common Mistakes and Troubleshooting.

Signed URLs

How do you let someone download one private file without giving them an account? A signed URL is a link that carries a signature and an expiry time. Whoever holds the link can perform the specified action (usually GET or PUT) on that one object until it expires. The signature is created with a service account credential:

Terminal
gcloud storage sign-url gs://my-demo-bucket-4821/reports/q3.pdf \
  --duration=15m \
  --impersonate-service-account[email protected]

Keep expiry short, because anyone with the URL can use it. Applications usually create signed URLs in code with a client library, then hand them to the browser for direct uploads or downloads, so large files do not pass through your servers.

Why a signature protects the URL

The model below uses an HMAC from Node's node:crypto to show the principle: the signature covers the object path and the expiry, so changing either makes it invalid. Google's real V4 signing is different in detail (it signs a canonical request with the service account's private key), so treat this as an analogy.

Node.js Stand-in: signed URLs with an expiry

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Both tampering attempts fail because the signature no longer matches. Everything else, such as who the signer is and which methods are allowed, is added in the real system.

Cloud storage in general: security, benefits and drawbacks

"Cloud storage" in general means keeping data on a provider's servers and reaching it over the network. Whatever the product, a service like this has a front-end layer that authenticates requests and routes them, a metadata layer that knows where things are, and a storage layer that holds the bytes on many machines with several copies.

Is it secure? Providers encrypt data in transit and at rest, control access with role-based access control (IAM), keep audit logs, and submit to independent third-party audits. The common failures are on the customer side: a public bucket, an over-broad role, or a leaked key.

Advantages: it scales without buying hardware, you can sync and share across devices and collaborators, you pay for what you use ("cost-efficient" and a good fit for FinOps practices), and durability is far higher than a single disk. Disadvantages: it needs a network connection, egress fees make moving data out cost money, performance depends on distance, vendor lock-in grows with proprietary features, and some regulations limit where data may live. Cost control is part of using it: lifecycle rules, the right storage class and budget alerts (Cost Management).

How Google stores data: from GFS to Colossus

Google's own storage grew out of the Google File System (GFS), a 2003 design for huge files on cheap, failure-prone machines. Today's internal successor, Colossus, sits underneath Cloud Storage, Bigtable, Spanner and persistent disks. The original ideas still explain how distributed storage works:

  • Files are split into large fixed-size chunks (64 MB in GFS) stored on many chunkservers, and each chunk is replicated (three copies by default) on different machines and racks.
  • A master holds only the metadata: file names, the list of chunks per file, and which servers hold each chunk. It does not carry file data, so it does not become a bottleneck.
  • Chunkservers send the master regular heartbeat messages. If one dies, the master notices and makes new copies of its chunks elsewhere.
  • Reading: the client asks the master "where is chunk 2 of this file?", receives the chunkserver locations, caches them, and then reads directly from a chunkserver.
  • Data integrity: every chunk carries checksums, verified on reads, so corruption is detected and repaired from another replica.
  • Writes are designed around appending large amounts of data, not changing small bits in place, which suits logs and analytics.
Python A miniature master and chunkservers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which storage class suits data you read about once a year?

    Archive has the lowest storage price for rarely read data, with a 365-day minimum duration and higher read costs.
  2. 2. What do lifecycle rules do?

    A rule pairs conditions (such as age) with an action (such as SetStorageClass or Delete).
  3. 3. What happens if you enable versioning and overwrite an object?

    Versioning keeps previous generations so they can be restored, at the cost of extra storage.
  4. 4. What does a signed URL provide?

    The URL carries a signature and expiry, so the holder can do the allowed action on that object until it expires.

Summary

  • Buckets hold objects; names are flat strings, objects are immutable, and bucket location is permanent.
  • Standard, Nearline, Coldline and Archive trade storage price against read price and minimum duration.
  • Lifecycle rules move or delete data by age and other conditions; pair them with versioning.
  • Use uniform bucket-level access and public access prevention; share single files with short-lived signed URLs.

Next: Persistent Disk, Local SSD and Filestore