Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 38 ยท Compliance and Governance

Data Privacy and Residency in the Cloud

Learn data classification, personal data basics, data residency and sovereignty, retention, pseudonymisation and masking, and how to find sensitive data in the cloud.

Moving data to the cloud raises a simple question with a complicated answer: where is the data, who can see it and what rules apply to it? A customer record stored in a bucket in another country, copied into a log file and replicated to a backup region may be subject to several laws at once. Privacy engineering is the discipline of knowing your data and handling it in a way that respects both the law and the people the data describes.

This lesson covers data classification, the key privacy principles (using the EU GDPR as the best-known example), data residency and sovereignty, retention and deletion, and the technical tools of pseudonymisation and masking. The Python exercises use fake personal data only. It is a technical introduction, not legal advice.

Know your data: classification

You cannot protect data you have not identified. A classification scheme gives each kind of data a label that drives the controls it needs. A typical four-level scheme:

Level Example Typical controls
Public Marketing pages, published docs Integrity protection only
Internal Org charts, internal wiki Authenticated access
Confidential Contracts, customer lists, source code Encryption, least privilege, access logging
Restricted Payment cards, health records, secrets, government IDs Strong encryption with customer-managed keys, tight access, extra monitoring, short retention

Apply the label with tags on resources (data-class=restricted), so policies and scanners can use them: for instance, "restricted data may only live in approved regions and must not be publicly accessible". See policy as code.

Personal data and privacy principles

Personal data is any information relating to an identifiable person: names, e-mail addresses, IP addresses, device IDs, location, and also combinations of fields that identify someone indirectly. Some categories are sensitive (health, biometrics, religion, political views) and need extra care.

The GDPR and many similar laws share a set of principles that map directly onto engineering decisions:

  • Lawfulness, fairness and transparency: have a legal basis (such as consent or contract) and tell people what you do.
  • Purpose limitation: use data only for the purpose you collected it for.
  • Data minimisation: collect and keep only what you need. Data you never collected cannot leak.
  • Accuracy: keep it correct and let people fix it.
  • Storage limitation: delete data when you no longer need it.
  • Integrity and confidentiality: protect it with appropriate security, such as encryption and access control.
  • Accountability: be able to demonstrate compliance.

People also hold rights: to access their data, correct it, have it erased, restrict or object to processing and receive it in a portable format. Your systems must be able to find all the data about one person in order to honour a request, which is much harder if copies are scattered across buckets, logs and analytics tables.

Residency, sovereignty and transfers

  • Data residency says where data is physically stored (a country or region).
  • Data sovereignty says whose law applies to it, which can follow the provider's headquarters or the data subject's location, not only the data centre.
  • Cross-border transfers are restricted by some laws. Transfers out of the EU, for example, need a recognised mechanism (an adequacy decision, standard contractual clauses or similar).

Cloud providers let you choose regions, and some offer sovereign or "confidential" options. Remember that data moves in ways that are easy to forget: backups and replicas in other regions, logs sent to a central system, support access from other countries, CDN caches and analytics services. Use region restrictions at the organisation level (for example an allow-list of regions in an organisation policy) and review the data flows.

flowchart LR
    U["User in the EU"] --> A["App in eu-west"]
    A --> D[("Primary DB, eu-west")]
    D --> B[("Backup, eu-north")]
    A --> L["Logs to central SIEM"]
    A --> X["Analytics vendor, outside EU?"]
    X -.->|"needs a transfer mechanism"| Z["Review"]

Follow the data: every arrow can cross a border.

Try it: a residency policy check

Python Check that data stays in allowed regions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The backup and health export are the kind of copies people forget. A scheduled check like this, run against the real inventory, catches them.

Finding sensitive data

You often need to discover where personal data hides: in free-text fields, in logs, in exported files. Cloud services such as Amazon Macie, Microsoft Purview and Google Sensitive Data Protection scan storage for patterns (card numbers, national IDs, e-mail addresses) and report locations. The idea is pattern matching plus validation, shown here with fake data.

Python Find personal data in text and validate card numbers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Luhn checksum is the standard cheap test that separates real card numbers from other long digit strings. The first number is a well-known test card number that passes; the order reference does not.

Retention and deletion

Keep data only as long as needed. Define a retention period per data class, enforce it with lifecycle rules on storage (expire objects after N days), delete from backups on a schedule and remember logs: they contain personal data too, so keep them long enough for security needs and legal duties, and no longer. When honouring an erasure request, define how deletion propagates to replicas, caches and backups (often by letting backups expire within a documented period).

Technical protections: pseudonymisation and masking

  • Pseudonymisation replaces direct identifiers with a stable token, so analysts can still count and join records without seeing who they are. The mapping key is kept separately and protected. It is still personal data under the GDPR, because it can be reversed with the key.
  • Anonymisation removes the link permanently. This is hard to achieve: combinations of age, postcode and job often identify people.
  • Masking hides part of a value in displays and logs (****1111).
  • Encryption protects data at rest and in transit; consider customer-managed keys and field-level encryption for restricted fields.
  • Access controls and logging limit and record who reads personal data.
Python Pseudonymise with a keyed hash and mask for display

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same person gets the same token regardless of capitalisation, so totals still add up (40 + 15 for Jane), but the analytics table contains no e-mail addresses. Using a keyed hash (HMAC) is important: a plain hash of an e-mail address can be reversed by hashing a list of guesses.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Data minimisation means:

    Data you never collect cannot be leaked, misused or demanded in a request. Minimisation is both a privacy principle and a security measure.
  2. 2. Why can backups and logs break a residency policy?

    Residency covers every copy: replicas, backups, logs, caches and vendor integrations.
  3. 3. Is pseudonymised data still personal data under the GDPR?

    Pseudonymisation reduces risk but the data can still be attributed to a person with additional information, so it remains personal data.
  4. 4. Why use an HMAC instead of a plain hash for pseudonymising e-mail addresses?

    A keyed hash cannot be recomputed by someone without the key, which blocks guess-and-compare attacks.

Summary

  • Classify data and tag resources so controls and policies can follow the label.
  • Privacy principles (minimisation, purpose limitation, storage limitation, integrity and confidentiality) translate into concrete engineering rules.
  • Residency and sovereignty cover every copy of the data, including backups, logs and vendors; enforce regions with policy and check them regularly.
  • Discover sensitive data with pattern matching and validation, retain it only as long as needed and protect it with pseudonymisation, masking, encryption and access logging.

Next: Shift-Left Security