Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 36 ยท Unity Catalog and Governance

The Unity Catalog Namespace

Understand Unity Catalog's three-level namespace (catalog.schema.table), the metastore, securable objects, managed versus external tables, and how to organise catalogs.

Before Unity Catalog, each Databricks workspace had its own table list and its own permissions, so the same data had to be governed again in every workspace. Unity Catalog is the central governance layer for data and AI assets across workspaces: one place for names, access rules, lineage and audit.

Databricks cannot run on this site. The SQL below is read-only and described; the runnable blocks are plain Python stand-ins that model the namespace and name resolution.

The object model

flowchart TD
    M["Metastore (one per region)"] --> C1["Catalog: main"]
    M --> C2["Catalog: dev"]
    C1 --> S1["Schema: sales"]
    C1 --> S2["Schema: hr"]
    S1 --> T["Tables and views"]
    S1 --> V["Volumes (files)"]
    S1 --> F["Functions"]
    S1 --> MD["Registered models"]

One metastore holds catalogs; catalogs hold schemas; schemas hold tables, views, volumes, functions and models.

  • Metastore: the top container, one per cloud region, attached to many workspaces.
  • Catalog: the first level, usually per environment (dev, prod), business domain (finance) or team.
  • Schema (also called database): groups related objects, for example bronze, silver, gold.
  • Table, view, volume, function, model: the objects you use.

A full name has three parts: catalog.schema.object, for example main.sales.orders.

SQL
CREATE CATALOG IF NOT EXISTS prod;
CREATE SCHEMA  IF NOT EXISTS prod.sales COMMENT 'Sales domain';

CREATE TABLE prod.sales.orders (
  order_id BIGINT,
  customer_id BIGINT,
  amount DECIMAL(12,2),
  order_ts TIMESTAMP
);

SELECT * FROM prod.sales.orders LIMIT 10;

Current catalog and schema

To avoid typing three-part names, set defaults for the session:

SQL
USE CATALOG prod;
USE SCHEMA sales;
SELECT * FROM orders;      -- resolves to prod.sales.orders
SELECT current_catalog(), current_schema();

In Python, spark.sql("USE CATALOG prod") does the same, and spark.read.table("orders") then resolves relative to the current schema. Name resolution is simply "fill in the missing parts from the session":

Python Stand-in: resolving one-, two- and three-part names

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Managed and external tables

Managed table External table
Where files live Unity Catalog-managed storage (catalog or schema location) A path you own, registered through an external location
DROP TABLE Removes metadata and, after a retention period, the data Removes metadata only; files stay
Best for Most new tables: Databricks handles layout and optimisation Data shared with other tools that write the same files
SQL
-- managed: no LOCATION
CREATE TABLE prod.sales.orders_clean AS SELECT * FROM prod.sales.orders WHERE amount > 0;

-- external: needs an external location the user may use
CREATE TABLE prod.sales.orders_ext
LOCATION 's3://company-lake/sales/orders';

Managed tables are Delta by default; Iceberg is also supported. See Volumes and External Locations for the storage objects behind this.

Where to put things: a layout

A simple, common convention:

Catalog Schemas Meaning
dev bronze, silver, gold, sandbox Experiments and CI
test same Pre-production checks
prod same Real pipelines and published data

Or one catalog per domain (finance, marketing) with environment encoded in the schema. Pick one rule and keep it. Access is easiest to manage at catalog or schema level, so the layout should follow who should see what.

Python Stand-in: a catalog tree and fully qualified names

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other securable objects

Besides data objects, Unity Catalog governs storage credentials, external locations, connections (to external databases for federation), shares (Delta Sharing) and service principals as identities. Everything that can have permissions is a securable object, and permissions inherit downwards (see Grants and Privileges).

Discovering data

  • Catalog Explorer in the workspace shows the tree, sample data, schema, permissions and lineage.
  • SHOW CATALOGS, SHOW SCHEMAS IN prod, SHOW TABLES IN prod.sales, and DESCRIBE EXTENDED prod.sales.orders work in SQL.
  • COMMENT ON TABLE ... IS '...' and column comments make search useful.
  • The system.information_schema views list tables and columns for SQL-based discovery.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What are the three levels of a Unity Catalog table name?

    A table is addressed as catalog.schema.table.
  2. 2. What happens to the data files when you drop an EXTERNAL table?

    External tables leave the underlying files in place.
  3. 3. Which statement sets the default catalog for the session?

    USE CATALOG changes the default used to resolve short names.
  4. 4. How many metastores does a region normally have?

    A single metastore per region is attached to many workspaces.

Summary

  • Unity Catalog is one governance layer across workspaces, built on a metastore.
  • Names are catalog.schema.object; defaults come from USE CATALOG and USE SCHEMA.
  • Prefer managed tables; use external tables when other systems own the files.
  • Choose a catalog/schema layout that matches who needs access.

Next: Grants and Privileges