Skip to content
Tutorials Center
My learning

Topics

Lesson 46 of 80 ยท Secure Coding

Secrets, Logging and Error Handling

Keep API keys and passwords out of source code and logs. Learn environment config, redaction filters, structured logging and safe error messages in Python.

Applications run on secrets: database passwords, API keys, signing keys, tokens. They also produce a constant stream of text, in logs and error messages, that is copied into files, dashboards and support tickets, often with much weaker protection than the secrets themselves. Two mistakes cause a large share of real-world leaks: committing a secret into source control, and writing it out through a log line or an error page.

In this lesson you'll learn where secrets should live, how to stop them reaching your logs with a redaction filter, what to log and what never to log, how structured logging helps defenders, and how to show users safe error messages while keeping the details for yourself. For storing and rotating keys, see Randomness, Key Management and Secrets; this lesson is about the code that uses them.

Keep secrets out of code

A secret written into source code (API_KEY = "sk_live_...") ends up in version control history, in every developer's clone, in build logs, in container images and in backups. Deleting the line later does not help: the old commit still contains it, and anyone who ever had access to the repository, or a leaked copy, keeps it. Bots scan public repositories for fresh keys within minutes of a push.

The usual rules:

  • Load secrets at runtime from environment variables, a secrets manager or a vault (cloud secret stores, HashiCorp Vault) injected by the platform.
  • Never commit .env files; add them to .gitignore and commit a .env.example with fake placeholders.
  • Use different secrets per environment (development, staging, production), so a leaked development key opens nothing important.
  • Give each secret the smallest scope and a short life; rotate on a schedule and immediately after any suspected exposure.
  • Fail loudly when a secret is missing, rather than falling back to a default like "changeme".
  • Scan commits for secrets before they land, using a pre-commit hook or CI step (the scanner idea is in the earlier lesson).
Python Load configuration from the environment

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The __repr__ hides the values on purpose. Debug output such as print(settings) or an exception that includes the object would otherwise leak every secret in it. Many libraries offer a SecretStr-style wrapper type for the same reason.

What not to log

Logs are for answering "what happened?" without becoming a second database of everything sensitive. Never log:

  • Passwords (even failed attempts: users often mistype one character of the real one), API keys, session IDs, tokens and private keys.
  • Full payment card numbers, government IDs, health data, and other regulated data (see Privacy and Data Protection).
  • Full request bodies and headers by default: Authorization and Cookie headers carry credentials.
  • URLs with secrets in query strings (another reason tokens belong in headers or bodies, not URLs).

Do log security-relevant events: logins (success and failure), permission denials, password changes, token creation, input rejections, configuration changes. Include the time, an identifier for the user and request, the source IP and the outcome (see Logging, Log Analysis and SIEM).

Log injection is a related risk. If a username is written verbatim into a plain-text log, an attacker can include a newline and a fake entry such as \nINFO login ok user=admin. Encode or escape user-controlled values, or write logs as JSON, where newlines are escaped automatically.

Redaction filters

People make mistakes, so add a safety net: a filter that scrubs known patterns from every record before it is written. Python's logging module supports this with a logging.Filter:

Python A logging filter that redacts secrets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Redaction by pattern is a backstop, not a plan. It misses secrets that do not match your patterns, so the primary defense stays the same: do not put secrets into log calls in the first place. Treat a redaction hit in production as a bug to fix at the source.

Structured logging

Plain-text logs are written for humans and parsed with fragile regular expressions. Structured logging emits one JSON object per event with consistent fields. It is easier to search, filter and alert on, and it handles special characters safely.

Python JSON logs with safe fields

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A fixed schema (time, event name, actor, source, outcome, request ID) is what lets a SIEM correlate events later. A request ID shared across services lets you follow one user action through the whole system without logging personal data.

Safe error messages

Errors come in two flavors, and they have different audiences:

Audience Needs Example
The user A short, helpful, safe message and a reference "Something went wrong. Reference 8f3a21."
The developer The exception, stack trace, request context OperationalError: no such table: invoices

Detailed errors shown to the public leak file paths, framework versions, SQL fragments, internal host names and sometimes credentials from connection strings. They also help attackers tune their attacks. Production should show a generic page and write the detail to a protected log. Turn off framework debug pages (DEBUG = False), and do not return str(exception) in API responses.

Python Generic response, detailed private log

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the split: errors caused by the user's own input can say what is wrong (without echoing hostile content back unescaped), authorization errors stay short, and anything unexpected becomes a generic message with a reference that support can look up. Beyond leaking, protect the log store itself: restrict who can read it, keep it off the web root, retain it only as long as needed and ship it to a separate system so an intruder cannot easily erase their tracks.

If a secret leaks anyway

Assume it will happen and prepare: revoke or rotate the secret first (before cleaning the history, because copies already exist), check access logs for misuse during the exposure window, remove it from the code and, if you must, scrub history, then add a scanner so it cannot be committed again. Rotation is only fast if secrets live in one place and applications reload them without a redeploy, which is another argument for a secrets manager.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A developer accidentally commits an API key and then deletes the line in the next commit. What should happen next?

    Version control keeps old commits, and clones and bots may already have the key. Rotation is the only reliable fix.
  2. 2. Which of these should NOT appear in a log line?

    Authorization headers carry credentials; logs are widely readable and long-lived. The other items are useful security context.
  3. 3. Why is a redaction filter described as a backstop?

    Pattern-based scrubbing misses unknown formats. The primary defense is not logging secrets at all.
  4. 4. What is the best response for an unexpected exception in production?

    Users get a safe message and a reference; developers get the full detail from the protected log.
  5. 5. How does JSON-structured logging help against log injection?

    A JSON encoder turns a newline in a value into an escape sequence, so one event always remains one physical line.

Summary

  • Keep secrets out of source code and images: load them at runtime from the environment or a secrets manager, use separate secrets per environment, and fail when one is missing.
  • Never log passwords, tokens, keys, full card numbers or credential headers; do log security events with time, actor, source and outcome.
  • Redaction filters are a safety net; structured JSON logs make logs searchable and resistant to injection.
  • Show users generic errors with a reference ID and keep stack traces in a protected log.
  • If a secret leaks, rotate it first, then investigate and clean up.

Next: Security Code Review, SAST and Testing, where you'll check code for these problems systematically.