Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 37 ยท Apps, Dynos and Buildpacks

Dyno Lifecycle and the Ephemeral Filesystem

Learn how dynos start, restart and stop, why Heroku's filesystem is ephemeral, how to handle SIGTERM gracefully, and where to keep data that must survive.

On your laptop a program starts, runs, and leaves its files behind. On Heroku, a dyno is disposable: it is created from a slug, runs for a while, and is replaced. This is a deliberate design that makes deploys, scaling and failure recovery simple, but it changes how you write code. Anything you assume will "just be there next time" (a file you saved, an in-memory counter, a cached result) may be gone at any moment.

This lesson explains when dynos restart, what happens to a process when it is told to stop, and how to design around an ephemeral filesystem.

The states of a dyno

A dyno moves through a few states that you see in heroku ps:

  • starting: the container is being created and the process launched.
  • up: the process is running (for web dynos, it has bound its port).
  • crashed: the process exited with an error or never bound its port.
  • idle: an Eco web dyno that is asleep.
  • down: the dyno has been stopped (for example, scaled to zero).
stateDiagram-v2
  [*] --> starting
  starting --> up: process ready
  starting --> crashed: exits or boot timeout
  up --> crashed: process exits
  up --> starting: restart or cycling
  up --> down: scaled to 0
  crashed --> starting: automatic restart

Crashed dynos are restarted automatically, with an increasing delay if they keep failing.

What causes a restart

A dyno is restarted (and its filesystem reset) when:

  1. You deploy. New dynos start from the new slug and the old ones stop.
  2. You change config vars or add-ons. These create a new release.
  3. You run heroku ps:restart.
  4. The platform cycles it. Every dyno is restarted at least once per day, at a time that varies per dyno, so a fleet does not restart all at once.
  5. The process crashes. The platform starts a replacement. If it crashes repeatedly in a short time, the restarts back off to every ten minutes or so.
  6. Platform maintenance moves it to different hardware.

You cannot opt out of daily cycling. Treat "my app restarts about once a day, and sometimes at random" as part of the environment, and your design will be fine.

Graceful shutdown with SIGTERM

When a dyno is asked to stop, Heroku sends the process a SIGTERM signal. Your process then has about 30 seconds to finish up. If it is still running after that, the platform sends SIGKILL, which cannot be caught.

A well-behaved process on receiving SIGTERM:

  • stops accepting new work (a web server stops accepting new connections, a worker stops fetching new jobs),
  • finishes or re-queues what it is doing,
  • flushes logs and closes connections,
  • exits promptly.

Most frameworks and servers (gunicorn, puma, Node's server.close()) support this, but only if your command is the actual process. If the Procfile starts your app through a shell script that does not forward signals, the signal can be lost. Prefix with exec in scripts so the app replaces the shell.

JavaScript
// Node.js: shut down on SIGTERM
process.on("SIGTERM", () => {
  console.log("SIGTERM received, closing server");
  server.close(() => process.exit(0));
});

The ephemeral filesystem

Each dyno has its own writable disk, created fresh from the slug. You can write to it during the dyno's life, but:

  • Files written on one dyno are not visible to other dynos, including another copy of the same web process.
  • Files are lost on every restart, deploy, config change and daily cycle.
  • One-off dynos (heroku run) get their own separate disk, so a file created there never reaches the web dyno.

This is why none of these patterns work on Heroku:

  • storing user uploads in a folder on the dyno,
  • using SQLite as a production database,
  • keeping sessions in local files,
  • writing logs to a file you plan to read later.

What to do instead:

Need Use
Relational data Heroku Postgres (see the data chapter)
Uploaded files Object storage such as Amazon S3 or a compatible service
Cache and sessions Redis (Heroku Key-Value Store) or the database
Logs Write to stdout; the platform collects them (see CLI essentials)
Temporary scratch The dyno's disk is fine, as long as you can recreate it

Temporary files are legitimate: image resizing, building a PDF before streaming it, unpacking an archive. The rule is "scratch space, not storage."

Stand-in: watch the filesystem disappear

This model gives each dyno its own dictionary as a "disk." Restarting replaces the dictionary, and a second dyno never sees the first one's files.

Python Stand-in: why uploads on the dyno vanish

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in: how much work is lost at a shutdown?

A worker that takes longer than the grace period to finish its job is killed in the middle of it. This stand-in simulates the 30-second window with whole seconds, so you can see why long jobs should be split up or made resumable.

Python Stand-in: the SIGTERM grace period

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The lesson for real workers: make jobs idempotent (safe to run twice) and let the queue retry anything that was cut off.

Designing for disposability

  • Start fast. Slow boot makes every restart a visible blip.
  • Keep state in backing services so any dyno can serve any request.
  • Make background jobs idempotent and retryable.
  • Handle SIGTERM, and log it so you can tell restarts from crashes.
  • Never rely on a file or an in-memory value that must survive. If you must hold something in memory (a cache), be able to rebuild it.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. How often is each dyno restarted at least?

    Heroku cycles every dyno at least once per day, so apps must tolerate regular restarts.
  2. 2. What signal does a dyno's process receive first when it is asked to stop?

    SIGTERM asks the process to exit; SIGKILL follows if it is still running after about 30 seconds.
  3. 3. A user uploads an avatar and the app saves it under ./uploads. What is the problem?

    The dyno filesystem is ephemeral and per dyno. Use object storage for uploads.
  4. 4. Which is a good use of the dyno's disk?

    Scratch space that can be recreated is fine. Anything that must persist belongs in a backing service.
  5. 5. Why should background jobs be idempotent?

    Restarts can interrupt a job, and queues retry it. Idempotent jobs produce the same result when run again.

Summary

  • Dynos are disposable: they restart on deploys, config changes, crashes and at least daily.
  • A stopping process gets SIGTERM and roughly 30 seconds before SIGKILL; handle it and exit cleanly.
  • The filesystem is per dyno and temporary; use Postgres, Redis and object storage for anything that must last.
  • Design for disposability: fast start, stateless processes, idempotent jobs.

Next: Git deploys and slugs