Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 50 ยท Quality, Testing and Delivery

Deployment, Maintenance and Software Evolution

Learn release strategies such as canary and blue-green, how monitoring and incidents work, Lehman's laws of software evolution, legacy systems and end of life.

Shipping the first version is the beginning of a program's life, not the end. Studies of real projects repeatedly find that most of the total cost of software is spent after the first release: fixing faults, adapting to new platforms, adding features and keeping the system secure. Software engineering therefore cares as much about how a system is released, watched and changed over the years as about how it is first built.

This lesson covers four connected topics: how to release changes safely, how to notice and handle problems in production, what "software evolution" means, and what to do with old systems. Real deployment tools and monitoring services cannot run on this site, so the JavaScript examples are stand-ins with invented numbers: a canary rollout controller and a small availability calculator. They show the decision logic, not a real platform. The CI/CD track covers real tooling, for example deployment strategies and observability and metrics.

Releasing safely

Deploying means putting a new version in front of real users. The risk is that a bug you did not find in testing reaches everyone at once. Release strategies reduce that blast radius.

  • Big-bang release. Replace the old version everywhere in one step. Simple, but any fault hits all users and there may be downtime.
  • Rolling release. Replace servers a few at a time. Capacity stays up, and you can stop midway.
  • Blue-green release. Run two complete environments. "Blue" serves users while you deploy and test "green"; then you switch traffic over. Rolling back means switching back, which is fast. The cost is running two environments.
  • Canary release. Send a small share of traffic (say 5%) to the new version, watch the numbers, and increase the share only while things look healthy. If they do not, route everyone back to the old version.
  • Feature flags. Deploy the code switched off and turn it on later, for some users or all. This separates "deploying code" from "releasing a feature".

Four ways to release

Big bang Everyone at once. Simple and risky.
Rolling A few servers at a time.
Blue-green Switch between two full environments.
Canary A small slice of users goes first.

Each step trades speed and cost against blast radius.

1 / 2

Database changes deserve a special warning. Code can be rolled back in seconds, but data written in a new format cannot. A common technique is the expand and contract pattern: first add the new column (old code ignores it), then deploy code that writes both, then migrate, and only much later remove the old column. Each step is safe to roll back.

A canary rollout stand-in

A canary controller repeats a simple loop: move more traffic to the new version, measure its error rate, compare it with the old version, and either continue, hold or roll back. The measurements below are a fixed list standing in for what a monitoring system would report.

JS Canary rollout controller

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The second release looks fine at 5% (a small sample can hide a rare bug) and fails at 25%. Note what the strategy bought: the faulty release hurt at most a quarter of users for a few minutes instead of everyone for hours. The real work is choosing good signals (error rate, latency, business metrics) and thresholds, and making sure the sample is big enough to be meaningful.

Monitoring and incidents

Once software is running, the team must know when it misbehaves. Monitoring collects signals; teams commonly group them as:

  • Metrics: numbers over time, such as requests per second, error rate, latency percentiles, memory use.
  • Logs: timestamped records of events, useful for finding why something failed.
  • Traces: the path of one request through several services, useful for finding where time was spent.

Alerts should fire on symptoms users feel (errors, slow pages) rather than on every internal blip, otherwise people drown in noise and start ignoring them.

An incident is an unplanned event that degrades the service. A mature incident process looks like this:

  1. Detect. An alert or a user report. Measure how long this takes.
  2. Respond. Assign an incident lead, open a communication channel, and tell users what is known.
  3. Mitigate first, fix later. Roll back, switch a flag off or add capacity to stop the damage. Understanding the root cause can wait until users are safe.
  4. Recover and verify. Confirm the metrics are healthy again.
  5. Review. Hold a blameless postmortem: describe the timeline, the contributing causes and the follow-up actions. The goal is to fix the system, not to find a person to blame. People who fear blame hide mistakes, and hidden mistakes repeat (see working in a team).

Teams track a few numbers about reliability. Availability is the share of time the service works. MTTR (mean time to recover) is the average time from the start of an incident to recovery. Many teams also set a service level objective (SLO), for example "99.9% of requests succeed this month", and the gap between 100% and the objective is the error budget: the amount of failure the team may "spend" on risky releases.

JS Availability and error budget

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Software maintenance

Maintenance is every change made to software after delivery. It is usually split into four kinds:

Kind Purpose Example
Corrective Fix faults A rounding bug in invoices
Adaptive Keep up with a changing environment Support a new operating system or payment provider
Perfective Improve or extend A faster search, a new report
Preventive Make future change easier Refactoring, removing a dead library, adding tests

Most maintenance effort is perfective and adaptive, not bug-fixing. Preventive work is the easiest to postpone and the one that keeps the others affordable; see technical debt and refactoring.

Software evolution and Lehman's laws

In the 1970s and later, Meir Lehman and colleagues studied large long-lived systems and described regularities in how they change, now called Lehman's laws of software evolution. In paraphrase, the best known are:

  • Continuing change. A system used in the real world must keep changing, or it becomes less and less useful as the world moves on.
  • Increasing complexity. As a system changes, its structure tends to degrade and become more complex unless work is spent to reduce that.
  • Continuing growth. Functionality must keep growing to keep users satisfied.
  • Declining quality. Quality seems to decline unless the system is actively maintained and adapted.

These are observations from certain kinds of systems, not mathematical laws, but they match everyday experience. The practical message: a codebase that is only ever extended and never cleaned will slow the team down. Budget for refactoring the way you budget for features.

The stand-in below makes the "increasing complexity" law visible with a toy model. Each release adds features, which adds complexity; a team can spend part of its effort on cleanup, which removes some. All coefficients are invented for illustration.

JS Complexity growth with and without cleanup

Press Run (or Ctrl + Enter) to see the console output here.

No output.

With no cleanup the index keeps rising faster and faster. A steady cleanup share slows the climb (40% grows far less than 0%), at the cost of fewer new features per release, which is exactly the trade a team has to choose on purpose.

Legacy systems

A legacy system is an old system that still matters to the business but is hard to change: outdated technology, missing tests, no documentation, and often nobody left who understands all of it. "Legacy" does not mean "bad". It often means "valuable enough to have survived". Options for handling one are:

  1. Maintain as is. Fix only what must be fixed. Cheap now, riskier every year.
  2. Wrap. Put a modern interface (an API) around the old system so new code can use it without touching the inside.
  3. Refactor or modernize. Add tests around current behavior (characterization tests record what the code does today, right or wrong), then improve it in small safe steps.
  4. Replace incrementally. Build new functionality beside the old system and move traffic piece by piece (often called the strangler approach), retiring old parts as new ones take over.
  5. Rewrite from scratch. Tempting and often disastrous: the old code embodies years of hidden rules and bug fixes, and the business cannot pause while you rebuild.

End of life

Every system eventually stops being worth supporting. End of life (EOL) is the announced date after which a product or version gets no more fixes or security patches. A responsible retirement plan includes:

  • An early announcement with a date, and a clear migration path.
  • A period of read-only or limited operation, where appropriate.
  • Data export and archival, including legal retention requirements.
  • Shutting down integrations, accounts and servers so nothing forgotten keeps running (and keeps costing money or exposing risk).

The same applies to the dependencies you use. Frameworks and runtimes have EOL dates too, and running an unsupported version means no security fixes. Track them as part of maintenance.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the main advantage of a canary release over a big-bang release?

    A canary exposes the new version to a small slice of traffic first, which limits the blast radius.
  2. 2. During an incident, what should the team usually do first?

    Stop the damage first; the root cause can be investigated once users are safe.
  3. 3. A service has a 99.9% objective over a 30-day month (43,200 minutes). About how much downtime is allowed?

    0.1% of 43,200 minutes is 43.2 minutes.
  4. 4. Moving a service to a new operating system version is which kind of maintenance?

    Adaptive maintenance keeps software working as its environment changes.
  5. 5. Which approach to a legacy system is usually the riskiest?

    Old code holds years of hidden rules and fixes; rewriting it all at once usually takes longer and loses behavior.

Summary

  • Most software cost comes after the first release, so releasing, monitoring and maintaining matter as much as building.
  • Rolling, blue-green and canary releases and feature flags limit the blast radius of a bad change; keep rollback easy, including for data.
  • Monitor metrics, logs and traces; respond to incidents by mitigating first and holding blameless postmortems; track availability, MTTR and error budgets.
  • Maintenance is corrective, adaptive, perfective or preventive; Lehman's laws say a living system must keep changing and tends to grow more complex unless you clean it.
  • Legacy systems can be wrapped, modernized or replaced step by step; plan end of life with dates, migration and data retention.

Next: Project Management Basics.