Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 32 ยท Chains and LCEL

Retries, Fallbacks and Config

Make chains reliable with with_retry and with_fallbacks, adjust behavior per call with configurable fields and bind, and observe runs with callbacks and tags, plus stand-ins.

A chain that works in a demo meets rate limits, timeouts, malformed replies and provider outages in production. LCEL gives every Runnable a set of reliability wrappers: with_retry, with_fallbacks, bind, configurable_fields and with_config. Each returns a new Runnable, so you can attach them to any step and keep composing. This lesson covers them with runnable stand-ins of the underlying behavior.

with_retry

with_retry() re-runs a step when it raises an exception, usually with exponential backoff and jitter:

Python
robust_model = model.with_retry(
    retry_if_exception_type=(TimeoutError, ConnectionError),
    wait_exponential_jitter=True,
    stop_after_attempt=4,
)
chain = prompt | robust_model | parser

Retry only errors that are worth retrying: network failures and rate limits, not a ValueError that will fail the same way every time. Retrying a whole chain is also valid, for example to re-run a parser failure:

Python
chain.with_retry(stop_after_attempt=3)

Note that this runs the model again, and you pay for each attempt.

with_fallbacks

A fallback is a different Runnable to try when the first fails: another provider, a cheaper model, a cached answer.

Python
primary = init_chat_model("openai:gpt-4o")
backup = init_chat_model("anthropic:claude-sonnet-4-5")

model = primary.with_fallbacks([backup])

If primary raises, LangChain calls backup with the same input. List several fallbacks in order. You can restrict which errors trigger them with exceptions_to_handle, and put fallbacks on a whole chain, for example a strict structured-output chain that falls back to a looser one.

Combining both is typical: retry the primary a couple of times, then fall back.

Python
model = primary.with_retry(stop_after_attempt=2).with_fallbacks([backup])
flowchart LR
    I["Input"] --> P["Primary model<br/>retry x2"]
    P -->|"success"| O["Output"]
    P -->|"still failing"| B["Fallback model"]
    B --> O

Retry handles brief blips; a fallback handles an outage.

The two ideas in plain Python (runnable)

This stand-in builds both wrappers as functions around a fake flaky service, so you can watch the order of events.

Python Stand-in: retry, then fall back

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The log shows two primary attempts, then the backup, the same sequence as the real chain.

bind: fixing call arguments

bind(**kwargs) returns a Runnable with extra keyword arguments pre-applied to every call. Common uses are stop sequences and tool definitions:

Python
short_model = model.bind(max_tokens=50, stop=["\n\n"])
tool_model = model.bind_tools([get_weather])      # shown in the agents chapter

A Python analogue is functools.partial.

Python Real functools.partial: the idea behind bind

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Configurable fields and alternatives

Sometimes you want one chain whose model or temperature can change per request, driven by the caller or a config file, without rebuilding the chain. LangChain's configurable features do that:

Python
from langchain_core.runnables import ConfigurableField

model = init_chat_model("openai:gpt-4o-mini", temperature=0).configurable_fields(
    temperature=ConfigurableField(id="temp", name="Sampling temperature"),
)
chain = prompt | model | StrOutputParser()

chain.invoke({"topic": "tides"})                                          # temperature 0
chain.invoke({"topic": "tides"}, config={"configurable": {"temp": 0.9}})  # temperature 0.9

configurable_alternatives swaps whole components, such as a different provider's model, by a key:

Python
model = ChatAnthropic(model="claude-sonnet-4-5").configurable_alternatives(
    ConfigurableField(id="llm"),
    default_key="anthropic",
    openai=ChatOpenAI(model="gpt-4o-mini"),
)
chain.invoke({...}, config={"configurable": {"llm": "openai"}})

This is handy for A/B tests and for letting users choose a model, while keeping a single chain definition.

Python Stand-in: per-call configuration

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Callbacks, tags and metadata

Callbacks are hooks that fire as a chain runs: when a step starts, ends, errors, or a model emits a token. LangSmith tracing is itself a callback. You can pass your own through config:

Python
from langchain_core.callbacks import BaseCallbackHandler

class PrintHandler(BaseCallbackHandler):
    def on_llm_start(self, serialized, prompts, **kwargs):
        print("LLM called with", len(prompts), "prompt(s)")
    def on_llm_end(self, response, **kwargs):
        print("LLM finished")

chain.invoke({"topic": "tides"}, config={"callbacks": [PrintHandler()],
                                         "tags": ["demo"], "metadata": {"user": "u1"}})

Use callbacks for logging, metrics (latency, tokens) and streaming to a UI. Tags and metadata make runs filterable in LangSmith.

Python Stand-in: a callback handler observing steps

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reliability checklist

  • Set timeout and max_retries on models; add with_fallbacks for outages.
  • Make failures visible: trace with LangSmith or callbacks, never swallow exceptions silently.
  • Retry only transient errors, and cap attempts to bound cost.
  • Keep a deterministic fallback (a canned message) for the case where everything fails.
  • Test the failure paths on purpose by raising errors from fake models, the way this lesson does.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does model.with_fallbacks([backup]) do?

    The fallback is tried with the same input only when the primary fails.
  2. 2. Which errors should with_retry normally retry?

    Retrying a deterministic error just wastes money; transient failures can succeed on a retry.
  3. 3. What is bind(max_tokens=50) used for?

    bind pre-fills call arguments, similar to functools.partial.
  4. 4. How do you change a configurable field for one call only?

    The configurable dict in config overrides the field for that single run.

Summary

  • with_retry re-runs a step on transient errors; with_fallbacks tries alternative Runnables.
  • Combine them: retry the primary, then fall back to another provider.
  • bind pre-applies call arguments; configurable_fields and configurable_alternatives change behavior per call.
  • Callbacks, tags and metadata observe runs and feed tracing.
  • Test failure paths deliberately.

Next: Retrieval-Augmented Generation: The Big Picture