Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 32 ยท Getting Started

The Runnable Interface: invoke, stream, batch

Learn the Runnable interface that every LangChain component shares: invoke, batch, stream and their async versions, plus config, with a runnable stand-in class.

Prompt templates, chat models, parsers, retrievers and whole chains all look different, yet you call them the same way. That is because they implement Runnable, the central interface of langchain-core. Learn it once and you know how to run every component in the library. It is also the reason the pipe syntax in the LCEL lesson works.

The six methods

A Runnable takes one input and produces one output, and offers these ways to run:

Method Use it to Returns
invoke(input) Run once and wait The output
batch([in1, in2, ...]) Run many inputs in parallel threads A list of outputs, same order
stream(input) Get output piece by piece as it is produced An iterator of chunks
ainvoke, abatch, astream The same three, as async coroutines Awaitables or async iterators

There is also astream_events for fine-grained progress events from inside a chain, which you meet in the evaluation chapter.

Python
# Real LangChain (read-only: needs a provider)
model = init_chat_model("openai:gpt-4o-mini")

model.invoke("Hi")                                  # one AIMessage
model.batch(["Hi", "Bonjour", "Hola"])              # three AIMessages, concurrently
for chunk in model.stream("Write a haiku."):        # AIMessageChunk objects
    print(chunk.content, end="", flush=True)

# inside async code
reply = await model.ainvoke("Hi")

Streaming matters for user experience: a chat interface that shows words as they arrive feels fast even if the full answer takes ten seconds.

Input and output types

Each Runnable declares what it accepts and returns:

Component Input Output
Prompt template dict of variables PromptValue (list of messages)
Chat model PromptValue, str or list of messages AIMessage
Output parser AIMessage or str Parsed value (str, dict, a Pydantic object ...)
Retriever str query List of Document
Tool dict of arguments Tool result

When you chain Runnables, each output becomes the next input, so types must line up. Most "my chain crashes" errors are a mismatch here: a prompt expecting a dict receiving a string, for example.

The optional config argument

Every method takes a second argument, config, a dictionary that travels down through a whole chain:

Python
chain.invoke(
    {"topic": "tides"},
    config={
        "tags": ["tutorial"],              # labels for filtering traces
        "metadata": {"user_id": "u-42"},   # free-form data for traces
        "run_name": "explain-topic",       # name shown in LangSmith
        "max_concurrency": 4,              # for batch
    },
)

You can ignore config at first. It becomes the control panel for tracing, concurrency, callbacks and per-run settings.

Building a Runnable from scratch (runnable)

A real Runnable is an abstract class that requires one method, invoke, and supplies batch and stream defaults built on it. This stand-in shows the pattern with a few lines of plain Python.

Python Stand-in: a minimal Runnable base class

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The real interface works the same way: a component must define the one-shot behavior, and gets batching and streaming for free; components that can truly stream (chat models) override stream to yield tokens as they come.

Wrapping a function with RunnableLambda

You rarely subclass Runnable. To plug any Python function into a chain, wrap it:

Python
from langchain_core.runnables import RunnableLambda

shout = RunnableLambda(lambda s: s.upper())
shout.invoke("hello")        # "HELLO"
shout.batch(["a", "b"])      # ["A", "B"]

A function decorated with @chain does the same thing. Anything that is a Runnable can be composed with |, given retries, traced and streamed.

A stand-in with logging and config (runnable)

This second stand-in adds a config argument that carries a run name down to the step, the way tracing metadata flows through a real chain.

Python Stand-in: passing config through a step

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(Timings print as tiny numbers and vary run to run; only the structure matters.)

Streaming and why not everything streams

Not every step of a chain can produce partial output. A prompt template finishes instantly, a model streams tokens, and a parser such as the JSON parser can stream partial objects. A step that needs the whole input before it can output anything, like a sorter, blocks streaming for everything after it. When you design chains for chat interfaces, keep the final steps stream-friendly.

flowchart LR
    A["Input dict"] --> B["Prompt<br/>instant"] --> C["Model<br/>streams tokens"] --> D["Parser<br/>passes chunks on"] --> E["UI shows text as it arrives"]

Chunks flow through the pipeline as soon as the model produces them.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which method runs a Runnable on many inputs concurrently?

    batch takes a list of inputs and returns a list of outputs in the same order, running them in parallel.
  2. 2. What is the main reason to use stream() in a chat application?

    Streaming shows partial output early; total generation time is about the same.
  3. 3. What should you do to use a normal Python function inside a LangChain pipeline?

    RunnableLambda turns a function into a Runnable, so it gains invoke, batch, stream and composition.
  4. 4. What does the config argument carry?

    Config is a dict passed down through the whole chain, used for tracing, callbacks and concurrency.

Summary

  • Every LangChain component is a Runnable with invoke, batch, stream and async twins.
  • Output of one Runnable becomes input of the next, so input and output types must match.
  • RunnableLambda (or @chain) wraps plain functions.
  • config carries tags, metadata, run name and concurrency through a chain.
  • Streaming works best when every later step can pass chunks along.

Next: Chat Models in Depth