Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 64 ยท Transforming Data

apply, map and replace

Learn when to use pandas Series.map, apply on columns and rows, replace and DataFrame.map, and why apply is often slower than a vectorised alternative.

Most column work in pandas needs no custom code: vectorised operations handle arithmetic, comparisons and cumulative values. Sometimes, though, you have a rule that is not built in: translate country codes to names, apply your own function to every value, or compute something from several columns of a row. For those cases pandas offers map, apply and replace.

They look similar, and people mix them up. This lesson shows what each one is for, how they differ, and, just as important, when to avoid them because a faster tool exists.

Series.map: translate values

Series.map transforms a Series value by value. Its argument can be:

  • a dictionary: each value is looked up as a key (values that are not in the dictionary become NaN),
  • a function: called once per value,
  • another Series, used like a dictionary.

The dictionary form is the standard way to translate codes into labels.

Python map with a dict and with functions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The missing country XX shows the main trap of dictionary mapping: anything you forgot silently turns into NaN. If you want unmatched values to stay as they were, use replace (below) instead, or fill afterwards as in the example.

The function form is called once per element, in Python. A function such as str.upper works, but pandas has dedicated, faster tools for that case, such as df["name"].str.upper() (see Working with Text).

Series.apply and DataFrame.apply

apply is the general-purpose tool. Its behaviour depends on what you call it on.

Call The function receives Result
series.apply(f) each single value a Series
df.apply(f) (default axis=0) each column as a Series one value per column, or a frame
df.apply(f, axis=1) each row as a Series one value per row

On a Series, apply and map do nearly the same thing when you pass a function. Use map for dictionaries and apply when the function needs extra arguments (apply(f, args=(...))).

flowchart LR
    subgraph C["axis=0 (default)"]
        direction TB
        C1["math column"] --> CF["f(column)"]
        C2["art column"] --> CF
        CF --> CR["one result per column"]
    end
    subgraph R["axis=1"]
        direction TB
        R1["row 0"] --> RF["f(row)"]
        R2["row 1"] --> RF
        RF --> RR["one result per row"]
    end

With axis=0 your function sees whole columns; with axis=1 it sees one row at a time.

Python apply on columns and on rows

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that df[subjects].apply(lambda col: ...) returned one number per column, and axis=1 returned one per row. When the function returns a Series, apply assembles the results into a DataFrame instead.

replace: swap specific values

replace swaps chosen values for others and, unlike map, leaves everything else alone. It accepts a single pair, a dictionary, two lists, or a regular expression with regex=True. It works on both Series and DataFrames.

Python replace values, sentinels and patterns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Compare with map: s.map({"a": "alpha"}) would turn b, c and d into NaN, while s.replace({"a": "alpha"}) keeps them. If you only want to change a few values, replace is usually the safer choice. Replacing -999 with NaN is a common first cleaning step; see Missing Data.

On a column, replace with a string pattern and regex=True treats the whole cell as the search text. For many text edits on a column, the .str.replace() accessor is clearer; it only works on strings and is covered in Working with Text.

DataFrame.map: every cell

DataFrame.map(f) calls f on every single cell of the frame and returns a frame of the same shape. It is the right tool for formatting or converting a whole table of values.

Python DataFrame.map for cell-by-cell changes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Older code and tutorials use DataFrame.applymap. It was renamed to DataFrame.map in pandas 2.1 and the old name is deprecated. Write map in new code.

When apply is the wrong tool

apply with a Python function runs that function once per element or per row, so it is essentially a loop in disguise. It has no special speed. Before reaching for it, check this list:

  1. Arithmetic or comparisons? Use column operators: df["a"] + df["b"].
  2. If/else on a condition? Use np.where or np.select.
  3. Text work? Use the .str accessor. Dates? Use .dt.
  4. Translating codes? Use map with a dict.
  5. Per-group results? Use groupby (groupby Basics).

Reserve apply for logic that truly needs Python: many branches, helper libraries, or calls into custom code.

Python apply versus vectorised, measured

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second comparison is the dramatic one. axis=1 makes pandas build a Series object for each row before calling your function, so it is typically the slowest approach in the whole library. If you must work row by row, selecting only the columns you need first (as df[subjects].apply(...) above) keeps the cost down.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What happens to values missing from the dictionary in s.map({'a': 1})?

    map looks every value up in the dictionary and gives NaN when the key is absent. replace is the method that leaves unmatched values alone.
  2. 2. What does df.apply(f, axis=1) pass to f?

    axis=1 means the function is called once per row, receiving that row as a Series indexed by the column names. The default axis=0 passes columns.
  3. 3. Which method calls a function on every cell of a DataFrame?

    DataFrame.map is cell by cell (it replaced the older applymap). Series.map is for one column.
  4. 4. You want to turn every -999 in a column into NaN and leave other values unchanged. Which is the best fit?

    replace changes only the values you name. The map version would turn every other number into NaN.
  5. 5. Why is df.apply(lambda row: row['a'] + row['b'], axis=1) a poor choice?

    The apply version runs Python once per row and builds a Series each time. The column expression is one vectorised operation.

Summary

  • Series.map(dict) translates values; anything not in the dictionary becomes NaN.
  • apply runs a function per value (Series), per column (axis=0) or per row (axis=1).
  • replace swaps only the values you name, with optional regex, and leaves the rest alone.
  • DataFrame.map applies a function to every cell and replaces the deprecated applymap.
  • apply is a Python loop in disguise: prefer vectorised arithmetic, np.where/np.select, .str, .dt or groupby when they fit.

Next: Binning and Discretising, which turns numbers into labelled ranges.