Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 64 ยท Selecting and Filtering

Utility Methods: sample, squeeze, add_prefix, truncate, items, pop, insert

Learn the small pandas helpers you reach for every week: sample, squeeze, add_prefix and add_suffix, truncate, items, itertuples, pop and insert, with runnable examples.

Besides the big topics like groupby and merge, pandas has a drawer full of small tools. Each one solves a tiny, specific problem: take a random sample, turn a one-column table into a plain Series, label columns before a join, cut a sorted table at two labels, loop over columns, pull a column out, or put a new one in at an exact position. You can live without them, but then you write five lines where one would do.

This lesson collects them in one place. All of them return new objects (except pop and insert, which change the DataFrame, and the lesson says so loudly). It builds on Selecting Columns and Rows and Selecting with loc and iloc.

We will use one small table of cities for the examples.

Python The example table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

sample: random rows

df.sample(n=3) returns three random rows; df.sample(frac=0.5) returns half of them. Without random_state the result differs on every run, so pass a number whenever you want a reproducible result (in tests, tutorials and reports).

Python Random samples

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Four options cover almost every use:

  • frac=1 is the standard way to shuffle a table. Add .reset_index(drop=True) for a clean index.
  • replace=True samples with replacement, which is what bootstrapping needs.
  • weights="column" makes rows with bigger weights more likely. Without replace=True the weights must allow the sample size you ask for (one very heavy row cannot be drawn five times), otherwise pandas raises a ValueError.
  • axis=1 samples columns instead of rows.

squeeze: from table to scalar

A DataFrame with one column is still a table, and a Series with one value is still a Series. squeeze() removes the dimensions of length one, so you get the simplest object: a Series from a one-column frame, or a plain number from a one-value Series.

Python Squeezing dimensions away

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

squeeze only reduces length-one dimensions, so it is safe when you are sure there is exactly one match, and it quietly returns a Series if there are more. If you need a guarantee, check len(result) == 1 or use .item(), which raises an error unless there is exactly one value.

add_prefix and add_suffix

These add text to every column label (or to the index labels of a Series). Their main job is to avoid name clashes before combining tables, and to mark where columns came from.

Python Labelling columns before a join

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note that add_prefix touches every column, including keys, so set the key as the index first (or rename it back, as the first example does). For a join with clashing names, suffixes=("_a", "_b") in merge is often simpler.

truncate: cut a sorted index

truncate(before=..., after=...) keeps the rows between two index labels (inclusive). It requires a sorted index, and it is mostly used with dates. For anything else, .loc[start:end] does the same and is the more common spelling.

Python Truncating by labels

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Looping: items, iterrows and itertuples

Most of the time you should not loop over a DataFrame at all, because vectorised code is far faster (see Fast pandas). When you do need a loop, for output, for calling an outside function or for a quick check, pick the right tool:

Method Gives you Notes
df.items() (column name, Series) pairs loops over columns
df.iterrows() (index, Series) pairs slow; each row becomes a Series, so dtypes can be upcast
df.itertuples() one named tuple per row much faster, keeps dtypes
Python Three ways to loop

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the type of rank: iterrows converts the whole row into one Series, which has one dtype, so the integer is turned into a float to match the float column next to it. itertuples keeps each value's own type. Prefer itertuples for row loops.

pop and insert: move a column out and in

df.pop("col") removes a column from the DataFrame itself and returns it as a Series. It is the tidy way to take out a target column: y = df.pop("price") leaves the features in df. df.insert(loc, "name", values) puts a new column at a given position (0 is the first column) and also changes df in place. A plain df["new"] = ... always appends at the end.

Python pop and insert change the frame in place

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both are rare in clean pipelines, which usually prefer methods that return new objects, such as assign and drop. They are handy in notebooks and for column order. To reorder columns without them, use df[["city", "pop_m", "area_km2"]].

Tip: pop also works on dictionaries and lists, and DataFrame pop raises KeyError for a missing column, so a typo is caught immediately.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does df.sample(frac=1, random_state=0) do?

    frac=1 means 100 percent of the rows, so the result is the whole table shuffled. random_state makes the shuffle repeatable.
  2. 2. What does squeeze() return for a Series with exactly one value?

    squeeze removes dimensions of length one, so a one-value Series becomes a scalar.
  3. 3. Which statement about pop and insert is correct?

    pop removes a column from the frame itself and returns it; insert adds a column to the frame itself at a position and returns None.
  4. 4. Why is itertuples usually better than iterrows for looping over rows?

    iterrows builds a Series for every row, which is slow and may upcast mixed dtypes. itertuples yields lightweight named tuples that keep the types.
  5. 5. What does cities.truncate(before=20, after=40) require?

    truncate selects by index labels and needs the index to be sorted (increasing or decreasing). It works for numbers, strings and dates.

Summary

  • sample(n=..., frac=..., random_state=..., replace=..., weights=...) draws random rows; frac=1 shuffles.
  • squeeze() turns a one-column DataFrame into a Series and a one-value Series into a scalar.
  • add_prefix and add_suffix label every column; set key columns as the index first.
  • truncate(before, after) cuts a sorted index; .loc[a:b] is the usual alternative.
  • Prefer vectorised code, but for loops use items (columns) or itertuples (rows) over iterrows.
  • pop and insert change the DataFrame in place, taking a column out or placing it at a position.

Next: Boolean Filtering shows how to keep only the rows that match a condition.