Skip to content
Tutorials Center
My learning

Topics

Lesson 63 of 64 ยท Practice and Review

pandas Interview Questions

38 common pandas interview questions with concise answers and runnable examples: loc vs iloc, merge types, apply, copy-on-write, transform and memory.

pandas interviews, whether for data analyst, data scientist or backend roles, check two things: whether you can manipulate tables correctly, and whether you can explain why a piece of code behaves the way it does. This page collects 38 questions that come up again and again, grouped by topic, with short answers you can expand in your own words and eight small code-reading puzzles you can run.

Try to answer each question out loud before opening it. If you cannot explain an idea in two or three sentences, follow the link to its lesson and review it. For the code-reading puzzles, decide what the output will be first, then run the editor.

Tip: in an interview, a short correct answer plus a tiny example beats a long recital. Name the trade-off ("this is vectorized and fast, but it needs a numeric column") and mention what you would check on real data: row counts, missing values and dtypes.

flowchart TD
    Q{"After grouping,<br/>what shape do you need?"}
    Q -- "one row per group" --> A["agg"]
    Q -- "same length as the input" --> T["transform"]
    Q -- "keep or drop whole groups" --> F["filter"]
    Q -- "anything else" --> P["apply (last resort)"]

Choosing between agg, transform, filter and apply is one of the most common interview topics: ask for the output shape first.

Core objects and indexing

What is the difference between a Series and a DataFrame?

A Series is a one-dimensional labeled array with a single dtype. A DataFrame is a two-dimensional table of Series that share one index, and each column can have its own dtype. Selecting one column of a DataFrame gives a Series. See Series and DataFrame Basics.

What is the index, and why does it matter?

The index holds the row labels. It is used for selection with loc, for fast lookups, and above all for alignment: when you combine Series or DataFrames with arithmetic, pandas matches values by label, not by position. A good index (unique, meaningful, sorted when you slice) makes code shorter; a bad one causes surprising NaNs. See The Index.

What is the difference between loc and iloc?

loc selects by label and its slices include the end label. iloc selects by integer position and its slices exclude the end, like normal Python. Both accept scalars, lists, slices and boolean arrays. On a table with a default RangeIndex the two look similar, which hides bugs until the index changes after a filter or sort. See loc and iloc.

Code reading: three ways to index

What does this print? Remember that the index here is not 0, 1, 2...

Python loc, iloc and plain brackets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

It prints ['b', 'c', 'd'] for the label slice (end included), ['b', 'c'] for the position slice (end excluded), and KeyError 1: with an integer index, plain brackets look up the label 1, which does not exist. Always say loc or iloc to make your intent clear.

Why do you write (a > 1) & (b < 3) instead of a > 1 and b < 3?

and, or and not ask Python for a single True/False value, and a whole Series has no single truth value, so pandas raises a ValueError. Use the element-wise operators &, | and ~. They bind tighter than comparisons, so each comparison needs its own parentheses. See Filtering with Conditions.

How do you take a first look at a new DataFrame?

df.shape, df.head(), df.dtypes and df.info() for structure; df.describe() for numeric ranges; df.isna().sum() for missing values; df.nunique() and df["col"].value_counts() for categories; and df.duplicated().sum() for repeated rows. See the exploratory analysis workflow.

What is the difference between value_counts, nunique and unique?

unique() returns the distinct values, nunique() returns how many there are, and value_counts() returns how often each occurs, sorted by frequency. value_counts(normalize=True) gives proportions, and dropna=False counts missing values too.

Types and missing data

What are the main dtypes, and what changed in pandas 3?

Numeric (int64, float64), bool, datetime64, timedelta64, category, and the nullable types (Int64, boolean, string). In pandas 3 text columns get the dedicated str dtype by default instead of the old catch-all object, so string operations are faster and type checks are clearer. object still exists for mixed content. See Data Types and Conversion.

What is the difference between NaN, None and pd.NA?

NaN is a floating-point "not a number" and is what numeric and str columns use for missing values. None is Python's null object; pandas usually converts it to NaN in numeric columns. pd.NA is the newer, dtype-independent missing marker used by nullable dtypes such as Int64. Missing values never compare equal, not even to themselves, so use isna() and notna(), never == np.nan. See Missing Data.

Code reading: missing values

Python Integers with holes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

float64, Int64, [1, 2, <NA>], False, [False, True]. A plain integer column cannot hold NaN, so pandas silently converts it to floats. The nullable Int64 dtype keeps integers and uses pd.NA. And np.nan == np.nan is False, which is why isna() exists.

How do you handle missing values?

First find out why they are missing, then choose. Drop them (dropna, with subset and how) when few rows are affected; fill them with a constant, a statistic or a group statistic (fillna, groupby().transform("median")); carry values forward or backward in time series (ffill, bfill) or interpolate; or keep them and use a nullable dtype. Document the decision, because every choice changes the statistics.

How do you convert a messy column to numbers?

Clean the text (.str.replace, .str.strip) and then call pd.to_numeric(col, errors="coerce"), which turns unparseable values into NaN so you can count and inspect them. astype(float) raises an error on the first bad value, which is sometimes exactly what you want in a validated pipeline.

When should you use the category dtype?

For columns with few distinct values repeated many times (country, status, plan). It stores each distinct value once plus small integer codes, which saves memory and speeds up grouping. It also supports a custom order for sorting. It is not worth it for mostly unique text. See Categorical Data.

Transforming data

Why are loops and apply slower than vectorized operations?

A vectorized expression like df["a"] * df["b"] runs one compiled loop in C over the whole array. apply and iterrows call a Python function once per element or row, paying interpreter overhead each time; the difference is often 10x to 1000x. Prefer built-in methods, np.where, np.select and the .str and .dt accessors. See Vectorised Operations.

What is the difference between map, apply and replace?

Series.map transforms values with a function or a dict (unmatched keys become NaN). apply runs a function over a Series, over each column, or over each row (axis=1). replace swaps specific values and leaves everything else unchanged. DataFrame.map applies a function to every cell. See apply, map and replace.

How do you create a column based on conditions?

For two outcomes use np.where(condition, a, b). For several, use np.select([cond1, cond2], [value1, value2], default=...), which returns the first matching value. For ranges of numbers use pd.cut or pd.qcut. Avoid apply(axis=1) with an if/else; it does the same job much more slowly.

What is the difference between cut and qcut?

cut makes bins of equal width (or at edges you choose); qcut makes bins with (roughly) equal numbers of rows, using quantiles. Use cut for meaningful boundaries such as age groups and qcut for ranks such as quintiles. See Binning and Discretising.

How do you find and remove duplicates?

df.duplicated(subset=..., keep=...) marks repeated rows and drop_duplicates removes them. keep="first" keeps the first occurrence, "last" the last, and False marks all copies. To keep the most recent record per key, sort by the timestamp first and use keep="last". See Duplicates and Outliers.

Copy-on-write and common traps

What is Copy-on-Write, and what does it change in pandas 3?

With Copy-on-Write (CoW), always on in pandas 3, every DataFrame or Series that derives from another one behaves as an independent copy: pandas shares the memory lazily and copies only when one side is modified. The result is predictable semantics: changing a derived object never changes its parent, and there are no more views-versus-copies surprises. See Creating and Changing Columns.

Code reading: modifying a derived Series

Python Does the original change?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

It prints [1, 2, 3] [99, 2, 3] and then [10, 0, 0]. Under Copy-on-Write, col is independent of df, so changing it leaves the DataFrame alone. To really change the DataFrame, assign through df itself: df.loc[mask, "b"] = 0 sets the values in one step.

Why does df[mask]["b"] = 0 not work, and what was SettingWithCopyWarning?

df[mask] creates an intermediate object, and with CoW that object is always a copy, so the second assignment modifies the copy and the original never changes. pandas 3 warns about this chained assignment. In older versions the same line sometimes worked and sometimes did not, and SettingWithCopyWarning tried to flag the unreliable cases. The fix has always been a single loc assignment. See Common pandas Pitfalls.

Is inplace=True faster or better?

No. In most cases inplace=True still creates a new object internally and then swaps it in, so it saves nothing, and it prevents method chaining. It is being de-emphasized by the pandas team. Assign the result instead: df = df.dropna().

Do you need df.copy() with Copy-on-Write?

Rarely. Because derived objects already behave as copies, the defensive .copy() calls that were once needed to avoid warnings are no longer necessary. Use copy() when you want a clear independent snapshot, for example before passing a frame to code you do not trust.

Code reading: index alignment

Python Adding two Series

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

The first sum matches by label: y is 12, z is 23, and x and w are NaN because each exists on one side only (the union of the labels is sorted: w, x, y, z). add(b, fill_value=0) treats a missing side as 0, giving w 30 and x 1. The last line strips the labels with to_numpy(), so values are added by position and the result is 11, 22, 33 on the labels of a. Alignment is helpful until you forget it exists.

Grouping and reshaping

Explain groupby in one sentence, and the difference between size and count.

groupby splits the rows by key, applies a function to each group and combines the results (split-apply-combine). size() counts rows per group including missing values; count() counts non-missing values per column. See groupby Basics.

What is the difference between agg, transform, filter and apply?

agg returns one row per group. transform returns a result with the same length as the input, so you can add it as a column (for example a group mean or a share of the group total). filter keeps or drops whole groups based on a condition. apply takes a function that sees each group as a DataFrame and can return anything; it is the most flexible and the slowest, so use it last. See agg, transform and filter.

Code reading: agg versus transform

Python Totals per group

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

[3, 10] has one value per group, while [3, 3, 10] repeats each group's total on every row of that group, so it lines up with d. That is what makes the share column possible: 0.33 and 0.67 for group a, and 1.0 for the single row of b.

Why do rows with a missing key disappear from a groupby, and how do you keep them?

By default groupby drops groups whose key is NaN, which can silently lose data from totals. Pass dropna=False to keep them as their own group. Always compare the grand total before and after grouping.

What is the difference between pivot and pivot_table?

pivot only reshapes: it needs each (index, column) pair to be unique and raises an error otherwise. pivot_table aggregates duplicates with aggfunc (default mean) and supports margins, fill_value and several value columns. It is groupby plus unstack. See Pivot Tables and Crosstabs.

What are wide and long formats, and how do you convert between them?

Wide data has one column per measurement (jan, feb, mar); long (tidy) data has one row per observation with columns such as month and value. Long data is easier to group, filter and plot. melt goes from wide to long; pivot or unstack go back. See Reshaping.

What does get_dummies do?

It turns a categorical column into one 0/1 (or True/False) column per category, known as one-hot encoding, which many machine learning models require. Use drop_first=True to avoid perfectly redundant columns in linear models.

Combining data

Explain the merge types.

An inner merge keeps keys found in both tables. Left keeps all rows of the left table and fills missing right-hand values with NaN; right is the mirror image. Outer keeps everything from both. A cross join pairs every row with every row. Choose by asking which table is the "truth" you must not lose rows from. See Merge and Join.

Code reading: row counts after a merge

Python How many rows?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

inner 4, left 5, right 5, outer 6. Key 1 appears twice on each side, so it produces 2 x 2 = 4 rows (a many-to-many match). Key 2 exists only on the left and key 3 only on the right. Inner: 4. Left adds the unmatched 2: 5. Right adds the unmatched 3: 5. Outer adds both: 6.

How do you avoid duplicated rows after a merge?

Check key uniqueness before merging and pass validate="one_to_one", "one_to_many" or "many_to_one" so pandas raises an error when the assumption is wrong. Compare row counts before and after, and use indicator=True to see which rows matched.

What is the difference between merge, join and concat?

merge combines tables by matching column values (database-style). join is a shortcut that matches on the index by default. concat stacks tables along an axis without matching keys: rows on top of each other (axis=0) or columns side by side aligned on the index (axis=1). See Concatenating DataFrames. Note that DataFrame.append was removed in pandas 2.0; use concat.

When would you use merge_asof?

When keys will not match exactly but you want the nearest earlier (or later) match, such as attaching the latest price quote to each trade by timestamp. Both tables must be sorted by the key. See Advanced Merges.

Time series

How do you work with dates in pandas?

Parse with pd.to_datetime (pass format= to avoid ambiguity), use the .dt accessor for parts such as year or weekday name, put the dates in the index for slicing like ts["2025-03"], and do arithmetic with Timedelta. See Dates and Times.

What is the difference between resample, rolling and groupby on dates?

resample groups by calendar periods (days, weeks, months) and aggregates each, so the output has fewer rows. rolling slides a fixed window over the data and returns a value for every row. groupby with dt parts (such as dt.month) groups all rows with the same part, even from different years. See Resampling and Rolling Windows.

What is the difference between naive and time-zone-aware timestamps?

A naive timestamp has no zone attached and cannot be compared with an aware one. Aware timestamps know their zone, handle daylight saving time, and convert with tz_convert. A common practice is to store everything in UTC and convert only for display. Attach a zone to naive data with tz_localize. See Time Zones.

Performance and scale

How do you reduce the memory usage of a DataFrame?

Measure first with df.memory_usage(deep=True). Then read only the columns you need (usecols), choose smaller numeric types (pd.to_numeric(..., downcast="integer")), convert repeated text to category, and drop columns you no longer use. See Memory Usage and Smaller Dtypes.

Code reading: does category really save memory?

Python Category versus text

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

It prints str category, category is smaller: True and int8. For two distinct values repeated 2,000 times the category column stores two strings plus 2,000 one-byte codes. The exact byte counts depend on the environment, so compare rather than quote them. Values 1 to 3 fit in one byte instead of eight.

Why should you avoid iterrows?

iterrows builds a Series for every row, which is slow and can change dtypes (an integer next to a float turns into float). Prefer vectorized expressions; if you must loop, itertuples is much faster, and zip over the columns is faster still. See Writing Fast pandas Code.

How would you process a CSV that does not fit in memory?

Read it in chunks with pd.read_csv(..., chunksize=...), aggregate each chunk and combine the partial results; load only needed columns with usecols and compact dtypes with dtype. For repeated analysis, convert to a columnar format such as Parquet. If it is still too big, consider Polars, DuckDB or Dask. See When Data Gets Big.

How do you compare two DataFrames in tests?

Use pandas.testing.assert_frame_equal(left, right), which checks values, dtypes, index and column labels and gives a readable diff; pass check_exact=False or a tolerance for floats. Comparing with == returns a frame of booleans, and NaN never equals NaN. See Testing and Validating DataFrames.

What is the difference between query and boolean indexing?

They select the same rows. df.query("age > 30 and city == 'Paris'") takes a string that is easier to read in long chains and can use @variable for local variables; boolean indexing (df[(df.age > 30) & (df.city == "Paris")]) works with any expression and is easier to debug and to type-check. Choose by readability.

Code reading: groupby and missing keys

Python The vanishing group

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Show the answer

The first line is {'a': 4, 'b': 4}: the row with the missing key (value 2) is gone, so the groups add up to 8 while the real total (last line) is 10. With dropna=False the missing key becomes its own group, with a total of 2. Reconcile grouped totals against the grand total whenever the key may contain gaps.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does s.loc[20:40] do on a Series whose index is [10, 20, 30, 40, 50]?

    loc slices by label and includes the end label, so 20, 30 and 40 are returned. iloc would use positions and exclude the end.
  2. 2. Under Copy-on-Write in pandas 3, what happens when you run df[df['a'] > 1]['b'] = 0?

    The first brackets create an intermediate object that behaves as a copy, so the assignment never reaches df. Use df.loc[df['a'] > 1, 'b'] = 0 instead.
  3. 3. You want each row to show its group's total, for example to compute a share. Which tool fits?

    transform returns a result with the same length and index as the input, so it can be added back as a column. agg gives one row per group.
  4. 4. Which merge keeps every row of the left table, whether or not it has a match?

    A left merge keeps all rows of the left table and fills the right-hand columns with NaN where nothing matches. Inner drops unmatched rows on both sides.
  5. 5. Why is vectorized code usually much faster than apply with a Python function?

    Built-in vectorized operations loop in compiled code, while apply pays the Python function-call overhead for each element or row.

Summary

  • Know the vocabulary: Series, DataFrame, index, alignment, dtype, missing values, and the difference between loc (labels, end included) and iloc (positions, end excluded).
  • Prefer vectorized operations; know why apply and iterrows are slow and what to use instead.
  • pandas 3 uses Copy-on-Write: derived objects are independent copies, chained assignment never updates the original, and inplace=True buys nothing.
  • For groups, ask for the output shape: agg (one row per group), transform (same length), filter (whole groups), apply (last resort).
  • Check row counts and totals after every merge and groupby, and use validate=, indicator= and dropna=False to catch silent loss or duplication.
  • Measure memory and speed before optimizing, and say what you would do when the data stops fitting in memory.

Finally, collect the most useful snippets on one page: the pandas Cheat Sheet.