pandas Tutorial
Learn pandas 3 from your first DataFrame to cleaning, grouping, joins, time series, visualisation, performance and real projects. 64 lessons with live examples, quizzes and projects.
Start learning →
of lessons completed
Getting Started
- What Is pandas? Learn what pandas is, what Series and DataFrame objects are, when to choose pandas over NumPy or SQL, and how to run every example in your browser.
- Installing pandas and Using Jupyter Set up pandas on your own computer: create a virtual environment, install pandas with pip or uv, start JupyterLab, and learn how notebook cells and kernels work.
- Series Learn how to create a pandas Series from lists, dicts and arrays, read its index, values and dtype, do vectorized math and understand index alignment.
- DataFrame Basics Create a pandas DataFrame from dicts, records and arrays, then inspect it with shape, dtypes, head, tail, info and describe. Includes live examples.
- Reading CSV Data Learn pandas read_csv: separators, headers, names, usecols, index_col, dtype, parse_dates, na_values, nrows, skiprows and reading big files in chunks.
- Reading Text and Fixed-Width Files Read tab-separated, whitespace-aligned and fixed-width text files with pandas read_table, read_fwf and read_csv options, and convert them to clean CSV.
- Writing and Formatting Data Export a pandas DataFrame with to_csv, to_json, to_dict, to_string and to_markdown, control formats, and check that data survives a round trip.
- Working with Excel Files Read and write Excel workbooks with pandas: read_excel options, several sheets, ExcelWriter, number formats, column widths, frozen headers and common pitfalls.
- JSON and Nested Data Turn JSON from APIs into tables with pandas read_json, json_normalize, record_path, meta, explode and flattening of nested dicts, step by step.
- Reading HTML, XML, Pickle, Clipboard, HDF5 and Parquet Read and write less common formats with pandas: HTML tables, XML, pickle, the clipboard, HDF5, Parquet and Feather, with trade-offs for size, speed and safety.
- pandas for Excel Users: VLOOKUP, Pivot, Filters and Formulas Translate what you know from Excel into pandas: VLOOKUP and XLOOKUP with merge, IF and SUMIF with where and groupby, filters, text to columns, sorting and pivot tables.
Selecting and Filtering
- Selecting Columns and Rows Pick columns with df["col"] and df[["a", "b"]], slice rows, preview with head and sample, reorder and drop columns, and avoid the attribute-access traps.
- loc and iloc Learn the difference between loc (labels) and iloc (positions), how slices behave, how to filter and assign with masks, and how at and iat read single cells.
- Filtering with Conditions Build boolean masks, combine them with &, | and ~, use isin, between, str.contains, isna and query, and learn where and mask for conditional replacement.
- Utility Methods: sample, squeeze, add_prefix, truncate, items, pop, insert Learn the small pandas helpers you reach for every week: sample, squeeze, add_prefix and add_suffix, truncate, items, itertuples, pop and insert, with runnable examples.
- Sorting and Ranking Sort DataFrames with sort_values and sort_index, use multiple keys and orders, handle missing values and ties, pick top rows with nlargest, and rank values.
- The Index Understand the pandas Index, use set_index and reset_index, name indexes, reindex and align data by label, deal with duplicate labels, and choose index or column.
- MultiIndex Create hierarchical indexes with set_index, from_tuples and from_product, select with loc, xs and slices, sort levels, swap them, and flatten back to columns.
Data Types
- Data Types and Conversion Learn pandas data types: int, float, bool, str, category and datetime. Convert columns with astype, to_numeric and convert_dtypes, and use nullable integers.
- Missing Data Find, count, drop and fill missing values in pandas. Compare NaN, NA and None, use dropna, fillna, ffill, bfill and interpolate, and avoid common traps.
- Working with Text Clean text columns in pandas with the .str accessor: lower, strip, replace, split, extract with regex, contains, cat and padding, plus missing-value pitfalls.
- Dates and Times Parse text with to_datetime, use the .dt accessor, do date arithmetic with Timedelta, build ranges with date_range, and understand periods and time zones.
- Categorical Data Use the pandas category dtype for repeated labels: save memory, define an order, rename and reorder categories, and handle unobserved categories in groupby.
Transforming Data
- Creating and Changing Columns Add and change columns with assignment, assign(), insert(), rename() and drop(), and learn how pandas 3 Copy-on-Write ends chained-assignment surprises.
- Vectorised Operations Learn vectorised pandas operations: column arithmetic, np.where and np.select, clip, round, cumsum, diff, pct_change, shift, and why Python loops are slow.
- apply, map and replace Learn when to use pandas Series.map, apply on columns and rows, replace and DataFrame.map, and why apply is often slower than a vectorised alternative.
- Binning and Discretising Learn pandas binning with cut and qcut: custom edges and labels, open and closed bin edges, equal-width versus equal-count bins, and counting values per bin.
- Normalizing and Scaling Columns Scale pandas columns with min-max, z-score, robust and log scaling, normalize within groups, turn values into shares and indexes, and avoid data leakage.
- Duplicates and Outliers Find and handle duplicate rows with duplicated and drop_duplicates, detect outliers with the IQR and z-score rules, then flag, clip or remove them on purpose.
Grouping and Reshaping
- groupby Basics Learn pandas groupby from scratch: split-apply-combine, grouping by one or many keys, size versus count, iterating groups, as_index, dropna and observed.
- agg, transform and filter Master groupby agg with named aggregation, transform for group-relative values such as shares and z-scores, filter groups, group ranks, and when to use apply.
- Pivot Tables and Crosstabs Summarise a DataFrame as a grid with pivot_table and crosstab. Learn aggfunc, margins, fill_value, normalize, and when pivot or unstack is the better tool.
- Reshaping: melt, stack, wide to long Change the shape of a DataFrame between wide and long with melt, stack, unstack and wide_to_long, then explode lists and one-hot encode with get_dummies.
Combining Data
- Concatenating DataFrames Combine DataFrames with pd.concat. Stack rows, join columns, handle differing columns with join modes, track the source with keys, and avoid appending in loops.
- Merge and Join Match rows from two DataFrames by key with merge and join. Learn inner, left, right and outer joins, suffixes, validate, indicator, and how to avoid row explosions.
- Advanced Merges Go beyond basic joins in pandas. Use merge_asof for nearest-time matches, merge_ordered, cross joins, anti-joins, self-joins, combine_first, update and compare.
Time Series
- Time Series Basics Learn the basics of time series in pandas. Use DatetimeIndex, select by date strings and slices, shift and diff, rolling date windows, PeriodIndex and frequency strings.
- Resampling and Time Windows Learn pandas resample() to turn daily data into weekly or monthly totals, fill gaps when upsampling, group with Grouper and handle business days.
- Rolling, Expanding and EWM Learn pandas rolling(), expanding() and ewm() to smooth noisy data, build moving averages, use time-based windows and avoid look-ahead mistakes.
- Time Zones Learn pandas time zones step by step. Localize naive timestamps, convert between zones, handle daylight saving gaps and store everything in UTC.
Statistics and Analysis
- Descriptive Statistics Summarize data with pandas: mean, median, std, quantile, mode, skew, describe(), value_counts, nunique, corr, cov and weighted averages, with live examples.
- Finding Correlation Measure how columns move together with pandas corr, corrwith and rank-based Spearman correlation, draw a correlation heatmap, and avoid the classic traps.
- Exploratory Data Analysis Workflow A repeatable pandas checklist for exploring new data: shape, dtypes, missing values, distributions, duplicates, outliers, correlations and a profile function.
- Data Cleaning in Practice Clean messy data with pandas: fix headers, whitespace, inconsistent categories, mixed types, bad dates and units, then chain the steps into a pipe() pipeline.
Visualisation
- Plotting with pandas Draw charts straight from a DataFrame with df.plot. Make line, bar, histogram, box, scatter, area and pie charts, add titles and labels, and use subplots.
- Charts for Analysis Use charts to answer questions about data. Draw distributions, grouped bars, rolling-mean trends, scatter plots with trend lines and correlation heatmaps.
- Seaborn Plots from DataFrames: Box, KDE, Pairplot and Heatmap Draw statistical charts straight from a pandas DataFrame with seaborn: box plots, KDE curves, pair plots and heatmaps, plus pandas' own boxplot and scatter_matrix.
- Styling and Displaying Tables Control how a DataFrame prints. Use display options, number formatting, to_string, option_context and the Styler class for conditional formatting.
Performance and Scale
- Memory Usage and Smaller Dtypes Measure how much memory a DataFrame uses and shrink it with smaller numeric dtypes, category columns, sparse arrays, usecols and chunked reading.
- Writing Fast pandas Code Make pandas code faster. Replace loops and iterrows with vectorised operations, avoid repeated concat, use query and eval, time code with perf_counter.
- When Data Gets Big Learn what to do when a DataFrame no longer fits in memory. Chunked processing, sampling, Parquet, PyArrow, Polars, DuckDB and Dask, and when to leave pandas.
Quality and Production
- Common pandas Pitfalls The pandas surprises that bite everyone: chained assignment, index alignment, object dtype, integer NaN, dropped groupby keys, inplace myths and float precision.
- Testing and Validating DataFrames Test pandas code with assert_frame_equal and assert_series_equal, tolerances, hand-written schema checks, small fixtures and property-style checks, plus a pytest example.
- Building Data Pipelines Build clean pandas pipelines: small functions that take and return DataFrames, pipe and method chaining, row-count logging, idempotent steps and configuration.
Projects
- Project: Sales Analysis Clean order data, compute revenue by month and category, rank top products, measure growth and chart it with pandas, then write findings that match the numbers.
- Project: Cleaning a Messy Survey Clean a messy survey with pandas: fix typos in categories, parse mixed dates and money, handle impossible ages, derive columns and print a validation report.
- Project: Sensor and Time-Series Report Turn gappy sensor readings into a report with pandas: resample to a regular grid, interpolate short gaps, flag anomalies with a rolling median and summarise by day.
- Project: Customer Segments with RFM Build an RFM customer segmentation in pandas: merge orders and customers, compute recency, frequency and monetary value, score with qcut and label segments.
- Project: Ratings and Recommendations Build a small movie recommender in pandas: ratings tables, pivot to a user-item matrix, fair popularity ranking, correlation-based similar items and top-N lists.
- Project: Passenger Survival Analysis Analyze a Titanic-style passenger table with pandas: audit missing values, fill ages by group, bin ages, compare survival rates with groupby and pivot tables, and report.
- Project: Analysing Used Car Selling Prices Clean a messy used-car listings file with pandas, explore prices with groupby and correlation, chart the drivers of price and estimate depreciation per year of age.
Practice and Review
- Practice Exercises with Solutions 25 graded pandas exercises with solutions on selecting, grouping, merging, time series and data cleaning. Practice in your browser with ready-made data.
- pandas Interview Questions 38 common pandas interview questions with concise answers and runnable examples: loc vs iloc, merge types, apply, copy-on-write, transform and memory.
- pandas Cheat Sheet A one-page pandas 3 cheat sheet with runnable examples: create, inspect, select, clean, group, merge, reshape, time series, plotting and performance.