Lesson 16 of 64 ยท Selecting and Filtering
Sorting and Ranking
Sort DataFrames with sort_values and sort_index, use multiple keys and orders, handle missing values and ties, pick top rows with nlargest, and rank values.
Sorting puts rows in a meaningful order: newest first, highest score first, alphabetical by name. Ranking answers a related question: where does each row stand compared with the others? pandas has a small, consistent set of tools for both, and a few details (missing values, ties, stability, text case) are worth knowing before they surprise you.
In this lesson you will sort by one or several columns, sort by the index, take the top or bottom rows quickly with nlargest and nsmallest, assign ranks with different tie rules, and use the key= argument for custom orderings. It continues from Filtering with Conditions.
sort_values: order by a column
sort_values(by=...) returns a new DataFrame ordered by one or more columns. The default is ascending; pass ascending=False for the reverse. Notice that every row keeps its original index label, so you can still tell where it came from.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
Do not rely on inplace=True. Assigning the result (players = players.sort_values(...)) is just as fast and chains better.
Several keys and mixed directions
Pass a list to by to break ties: rows are ordered by the first key, and rows equal in the first key are ordered by the second, and so on. ascending can also be a list, one entry per key.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
flowchart TD
A["Rows"] --> B["Sort by first key"]
B --> C{"Equal first keys?"}
C -->|"No"| D["Order is decided"]
C -->|"Yes"| E["Compare second key"]
E --> F["Still equal: keep original order<br/>(if the sort is stable)"]Later keys only matter among rows that tie on the earlier ones.
Missing values and the sort kind
Missing values (NaN) go to the end by default, whatever the direction. Change that with na_position="first".
A stable sort keeps equal rows in their original relative order. For a single key, sort_values uses a fast quicksort by default, which is not guaranteed to be stable. When you sort on one key and the order of ties matters, pass kind="stable".
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
Stability is also what makes a "sort twice" trick work: sort by the minor key first, then by the major key with kind="stable". A single call with a list of keys is simpler, though.
Choosing the sorting algorithm with kind
sort_values and sort_index accept kind= with four values: "quicksort" (the default), "mergesort", "heapsort" and "stable". They all give a correctly ordered result; they differ in speed, memory and in one guarantee.
- quicksort is usually the fastest on average, but it is not stable. Rows with equal keys may come out in any order.
- mergesort is stable: equal keys keep their original relative order. It needs extra memory. In practice pandas treats
"mergesort"and"stable"as the same request, and"stable"is the clearer name to write. - heapsort needs no extra memory and has a predictable worst case, but it is usually slower and is not stable.
kind only matters when you sort on one column (or one index). With a list of columns, pandas always uses a stable sort and ignores kind.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
On a table this small every algorithm happens to give the same answer. Do not rely on that: if the order of tied rows matters, ask for kind="stable" explicitly, because the default can change on larger data.
sort_index: order by the labels
sort_index() orders rows by the index instead of a column. Use axis=1 to sort the columns by name. This is most useful after operations that scramble the labels, such as filtering, concat or groupby results.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
Sorted indexes also make label slicing faster and more predictable, which matters for the index and for MultiIndex.
nlargest and nsmallest
To get the top few rows you could sort and call head. nlargest(n, columns) and nsmallest(n, columns) say it directly and avoid a full sort on big tables. The keep argument decides what happens with ties at the boundary.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
With keep="all" the result can be longer than n, because every row tied with the cut-off value is kept. nlargest works with numeric (and datetime) columns, not with text.
rank: positions with tie rules
rank() replaces each value by its position in the sorted order. Ties are the interesting part, and the method argument controls them:
| method | Tied values get | Example for scores 95, 95, 88 (descending) |
|---|---|---|
average (default) |
the mean of the positions | 1.5, 1.5, 3.0 |
min |
the lowest position (competition rank) | 1, 1, 3 |
max |
the highest position | 2, 2, 3 |
dense |
next integer, with no gaps | 1, 1, 2 |
first |
order of appearance | 1, 2, 3 |
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
By default the smallest value gets rank 1, so use ascending=False for "highest is first". rank always returns floats; convert with astype(int) when there are no missing values. Missing values stay NaN unless you pass na_option="bottom" or "top". Group-wise ranks (rank within each team, for example) arrive in agg, transform and filter.
Custom orders with key=
sort_values and sort_index accept a key function. It is applied to the column before comparing, but the original values stay in the result. The function must be vectorised: it receives a whole Series and returns a Series of the same length.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
For a fixed custom order that you reuse, an ordered categorical column is cleaner; see Categorical Data.
Check your understanding
Quick check ยท 5 questions
of 5 answered You scored / 5 ยท perfect!
-
1. Where do missing values go with
df.sort_values('x', ascending=False)?na_positiondefaults to'last'regardless of the sort direction. Passna_position='first'to move them to the top. -
2. Which call sorts by
teamA-Z and then byscorefrom high to low?ascendingcan be a list with one flag per key. Sorting twice without a stable sort would lose the team order. -
3. Scores are 90, 90, 80 and you call
rank(method='dense', ascending=False). What ranks do you get?densegives tied values the same rank and does not skip numbers afterward.minwould give 1, 1, 3. -
4. What does the
keyargument ofsort_valuesdo?keyis applied to the sort column only to decide the order, for examplestr.lowerfor a case-insensitive sort. -
5. When might
nlargest(3, 'score', keep='all')return more than 3 rows?keep='all'keeps every row tied at the boundary value.
Summary
sort_values(by, ascending=...)returns a new, ordered DataFrame; lists of keys and flags give multi-key sorts with mixed directions.- Missing values go last unless
na_position="first"; usekind="stable"when ties must keep their order. sort_indexorders by the labels, or by column names withaxis=1.nlargestandnsmallestfetch the extreme rows without a full sort and have akeeprule for ties.rankoffersaverage,min,max,denseandfirsttie rules, pluspct=Truefor percentiles.key=customizes the comparison, for example case-insensitive or custom category order.
Next up: take control of row labels in The Index.
Track your progress
Create a free account to tick off lessons and pick up where you left off on any device.
of pandas lessons completed