Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 39 ยท Filtering, Pagination and Throttling

Pagination in Django REST Framework

Paginate Django REST Framework list endpoints with PageNumberPagination, LimitOffsetPagination and CursorPagination, plus custom page sizes, response shapes and when to choose which style.

A list endpoint that returns every row works on your laptop and fails in production. A table with a million rows produces a response of hundreds of megabytes, ties up a database connection and a worker for seconds, and no client wants that much data anyway. Pagination splits a result into pages and tells the client how to ask for the next one.

DRF has three built-in styles. They differ in how the client moves through the data and in what they cost the database. This lesson shows all three on the same twelve-book catalog from the filtering lesson, then builds a custom response shape.

Turning pagination on

You can set a default style for every list view in settings.py, or set pagination_class on one view:

Python
# settings.py (read-only)
REST_FRAMEWORK = {
    "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
    "PAGE_SIZE": 5,
}

Pagination applies to ListModelMixin (so ListAPIView, ModelViewSet and friends) automatically. In your own APIView you must call the paginator yourself; the last section shows how.

Important detail: if you set DEFAULT_PAGINATION_CLASS but forget PAGE_SIZE, PageNumberPagination does nothing and returns a plain list. A page size is what switches paging on.

Three ways to page

Page number ?page=3 โ€” simple, great for websites
Limit and offset ?limit=5&offset=10 โ€” flexible windows
Cursor An opaque bookmark โ€” fast and stable

Pick by how clients navigate and how big the table is

1 / 2

PageNumberPagination

The client sends ?page=2. The response wraps the items in an object with count, next, previous and results. The page size comes from page_size; if you set page_size_query_param, clients may choose their own size up to max_page_size.

Python PageNumberPagination with a client-chosen page size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Output:

TEXT
{} -> count 12 | next: http://testserver/books/?page=2 | previous: None
    ['Night Trains', 'Salt and Iron', 'The Quiet Harbor', 'Learning Rust Slowly', 'Gardens of Glass']
{'page': 3} -> count 12 | next: None | previous: http://testserver/books/?page=2
    ['Cells and Stars', 'Deep Roots']
{'size': 4, 'page': 2} -> count 12 | next: http://testserver/books/?page=3&size=4 | previous: http://testserver/books/?size=4
    ['Gardens of Glass', 'Python by Doing', 'Small Kitchens', 'Bread at Dawn']
{'size': 100} -> count 12 | next: http://testserver/books/?page=2&size=100 | previous: None
    ['Night Trains', 'Salt and Iron', 'The Quiet Harbor', 'Learning Rust Slowly', 'Gardens of Glass', 'Python by Doing', 'Small Kitchens', 'Bread at Dawn']
{'page': 9} -> 404 {'detail': 'Invalid page.'}

Things to notice:

  • The next and previous values are full URLs that keep the other query parameters (such as size), so a client can simply follow the link instead of building URLs.
  • Asking for a page that does not exist returns 404 with {"detail": "Invalid page."}.
  • count makes the database run an extra SELECT COUNT(*). On very large tables that count can be the slowest part of the request.

LimitOffsetPagination

Here the client controls a window directly: ?limit=5&offset=10 means "give me five rows, skipping ten". It maps straight onto SQL LIMIT and OFFSET. It suits APIs where the client may want to jump to an arbitrary position or load "the next 20" in an infinite list.

Python LimitOffsetPagination

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Output:

TEXT
{} -> count 12 [1, 2, 3, 4]
    next: http://testserver/books/?limit=4&offset=4
{'limit': 3, 'offset': 6} -> count 12 [7, 8, 9]
    next: http://testserver/books/?limit=3&offset=9
{'limit': 50} -> count 12 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    next: http://testserver/books/?limit=10&offset=10
{'offset': 11} -> count 12 [12]
    next: None

The cost of large offsets

Both styles above use OFFSET. To return rows 100,000 to 100,019 the database must still walk past the first 100,000. Deep pages therefore get slower and slower. There is a second problem: if rows are inserted or deleted between two requests, offset-based pages shift, so a client can see a row twice or miss one entirely.

CursorPagination

CursorPagination fixes both problems. Instead of a page number it returns an opaque cursor that remembers the position of the last row seen, and the next query becomes WHERE created < <last value>, which uses an index and never scans skipped rows. Inserts at the head of the list do not disturb pages the client already received.

The price: no count, no jumping to page 7, only next and previous. You must also provide a stable, unique-ish ordering field (by default created; set ordering to a field such as -id or -created).

Python CursorPagination: follow the next links

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Output:

TEXT
page 1 ids: [12, 11, 10, 9, 8] | has next: True | has previous: False
page 2 ids: [7, 6, 5, 4, 3] | has next: True | has previous: True
page 3 ids: [2, 1] | has next: False | has previous: True
keys in a cursor response: ['next', 'previous', 'results']

A cursor is opaque by design: do not tell clients to build or parse it. They must just follow next. Treat the layout as private so you can change it later.

Which style should you choose?

Need Style
Website with numbered pages and a total count PageNumberPagination
Client wants any window, or "load more" with known offsets LimitOffsetPagination
Huge table, feed or timeline, data changes while reading CursorPagination
Public API where deep pages must stay fast CursorPagination

A custom response shape and pagination in your own views

Teams often want a different envelope, for example {"meta": {...}, "data": [...]}. Override get_paginated_response(). And if you write an APIView instead of a generic view, you drive the paginator yourself with two calls: paginate_queryset() and get_paginated_response().

Python A custom envelope inside a plain APIView

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Output:

TEXT
{} -> {'total': 12, 'page': 1, 'pages': 3, 'has_next': True} [1, 2, 3, 4, 5]
{'page': 3} -> {'total': 12, 'page': 3, 'pages': 3, 'has_next': False} [11, 12]

Practical advice

  • Always paginate public list endpoints. An unpaginated list is a denial-of-service waiting to happen.
  • Order before you paginate. Without a defined order, the database may return rows in any order, and pages can overlap. Set ordering on the queryset (or a filter backend) with a unique tie-breaker such as id.
  • Cap the page size. Use max_page_size or max_limit so one request cannot pull the whole table.
  • Keep queries cheap per page. Use select_related() and prefetch_related() so the page itself does not trigger a query per row (see Query optimization).
  • Combine with filtering. Filters narrow the queryset first, then pagination slices it. See Search and Ordering.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. You set DEFAULT_PAGINATION_CLASS to PageNumberPagination but no PAGE_SIZE. What happens?

    PageNumberPagination needs a page size to turn paging on. Without one it returns the plain list.
  2. 2. Which style gives you no total count and no way to jump to page 7?

    CursorPagination only offers next and previous links, which is the price for fast, stable paging.
  3. 3. Why does a very large OFFSET get slow?

    OFFSET n makes the database read and discard n rows before returning the page.
  4. 4. In a plain APIView, how do you paginate a queryset?

    You instantiate the pagination class and call those two methods yourself.
  5. 5. What should a client do with the cursor value in a next link?

    The cursor format is private. Following next is the only supported use.

Summary

  • Pagination keeps list responses small; turn it on globally or per view, and always set a page size.
  • PageNumberPagination gives ?page= and a total; LimitOffsetPagination gives ?limit=&offset=; CursorPagination gives opaque, fast, stable next/previous links without a total.
  • Large offsets are slow and unstable; prefer cursors for big or fast-changing tables.
  • Override get_paginated_response() for a custom envelope, and call the paginator yourself in an APIView.
  • Order by a unique field and cap the page size.

Next: Throttling and Rate Limits