Skip to content
Tutorials Center
My learning

Topics

Lesson 71 of 104 ยท Local Data

Paging 3 for Large Lists

Load large lists in pages with Paging 3: PagingSource, Pager, LazyPagingItems, RemoteMediator, prefetch distance and load states explained.

A feed with 50,000 rows should not be loaded in one go. It wastes memory, delays the first screen and downloads data the user will never scroll to. The better approach is to load a small page, show it and fetch the next page just before the user reaches the end. That is paging, and Paging 3 is the Jetpack library that does it with coroutines and Flow.

In this lesson you will meet the four main parts, PagingSource, Pager, PagingData and the Compose adapter, then RemoteMediator for network plus database, and the load states your UI reacts to. Paging needs the Android build, so those snippets are read-only. The TryIt blocks are Kotlin stand-ins: a paginator that loads pages when you scroll near the end, so you can see the prefetch rules and the load states in action.

The parts of Paging 3

flowchart LR
    PS["PagingSource<br/>(loads one page)"] --> P["Pager<br/>(config + triggers loads)"]
    P -->|"Flow of PagingData"| VM["ViewModel<br/>(cachedIn)"]
    VM --> UI["LazyPagingItems<br/>(LazyColumn)"]
    UI -->|"scrolling near the end"| P

The UI reports how far the user scrolled; the Pager asks the PagingSource for more.

  • PagingSource<Key, Value> knows how to load one page of data from one place, such as a database table or a REST endpoint.
  • Pager takes a PagingConfig and a factory for the source and exposes a Flow<PagingData<Value>>.
  • PagingData is the stream of pages and their changes, collected by the UI.
  • LazyPagingItems is the Compose side: it behaves like a list you can read by index and tells the Pager what is being displayed.

Writing a PagingSource

A source implements load, which receives the key of the page to load and a size, and getRefreshKey, which says where to restart after the data is invalidated:

Kotlin
// read-only example: needs the Paging library
class ArticlePagingSource(private val api: NewsApi) : PagingSource<Int, Article>() {

    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Article> {
        val page = params.key ?: 1
        return try {
            val articles = api.getArticles(page = page, perPage = params.loadSize)
            LoadResult.Page(
                data = articles,
                prevKey = if (page == 1) null else page - 1,
                nextKey = if (articles.isEmpty()) null else page + 1
            )
        } catch (e: IOException) {
            LoadResult.Error(e)
        } catch (e: HttpException) {
            LoadResult.Error(e)
        }
    }

    override fun getRefreshKey(state: PagingState<Int, Article>): Int? =
        state.anchorPosition?.let { position ->
            state.closestPageToPosition(position)?.prevKey?.plus(1)
                ?: state.closestPageToPosition(position)?.nextKey?.minus(1)
        }
}

LoadResult.Page carries the data and the keys for its neighbors; a null key means "no more in this direction", which is how the library knows the list has ended. Errors are returned as LoadResult.Error, never thrown, so the UI can show a retry button. Note that loadSize for the first load is larger than the page size by default (three pages' worth), which is why you should compute keys from what you received, not assume each load is exactly one page.

Room can create a PagingSource for you: a DAO method that returns PagingSource<Int, Item> pages a query automatically and invalidates itself when the table changes.

Pager, ViewModel and Compose

Kotlin
// read-only example: needs the Paging library
class FeedViewModel(private val api: NewsApi) : ViewModel() {
    val articles: Flow<PagingData<Article>> = Pager(
        config = PagingConfig(
            pageSize = 20,
            prefetchDistance = 5,
            enablePlaceholders = false
        ),
        pagingSourceFactory = { ArticlePagingSource(api) }
    ).flow.cachedIn(viewModelScope)
}

@Composable
fun FeedScreen(viewModel: FeedViewModel = viewModel()) {
    val items = viewModel.articles.collectAsLazyPagingItems()

    LazyColumn {
        items(count = items.itemCount, key = items.itemKey { it.id }) { index ->
            val article = items[index]
            if (article != null) ArticleRow(article) else PlaceholderRow()
        }
        if (items.loadState.append is LoadState.Loading) {
            item { CircularProgressIndicator() }
        }
    }
}

cachedIn(viewModelScope) is important. It keeps the loaded pages in memory for the ViewModel's life, so rotating the screen does not reload everything, and it lets several collectors share one stream. Reading items[index] is what triggers loading: when the index you touch gets within prefetchDistance of the end of what is loaded, the Pager requests the next page. With enablePlaceholders = true the list reports its full size up front and gives null for unloaded positions, which is only possible when the source knows the total count.

The first TryIt models that mechanism. FakeSource is the PagingSource, and Paginator.get(index) plays the role of reading items[index] from a lazy list: it records which index is displayed and loads the next page when that index is within the prefetch distance of the end of the loaded data.

Kotlin Prefetch distance in action (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readln()), it gets the next line from this box.

Real Paging also drops pages that are far off screen (maxSize), cancels loads that are no longer needed and runs on a background dispatcher. The stand-in only shows the trigger.

Load states

A paged list is never simply "loaded". loadState describes three independent directions: refresh (the initial load or a pull to refresh), prepend (loading before the first item) and append (loading after the last one). Each is one of three values:

  • LoadState.Loading: a request is in flight.
  • LoadState.Error: it failed; the exception is attached and items.retry() repeats it.
  • LoadState.NotLoading(endOfPaginationReached): idle, and the flag tells you whether there is more.
Kotlin
// read-only example: needs the Paging library
when (val refresh = items.loadState.refresh) {
    is LoadState.Loading -> FullScreenSpinner()
    is LoadState.Error -> ErrorMessage(refresh.error, onRetry = { items.retry() })
    is LoadState.NotLoading ->
        if (items.itemCount == 0) EmptyState() else FeedList(items)
}

Show a full-screen spinner only for refresh; show a small footer spinner or retry row for append. Mixing those two is the most common Paging UI bug.

The second TryIt is a stand-in for the state machine. A source fails on one page the first time it is asked, the paginator exposes the append state, and retry() repeats the failed load.

Kotlin Load states and retry (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readln()), it gets the next line from this box.

RemoteMediator: network plus database

Often you want offline support: the database is the single source of truth, and the network only fills it. RemoteMediator does exactly that. The Pager reads pages from a Room PagingSource, and when the database runs out of rows near the edge of the list, Paging calls the mediator's load, which fetches the next network page and writes it to Room. Room's invalidation then pushes the new rows to the UI.

Kotlin
// read-only example: needs the Paging library and Room
@OptIn(ExperimentalPagingApi::class)
class ArticleRemoteMediator(
    private val api: NewsApi,
    private val db: AppDatabase
) : RemoteMediator<Int, ArticleEntity>() {

    override suspend fun load(loadType: LoadType, state: PagingState<Int, ArticleEntity>): MediatorResult {
        val page = when (loadType) {
            LoadType.REFRESH -> 1
            LoadType.PREPEND -> return MediatorResult.Success(endOfPaginationReached = true)
            LoadType.APPEND -> db.remoteKeyDao().nextPage() ?: return MediatorResult.Success(true)
        }
        return try {
            val response = api.getArticles(page, state.config.pageSize)
            db.withTransaction {
                if (loadType == LoadType.REFRESH) db.articleDao().clearAll()
                db.articleDao().insertAll(response.map { it.toEntity() })
                db.remoteKeyDao().save(nextPage = if (response.isEmpty()) null else page + 1)
            }
            MediatorResult.Success(endOfPaginationReached = response.isEmpty())
        } catch (e: IOException) {
            MediatorResult.Error(e)
        }
    }
}

You pass it to the pager with remoteMediator = ArticleRemoteMediator(api, db) and a pagingSourceFactory that returns db.articleDao().pagingSource(). You will build the full pattern in Project: News Reader with Paging, and the offline-first idea is explained in Error Handling and Offline-first.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a null nextKey in LoadResult.Page tell Paging?

    A null key marks the end of the data in that direction, so Paging stops requesting more pages and reports endOfPaginationReached.
  2. 2. Why call cachedIn(viewModelScope) on the Pager flow?

    cachedIn keeps the loaded pages in memory for the scope's lifetime, so rotation does not reload and the flow can be collected more than once safely.
  3. 3. Which load state should drive a full-screen progress spinner?

    refresh covers the initial load and pull to refresh. A small footer indicator is the right place for append, because the user is already looking at data.
  4. 4. In an offline-first Paging setup, what does the RemoteMediator do?

    The database stays the single source of truth. The mediator only fills it when more data is needed, and Room invalidation updates the UI.

Summary

  • Paging loads data in small pages and fetches the next one when the user scrolls within prefetchDistance of the end.
  • A PagingSource loads one page and returns LoadResult.Page (with prev and next keys) or LoadResult.Error.
  • Pager(...).flow.cachedIn(viewModelScope) exposes PagingData; collectAsLazyPagingItems() feeds a LazyColumn.
  • Read loadState.refresh, prepend and append separately, and offer retry() on errors.
  • RemoteMediator combines network and Room so the database is the source of truth.
  • The Paging code is read-only here; the TryIt blocks are stand-ins for the prefetch trigger and the load states.

Next: Networking with Retrofit and OkHttp