Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 42 ยท Quality and Production

Indexers and Caching

Understand Magento 2.4 indexers (Update on Save vs Update by Schedule, mview), cache types, cache tags and the full page cache, with runnable tagged-cache and changelog stand-ins.

A Magento store is slow by design if you look at the raw data. A product's final price depends on tax, catalog price rules, customer group and website. A category page needs the products in that category, sorted by position, filtered by attributes. Computing all this from the normal tables on every request would be far too expensive. Magento solves the problem twice: indexers precompute flat, query-friendly tables ahead of time, and caches remember rendered or loaded results so they are not computed again.

Both tricks have the same price: the stored copy can become stale. Most "I changed the price in the admin but the storefront still shows the old one" tickets come from an index that has not caught up or a cache that was not invalidated. This lesson explains how each system decides when to refresh, which commands you use, and how your own module takes part. If you have not met caching in plain PHP, Caching Strategies is good background.

Indexers

An indexer reads source data (products, categories, prices, stock, rules) and writes a denormalised copy. Core indexers include catalog product and category data, product price, inventory, catalog search, and URL rewrites. You can list them with bin/magento indexer:status, and rebuild them with bin/magento indexer:reindex (all of them, or names such as catalog_product_price). The exact list depends on your version and the modules installed.

Each indexer runs in one of two modes, shown by bin/magento indexer:show-mode:

  • Update on Save (the realtime mode): saving an entity reindexes the affected rows in the same request. Simple, but a save in the admin can become slow, and large imports are painful.
  • Update by Schedule (the schedule mode): saving only records that something changed. A cron job reindexes the changes in the background. Changing mode is done with bin/magento indexer:set-mode schedule (or realtime) with optional indexer names. Production stores generally use schedule mode with cron running.

How schedule mode works: mview

Schedule mode is implemented by mview (materialized view). When an indexer is switched to schedule mode, Magento creates database triggers on the source tables. Each trigger writes the id of the changed row to a changelog table, named after the subscribed table with a _cl suffix (for example catalog_product_entity_cl). A cron job then reads the changelog from the last processed version, collects the unique ids, reindexes only those rows, and moves its version marker forward. If you changed one product, only one product is reindexed.

The wiring lives in an etc/mview.xml file in the module, and the indexer itself is declared in etc/indexer.xml. These are read-only real-Magento sketches:

XML
<!-- etc/indexer.xml -->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Indexer/etc/indexer.xsd">
    <indexer id="acme_badge" view_id="acme_badge" class="Acme\Badge\Model\Indexer\Badge">
        <title translate="true">Product badges</title>
        <description translate="true">Rebuilds the badge for each product</description>
    </indexer>
</config>

<!-- etc/mview.xml -->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Mview/etc/mview.xsd">
    <view id="acme_badge" class="Acme\Badge\Model\Indexer\Badge" group="indexer">
        <subscriptions>
            <table name="catalog_product_entity" entity_column="entity_id"/>
        </subscriptions>
    </view>
</config>

The indexer class implements Magento\Framework\Indexer\ActionInterface and Magento\Framework\Mview\ActionInterface, with methods such as executeFull(), executeList(array $ids) and executeRow($id). Full reindexing rebuilds everything, and the list and row methods rebuild specific ids, which is what mview calls.

flowchart LR
    A["Admin saves a product"] --> B["Trigger writes id<br/>to the changelog table"]
    B --> C["Cron: indexer_update_all_views"]
    C --> D["Read ids after last version"]
    D --> E["executeList(ids)"]
    E --> F["Index table is fresh"]

In schedule mode a save only writes to the changelog; cron does the actual reindexing later.

A changelog stand-in

The stand-in below uses SQLite, which supports triggers, to copy the idea. A price_index table holds the final price with 20 percent tax added. In schedule mode, a trigger appends to products_cl, and processChangelog() plays the role of the cron job. In realtime mode the stand-in reindexes immediately. Real Magento creates and drops its triggers when you switch modes, which setMode() imitates.

PHP Realtime versus scheduled indexing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with fgets(STDIN)), it gets the next line from this box.

Two things are worth noticing. The mug was saved twice but was reindexed once, because the cron job works on the distinct ids. And between the save and the cron run the index shows the old price. That gap is the price of cheap saves, so a store that uses schedule mode must keep cron healthy, and indexer:status is the first place to look when data seems stale.

Do not run a full indexer:reindex on a busy production store at peak time: it rebuilds whole tables. Prefer schedule mode for incremental updates and plan full reindexing for quiet periods.

Cache types

Magento keeps several separate caches, called cache types, so a change in one area does not force you to throw away everything. You see them with bin/magento cache:status. Examples include config (merged XML configuration), layout (merged layout XML), block_html (rendered blocks), collections (database query results), reflection (API data reflection), eav (attribute metadata), translate, and full_page (whole rendered pages). Developers also meet db_ddl and compiled_config, and newer versions may add more.

The commands you will use daily:

Command What it does
bin/magento cache:status Lists cache types and whether each is enabled
bin/magento cache:enable / cache:disable Turns types on or off (optionally by name)
bin/magento cache:clean Removes entries of the enabled cache types that Magento tags as its own
bin/magento cache:flush Empties the whole cache storage, including entries other applications put there

Use cache:clean by default. cache:flush is the heavy tool: if Redis is shared with other applications or sessions, flushing it clears their data as well. After a di.xml or layout change, cleaning is usually enough. Never disable the full page cache on a production store to "fix" a stale page, since that sends every visitor to PHP.

Cache tags and invalidation

Clearing everything on every product save would defeat the purpose of caching. Instead each cache entry can carry tags, and invalidation removes entries by tag. A category page that lists products 1 and 2 is stored with tags like cat_c_5 (category 5), cat_p_1 and cat_p_2. When product 1 changes, Magento cleans the tag cat_p_1, and every cached page that mentions product 1 goes away, while unrelated pages stay warm. Magento builds these ids from the getIdentities() method of models and blocks, using prefixes such as cat_p for products and cat_c for categories.

Your own blocks take part by implementing Magento\Framework\DataObject\IdentityInterface:

PHP
<?php
namespace Acme\Badge\Block;

use Magento\Framework\DataObject\IdentityInterface;
use Magento\Framework\View\Element\Template;

class Badge extends Template implements IdentityInterface
{
    public function getIdentities(): array
    {
        return ['acme_badge_' . $this->getData('product_id')];
    }
}

The full page cache collects the identities of every block that was rendered into a page and stores them as that page's tags. Your code (an observer on save, or a model's afterSave) later invalidates by tag with the cache manager. When the page is served by Varnish instead of the built-in cache, Magento sends the tags to Varnish in a response header, and a purge request with the tag removes the matching pages. Varnish is a fast HTTP cache in front of the web server, recommended for production, and its VCL file is generated from the admin (Stores > Configuration > Advanced > System > Full Page Cache).

The stand-in below implements tagged storage in 30 lines. Pages are saved with tags, hits and misses are counted, and a product change cleans by tag.

PHP A tagged cache

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with fgets(STDIN)), it gets the next line from this box.

After the change only the category page and product-1 were removed. product-2 stayed cached and served as a hit, which is the whole point of tags.

Making your module cache-friendly

  • Give custom blocks that show entity data a getIdentities() method so the full page cache can invalidate them.
  • Mark user-specific content as private. Anything customer-specific must not end up in the shared page cache: Magento loads it by JavaScript from the customer section data.
  • Avoid cacheable="false" in layout XML. A single uncacheable block disables the full page cache for every page that includes it.
  • Do not change the cache by hand in the code that handles a normal request unless you know the tag you are invalidating.
  • After editing configuration, XML or templates in development, clean the affected cache type. Developer mode does not remove the need to clean.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does Update by Schedule indexing do when a product is saved?

    In schedule mode, mview triggers write the changed id to a changelog table. Cron then reindexes only those ids.
  2. 2. A store uses schedule mode but prices on the storefront never update. What should you check first?

    Schedule mode relies on cron jobs that process the changelog. If cron stops, indexes become stale.
  3. 3. Why does Magento attach tags such as cat_p_1 to cache entries?

    Tags let Magento clean exactly the pages and blocks that mention a changed product or category, instead of flushing everything.
  4. 4. Which statement about cache:clean and cache:flush is correct?

    cache:clean removes Magento's tagged entries of enabled types. cache:flush clears the entire backend, which can wipe other data on a shared Redis.
  5. 5. Why is cacheable="false" on a layout block a performance risk?

    One uncacheable block prevents the full page cache from storing the whole page, so each visit runs PHP.

Summary

  • Indexers precompute query-friendly data; caches remember computed results. Both can go stale, and you must know how each refreshes.
  • Indexers run On Save or By Schedule; schedule mode uses mview triggers, _cl changelog tables and cron.
  • Cache types separate configuration, layout, blocks, collections and full pages. Prefer cache:clean to cache:flush.
  • Cache tags let Magento invalidate exactly the entries that depend on a changed entity; custom blocks join in with getIdentities().
  • Avoid cacheable="false", keep customer-specific data private, and keep cron healthy in production.

Next: Testing Magento Code