Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 33 ยท Projects

Project: Syncing Products with an External System

Sync products between Medusa 2 and an external ERP: a scheduled job, subscribers, idempotent upserts, inbound webhooks, retries and a dead-letter queue.

Most real stores do not create their catalog in Medusa. Products live in an ERP, a PIM or a supplier feed, and Medusa has to follow them: new items appear, prices and titles change, stock moves. A sync looks easy ("loop and save") until it runs twice, half-fails, receives the same webhook three times, or meets a product it created itself last week. This project builds a sync that survives all of that.

You will combine a scheduled job (pull), an inbound webhook route (push), idempotent upserts keyed by an external id, retry and dead-letter handling, and subscribers that react to changes. The Medusa code is read-only; the stand-ins implement the diffing engine and the retry logic so you can verify the behavior, including one that runs on the Node.js runner and uses real HMAC signatures.

The architecture

flowchart LR
    ERP["External system"] -- "poll every 15 min" --> JOB["Scheduled job"]
    ERP -- "webhook push" --> ROUTE["Inbound webhook route"]
    JOB --> Q["Sync queue"]
    ROUTE --> Q
    Q --> UP["Upsert workflow<br/>external_id to product"]
    UP --> MED[("Medusa products")]
    UP -- "fails 5 times" --> DLQ[("Dead-letter table")]
    MED -- "product.updated" --> SUB["Subscriber<br/>reindex, notify"]

Pull and push both feed one idempotent upsert; poison items end up in a dead-letter table instead of blocking the rest.

Design principles:

  1. One write path. Job and webhook call the same workflow, so rules live in one place.
  2. Idempotent. Running the sync twice, or receiving an event twice, changes nothing the second time.
  3. The external system is the source of truth for fields it owns (title, SKU, base price); Medusa owns the rest (merchandising, images you edit). Decide per field and write it down.
  4. Never lose an item silently. A failure is retried, then recorded.

Mapping external ids

Every synced product needs a stable link to its source row. The quickest approach is the product's metadata (read-only):

TypeScript
metadata: { external_id: "ERP-1042", external_hash: "9c1f...", synced_at: "2026-10-30T10:00:00Z" }

Store a content hash of the fields you sync. If the hash from the ERP equals the stored one, the product has not changed and you skip the write, which makes frequent polling cheap. For a large catalog, filtering and joining on metadata is not efficient, so a dedicated mapping model in a small custom module (external_id indexed and unique, product_id, hash) scales better (Custom Modules and Data Models). Variants need the same treatment: the SKU is the natural key, and Medusa's variant SKU is unique.

The upsert workflow

An upsert creates a product when the external id is unknown and updates it otherwise. Use the core workflows rather than writing straight to the module, so prices, inventory links and events stay consistent (Hooks and Built-in Workflows). A simplified, read-only sketch:

TypeScript
// src/workflows/upsert-external-product.ts
import { createWorkflow, WorkflowResponse, when } from "@medusajs/framework/workflows-sdk"
import {
  createProductsWorkflow,
  updateProductsWorkflow,
} from "@medusajs/medusa/core-flows"

type Input = { item: ExternalProduct; existing?: { id: string; hash: string } }

export const upsertExternalProductWorkflow = createWorkflow(
  "upsert-external-product",
  (input: Input) => {
    const created = when(input, (i) => !i.existing).then(() =>
      createProductsWorkflow.runAsStep({
        input: { products: [toMedusaProduct(input.item)] },
      })
    )

    const updated = when(input, (i) => !!i.existing && i.existing.hash !== i.item.hash).then(() =>
      updateProductsWorkflow.runAsStep({
        input: {
          selector: { id: input.existing!.id },
          update: toMedusaUpdate(input.item),
        },
      })
    )

    return new WorkflowResponse({ created, updated })
  }
)

The details of when/then and runAsStep are in the workflow reference for your version; the lesson is the shape: decide create, update or skip from the external id and hash.

The scheduled job (pull)

A job runs on a cron schedule (Scheduled Jobs). It should only fetch and enqueue; heavy work belongs to workflows (read-only):

TypeScript
// src/jobs/sync-erp-products.ts
import type { MedusaContainer } from "@medusajs/framework/types"
import { syncFromErp } from "../lib/sync-from-erp"

export default async function syncErpProducts(container: MedusaContainer) {
  const logger = container.resolve("logger")
  const result = await syncFromErp(container)
  logger.info(`ERP sync: ${result.created} created, ${result.updated} updated, ${result.skipped} unchanged, ${result.failed} failed`)
}

export const config = {
  name: "sync-erp-products",
  schedule: "*/15 * * * *",
}

Two cautions from the scheduling lesson apply: run jobs on the worker instance only, so scaled-out servers do not all sync at once (Deploying Medusa), and keep each run bounded (fetch "changed since the last cursor", in pages) so a slow run does not overlap the next one.

The inbound webhook route (push)

Many systems can notify you when a product changes, which is faster than polling. The route must verify the signature of the raw body, reject unknown or replayed events and respond quickly (Webhooks and signatures). Read-only sketch:

TypeScript
// src/api/hooks/erp/route.ts
import type { MedusaRequest, MedusaResponse } from "@medusajs/framework/http"
import crypto from "node:crypto"

export async function POST(req: MedusaRequest, res: MedusaResponse) {
  const signature = req.headers["x-erp-signature"] as string
  const expected = crypto
    .createHmac("sha256", process.env.ERP_WEBHOOK_SECRET!)
    .update(req.rawBody as Buffer)
    .digest("hex")

  const ok =
    signature?.length === expected.length &&
    crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))
  if (!ok) return res.status(401).json({ message: "bad signature" })

  await enqueue(req.scope, req.body) // store, then answer fast
  res.status(202).json({ received: true })
}

Reading req.rawBody requires configuring the body parser for that route with preserveRawBody: true in defineMiddlewares (Custom API Routes and Middleware). Answer 202 immediately and process the item from the queue; slow handlers cause the sender to retry, which creates duplicates.

Subscribers: reacting to changes

When the upsert workflow runs, core workflows emit events such as product.created and product.updated. A subscriber can push the item to a search index, clear a cache or notify merchandisers (Events and Subscribers). Subscribers are at-least-once, so make them idempotent. Beware loops: if your own sync writes trigger a subscriber that writes back to the ERP, which then sends a webhook, you have created a ping-pong. Mark writes that came from the ERP (for example by comparing the stored hash) and do not send them back.

Stand-in 1: the diffing engine

This stand-in computes what a pull-sync should do: create unknown external ids, update changed ones, skip identical ones and report products that vanished from the feed. Running it twice proves idempotency. The hash is a simple deterministic function standing in for a real SHA-256 digest.

JS Diff an ERP feed against local products

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Run 2 changes nothing (skipped: 2). Run 3 shows one update, one create and ERP-2 reported as missing. Decide deliberately what "missing" means: archive it, unpublish it or just alert, but never delete automatically on a single missing run, because a partial feed would wipe your catalog.

Stand-in 2: webhooks, retries and a dead-letter queue

This second stand-in runs on the Node.js runner so it can use real node:crypto HMAC signatures. It verifies each inbound event, ignores duplicates by event id, processes with a bounded number of attempts and moves items that keep failing into a dead-letter list. A fake clock keeps the output deterministic.

Node.js Verified webhooks with retry and dead-letter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Read the timeline: TOTE-1 succeeds at once, FLAKY fails and succeeds on the retry, and BAD exhausts its three attempts and lands in the dead-letter list where a human can look at it. The forged request never reaches the queue.

Dead letters and operations

A dead-letter table (a small model with payload, error, attempts, created_at) is only useful if someone looks at it. Add an admin page or a daily alert, plus a "retry" button that re-enqueues an item after the cause is fixed. Log each sync run with its counts, alert when failures jump, and keep the cursor ("changed since") in the database so a crash resumes where it stopped. For stock, apply absolute quantities from the ERP rather than deltas, so a repeated message cannot double-count (Inventory reservations).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why store a content hash next to the external id?

    Comparing hashes lets the sync decide create, update or skip without comparing every field, and makes repeated runs do no work.
  2. 2. What makes a sync run idempotent?

    With a stable key, repeated runs and duplicate events map to the same record, so the end state is the same no matter how often the input arrives.
  3. 3. Why should the webhook route answer 202 quickly and process from a queue?

    Senders retry when they do not get a fast success response. Acknowledging first and processing asynchronously avoids duplicate deliveries.
  4. 4. What should happen when an item fails all its retry attempts?

    Dead-lettering stops a poison item from blocking others while keeping it visible for investigation and manual retry.
  5. 5. An item disappears from one feed run. What is the safest default?

    A partial or failed feed would look like mass deletions. Treat missing items as a signal, not an instruction to delete.

Summary

  • Map external records to Medusa products by a stable external id (metadata for small catalogs, a mapping module for large ones) plus a content hash.
  • Pull with a worker-only scheduled job, push with a verified webhook route, and send both through one idempotent upsert workflow built on core workflows.
  • Verify webhook signatures on the raw body, dedupe by event id and acknowledge fast.
  • Retry with backoff, then dead-letter, and make failures visible and re-runnable.
  • Decide field ownership, avoid sync loops through subscribers and treat missing items carefully.

Next: Practice: Exercises with Solutions