Skip to content
Tutorials Center
My learning

Topics

Lesson 47 of 55 ยท Testing, Deployment and Operations

Monitoring, Logging and Scaling

Run a Shopify app in production: structured logs, error tracking, queue workers with backpressure, rate-limit-aware scheduling and install and uninstall analytics.

Once your app is live, the merchant is no longer in the room. Nobody tells you that a webhook failed at 3 a.m., that one shop's import is hammering the API, or that installs doubled last week and uninstalls tripled. If you cannot see it, you cannot fix it, and Shopify's reviewers and merchants will notice before you do.

This lesson covers the four things a production app needs: logs that can be searched, error tracking and alerts, background workers that do slow work without overloading anything, and business metrics about installs and uninstalls. The runnable examples are stand-ins written in plain JavaScript (a structured logger, a worker queue with backpressure and rate-limit awareness, and an install funnel counter); the vendor products you would use in production are named only as categories because they change. Earlier lessons on Rate Limits and Query Cost and Reliable Webhook Handlers are the background for this one.

What to watch

flowchart TB
  A["Your app"] --> L["Logs<br/>what happened"]
  A --> E["Errors<br/>what broke"]
  A --> M["Metrics<br/>how much, how fast"]
  A --> B["Business events<br/>installs, uninstalls, plans"]
  L --> S["Search and alerts"]
  E --> S
  M --> S
  B --> D["Dashboard"]

Four kinds of signals; the first three tell you the app is healthy, the fourth tells you whether it is succeeding.

A short list of signals worth having from day one:

  • Webhook outcomes: received, verified, processed, failed, duplicate, by topic. A rising failure rate is an early warning, because Shopify retries and may eventually stop delivering to a failing endpoint.
  • API calls: count, latency, throttled responses, the remaining cost budget.
  • Job queue: depth, age of the oldest job, failures, retries.
  • Request latency and error rate of embedded pages, since admin performance is part of App Store review.
  • Install, uninstall and plan changes per day.

Structured logs

A log line like Failed to update product is hard to use. When you have a thousand shops, you need to ask "show me every failure for shop X in the last hour". That works when each line is a structured record, normally one JSON object per line, with consistent fields:

JSON
{"time":"2026-10-30T10:15:03.120Z","level":"error","msg":"product update failed","shop":"alpha.myshopify.com","topic":"products/update","webhookId":"b54557e4-...","attempt":2,"error":"Throttled"}

Good fields: time, level, msg, shop, a request or webhook id to correlate lines, the operation, durations and error codes. Equally important is what you must not log:

  • Access tokens, the API secret, session tokens and webhook signatures.
  • Customer personal data such as names, emails and addresses. Privacy rules in Privacy, GDPR and Mandatory Webhooks apply to logs too; they are stored, copied and retained.
  • Full request bodies from webhooks. Log ids and topic instead.

The safest design is a logger that redacts known secret field names automatically, so one careless log.info(request) cannot leak a token. The stand-in below does that and adds the shop to every line through a child logger.

Node.js A structured logger with redaction and shop context

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Every line carries the app, shop and webhook id without the caller repeating them, and secret-looking fields come out as [redacted]. Real projects use an established logging library for the same idea; what matters is the habit of structured fields plus automatic redaction.

Error tracking and alerts

Logs are for investigating; error tracking is for being told. Connect an error-tracking service (several exist; pick one you can afford) to both the server and the browser, group identical errors, and attach the shop and app version as tags so you can answer "is this one merchant or all of them?". Then set a small number of alerts that wake you only when it matters:

  • The webhook failure rate is above a few percent for ten minutes.
  • The oldest queued job is older than your promise (say, fifteen minutes).
  • The Admin API returns authentication errors for many shops at once (a secret or scope problem on your side).
  • Request errors on embedded pages exceed a baseline.

Too many alerts teach you to ignore them. Start with three and add one only after an incident shows it would have helped. Add a health endpoint that checks the database and returns quickly, and let your host or an external uptime monitor call it.

Workers: do slow work away from the request

A web request must answer quickly: the webhook rules from earlier lessons require a fast 200, and a merchant staring at the admin does not want to wait for your import. The pattern is a queue and workers: the request enqueues a job (a row in the database or an entry in a queue service) and a worker process picks it up later. You also get retries, scheduling and the ability to scale workers separately from web servers.

Two problems appear as soon as you run a queue against Shopify:

  1. Backpressure. If webhooks arrive faster than workers can handle them, the queue grows without limit. You need a policy: a maximum size, and a decision when it is full (answer 503 so Shopify retries later, drop low-value jobs, or scale out). Silent unbounded growth ends in memory exhaustion or very stale data.
  2. Rate limits are per shop. One merchant's huge import should not use up the budget that other merchants need, and your worker must slow down for a throttled shop without blocking everyone. Review the leaky-bucket model in Rate Limits and Query Cost: each shop has its own bucket.

The stand-in below combines both. It has a bounded queue, a fair scheduler that picks the next runnable job across shops, and a simple per-shop budget that refills over time. The clock is fake so the result is the same every run.

Node.js A worker queue with backpressure and per-shop budgets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Look at the output: the small shop's two jobs finish in the first rounds even though the big shop queued five jobs earlier, because the scheduler skips shops that are out of budget instead of waiting. The eighth job is refused with a 503 rather than growing the queue. In production the "budget" is the real throttle status returned in the GraphQL cost extensions, and the queue lives in a database or a managed queue service so a restart loses nothing. If you use a queue service, remember that most deliver at least once, so jobs must be idempotent, a rule from Idempotency.

Retries

Retry only errors that can succeed later: network failures, 5xx, throttling. Wait longer each time (exponential backoff) and add a small random jitter so thousands of jobs do not retry in lockstep. After a limit, move the job to a dead-letter list that a human can inspect. Never retry validation errors or userErrors that say the input is wrong; they will fail forever (Mutations and userErrors).

Scaling in steps

Do not scale before you measure. A sensible order:

  1. Fix the slow query or the N+1 call. Most "scaling" problems are one expensive GraphQL query (Pagination with Cursors) or a missing database index.
  2. Cache what is stable, such as parsed settings per shop, and invalidate on change. Do not cache anything that must be fresh, like stock counts.
  3. Move work off the request path into workers, as above.
  4. Use bulk operations instead of thousands of single calls for large reads and writes (Bulk Operations).
  5. Add instances of the web server and workers separately. Keep servers stateless: no in-memory sessions, no local files that matter.
  6. Scale the database with a connection pooler and read replicas only when metrics show it is the bottleneck.

Business metrics: installs and uninstalls

Technical health is not the whole picture. For a public app, the most useful numbers come from a handful of events: installed, first_use (the merchant did the key action), plan_started, plan_changed, uninstalled. From them you can see activation (what fraction of installs reach first use) and retention (who is still installed after 7 and 30 days), and you can see when people leave: uninstalls on day zero point to a confusing onboarding, uninstalls right after a billing change point to pricing.

You already receive the app/uninstalled webhook, and install time is known when the offline session is created. Record each event with a shop id, date and plan, but not personal data. The stand-in computes a funnel and uninstall timing from a small list of events.

Node.js Install funnel and uninstall timing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Keep the analytics in your own database or an analytics tool; do not put merchants' customer data in them. Aggregate numbers are enough to guide decisions.

Incident habits

When something does break:

  1. Stop the bleeding (roll back the release, pause the worker, disable the feature flag) before searching for the cause.
  2. Use the logs by shop and id, as above, to find the first failure, not the loudest one.
  3. Reprocess missed events. Webhooks can be lost during an outage, so keep a reconciliation job that re-reads recent changes through the API or a bulk operation.
  4. Tell affected merchants when their data was wrong or late, and write down what you will change.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why log JSON objects with fields such as shop and webhookId instead of free text?

    Consistent fields let you ask precise questions, such as every error for one shop in the last hour.
  2. 2. Which of these must never appear in logs?

    Tokens and personal data are secrets or regulated data. Logs are stored and copied, so they must be redacted.
  3. 3. What does a bounded queue with a 503 answer protect you from?

    Backpressure is a deliberate refusal when the system is full. A 503 tells Shopify to try again later instead of piling work up.
  4. 4. Why does the scheduler skip a shop that has no rate-limit budget instead of waiting?

    Rate limits apply per shop. Skipping a throttled shop keeps the workers busy with jobs that can run now.
  5. 5. What does a spike of same-day uninstalls usually suggest?

    Merchants who leave immediately did not get value or understanding in the first minutes, which points to onboarding and expectations.

Summary

  • Log structured JSON with a shop and request id, and redact tokens and personal data automatically.
  • Use error tracking with a few meaningful alerts, plus a fast health endpoint.
  • Move slow work to workers; bound the queue, answer 503 when full and keep jobs idempotent.
  • Treat rate limits as per-shop, schedule fairly and retry with backoff, jitter and a dead-letter list.
  • Scale by measuring: fix queries, cache, queue, bulk, then add instances.
  • Track installs, activation and uninstalls to see whether the app is succeeding, not only running.

Next, put these ideas into practice in Project: Bulk Product Tagger.