Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 28 ยท Projects

Project: Order Export Job and Integration

Build an order export for Salesforce B2C Commerce: a custom job step, an order query, a service call, retries and export statuses, with an idempotent Node stand-in.

Almost every shop sends its orders somewhere else: an ERP, a warehouse system, an accounting tool. In Salesforce B2C Commerce the classic way to do that is a scheduled job that finds new orders, posts each one to the other system and marks it as exported. It sounds simple, and it is, until the other system is down for ten minutes, answers "yes" but your job crashes before it hears it, or rejects one order because of a bad address. This project builds the export with those failures in mind.

You will combine four things from earlier lessons: a custom job step (Jobs, Feeds and Import/Export), order queries (Basket, Checkout and Order Flow), the service framework (Service Framework and Third-Party Integrations) and custom attributes (Custom Objects and System Object Extensions). The general reasoning about safe retries is in Idempotency.

The real platform code here (job metadata, dw.* scripts) cannot run on this site, so it appears in read-only blocks. The runnable parts are Node.js stand-ins: a fake ERP, a fake order table and a job loop that follow the same design. They are not the Script API, and the ERP is invented. Where the platform's exact behavior matters, check the current documentation for your release.

The design in one picture

flowchart TD
    A["Scheduled job"] --> B["Query orders<br/>exportStatus = READY"]
    B --> C["Build payload<br/>for one order"]
    C --> D["POST to ERP<br/>with idempotency key"]
    D -->|"2xx"| E["Mark EXPORTED"]
    D -->|"5xx or timeout"| F["Count attempt<br/>stay READY"]
    D -->|"4xx"| G["Mark FAILED<br/>alert a human"]
    F -->|"too many attempts"| G

Each order ends in one of three places: exported, waiting for the next run, or failed for a person to look at.

The key decisions are these:

  • The order carries its own state. Platform orders have an export status (not exported, ready, exported, failed). The job reads and writes that field, so the job itself stays stateless and any run can pick up where the last one stopped.
  • Errors are sorted into two kinds. Transient errors (server error, timeout, service unavailable) deserve a retry on the next run. Permanent errors (the ERP says the data is invalid) will never succeed by waiting, so the order is marked failed and surfaced.
  • The call is idempotent. The order number is sent as an idempotency key, so a second delivery of the same order is harmless.
  • Attempts are counted. After a few tries a transient error becomes a failure for a human, otherwise one broken order is retried forever.

Step 1: describe the job step

A custom job step is declared in a steptypes.json file in your cartridge (check the documentation for the exact location in your project layout) and points to a module and a function:

JSON
{
  "step-types": {
    "script-module-step": [
      {
        "@type-id": "custom.ExportOrders",
        "@supports-parallel-execution": "false",
        "@supports-site-context": "true",
        "@supports-organization-context": "false",
        "description": "Exports orders that are ready to the ERP system",
        "module": "int_erp/cartridge/scripts/jobs/exportOrders.js",
        "function": "execute",
        "timeout-in-seconds": 900,
        "parameters": {
          "parameter": [
            {
              "@name": "maxOrders",
              "@type": "long",
              "@required": "false",
              "description": "Maximum number of orders per run"
            },
            {
              "@name": "maxAttempts",
              "@type": "long",
              "@required": "false",
              "description": "Give up after this many failed attempts"
            }
          ]
        },
        "status-codes": {
          "status": [
            { "@code": "OK", "description": "Finished without failures" },
            { "@code": "WARN", "description": "Some orders failed or will be retried" },
            { "@code": "ERROR", "description": "The step could not run" }
          ]
        }
      }
    ]
  }
}

After you upload the cartridge, the step type appears in Business Manager under Job Schedules, where an administrator adds it to a job and chooses a schedule. The parameters become fields in that screen, so values such as the batch size live in configuration, not in code.

Step 2: query the orders and call the service

The step function receives the parameters and the step execution, and returns a Status. The sketch below keeps the platform calls in one place and the business decisions in small helper functions, which is also what makes the logic testable (Testing, Debugging and Logging).

JavaScript
// int_erp/cartridge/scripts/jobs/exportOrders.js (read-only: needs the real platform)
'use strict';

var OrderMgr = require('dw/order/OrderMgr');
var Order = require('dw/order/Order');
var Status = require('dw/system/Status');
var Transaction = require('dw/system/Transaction');
var Logger = require('dw/system/Logger').getLogger('order-export', 'OrderExport');
var erpService = require('*/cartridge/scripts/services/erpService');
var payloadBuilder = require('*/cartridge/scripts/helpers/orderPayload');

function execute(parameters) {
    var maxOrders = parameters.maxOrders || 200;
    var maxAttempts = parameters.maxAttempts || 3;
    var counts = { exported: 0, retry: 0, failed: 0 };

    var orders = OrderMgr.queryOrders(
        'exportStatus = {0}',
        'creationDate asc',
        Order.EXPORT_STATUS_READY
    );

    try {
        var processed = 0;
        while (orders.hasNext() && processed < maxOrders) {
            var order = orders.next();
            processed++;
            exportOne(order, maxAttempts, counts);
        }
    } finally {
        orders.close();
    }

    Logger.info('Order export finished: {0}', JSON.stringify(counts));
    return counts.failed > 0 || counts.retry > 0 ? new Status(Status.OK, 'WARN') : new Status(Status.OK, 'OK');
}

function exportOne(order, maxAttempts, counts) {
    var attempts = (order.custom.exportAttempts || 0) + 1;
    var result = erpService.send(order.orderNo, payloadBuilder.build(order));

    Transaction.wrap(function () {
        order.custom.exportAttempts = attempts;
        if (result.ok) {
            order.setExportStatus(Order.EXPORT_STATUS_EXPORTED);
            counts.exported++;
        } else if (result.transient && attempts < maxAttempts) {
            counts.retry++; // stays READY; the next run tries again
        } else {
            order.setExportStatus(Order.EXPORT_STATUS_FAILED);
            order.custom.exportError = String(result.message).substring(0, 250);
            counts.failed++;
        }
    });
}

module.exports = { execute: execute };

A few details worth noticing:

  • The query asks only for orders whose export status is ready. Something must set that status when the order is accepted; many projects do it in a hook right after placeOrder, and the status constants (EXPORT_STATUS_READY, EXPORT_STATUS_EXPORTED, EXPORT_STATUS_FAILED) live on the Order class. Decide on one convention and write it down.
  • The iterator is closed in a finally block. Open iterators hold resources.
  • The service call happens outside the transaction. A transaction should be short, and a slow HTTP call inside it is a recipe for contention. Only the status update is wrapped.
  • exportAttempts and exportError are custom attributes you add to Order, as shown in the previous lessons on metadata.

The service wrapper decides what "transient" means:

JavaScript
// int_erp/cartridge/scripts/services/erpService.js (read-only)
'use strict';

var LocalServiceRegistry = require('dw/svc/LocalServiceRegistry');

var service = LocalServiceRegistry.createService('erp.orders', {
    createRequest: function (svc, args) {
        svc.setRequestMethod('POST');
        svc.addHeader('Content-Type', 'application/json');
        svc.addHeader('Idempotency-Key', 'order-' + args.orderNo);
        return JSON.stringify(args.payload);
    },
    parseResponse: function (svc, response) {
        return response.text;
    },
    mockCall: function () {
        return { statusCode: 201, statusMessage: 'Created', text: '{}' };
    }
});

function send(orderNo, payload) {
    var result = service.call({ orderNo: orderNo, payload: payload });
    if (result.ok) {
        return { ok: true };
    }
    // result.error holds the HTTP status code when the server answered
    var transient = result.status === 'SERVICE_UNAVAILABLE' || result.error >= 500 || result.error === 429;
    return { ok: false, transient: transient, message: result.errorMessage };
}

module.exports = { send: send };

The credentials, URL, timeout and circuit-breaker settings of erp.orders are configured in Business Manager, not in the script. Which header name the ERP uses for idempotency is up to that system; Idempotency-Key is a common convention.

Step 3: why the key matters

Imagine the ERP accepts order 1001 and replies "created", but the job is killed (a deployment, a quota, a timeout) before it writes EXPORTED. On the next run the order is still ready, so it is sent again. Without protection, the warehouse ships two parcels. With an idempotency key, the ERP recognizes order-1001 and answers "already have it", and the job marks the order exported. The platform cannot make "send an HTTP request" and "save a status" one atomic action, so the design has to tolerate the repeat.

Live stand-in 1: the job loop

The first stand-in runs the whole job three times against a fake ERP. Order 1002 hits a temporary outage once, order 1003 is rejected for good, and order 1004 was exported earlier and is never selected.

Node.js The export job: three runs, three outcomes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Read the three runs. Run 1 exports 1001 and 1005, leaves 1002 waiting after the outage and fails 1003 for good. Run 2 picks up only 1002, which now succeeds. Run 3 finds nothing to do and makes no calls at all: that is what an idempotent, status-driven job looks like when there is no work. The ERP ends up holding each order exactly once, and 1004 was never touched.

Live stand-in 2: crash after the ERP said yes

The second stand-in forces the dangerous moment: the ERP accepts the order, then the job "crashes" before saving the new status. It runs the scenario twice, once with the idempotency key and once without.

Node.js A crash between the call and the status update

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Without the key, run 2 creates a second copy, which in real life would be a second shipment. With the key, run 2 is acknowledged as a repeat and the status finally moves to EXPORTED.

Operating the export

A job that works in testing still needs care in production:

  • Alert on failed orders. An order in the failed state needs a person. Log it with a clear category, and report the count in the job status so that a monitoring rule or an email on job failure can notice it. Never log full customer details.
  • Make failed orders retryable on purpose. Add a small Business Manager or script path that resets FAILED orders to READY once the data is fixed.
  • Bound each run. The maxOrders parameter keeps a backlog from turning one run into a very long one; the next scheduled run continues.
  • Watch the clock. Choose a schedule that fits the ERP's capacity, and avoid running two overlapping executions of the same step; the metadata above sets parallel execution to false for that reason.
  • Respect privacy. The payload should contain what the ERP needs and nothing more (Security, Sessions and Compliance).
  • Think about order volume. At very high volume, a push triggered by an event may suit better than a polling job; the polling job is still a good fallback and repair tool.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is the service call made outside Transaction.wrap in the sketch?

    Only the status update needs to be atomic. Holding a transaction open during network I/O invites contention and long locks.
  2. 2. The ERP returns 422 with 'invalid address' for an order. What should the job do?

    A client error will not fix itself. Retrying wastes calls, and marking it exported would hide a real problem.
  3. 3. What problem does the Idempotency-Key header solve here?

    HTTP call and status save cannot be one atomic step. The key makes a repeat delivery harmless.
  4. 4. Why count attempts and give up after a limit?

    Without a limit, a problem that keeps looking temporary would be retried forever. After the limit it becomes a visible failure.

Summary

  • Model the export as state on the order (ready, exported, failed) so the job is stateless and every run is safe to repeat.
  • Declare a custom job step in steptypes.json, keep parameters such as batch size and attempt limit configurable, and return a meaningful Status.
  • Query with OrderMgr.queryOrders, always close the iterator, and keep service calls outside transactions.
  • Separate transient errors (retry, with an attempt limit) from permanent ones (fail and alert).
  • Send an idempotency key so a crash between the call and the status update cannot create duplicates.
  • Plan for operations: alerts, a way to requeue failed orders, bounded runs and minimal payloads.

Next, test your knowledge with the Practice Exercises with Solutions.