Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 36 ยท Patterns and Analysis

Data Parameterization and SharedArray in k6

Feed k6 tests with realistic data: load JSON and CSV with SharedArray, pick unique rows per VU and iteration, avoid memory blowups, generate random data and handle files and secrets safely.

If every virtual user logs in as the same account and searches for the same word, you are mostly testing your caches. Real traffic has thousands of different users, products and queries. Parameterization means feeding your script varied data. k6 gives you a handful of tools, and one important trap: memory.

Honesty note. k6 cannot run on this site, so k6/data code and shell commands are read-only. Runnable blocks are Node stand-ins that implement the same selection logic (round-robin, unique rows, weighted choice) so you can check it.

Where data can come from

Source How Notes
Environment variables __ENV.NAME with -e NAME=value Small configuration and secrets
Files (JSON, CSV, text) open('./data.json') Init context only, read as a string or bytes
Generated in the script Math.random(), randomString() Cheap, but can collide
Previous responses res.json() Correlation (see Authentication and Correlation)
Setup function return value of setup() Computed once, copied to each VU

The memory trap and SharedArray

open() can only be called in the init context, and the init code runs once per VU. If a script reads a 100 MB file at the top level, then 500 VUs hold 500 copies: 50 GB of RAM. The fix is SharedArray, which loads the data once and shares it read-only between all VUs:

JavaScript
import { SharedArray } from 'k6/data';
import http from 'k6/http';

const users = new SharedArray('users', function () {
  // This function runs once, not per VU.
  return JSON.parse(open('./users.json'));
});

export default function () {
  const user = users[Math.floor(Math.random() * users.length)];
  http.post(`${__ENV.BASE_URL}/login`, JSON.stringify(user), {
    headers: { 'Content-Type': 'application/json' },
  });
}

Rules:

  • The first argument is a unique name; the function must return an array.
  • The array is read-only and behaves like a normal array for indexing, length, map and so on.
  • It is only worth it for data bigger than a few kilobytes; for tiny arrays a plain constant is fine.

CSV

k6 has no built-in CSV parser, but the Papa Parse library in the official JS library collection works:

JavaScript
import papaparse from 'https://jslib.k6.io/papaparse/5.1.1/index.js';
import { SharedArray } from 'k6/data';

const rows = new SharedArray('products', function () {
  return papaparse.parse(open('./products.csv'), { header: true }).data;
});

With header: true each row becomes an object keyed by the column names. Watch for a trailing empty row; filter it out.

For very large files, k6's experimental file system module (k6/experimental/fs) can stream a file instead of loading it whole; check the documentation for your version.

Node.js Stand-in: parse CSV into rows and pick one per iteration

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Choosing a row: random, per VU, per iteration

Random choice is simple, but may select the same row twice at the same moment. When uniqueness matters (accounts that cannot log in twice, orders that need a fresh product), derive the index from a counter:

Index Expression Use
Per VU, fixed users[__VU - 1] One account per VU for the whole test
Per VU, rotating users[(__VU - 1 + __ITER * vus) % users.length] Different rows over time
Unique across the whole test users[exec.scenario.iterationInTest % users.length] One row per iteration, no repeats until wrap-around
Random users[Math.floor(Math.random() * users.length)] Independent, may collide

__VU (starting at 1) and __ITER (starting at 0 for each VU) are global variables. In scenarios with several executors prefer the k6/execution module:

JavaScript
import exec from 'k6/execution';

export default function () {
  const n = exec.scenario.iterationInTest;     // 0, 1, 2... across all VUs of this scenario
  const vu = exec.vu.idInTest;                 // unique VU id across the whole test
  const user = users[n % users.length];
}

If a test needs more unique rows than the file holds (50,000 iterations, 1,000 users), the wrap-around is intentional: say so and check that the application tolerates repeat logins.

Node.js Stand-in: unique rows per iteration with wrap-around

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Generating data

When you need fresh values (emails, names, ids) rather than reading them:

JavaScript
import { randomString, randomIntBetween, randomItem, uuidv4 } from 'https://jslib.k6.io/k6-utils/1.4.0/index.js';

const email = `load-${randomString(8)}@example.test`;
const qty = randomIntBetween(1, 5);
const method = randomItem(['card', 'paypal', 'invoice']);
const requestId = uuidv4();

For repeatable runs, remember that random data makes two runs differ. If you need determinism, derive values from __VU/__ITER instead of Math.random().

Weighted choice

Real users do not pick products uniformly. A few are very popular. Implement weighted picking to give your cache a realistic hit rate:

JavaScript
function weighted(items) {          // items: [[value, weight], ...]
  const total = items.reduce((s, [, w]) => s + w, 0);
  let r = Math.random() * total;
  for (const [value, w] of items) { if ((r -= w) < 0) return value; }
}
const category = weighted([['laptops', 60], ['phones', 30], ['cables', 10]]);
Node.js Stand-in: weighted choice follows the weights

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Data hygiene

  • Secrets stay out of the repo. Pass tokens with -e or CI secrets, never as data files in source control.
  • Test accounts, not real people. Never load production personal data into a load test.
  • Clean up. A load test that creates 200,000 orders leaves 200,000 orders. Use a disposable environment or a teardown() that deletes test data.
  • Keep data and script together so a test is reproducible, and version large data separately.
  • Match the data to production shape. A table with 100 rows behaves differently from one with 100 million; indexes, caches and query plans all change.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use SharedArray instead of a plain top-level array loaded with open()?

    Init code runs per VU, so a big file would be copied for every VU. SharedArray keeps one read-only copy.
  2. 2. Where can open() be called?

    open() works at init time only, which is why data is usually loaded with SharedArray at the top of the script.
  3. 3. Which expression gives a unique row for each iteration across the whole scenario?

    iterationInTest increments across all VUs of the scenario, so each iteration gets a different index until the data wraps around.
  4. 4. How do you pass a secret API token to a k6 script safely?

    Environment variables keep secrets out of the repository and the script text.

Summary

  • Use SharedArray for large read-only data; open() works only at init.
  • Pick rows with __VU, __ITER or k6/execution for uniqueness; random is simple but may collide.
  • Generate values with k6-utils; use weighted choice for realistic popularity.
  • Keep secrets in env vars, use disposable test data and clean up.

Next: Analyzing Results and Finding Bottlenecks