Skip to content
Tutorials Center
My learning

Topics

C Reference ยท string.h

C strtok() function

Reference for C strtok(): split a string at delimiter characters, why it modifies its input and keeps hidden state, empty fields, and strtok_r as the safe form.

strtok() splits a string into tokens separated by any of a set of delimiter characters. The first call receives the string; each following call receives NULL and continues where the previous one stopped. Each call returns a pointer to the next token, or NULL when none are left.

It works by overwriting delimiters with null bytes, so it modifies the string you give it, and it remembers its position in hidden static state. Both facts cause most of the bugs people have with it.

C
#include <string.h>

char *strtok(char *restrict str, const char *restrict delim);

Try it

C Splitting words and numbers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

C Empty fields vanish; the input is changed

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

Syntax

C
char *strtok(char *restrict str, const char *restrict delim);
Name Description
str The string to tokenize on the first call; NULL on later calls to continue the same string.
delim A string whose characters are all delimiters. It may differ from call to call.

Returns: a pointer to the next token (a null-terminated piece inside the original buffer), or NULL when the string has no more tokens.

Errors / undefined behavior: str must be modifiable. Passing a string literal is undefined behavior (typically a crash). Using strtok from several threads, or interleaving two tokenizations, corrupts the shared state.

Usage notes

  • It destroys the input. Copy the string first if you still need the original.
  • Runs of delimiters count as one. Leading delimiters are skipped and consecutive ones never produce empty tokens. That is helpful for words, wrong for CSV where a,,b has an empty middle field.
  • Hidden state means no reentrancy. Do not call strtok inside a loop that is already using it (for example, in a function you call from the loop), and do not use it from multiple threads. POSIX provides strtok_r(str, delim, &saveptr) with explicit state, and Microsoft's CRT offers strtok_s. C11 Annex K's strtok_s is optional and rarely implemented.
  • Tokens are pieces of the original buffer, not copies. They stay valid only as long as that buffer lives.
  • The delimiter set is characters, not a string. " ,;" means space, comma or semicolon, not the three-character sequence.
  • The first call matters. Pass the string once, then NULL. Passing the string again restarts the scan.
  • Alternatives: strspn and strcspn let you walk through a string without modifying it, and strchr handles single-character separators with empty fields.
  • Convert tokens to numbers with strtol rather than atoi, so you can detect bad input.