C Reference ยท string.h
C strtok() function
Reference for C strtok(): split a string at delimiter characters, why it modifies its input and keeps hidden state, empty fields, and strtok_r as the safe form.
strtok() splits a string into tokens separated by any of a set of delimiter characters. The first call receives the string; each following call receives NULL and continues where the previous one stopped. Each call returns a pointer to the next token, or NULL when none are left.
It works by overwriting delimiters with null bytes, so it modifies the string you give it, and it remembers its position in hidden static state. Both facts cause most of the bugs people have with it.
#include <string.h> char *strtok(char *restrict str, const char *restrict delim);
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.
Syntax
char *strtok(char *restrict str, const char *restrict delim);
| Name | Description |
|---|---|
str |
The string to tokenize on the first call; NULL on later calls to continue the same string. |
delim |
A string whose characters are all delimiters. It may differ from call to call. |
Returns: a pointer to the next token (a null-terminated piece inside the original buffer), or NULL when the string has no more tokens.
Errors / undefined behavior: str must be modifiable. Passing a string literal is undefined behavior (typically a crash). Using strtok from several threads, or interleaving two tokenizations, corrupts the shared state.
Usage notes
- It destroys the input. Copy the string first if you still need the original.
- Runs of delimiters count as one. Leading delimiters are skipped and consecutive ones never produce empty tokens. That is helpful for words, wrong for CSV where
a,,bhas an empty middle field. - Hidden state means no reentrancy. Do not call
strtokinside a loop that is already using it (for example, in a function you call from the loop), and do not use it from multiple threads. POSIX providesstrtok_r(str, delim, &saveptr)with explicit state, and Microsoft's CRT offersstrtok_s. C11 Annex K'sstrtok_sis optional and rarely implemented. - Tokens are pieces of the original buffer, not copies. They stay valid only as long as that buffer lives.
- The delimiter set is characters, not a string.
" ,;"means space, comma or semicolon, not the three-character sequence. - The first call matters. Pass the string once, then
NULL. Passing the string again restarts the scan. - Alternatives:
strspnandstrcspnlet you walk through a string without modifying it, andstrchrhandles single-character separators with empty fields. - Convert tokens to numbers with
strtolrather thanatoi, so you can detect bad input.
Related
strspnandstrcspn,strpbrk,strchr,strdup- Lessons: String functions, String conversions