Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 37 · Getting Started

The Memcached Text Protocol

Speak to Memcached by hand: the line-based text protocol, the set and get formats, response codes like STORED and END, and a Python parser that mimics a server.

Memcached clients hide the network from you, but underneath there is a very small protocol, and understanding it removes most of the mystery. The classic text protocol is line-based: you send a command line, optionally a data block, and the server answers with a line or two. You can type it into telnet or nc and see everything that happens.

In this lesson you will read real protocol sessions, learn the format of the storage and retrieval commands, and then run a small parser written in Python that behaves like a server for the same lines.

A real session

With a server on localhost, connect and type these lines. Each line ends with the two characters carriage return and line feed (\r\n); nc and telnet send that when you press Enter, depending on the options.

TEXT
$ nc 127.0.0.1 11211
set greeting 0 300 5
hello
STORED
get greeting
VALUE greeting 0 5
hello
END
get missing
END
delete greeting
DELETED
delete greeting
NOT_FOUND
quit

Read it line by line:

  1. set greeting 0 300 5 is the command line. Then comes a data block of exactly 5 bytes, hello, followed by \r\n.
  2. STORED means the item is saved.
  3. get greeting returns a header VALUE <key> <flags> <bytes>, then the data, then END.
  4. A miss is not an error: the answer is just END with no VALUE lines.
  5. delete answers DELETED or NOT_FOUND.

The storage command format

All storage commands share one shape:

TEXT
<command> <key> <flags> <exptime> <bytes> [noreply]\r\n
<data block>\r\n
Field Meaning
key Up to 250 bytes, no spaces or control characters
flags A 32-bit number the client can use for anything (often a type tag)
exptime Seconds until expiry; 0 means never; more than 30 days is a Unix timestamp; negative means already expired
bytes Length of the data block in bytes, not characters
noreply Optional: the server sends no response

The bytes count is a frequent source of bugs. The server reads exactly that many bytes, so if you say 5 and send 6, the extra byte confuses the next command; if you say 5 and send 4, the server waits for more. Non-ASCII text is longer in bytes than in characters: "café" is 4 characters but 5 bytes in UTF-8.

Retrieval commands

TEXT
get <key> [<key> ...]
gets <key> [<key> ...]

get can take many keys at once and returns one VALUE block per hit, in any order, then END. gets is the same but adds a CAS unique number to each header, which you will use for safe concurrent updates in CAS and Concurrent Updates.

Response words

Response Meaning
STORED The item was saved
NOT_STORED add found the key, or replace/append/prepend did not
EXISTS A cas token was out of date
NOT_FOUND The key does not exist
DELETED, TOUCHED Success for delete and touch
ERROR Unknown command
CLIENT_ERROR <msg> You sent something malformed (key too long, bad data length)
SERVER_ERROR <msg> The server failed (for example out of memory)

A protocol parser in Python

The block below is a stand-in: a teaching parser that handles set, add, get, delete and version over a byte string, the same way a tiny server would. It is not the real server and skips many details (key limits, noreply, other commands), but the message formats are the real ones.

Python Talk to a pretend server

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the second command, add greeting, answers NOT_STORED because the key already exists; set would have overwritten it. You will meet this difference again in Storage Commands.

The byte-count trap, demonstrated

Here is the bytes field going wrong. The text contains an accented letter, so counting characters gives the wrong length.

Python Characters are not bytes

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real client libraries count bytes for you. When you write protocol by hand or debug with nc, always count bytes.

Binary and meta protocols

Besides the text protocol, older servers also speak a binary protocol. It is deprecated: Memcached 1.6 moved toward a newer meta protocol, still text-based, with commands mg (meta get), ms (meta set), md (meta delete), ma (meta arithmetic) and mn (meta no-op). Meta commands let a client ask only for the pieces it needs (value, TTL, CAS id) and support features like stale-while-revalidate that fight cache stampedes. You will meet the stampede problem in Stampede Protection. Everything in this course uses the classic text commands because every client supports them.

TEXT
ms greeting 5 T300
hello
HD
mg greeting v t
VA 5 t298
hello

Meta commands, read-only: HD is "stored", VA introduces the value, and t298 is the remaining TTL.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. What does the server reply to get for a key that does not exist?

    A miss on get is not an error. The server just sends END. NOT_FOUND is for delete, touch, incr and similar commands.
  2. 2. In set k 0 60 5, what does the 5 mean?

    The last number is the byte length of the data block that follows. The 60 is the exptime and the 0 the flags.
  3. 3. Which response means a cas token was out of date?

    EXISTS tells you someone changed the item since you read it. NOT_FOUND would mean it was gone entirely.
  4. 4. How many bytes is the string café in UTF-8?

    The letters c, a and f take one byte each and the accented é takes two, so the total is 5 bytes.

Summary

  • The text protocol is line-based: a command line, an optional data block, a short reply.
  • Storage commands take key flags exptime bytes, and the byte count must be exact.
  • A get miss returns just END; other commands use words like NOT_FOUND and NOT_STORED.
  • gets adds a CAS token; the newer meta protocol (mg, ms) is more flexible.
  • Client libraries hide all of this, but it is how you debug.

Next: Your First Client in Python