


Network Working Group                                   Aaron Stone, Ed.
Internet-Draft                                           Six Apart, Ltd.
Intended status: Informational                         December 14, 2007
Expires: June 16, 2008


                        Memcache Binary Protocol
                     draft-stone-memcache-binary-01

Status of this Memo

   This document is an Internet-Draft and is NOT offered in accordance
   with Section 10 of RFC 2026, and the author does not provide the IETF
   with any rights other than to publish as an Internet-Draft.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on June 16, 2008.

Abstract

   This memo explains the memcache binary protocol for informational
   purposes.

   Memcache is a high performance key-value cache.  It is intentionally
   a dumb cache, optimized for speed only.  Applications using memcache
   do not rely on it for data -- a persistent database with guaranteed
   reliability is strongly recommended -- but applications can run much
   faster when cached data is available in memcache.








Aaron Stone               Expires June 16, 2008                 [Page 1]

Internet-Draft          Memcache Binary Protocol           December 2007


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
     1.1.  Conventions Used In This Document  . . . . . . . . . . . .  3
   2.  Packet Structure . . . . . . . . . . . . . . . . . . . . . . .  3
   3.  Defined Values . . . . . . . . . . . . . . . . . . . . . . . .  5
     3.1.  Magic Byte . . . . . . . . . . . . . . . . . . . . . . . .  5
     3.2.  Response Status  . . . . . . . . . . . . . . . . . . . . .  5
     3.3.  Command Opcodes  . . . . . . . . . . . . . . . . . . . . .  5
     3.4.  Data Types . . . . . . . . . . . . . . . . . . . . . . . .  6
   4.  Commands . . . . . . . . . . . . . . . . . . . . . . . . . . .  6
     4.1.  Get, Get Quietly . . . . . . . . . . . . . . . . . . . . .  6
     4.2.  Delete . . . . . . . . . . . . . . . . . . . . . . . . . .  7
     4.3.  Set, Add, Replace  . . . . . . . . . . . . . . . . . . . .  7
     4.4.  noop . . . . . . . . . . . . . . . . . . . . . . . . . . .  8
     4.5.  Increment, Decrement . . . . . . . . . . . . . . . . . . .  8
   5.  Example Session  . . . . . . . . . . . . . . . . . . . . . . .  9
   6.  Security Considerations  . . . . . . . . . . . . . . . . . . . 13
   7.  Normative References . . . . . . . . . . . . . . . . . . . . . 13
   Appendix A.  Acknowledgments . . . . . . . . . . . . . . . . . . . 13
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 13






























Aaron Stone               Expires June 16, 2008                 [Page 2]

Internet-Draft          Memcache Binary Protocol           December 2007


1.  Introduction

   Memcache is a high performance key-value cache.  It is intentionally
   a dumb cache, optimized for speed only.  Applications using memcache
   do not rely on it for data -- a persistent database with guaranteed
   reliability is strongly recommended -- but applications can run much
   faster when cached data is available in memcache.

   Memcache was originally written to make LiveJournal [LJ] go faster.
   It now powers all of the fastest web sites that you love.

1.1.  Conventions Used In This Document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [KEYWORDS].


2.  Packet Structure

   General format of a packet:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0/ HEADER                                                        /
       /                                                               /
       /                                                               /
       /                                                               /
       +---------------+---------------+---------------+---------------+
     16/ COMMAND-SPECIFIC EXTRAS (as needed)                           /
      +/  (note length in th extras length header field)               /
       +---------------+---------------+---------------+---------------+
      m/ Key (as needed)                                               /
      +/  (note length in key length header field)                     /
       +---------------+---------------+---------------+---------------+
      n/ Value (as needed)                                             /
      +/  (note length is total body length header field, minus        /
      +/   sum of the extras and key length body fields)               /
       +---------------+---------------+---------------+---------------+
      Total 16 bytes









Aaron Stone               Expires June 16, 2008                 [Page 3]

Internet-Draft          Memcache Binary Protocol           December 2007


   Request header:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| Magic         | Opcode        | Key length                    |
       +---------------+---------------+---------------+---------------+
      4| Extras length | Data type     | Reserved                      |
       +---------------+---------------+---------------+---------------+
      8| Total body length                                             |
       +---------------+---------------+---------------+---------------+
     12| Message ID                                                    |
       +---------------+---------------+---------------+---------------+
     Total 16 bytes

   Response header:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| Magic         | Opcode        | Status                        |
       +---------------+---------------+---------------+---------------+
      4| Extras length | Data type     | Reserved                      |
       +---------------+---------------+---------------+---------------+
      8| Total body length                                             |
       +---------------+---------------+---------------+---------------+
     12| Message ID                                                    |
       +---------------+---------------+---------------+---------------+
     Total 16 bytes

   Header fields:
   Magic               Magic number.
   Opcode              Command code.
   Key length          Length in bytes of the text key that follows the
                       command extras.
   Status              Status of the response (non-zero on error).
   Extras length       Length in bytes of the command extras.
   Data type           Reserved for future use (Sean is using this
                       soon).
   Reserved            Really reserved for future use (up for grabs).
   Total body length   Length in bytes of extra + key + value.
   Message ID          Will be copied back to you in the response.
                       FIXME: Can this be used to organize [UDP]
                       packets?





Aaron Stone               Expires June 16, 2008                 [Page 4]

Internet-Draft          Memcache Binary Protocol           December 2007


3.  Defined Values

3.1.  Magic Byte

   0x80    Request packet for this protocol version
   0x81    Response packet for this protocol version

   Magic byte / version.  For each version of the protocol, we'll use a
   different request/reponse value pair.  This is useful for protocol
   analyzers to know what a packet is in isolation from which direction
   it is moving.  Note that it is common to run a memcached instance on
   a host that also runs an application server.  Such a host will both
   send and receive memcache packets.

   The version should hopefully correspond only to different meanings of
   the command byte.  In an ideal world, we will not change the header
   format.  As reserved bytes are given defined meaning, the protocol
   version / magic byte values should be incremented.

   Traffic analysis tools are encouraged to identify memcache packets
   and provide detailed interpretation if the magic bytes are recognized
   and otherwise to provide a generic breakdown of the packet.  Note
   that the key and value positions can always be identified even if the
   magic byte or command opcode are not recognized.

3.2.  Response Status

   Possible values of this two-byte field:
   0x0000  No error
   0x0081  Unknown command
   0x0001  Key not found
   0x0002  Key exists

3.3.  Command Opcodes

   Possible values of the one-byte field:
   0x00    Get
   0x01    Set
   0x02    Add
   0x03    Replace
   0x04    Delete
   0x05    Increment
   0x06    Decrement
   0x07    Quit







Aaron Stone               Expires June 16, 2008                 [Page 5]

Internet-Draft          Memcache Binary Protocol           December 2007


   0x08    Flush
   0x09    GetQ
   0x0A    No-op
   0x0B    Version

3.4.  Data Types

   Possible values of the one-byte field:
   0x00    Raw bytes


4.  Commands

4.1.  Get, Get Quietly

      MUST have extras.
      MUST have key.
      MUST NOT have value.

   o  4 byte flags
   o  8 byte data version check

   Extra data for get/getq:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| Data version check                                            |
       |                                                               |
       +---------------+---------------+---------------+---------------+
      8| Flags                                                         |
       +---------------+---------------+---------------+---------------+
     Total 12 bytes

   The get command gets a single key.  The getq command is both mum on
   cache miss and quiet, holding its response until a non-quiet command
   is issued.

   You're not guaranteed a response to a getq cache hit until you send a
   non-getq command later, which uncorks the server which bundles up IOs
   to send to the client in one go.

   Clients should implement multi-get (still important for reducing
   network roundtrips!) as n pipelined requests, the first n-1 being
   getq, the last being a regular get. that way you're guaranteed to get
   a response, and you know when the server's done. you can also do the
   naive thing and send n pipelined gets, but then you could potentially



Aaron Stone               Expires June 16, 2008                 [Page 6]

Internet-Draft          Memcache Binary Protocol           December 2007


   get back a lot of "NOT_FOUND!" error code packets. alternatively, you
   can send 'n' getqs, followed by an 'echo' or 'noop' command.

4.2.  Delete

      MAY have extras (FIXME: Is it OK to issue a delete without
      extras?).
      MUST have key.
      MUST NOT have value.

   o  4 byte expiration time

   Extra data for delete:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| Expiration                                                    |
       +---------------+---------------+---------------+---------------+
     Total 4 bytes

   When allows you to 'reserve' a key.  When 'when' is set for, say, ten
   seconds in the future, the 'add' and 'replace' operations will fail
   for that key until ten seconds from now.  The 'set' operation will
   succeed regardless of any reserved deletes.  FIXME: Is the
   reservation also cancelled?  Say there's a delete with a 10 second
   hold.  Two seconds later, an 'add' is received.  It fails.  Two
   second later, a 'set' is received.  Is succeeds unconditionally.
   What if another 'add' is received two more seconds later (a total of
   six seconds since the original 10 second delete-hold, thus still
   within its purview).

4.3.  Set, Add, Replace

      MUST have extras.
      MUST have key.
      MUST have value.

   o  4 byte flags
   o  4 byte expiration time
   o  8 byte data version check









Aaron Stone               Expires June 16, 2008                 [Page 7]

Internet-Draft          Memcache Binary Protocol           December 2007


   Extra data for set/add/replace:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| Data version check                                            |
       |                                                               |
       +---------------+---------------+---------------+---------------+
      8| Flags                                                         |
       +---------------+---------------+---------------+---------------+
     12| Expiration                                                    |
       +---------------+---------------+---------------+---------------+
     Total 16 bytes

   If the Data Version Check is present and nonzero, the set MUST
   succeed if the key exists and has a version identifier identical to
   the provided value, and MUST NOT succeed if the key does not exist or
   has a different version identifier.  The set response packet will
   include the same values in all three fields.

   If the Data Version Check is zero, the set MUST succeed
   unconditionally.  The set response packet will include idential
   values for flags and expiration, and a new value for Data Version
   Check, which the client SHOULD keep track of.

   The key MAY be reserved according to Section 4.2, causing the set to
   fail.

4.4.  noop

      MUST NOT have extras.
      MUST NOT have key.
      MUST NOT have value.

   Used as a keep alive.  Flushes outstanding getq's.

4.5.  Increment, Decrement

      MUST have extras.
      MUST have key.
      MUST NOT have value.

   o  8 byte value to add / subtract (FIXME: Is this unsigned?)
   o  8 byte initial value (unsigned)
   o  4 byte expiration time





Aaron Stone               Expires June 16, 2008                 [Page 8]

Internet-Draft          Memcache Binary Protocol           December 2007


   Extra data for incr/decr:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| Amount to add                                                 |
       |                                                               |
       +---------------+---------------+---------------+---------------+
      8| Initial value                                                 |
       |                                                               |
       +---------------+---------------+---------------+---------------+
     16| Expiration                                                    |
       +---------------+---------------+---------------+---------------+
     Total 20 bytes

   incr/decr response body:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| 64-bit unsigned response.                                     |
       |                                                               |
       +---------------+---------------+---------------+---------------+
     Total 8 bytes

   These commands will either add or remove the specified amount to the
   requested counter.  If the counter does not exist, one of two things
   may happen:
   1.  If the expiration value is all one-bits (0xffffffff), the
       operation will fail with NOT_FOUND.
   2.  For all other expiration values, the operation will succeed by
       seeding the value for this key with the provided initial value to
       expire with the provided expiration time.

   Note that in the creation case, flags will be set to zero (FIXME:
   Should they be provided here as well?)


5.  Example Session

   We start up our application, and it asks for the value associated
   with the 'Hello' key.







Aaron Stone               Expires June 16, 2008                 [Page 9]

Internet-Draft          Memcache Binary Protocol           December 2007


   Get request:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| 0x80          | 0x00          | 5 in big endian (BE)          |
       +---------------+---------------+---------------+---------------+
       | 12 in BE      | 0x00          |                               |
       +---------------+---------------+---------------+---------------+
       | 17 in BE                                                      |
       +---------------+---------------+---------------+---------------+
       | 0xDEADBEEF                                                    |
       +---------------+---------------+---------------+---------------+
     16| 0x00000000                                                    |
       +---------------+---------------+---------------+---------------+
     24| 0xDECAF 0x15 0xBAD 0xC0FFEE                                   |
       |                                                               |
       +---------------+---------------+---------------+---------------+
     28| 'H'             'e'             'l'             'l'           |
       | 'o'           |
       +---------------+
     Total 33 bytes (16 header + 12 get-extras + 5 key)

   Since nobody has set this key, it returns not found.

   Get response:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| 0x81          | 0x00          | 0x0001                        |
       +---------------+---------------+---------------+---------------+
       | 0 in BE       | 0x00          |                               |
       +---------------+---------------+---------------+---------------+
       | 0 in BE                                                       |
       +---------------+---------------+---------------+---------------+
       | 0xDEADBEEF                                                    |
       +---------------+---------------+---------------+---------------+
     Total 16 bytes

   Well, looks like we need to set the key!  Let's set it to expire on
   December 15, 2007 at 9:51:09 PM.







Aaron Stone               Expires June 16, 2008                [Page 10]

Internet-Draft          Memcache Binary Protocol           December 2007


   Set request:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| 0x80          | 0x01          | 5 in BE                       |
       +---------------+---------------+---------------+---------------+
       | 16 in BE      | 0x00          |                               |
       +---------------+---------------+---------------+---------------+
       | 26 in BE                                                      |
       +---------------+---------------+---------------+---------------+
       | 0xDA7ABA5E                                                    |
       +---------------+---------------+---------------+---------------+
     16| 0x00000000                                                    |
       +---------------+---------------+---------------+---------------+
     20| 0xDCCB4674                                                    |
       +---------------+---------------+---------------+---------------+
     24| 0xDECAF 0x15 0xBAD 0xC0FFEE                                   |
       |                                                               |
       +---------------+---------------+---------------+---------------+
     32| 'H'             'e'             'l'             'l'           |
       | 'o'           | 'W'             'o'             'r'           |
       | 'l'             'd'           |
       +---------------+---------------+
     Total 42 bytes (16 header + 16 set-extras + 5 key + 5 value)

   The set succeeds.























Aaron Stone               Expires June 16, 2008                [Page 11]

Internet-Draft          Memcache Binary Protocol           December 2007


   Set response:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| 0x81          | 0x01          | 0x0000                        |
       +---------------+---------------+---------------+---------------+
       | 16 in BE      | 0x00          |                               |
       +---------------+---------------+---------------+---------------+
       | 16 in BE                                                      |
       +---------------+---------------+---------------+---------------+
       | 0xDA7ABA5E                                                    |
       +---------------+---------------+---------------+---------------+
     16| 0x00000000                                                    |
       +---------------+---------------+---------------+---------------+
     20| 0xDCCB4674                                                    |
       +---------------+---------------+---------------+---------------+
     24| 0xDECAF 0x15 0xBAD 0xC0FFEE                                   |
       |                                                               |
       +---------------+---------------+---------------+---------------+
     Total 32 bytes (16 header + 16 set-extras)

   If the original get request is sent again, the key would be found.

   Get response:

     Byte/     0       |       1       |       2       |       3       |
        /              |               |               |               |
       |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
       +---------------+---------------+---------------+---------------+
      0| 0x81          | 0x00          | 0x00          |               |
       +---------------+---------------+---------------+---------------+
       | 12 in BE      | 0x00          |                               |
       +---------------+---------------+---------------+---------------+
       | 17 in BE                                                      |
       +---------------+---------------+---------------+---------------+
       | 0xDEADBEEF                                                    |
       +---------------+---------------+---------------+---------------+
     16| 0xDCCB4674                                                    |
       +---------------+---------------+---------------+---------------+
     24| 0xDECAF 0x15 0xBAD 0xC0FFEE                                   |
       |                                                               |
       +---------------+---------------+---------------+---------------+
     28| 'W'             'o'             'r'             'l'           |
       | 'd'           |
       +---------------+
     Total 33 bytes (16 header + 12 get-extras + 5 value)



Aaron Stone               Expires June 16, 2008                [Page 12]

Internet-Draft          Memcache Binary Protocol           December 2007


6.  Security Considerations

   Memcache has no authentication or security layers whatsoever.  It is
   RECOMMENDED that memcache be deployed strictly on closed, protected,
   back-end networks within a single data center, within a single
   cluster of servers, or even on a single host, providing shared
   caching for multiple applications.  Memcache MUST NOT be made
   available on a public network.


7.  Normative References

   [KEYWORDS]
              Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [LJ]       Danga Interactive, "LJ NEEDS MOAR SPEED", 10 1999.

   [UDP]      Postel, J., "User Datagram Protocol", STD 6, RFC 768,
              August 1980.


Appendix A.  Acknowledgments

   Thanks to Brad Fitzpatrick, Anatoly Vorobey, Steven Grimm, and Dustin
   Sallings, for their work on the memcached server.

   Thanks to Sean Chittenden, Jonathan Steinert, Brian Aker, Evan
   Martin, Nathan Neulinger, Eric Hodel, Michael Johnson, Paul Querna,
   Jamie McCarthy, Philip Neustrom, Andrew O'Brien, Josh Rotenberg,
   Robin H. Johnson, Tim Yardley, Paolo Borelli, Eli Bingham, Jean-
   Francois Bustarret, Paul G, Paul Lindner, Alan Kasindorf, Chris
   Goffinet, Tomash Brechko, and others for their work reporting bugs
   and maintaining memcached client libraries and bindings in many
   languages.


Author's Address

   Aaron Stone (editor)
   Six Apart, Ltd.
   548 4th Street
   San Francisco, CA  94107
   USA

   Email: aaron@serendipity.palo-alto.ca.us





Aaron Stone               Expires June 16, 2008                [Page 13]

