|
0
|
1 Protocol
|
|
|
2 --------
|
|
|
3
|
|
|
4 Clients of memcached communicate with server through TCP connections.
|
|
|
5 (A UDP interface is also available; details are below under "UDP
|
|
|
6 protocol.") A given running memcached server listens on some
|
|
|
7 (configurable) port; clients connect to that port, send commands to
|
|
|
8 the server, read responses, and eventually close the connection.
|
|
|
9
|
|
|
10 There is no need to send any command to end the session. A client may
|
|
|
11 just close the connection at any moment it no longer needs it. Note,
|
|
|
12 however, that clients are encouraged to cache their connections rather
|
|
|
13 than reopen them every time they need to store or retrieve data. This
|
|
|
14 is because memcached is especially designed to work very efficiently
|
|
|
15 with a very large number (many hundreds, more than a thousand if
|
|
|
16 necessary) of open connections. Caching connections will eliminate the
|
|
|
17 overhead associated with establishing a TCP connection (the overhead
|
|
|
18 of preparing for a new connection on the server side is insignificant
|
|
|
19 compared to this).
|
|
|
20
|
|
|
21 There are two kinds of data sent in the memcache protocol: text lines
|
|
|
22 and unstructured data. Text lines are used for commands from clients
|
|
|
23 and responses from servers. Unstructured data is sent when a client
|
|
|
24 wants to store or retrieve data. The server will transmit back
|
|
|
25 unstructured data in exactly the same way it received it, as a byte
|
|
|
26 stream. The server doesn't care about byte order issues in
|
|
|
27 unstructured data and isn't aware of them. There are no limitations on
|
|
|
28 characters that may appear in unstructured data; however, the reader
|
|
|
29 of such data (either a client or a server) will always know, from a
|
|
|
30 preceding text line, the exact length of the data block being
|
|
|
31 transmitted.
|
|
|
32
|
|
|
33 Text lines are always terminated by \r\n. Unstructured data is _also_
|
|
|
34 terminated by \r\n, even though \r, \n or any other 8-bit characters
|
|
|
35 may also appear inside the data. Therefore, when a client retrieves
|
|
|
36 data from a server, it must use the length of the data block (which it
|
|
|
37 will be provided with) to determine where the data block ends, and not
|
|
|
38 the fact that \r\n follows the end of the data block, even though it
|
|
|
39 does.
|
|
|
40
|
|
|
41 Keys
|
|
|
42 ----
|
|
|
43
|
|
|
44 Data stored by memcached is identified with the help of a key. A key
|
|
|
45 is a text string which should uniquely identify the data for clients
|
|
|
46 that are interested in storing and retrieving it. Currently the
|
|
|
47 length limit of a key is set at 250 characters (of course, normally
|
|
|
48 clients wouldn't need to use such long keys); the key must not include
|
|
|
49 control characters or whitespace.
|
|
|
50
|
|
|
51 Commands
|
|
|
52 --------
|
|
|
53
|
|
|
54 There are three types of commands.
|
|
|
55
|
|
|
56 Storage commands (there are three: "set", "add" and "replace") ask the
|
|
|
57 server to store some data identified by a key. The client sends a
|
|
|
58 command line, and then a data block; after that the client expects one
|
|
|
59 line of response, which will indicate success or faulure.
|
|
|
60
|
|
|
61 Retrieval commands (there is only one: "get") ask the server to
|
|
|
62 retrieve data corresponding to a set of keys (one or more keys in one
|
|
|
63 request). The client sends a command line, which includes all the
|
|
|
64 requested keys; after that for each item the server finds it sends to
|
|
|
65 the client one response line with information about the item, and one
|
|
|
66 data block with the item's data; this continues until the server
|
|
|
67 finished with the "END" response line.
|
|
|
68
|
|
|
69 All other commands don't involve unstructured data. In all of them,
|
|
|
70 the client sends one command line, and expects (depending on the
|
|
|
71 command) either one line of response, or several lines of response
|
|
|
72 ending with "END" on the last line.
|
|
|
73
|
|
|
74 A command line always starts with the name of the command, followed by
|
|
|
75 parameters (if any) delimited by whitespace. Command names are
|
|
|
76 lower-case and are case-sensitive.
|
|
|
77
|
|
|
78 Expiration times
|
|
|
79 ----------------
|
|
|
80
|
|
|
81 Some commands involve a client sending some kind of expiration time
|
|
|
82 (relative to an item or to an operation requested by the client) to
|
|
|
83 the server. In all such cases, the actual value sent may either be
|
|
|
84 Unix time (number of seconds since January 1, 1970, as a 32-bit
|
|
|
85 value), or a number of seconds starting from current time. In the
|
|
|
86 latter case, this number of seconds may not exceed 60*60*24*30 (number
|
|
|
87 of seconds in 30 days); if the number sent by a client is larger than
|
|
|
88 that, the server will consider it to be real Unix time value rather
|
|
|
89 than an offset from current time.
|
|
|
90
|
|
|
91
|
|
|
92 Error strings
|
|
|
93 -------------
|
|
|
94
|
|
|
95 Each command sent by a client may be answered with an error string
|
|
|
96 from the server. These error strings come in three types:
|
|
|
97
|
|
|
98 - "ERROR\r\n"
|
|
|
99
|
|
|
100 means the client sent a nonexistent command name.
|
|
|
101
|
|
|
102 - "CLIENT_ERROR <error>\r\n"
|
|
|
103
|
|
|
104 means some sort of client error in the input line, i.e. the input
|
|
|
105 doesn't conform to the protocol in some way. <error> is a
|
|
|
106 human-readable error string.
|
|
|
107
|
|
|
108 - "SERVER_ERROR <error>\r\n"
|
|
|
109
|
|
|
110 means some sort of server error prevents the server from carrying
|
|
|
111 out the command. <error> is a human-readable error string. In cases
|
|
|
112 of severe server errors, which make it impossible to continue
|
|
|
113 serving the client (this shouldn't normally happen), the server will
|
|
|
114 close the connection after sending the error line. This is the only
|
|
|
115 case in which the server closes a connection to a client.
|
|
|
116
|
|
|
117
|
|
|
118 In the descriptions of individual commands below, these error lines
|
|
|
119 are not again specifically mentioned, but clients must allow for their
|
|
|
120 possibility.
|
|
|
121
|
|
|
122
|
|
|
123 Storage commands
|
|
|
124 ----------------
|
|
|
125
|
|
|
126 First, the client sends a command line which looks like this:
|
|
|
127
|
|
|
128 <command name> <key> <flags> <exptime> <bytes>\r\n
|
|
|
129
|
|
|
130 - <command name> is "set", "add" or "replace"
|
|
|
131
|
|
|
132 "set" means "store this data".
|
|
|
133
|
|
|
134 "add" means "store this data, but only if the server *doesn't* already
|
|
|
135 hold data for this key".
|
|
|
136
|
|
|
137 "replace" means "store this data, but only if the server *does*
|
|
|
138 already hold data for this key".
|
|
|
139
|
|
|
140 - <key> is the key under which the client asks to store the data
|
|
|
141
|
|
|
142 - <flags> is an arbitrary 16-bit unsigned integer (written out in
|
|
|
143 decimal) that the server stores along with the data and sends back
|
|
|
144 when the item is retrieved. Clients may use this as a bit field to
|
|
|
145 store data-specific information; this field is opaque to the server.
|
|
|
146 Note that in memcached 1.2.1 and higher, flags may be 32-bits, instead
|
|
|
147 of 16, but you might want to restrict yourself to 16 bits for
|
|
|
148 compatibility with older versions.
|
|
|
149
|
|
|
150 - <exptime> is expiration time. If it's 0, the item never expires
|
|
|
151 (although it may be deleted from the cache to make place for other
|
|
|
152 items). If it's non-zero (either Unix time or offset in seconds from
|
|
|
153 current time), it is guaranteed that clients will not be able to
|
|
|
154 retrieve this item after the expiration time arrives (measured by
|
|
|
155 server time).
|
|
|
156
|
|
|
157 - <bytes> is the number of bytes in the data block to follow, *not*
|
|
|
158 including the delimiting \r\n. <bytes> may be zero (in which case
|
|
|
159 it's followed by an empty data block).
|
|
|
160
|
|
|
161 After this line, the client sends the data block:
|
|
|
162
|
|
|
163 <data block>\r\n
|
|
|
164
|
|
|
165 - <data block> is a chunk of arbitrary 8-bit data of length <bytes>
|
|
|
166 from the previous line.
|
|
|
167
|
|
|
168 After sending the command line and the data blockm the client awaits
|
|
|
169 the reply, which may be:
|
|
|
170
|
|
|
171 - "STORED\r\n", to indicate success.
|
|
|
172
|
|
|
173 - "NOT_STORED\r\n" to indicate the data was not stored, but not
|
|
|
174 because of an error. This normally means that either that the
|
|
|
175 condition for an "add" or a "replace" command wasn't met, or that the
|
|
|
176 item is in a delete queue (see the "delete" command below).
|
|
|
177
|
|
|
178
|
|
|
179 Retrieval command:
|
|
|
180 ------------------
|
|
|
181
|
|
|
182 The retrieval command looks like this:
|
|
|
183
|
|
|
184 get <key>*\r\n
|
|
|
185
|
|
|
186 - <key>* means one or more key strings separated by whitespace.
|
|
|
187
|
|
|
188 After this command, the client expects zero or more items, each of
|
|
|
189 which is received as a text line followed by a data block. After all
|
|
|
190 the items have been transmitted, the server sends the string
|
|
|
191
|
|
|
192 "END\r\n"
|
|
|
193
|
|
|
194 to indicate the end of response.
|
|
|
195
|
|
|
196 Each item sent by the server looks like this:
|
|
|
197
|
|
|
198 VALUE <key> <flags> <bytes>\r\n
|
|
|
199 <data block>\r\n
|
|
|
200
|
|
|
201 - <key> is the key for the item being sent
|
|
|
202
|
|
|
203 - <flags> is the flags value set by the storage command
|
|
|
204
|
|
|
205 - <bytes> is the length of the data block to follow, *not* including
|
|
|
206 its delimiting \r\n
|
|
|
207
|
|
|
208 - <data block> is the data for this item.
|
|
|
209
|
|
|
210 If some of the keys appearing in a retrieval request are not sent back
|
|
|
211 by the server in the item list this means that the server does not
|
|
|
212 hold items with such keys (because they were never stored, or stored
|
|
|
213 but deleted to make space for more items, or expired, or explicitly
|
|
|
214 deleted by a client).
|
|
|
215
|
|
|
216
|
|
|
217
|
|
|
218 Deletion
|
|
|
219 --------
|
|
|
220
|
|
|
221 The command "delete" allows for explicit deletion of items:
|
|
|
222
|
|
|
223 delete <key> <time>\r\n
|
|
|
224
|
|
|
225 - <key> is the key of the item the client wishes the server to delete
|
|
|
226
|
|
|
227 - <time> is the amount of time in seconds (or Unix time until which)
|
|
|
228 the client wishes the server to refuse "add" and "replace" commands
|
|
|
229 with this key. For this amount of item, the item is put into a
|
|
|
230 delete queue, which means that it won't possible to retrieve it by
|
|
|
231 the "get" command, but "add" and "replace" command with this key
|
|
|
232 will also fail (the "set" command will succeed, however). After the
|
|
|
233 time passes, the item is finally deleted from server memory.
|
|
|
234
|
|
|
235 The parameter <time> is optional, and, if absent, defaults to 0
|
|
|
236 (which means that the item will be deleted immediately and further
|
|
|
237 storage commands with this key will succeed).
|
|
|
238
|
|
|
239 The response line to this command can be one of:
|
|
|
240
|
|
|
241 - "DELETED\r\n" to indicate success
|
|
|
242
|
|
|
243 - "NOT_FOUND\r\n" to indicate that the item with this key was not
|
|
|
244 found.
|
|
|
245
|
|
|
246 See the "flush_all" command below for immediate invalidation
|
|
|
247 of all existing items.
|
|
|
248
|
|
|
249
|
|
|
250 Increment/Decrement
|
|
|
251 -------------------
|
|
|
252
|
|
|
253 Commands "incr" and "decr" are used to change data for some item
|
|
|
254 in-place, incrementing or decrementing it. The data for the item is
|
|
|
255 treated as decimal representation of a 32-bit unsigned integer. If the
|
|
|
256 current data value does not conform to such a representation, the
|
|
|
257 commands behave as if the value were 0. Also, the item must already
|
|
|
258 exist for incr/decr to work; these commands won't pretend that a
|
|
|
259 non-existent key exists with value 0; instead, they will fail.
|
|
|
260
|
|
|
261 The client sends the command line:
|
|
|
262
|
|
|
263 incr <key> <value>\r\n
|
|
|
264
|
|
|
265 or
|
|
|
266
|
|
|
267 decr <key> <value>\r\n
|
|
|
268
|
|
|
269 - <key> is the key of the item the client wishes to change
|
|
|
270
|
|
|
271 - <value> is the amount by which the client wants to increase/decrease
|
|
|
272 the item. It is a decimal representation of a 32-bit unsigned integer.
|
|
|
273
|
|
|
274 The response will be one of:
|
|
|
275
|
|
|
276 - "NOT_FOUND\r\n" to indicate the item with this value was not found
|
|
|
277
|
|
|
278 - <value>\r\n , where <value> is the new value of the item's data,
|
|
|
279 after the increment/decrement operation was carried out.
|
|
|
280
|
|
|
281 Note that underflow in the "decr" command is caught: if a client tries
|
|
|
282 to decrease the value below 0, the new value will be 0. Overflow in
|
|
|
283 the "incr" command is not checked.
|
|
|
284
|
|
|
285 Note also that decrementing a number such that it loses length isn't
|
|
|
286 guaranteed to decrement its returned length. The number MAY be
|
|
|
287 space-padded at the end, but this is purely an implementation
|
|
|
288 optimization, so you also shouldn't rely on that.
|
|
|
289
|
|
|
290 Statistics
|
|
|
291 ----------
|
|
|
292
|
|
|
293 The command "stats" is used to query the server about statistics it
|
|
|
294 maintains and other internal data. It has two forms. Without
|
|
|
295 arguments:
|
|
|
296
|
|
|
297 stats\r\n
|
|
|
298
|
|
|
299 it causes the server to output general-purpose statistics and
|
|
|
300 settings, documented below. In the other form it has some arguments:
|
|
|
301
|
|
|
302 stats <args>\r\n
|
|
|
303
|
|
|
304 Depending on <args>, various internal data is sent by the server. The
|
|
|
305 kinds of arguments and the data sent are not documented in this vesion
|
|
|
306 of the protocol, and are subject to change for the convenience of
|
|
|
307 memcache developers.
|
|
|
308
|
|
|
309
|
|
|
310 General-purpose statistics
|
|
|
311 --------------------------
|
|
|
312
|
|
|
313 Upon receiving the "stats" command without arguments, the server sents
|
|
|
314 a number of lines which look like this:
|
|
|
315
|
|
|
316 STAT <name> <value>\r\n
|
|
|
317
|
|
|
318 The server terminates this list with the line
|
|
|
319
|
|
|
320 END\r\n
|
|
|
321
|
|
|
322 In each line of statistics, <name> is the name of this statistic, and
|
|
|
323 <value> is the data. The following is the list of all names sent in
|
|
|
324 response to the "stats" command, together with the type of the value
|
|
|
325 sent for this name, and the meaning of the value.
|
|
|
326
|
|
|
327 In the type column below, "32u" means a 32-bit unsigned integer, "64u"
|
|
|
328 means a 64-bit unsigner integer. '32u:32u' means two 32-but unsigned
|
|
|
329 integers separated by a colon.
|
|
|
330
|
|
|
331
|
|
|
332 Name Type Meaning
|
|
|
333 ----------------------------------
|
|
|
334 pid 32u Process id of this server process
|
|
|
335 uptime 32u Number of seconds this server has been running
|
|
|
336 time 32u current UNIX time according to the server
|
|
|
337 version string Version string of this server
|
|
|
338 rusage_user 32u:32u Accumulated user time for this process
|
|
|
339 (seconds:microseconds)
|
|
|
340 rusage_system 32u:32u Accumulated system time for this process
|
|
|
341 (seconds:microseconds)
|
|
|
342 curr_items 32u Current number of items stored by the server
|
|
|
343 total_items 32u Total number of items stored by this server
|
|
|
344 ever since it started
|
|
|
345 bytes 64u Current number of bytes used by this server
|
|
|
346 to store items
|
|
|
347 curr_connections 32u Number of open connections
|
|
|
348 total_connections 32u Total number of connections opened since
|
|
|
349 the server started running
|
|
|
350 connection_structures 32u Number of connection structures allocated
|
|
|
351 by the server
|
|
|
352 cmd_get 64u Cumulative number of retrieval requests
|
|
|
353 cmd_set 64u Cumulative number of storage requests
|
|
|
354 get_hits 64u Number of keys that have been requested and
|
|
|
355 found present
|
|
|
356 get_misses 64u Number of items that have been requested
|
|
|
357 and not found
|
|
|
358 evictions 64u Number of valid items removed from cache
|
|
|
359 to free memory for new items
|
|
|
360 bytes_read 64u Total number of bytes read by this server
|
|
|
361 from network
|
|
|
362 bytes_written 64u Total number of bytes sent by this server to
|
|
|
363 network
|
|
|
364 limit_maxbytes 32u Number of bytes this server is allowed to
|
|
|
365 use for storage.
|
|
|
366
|
|
|
367
|
|
|
368
|
|
|
369 Other commands
|
|
|
370 --------------
|
|
|
371
|
|
|
372 "flush_all" is a command with an optional numeric argument. It always
|
|
|
373 succeeds, and the server sends "OK\r\n" in response. Its effect is to
|
|
|
374 invalidate all existing items immediately (by default) or after the
|
|
|
375 expiration specified. After invalidation none of the items will be returned
|
|
|
376 in response to a retrieval command (unless it's stored again under the
|
|
|
377 same key *after* flush_all has invalidated the items). flush_all doesn't
|
|
|
378 actually free all the memory taken up by existing items; that will
|
|
|
379 happen gradually as new items are stored. The most precise definition
|
|
|
380 of what flush_all does is the following: it causes all items whose
|
|
|
381 update time is earlier than the time at which flush_all was set to be
|
|
|
382 executed to be ignored for retrieval purposes.
|
|
|
383
|
|
|
384 The intent of flush_all with a delay, was that in a setting where you
|
|
|
385 have a pool of memcached servers, and you need to flush all content,
|
|
|
386 you have the option of not resetting all memcached servers at the
|
|
|
387 same time (which could e.g. cause a spike in database load with all
|
|
|
388 clients suddenly needing to recreate content that would otherwise
|
|
|
389 have been found in the memcached daemon).
|
|
|
390
|
|
|
391 The delay option allows you to have them reset in e.g. 10 second
|
|
|
392 intervals (by passing 0 to the first, 10 to the second, 20 to the
|
|
|
393 third, etc. etc.).
|
|
|
394
|
|
|
395
|
|
|
396 "version" is a command with no arguments:
|
|
|
397
|
|
|
398 version\r\n
|
|
|
399
|
|
|
400 In response, the server sends
|
|
|
401
|
|
|
402 "VERSION <version>\r\n", where <version> is the version string for the
|
|
|
403 server.
|
|
|
404
|
|
|
405 "verbosity" is a command with a numeric argument. It always
|
|
|
406 succeeds, and the server sends "OK\r\n" in response. Its effect is to
|
|
|
407 set the verbosity level of the logging output.
|
|
|
408
|
|
|
409 "quit" is a command with no arguments:
|
|
|
410
|
|
|
411 quit\r\n
|
|
|
412
|
|
|
413 Upon receiving this command, the server closes the
|
|
|
414 connection. However, the client may also simply close the connection
|
|
|
415 when it no longer needs it, without issuing this command.
|
|
|
416
|
|
|
417
|
|
|
418 UDP protocol
|
|
|
419 ------------
|
|
|
420
|
|
|
421 For very large installations where the number of clients is high enough
|
|
|
422 that the number of TCP connections causes scaling difficulties, there is
|
|
|
423 also a UDP-based interface. The UDP interface does not provide guaranteed
|
|
|
424 delivery, so should only be used for operations that aren't required to
|
|
|
425 succeed; typically it is used for "get" requests where a missing or
|
|
|
426 incomplete response can simply be treated as a cache miss.
|
|
|
427
|
|
|
428 Each UDP datagram contains a simple frame header, followed by data in the
|
|
|
429 same format as the TCP protocol described above. In the current
|
|
|
430 implementation, requests must be contained in a single UDP datagram, but
|
|
|
431 responses may span several datagrams. (The only common requests that would
|
|
|
432 span multiple datagrams are huge multi-key "get" requests and "set"
|
|
|
433 requests, both of which are more suitable to TCP transport for reliability
|
|
|
434 reasons anyway.)
|
|
|
435
|
|
|
436 The frame header is 8 bytes long, as follows (all values are 16-bit integers
|
|
|
437 in network byte order, high byte first):
|
|
|
438
|
|
|
439 0-1 Request ID
|
|
|
440 2-3 Sequence number
|
|
|
441 4-5 Total number of datagrams in this message
|
|
|
442 6-7 Reserved for future use; must be 0
|
|
|
443
|
|
|
444 The request ID is supplied by the client. Typically it will be a
|
|
|
445 monotonically increasing value starting from a random seed, but the client
|
|
|
446 is free to use whatever request IDs it likes. The server's response will
|
|
|
447 contain the same ID as the incoming request. The client uses the request ID
|
|
|
448 to differentiate between responses to outstanding requests if there are
|
|
|
449 several pending from the same server; any datagrams with an unknown request
|
|
|
450 ID are probably delayed responses to an earlier request and should be
|
|
|
451 discarded.
|
|
|
452
|
|
|
453 The sequence number ranges from 0 to n-1, where n is the total number of
|
|
|
454 datagrams in the message. The client should concatenate the payloads of the
|
|
|
455 datagrams for a given response in sequence number order; the resulting byte
|
|
|
456 stream will contain a complete response in the same format as the TCP
|
|
|
457 protocol (including terminating \r\n sequences).
|
