Mercurial > hg > memcached

comparison doc/protocol.txt @ 0:30782bb1fc04 MEMCACHED_1_2_3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
memcached-1.2.3
author Maxim Dounin <mdounin@mdounin.ru>
date Sun, 23 Sep 2007 03:58:34 +0400
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:30782bb1fc04
1 Protocol
2 --------
3
4 Clients of memcached communicate with server through TCP connections.
5 (A UDP interface is also available; details are below under "UDP
6 protocol.") A given running memcached server listens on some
7 (configurable) port; clients connect to that port, send commands to
8 the server, read responses, and eventually close the connection.
9
10 There is no need to send any command to end the session. A client may
11 just close the connection at any moment it no longer needs it. Note,
12 however, that clients are encouraged to cache their connections rather
13 than reopen them every time they need to store or retrieve data. This
14 is because memcached is especially designed to work very efficiently
15 with a very large number (many hundreds, more than a thousand if
16 necessary) of open connections. Caching connections will eliminate the
17 overhead associated with establishing a TCP connection (the overhead
18 of preparing for a new connection on the server side is insignificant
19 compared to this).
20
21 There are two kinds of data sent in the memcache protocol: text lines
22 and unstructured data. Text lines are used for commands from clients
23 and responses from servers. Unstructured data is sent when a client
24 wants to store or retrieve data. The server will transmit back
25 unstructured data in exactly the same way it received it, as a byte
26 stream. The server doesn't care about byte order issues in
27 unstructured data and isn't aware of them. There are no limitations on
28 characters that may appear in unstructured data; however, the reader
29 of such data (either a client or a server) will always know, from a
30 preceding text line, the exact length of the data block being
31 transmitted.
32
33 Text lines are always terminated by \r\n. Unstructured data is _also_
34 terminated by \r\n, even though \r, \n or any other 8-bit characters
35 may also appear inside the data. Therefore, when a client retrieves
36 data from a server, it must use the length of the data block (which it
37 will be provided with) to determine where the data block ends, and not
38 the fact that \r\n follows the end of the data block, even though it
39 does.
40
41 Keys
42 ----
43
44 Data stored by memcached is identified with the help of a key. A key
45 is a text string which should uniquely identify the data for clients
46 that are interested in storing and retrieving it. Currently the
47 length limit of a key is set at 250 characters (of course, normally
48 clients wouldn't need to use such long keys); the key must not include
49 control characters or whitespace.
50
51 Commands
52 --------
53
54 There are three types of commands.
55
56 Storage commands (there are three: "set", "add" and "replace") ask the
57 server to store some data identified by a key. The client sends a
58 command line, and then a data block; after that the client expects one
59 line of response, which will indicate success or faulure.
60
61 Retrieval commands (there is only one: "get") ask the server to
62 retrieve data corresponding to a set of keys (one or more keys in one
63 request). The client sends a command line, which includes all the
64 requested keys; after that for each item the server finds it sends to
65 the client one response line with information about the item, and one
66 data block with the item's data; this continues until the server
67 finished with the "END" response line.
68
69 All other commands don't involve unstructured data. In all of them,
70 the client sends one command line, and expects (depending on the
71 command) either one line of response, or several lines of response
72 ending with "END" on the last line.
73
74 A command line always starts with the name of the command, followed by
75 parameters (if any) delimited by whitespace. Command names are
76 lower-case and are case-sensitive.
77
78 Expiration times
79 ----------------
80
81 Some commands involve a client sending some kind of expiration time
82 (relative to an item or to an operation requested by the client) to
83 the server. In all such cases, the actual value sent may either be
84 Unix time (number of seconds since January 1, 1970, as a 32-bit
85 value), or a number of seconds starting from current time. In the
86 latter case, this number of seconds may not exceed 60*60*24*30 (number
87 of seconds in 30 days); if the number sent by a client is larger than
88 that, the server will consider it to be real Unix time value rather
89 than an offset from current time.
90
91
92 Error strings
93 -------------
94
95 Each command sent by a client may be answered with an error string
96 from the server. These error strings come in three types:
97
98 - "ERROR\r\n"
99
100 means the client sent a nonexistent command name.
101
102 - "CLIENT_ERROR <error>\r\n"
103
104 means some sort of client error in the input line, i.e. the input
105 doesn't conform to the protocol in some way. <error> is a
106 human-readable error string.
107
108 - "SERVER_ERROR <error>\r\n"
109
110 means some sort of server error prevents the server from carrying
111 out the command. <error> is a human-readable error string. In cases
112 of severe server errors, which make it impossible to continue
113 serving the client (this shouldn't normally happen), the server will
114 close the connection after sending the error line. This is the only
115 case in which the server closes a connection to a client.
116
117
118 In the descriptions of individual commands below, these error lines
119 are not again specifically mentioned, but clients must allow for their
120 possibility.
121
122
123 Storage commands
124 ----------------
125
126 First, the client sends a command line which looks like this:
127
128 <command name> <key> <flags> <exptime> <bytes>\r\n
129
130 - <command name> is "set", "add" or "replace"
131
132 "set" means "store this data".
133
134 "add" means "store this data, but only if the server *doesn't* already
135 hold data for this key".
136
137 "replace" means "store this data, but only if the server *does*
138 already hold data for this key".
139
140 - <key> is the key under which the client asks to store the data
141
142 - <flags> is an arbitrary 16-bit unsigned integer (written out in
143 decimal) that the server stores along with the data and sends back
144 when the item is retrieved. Clients may use this as a bit field to
145 store data-specific information; this field is opaque to the server.
146 Note that in memcached 1.2.1 and higher, flags may be 32-bits, instead
147 of 16, but you might want to restrict yourself to 16 bits for
148 compatibility with older versions.
149
150 - <exptime> is expiration time. If it's 0, the item never expires
151 (although it may be deleted from the cache to make place for other
152 items). If it's non-zero (either Unix time or offset in seconds from
153 current time), it is guaranteed that clients will not be able to
154 retrieve this item after the expiration time arrives (measured by
155 server time).
156
157 - <bytes> is the number of bytes in the data block to follow, *not*
158 including the delimiting \r\n. <bytes> may be zero (in which case
159 it's followed by an empty data block).
160
161 After this line, the client sends the data block:
162
163 <data block>\r\n
164
165 - <data block> is a chunk of arbitrary 8-bit data of length <bytes>
166 from the previous line.
167
168 After sending the command line and the data blockm the client awaits
169 the reply, which may be:
170
171 - "STORED\r\n", to indicate success.
172
173 - "NOT_STORED\r\n" to indicate the data was not stored, but not
174 because of an error. This normally means that either that the
175 condition for an "add" or a "replace" command wasn't met, or that the
176 item is in a delete queue (see the "delete" command below).
177
178
179 Retrieval command:
180 ------------------
181
182 The retrieval command looks like this:
183
184 get <key>*\r\n
185
186 - <key>* means one or more key strings separated by whitespace.
187
188 After this command, the client expects zero or more items, each of
189 which is received as a text line followed by a data block. After all
190 the items have been transmitted, the server sends the string
191
192 "END\r\n"
193
194 to indicate the end of response.
195
196 Each item sent by the server looks like this:
197
198 VALUE <key> <flags> <bytes>\r\n
199 <data block>\r\n
200
201 - <key> is the key for the item being sent
202
203 - <flags> is the flags value set by the storage command
204
205 - <bytes> is the length of the data block to follow, *not* including
206 its delimiting \r\n
207
208 - <data block> is the data for this item.
209
210 If some of the keys appearing in a retrieval request are not sent back
211 by the server in the item list this means that the server does not
212 hold items with such keys (because they were never stored, or stored
213 but deleted to make space for more items, or expired, or explicitly
214 deleted by a client).
215
216
217
218 Deletion
219 --------
220
221 The command "delete" allows for explicit deletion of items:
222
223 delete <key> <time>\r\n
224
225 - <key> is the key of the item the client wishes the server to delete
226
227 - <time> is the amount of time in seconds (or Unix time until which)
228 the client wishes the server to refuse "add" and "replace" commands
229 with this key. For this amount of item, the item is put into a
230 delete queue, which means that it won't possible to retrieve it by
231 the "get" command, but "add" and "replace" command with this key
232 will also fail (the "set" command will succeed, however). After the
233 time passes, the item is finally deleted from server memory.
234
235 The parameter <time> is optional, and, if absent, defaults to 0
236 (which means that the item will be deleted immediately and further
237 storage commands with this key will succeed).
238
239 The response line to this command can be one of:
240
241 - "DELETED\r\n" to indicate success
242
243 - "NOT_FOUND\r\n" to indicate that the item with this key was not
244 found.
245
246 See the "flush_all" command below for immediate invalidation
247 of all existing items.
248
249
250 Increment/Decrement
251 -------------------
252
253 Commands "incr" and "decr" are used to change data for some item
254 in-place, incrementing or decrementing it. The data for the item is
255 treated as decimal representation of a 32-bit unsigned integer. If the
256 current data value does not conform to such a representation, the
257 commands behave as if the value were 0. Also, the item must already
258 exist for incr/decr to work; these commands won't pretend that a
259 non-existent key exists with value 0; instead, they will fail.
260
261 The client sends the command line:
262
263 incr <key> <value>\r\n
264
265 or
266
267 decr <key> <value>\r\n
268
269 - <key> is the key of the item the client wishes to change
270
271 - <value> is the amount by which the client wants to increase/decrease
272 the item. It is a decimal representation of a 32-bit unsigned integer.
273
274 The response will be one of:
275
276 - "NOT_FOUND\r\n" to indicate the item with this value was not found
277
278 - <value>\r\n , where <value> is the new value of the item's data,
279 after the increment/decrement operation was carried out.
280
281 Note that underflow in the "decr" command is caught: if a client tries
282 to decrease the value below 0, the new value will be 0. Overflow in
283 the "incr" command is not checked.
284
285 Note also that decrementing a number such that it loses length isn't
286 guaranteed to decrement its returned length. The number MAY be
287 space-padded at the end, but this is purely an implementation
288 optimization, so you also shouldn't rely on that.
289
290 Statistics
291 ----------
292
293 The command "stats" is used to query the server about statistics it
294 maintains and other internal data. It has two forms. Without
295 arguments:
296
297 stats\r\n
298
299 it causes the server to output general-purpose statistics and
300 settings, documented below. In the other form it has some arguments:
301
302 stats <args>\r\n
303
304 Depending on <args>, various internal data is sent by the server. The
305 kinds of arguments and the data sent are not documented in this vesion
306 of the protocol, and are subject to change for the convenience of
307 memcache developers.
308
309
310 General-purpose statistics
311 --------------------------
312
313 Upon receiving the "stats" command without arguments, the server sents
314 a number of lines which look like this:
315
316 STAT <name> <value>\r\n
317
318 The server terminates this list with the line
319
320 END\r\n
321
322 In each line of statistics, <name> is the name of this statistic, and
323 <value> is the data. The following is the list of all names sent in
324 response to the "stats" command, together with the type of the value
325 sent for this name, and the meaning of the value.
326
327 In the type column below, "32u" means a 32-bit unsigned integer, "64u"
328 means a 64-bit unsigner integer. '32u:32u' means two 32-but unsigned
329 integers separated by a colon.
330
331
332 Name Type Meaning
333 ----------------------------------
334 pid 32u Process id of this server process
335 uptime 32u Number of seconds this server has been running
336 time 32u current UNIX time according to the server
337 version string Version string of this server
338 rusage_user 32u:32u Accumulated user time for this process
339 (seconds:microseconds)
340 rusage_system 32u:32u Accumulated system time for this process
341 (seconds:microseconds)
342 curr_items 32u Current number of items stored by the server
343 total_items 32u Total number of items stored by this server
344 ever since it started
345 bytes 64u Current number of bytes used by this server
346 to store items
347 curr_connections 32u Number of open connections
348 total_connections 32u Total number of connections opened since
349 the server started running
350 connection_structures 32u Number of connection structures allocated
351 by the server
352 cmd_get 64u Cumulative number of retrieval requests
353 cmd_set 64u Cumulative number of storage requests
354 get_hits 64u Number of keys that have been requested and
355 found present
356 get_misses 64u Number of items that have been requested
357 and not found
358 evictions 64u Number of valid items removed from cache
359 to free memory for new items
360 bytes_read 64u Total number of bytes read by this server
361 from network
362 bytes_written 64u Total number of bytes sent by this server to
363 network
364 limit_maxbytes 32u Number of bytes this server is allowed to
365 use for storage.
366
367
368
369 Other commands
370 --------------
371
372 "flush_all" is a command with an optional numeric argument. It always
373 succeeds, and the server sends "OK\r\n" in response. Its effect is to
374 invalidate all existing items immediately (by default) or after the
375 expiration specified. After invalidation none of the items will be returned
376 in response to a retrieval command (unless it's stored again under the
377 same key *after* flush_all has invalidated the items). flush_all doesn't
378 actually free all the memory taken up by existing items; that will
379 happen gradually as new items are stored. The most precise definition
380 of what flush_all does is the following: it causes all items whose
381 update time is earlier than the time at which flush_all was set to be
382 executed to be ignored for retrieval purposes.
383
384 The intent of flush_all with a delay, was that in a setting where you
385 have a pool of memcached servers, and you need to flush all content,
386 you have the option of not resetting all memcached servers at the
387 same time (which could e.g. cause a spike in database load with all
388 clients suddenly needing to recreate content that would otherwise
389 have been found in the memcached daemon).
390
391 The delay option allows you to have them reset in e.g. 10 second
392 intervals (by passing 0 to the first, 10 to the second, 20 to the
393 third, etc. etc.).
394
395
396 "version" is a command with no arguments:
397
398 version\r\n
399
400 In response, the server sends
401
402 "VERSION <version>\r\n", where <version> is the version string for the
403 server.
404
405 "verbosity" is a command with a numeric argument. It always
406 succeeds, and the server sends "OK\r\n" in response. Its effect is to
407 set the verbosity level of the logging output.
408
409 "quit" is a command with no arguments:
410
411 quit\r\n
412
413 Upon receiving this command, the server closes the
414 connection. However, the client may also simply close the connection
415 when it no longer needs it, without issuing this command.
416
417
418 UDP protocol
419 ------------
420
421 For very large installations where the number of clients is high enough
422 that the number of TCP connections causes scaling difficulties, there is
423 also a UDP-based interface. The UDP interface does not provide guaranteed
424 delivery, so should only be used for operations that aren't required to
425 succeed; typically it is used for "get" requests where a missing or
426 incomplete response can simply be treated as a cache miss.
427
428 Each UDP datagram contains a simple frame header, followed by data in the
429 same format as the TCP protocol described above. In the current
430 implementation, requests must be contained in a single UDP datagram, but
431 responses may span several datagrams. (The only common requests that would
432 span multiple datagrams are huge multi-key "get" requests and "set"
433 requests, both of which are more suitable to TCP transport for reliability
434 reasons anyway.)
435
436 The frame header is 8 bytes long, as follows (all values are 16-bit integers
437 in network byte order, high byte first):
438
439 0-1 Request ID
440 2-3 Sequence number
441 4-5 Total number of datagrams in this message
442 6-7 Reserved for future use; must be 0
443
444 The request ID is supplied by the client. Typically it will be a
445 monotonically increasing value starting from a random seed, but the client
446 is free to use whatever request IDs it likes. The server's response will
447 contain the same ID as the incoming request. The client uses the request ID
448 to differentiate between responses to outstanding requests if there are
449 several pending from the same server; any datagrams with an unknown request
450 ID are probably delayed responses to an earlier request and should be
451 discarded.
452
453 The sequence number ranges from 0 to n-1, where n is the total number of
454 datagrams in the message. The client should concatenate the payloads of the
455 datagrams for a given response in sequence number order; the resulting byte
456 stream will contain a complete response in the same format as the TCP
457 protocol (including terminating \r\n sequences).