Mercurial > hg > ngx_http_upstream_keepalive
annotate t/stale.t @ 45:489c5d4318ff draft
Keepalive: "single" parameter deprecated.
The original idea was to optimize edge cases in case of interchangeable
backends, i.e. don't establish a new connection if we have any one
cached. This causes more harm than good though, as it screws up
underlying balancer's idea about backends used and may result in
various unexpected problems.
author | Maxim Dounin <mdounin@mdounin.ru> |
---|---|
date | Mon, 25 Jun 2012 22:55:53 +0400 |
parents | 2ee28064a04a |
children |
rev | line source |
---|---|
21
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
1 #!/usr/bin/perl |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
2 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
3 # (C) Maxim Dounin |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
4 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
5 # Test for stale events handling in upstream keepalive. |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
6 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
7 ############################################################################### |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
8 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
9 use warnings; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
10 use strict; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
11 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
12 use Test::More; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
13 use Test::Nginx; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
14 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
15 ############################################################################### |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
16 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
17 select STDERR; $| = 1; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
18 select STDOUT; $| = 1; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
19 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
20 eval { require Cache::Memcached; }; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
21 plan(skip_all => 'Cache::Memcached not installed') if $@; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
22 |
24
2ee28064a04a
Keepalive: correctly close connections on gracefull shutdown.
Maxim Dounin <mdounin@mdounin.ru>
parents:
21
diff
changeset
|
23 my $t = Test::Nginx->new()->has('rewrite')->has_daemon('memcached')->plan(2) |
21
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
24 ->write_file_expand('nginx.conf', <<'EOF'); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
25 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
26 %%TEST_GLOBALS%% |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
27 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
28 daemon off; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
29 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
30 worker_processes 2; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
31 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
32 events { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
33 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
34 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
35 http { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
36 %%TEST_GLOBALS_HTTP%% |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
37 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
38 upstream memd { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
39 server 127.0.0.1:8081; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
40 keepalive 1; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
41 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
42 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
43 server { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
44 listen 127.0.0.1:8080 sndbuf=32k; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
45 server_name localhost; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
46 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
47 location / { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
48 set $memcached_key $uri; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
49 memcached_pass memd; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
50 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
51 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
52 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
53 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
54 EOF |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
55 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
56 my $memhelp = `memcached -h`; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
57 my @memopts1 = (); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
58 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
59 if ($memhelp =~ /repcached/) { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
60 # repcached patches adds additional listen socket memcached |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
61 # that should be different too |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
62 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
63 push @memopts1, '-X', '8091'; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
64 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
65 if ($memhelp =~ /-U/) { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
66 # UDP ports no longer off by default in memcached 1.2.7+ |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
67 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
68 push @memopts1, '-U', '0'; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
69 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
70 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
71 $t->run_daemon('memcached', '-l', '127.0.0.1', '-p', '8081', @memopts1); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
72 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
73 $t->run(); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
74 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
75 $t->waitforsocket('127.0.0.1:8081') |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
76 or die "Unable to start memcached"; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
77 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
78 ############################################################################### |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
79 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
80 my $memd1 = Cache::Memcached->new(servers => [ '127.0.0.1:8081' ]); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
81 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
82 # It's possible that stale events occur, i.e. read event handler called |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
83 # for just saved upstream connection without any data available for |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
84 # read. We shouldn't close upstream connection in such situation. |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
85 # |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
86 # This happens due to reading from upstream connection on downstream write |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
87 # events. More likely to happen with multiple workers due to use of posted |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
88 # events. |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
89 # |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
90 # Stale event may only happen if reading response from upstream requires |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
91 # entering event loop, i.e. response should be big enough. On the other |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
92 # hand, it is less likely to occur with full client's connection output |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
93 # buffer. |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
94 # |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
95 # We use here 2 workers, 20k response and set output buffer on clients |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
96 # connection to 32k. This allows more or less reliably reproduce stale |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
97 # events at least on FreeBSD testbed here. |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
98 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
99 $memd1->set('/big', 'X' x 20480); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
100 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
101 my $total = $memd1->stats()->{total}->{total_connections}; |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
102 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
103 for (1 .. 100) { |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
104 http_get('/big'); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
105 } |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
106 |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
107 cmp_ok($memd1->stats()->{total}->{total_connections}, '<=', $total + 2, |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
108 'only one connection per worker used'); |
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
109 |
24
2ee28064a04a
Keepalive: correctly close connections on gracefull shutdown.
Maxim Dounin <mdounin@mdounin.ru>
parents:
21
diff
changeset
|
110 $t->stop(); |
2ee28064a04a
Keepalive: correctly close connections on gracefull shutdown.
Maxim Dounin <mdounin@mdounin.ru>
parents:
21
diff
changeset
|
111 |
2ee28064a04a
Keepalive: correctly close connections on gracefull shutdown.
Maxim Dounin <mdounin@mdounin.ru>
parents:
21
diff
changeset
|
112 like(`grep -F '[alert]' ${\($t->testdir())}/error.log`, qr/^$/s, 'no alerts'); |
2ee28064a04a
Keepalive: correctly close connections on gracefull shutdown.
Maxim Dounin <mdounin@mdounin.ru>
parents:
21
diff
changeset
|
113 |
21
9a4ee6fe1c6d
Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff
changeset
|
114 ############################################################################### |