Mercurial > hg > ngx_http_upstream_keepalive

annotate t/stale.t @ 21:9a4ee6fe1c6d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Keepalive: avoid closing upstream connection on stale events. Under some conditions it's possible that stale events occur, i.e. read event handler called for just saved upstream connection without any data available for read. We shouldn't close upstream connection in such situation. Reported by: Martin Fjordvald
author Maxim Dounin <mdounin@mdounin.ru>
date Tue, 14 Sep 2010 04:01:14 +0400
parents
children 2ee28064a04a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
21
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
1 #!/usr/bin/perl
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
2
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
3 # (C) Maxim Dounin
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
4
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
5 # Test for stale events handling in upstream keepalive.
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
6
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
7 ###############################################################################
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
8
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
9 use warnings;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
10 use strict;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
11
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
12 use Test::More;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
13 use Test::Nginx;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
14
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
15 ###############################################################################
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
16
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
17 select STDERR; $| = 1;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
18 select STDOUT; $| = 1;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
19
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
20 eval { require Cache::Memcached; };
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
21 plan(skip_all => 'Cache::Memcached not installed') if $@;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
22
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
23 my $t = Test::Nginx->new()->has('rewrite')->has_daemon('memcached')->plan(1)
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
24 ->write_file_expand('nginx.conf', <<'EOF');
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
25
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
26 %%TEST_GLOBALS%%
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
27
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
28 daemon off;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
29
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
30 worker_processes 2;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
31
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
32 events {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
33 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
34
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
35 http {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
36 %%TEST_GLOBALS_HTTP%%
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
37
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
38 upstream memd {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
39 server 127.0.0.1:8081;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
40 keepalive 1;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
41 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
42
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
43 server {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
44 listen 127.0.0.1:8080 sndbuf=32k;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
45 server_name localhost;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
46
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
47 location / {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
48 set $memcached_key $uri;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
49 memcached_pass memd;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
50 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
51 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
52 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
53
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
54 EOF
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
55
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
56 my $memhelp = `memcached -h`;
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
57 my @memopts1 = ();
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
58
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
59 if ($memhelp =~ /repcached/) {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
60 # repcached patches adds additional listen socket memcached
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
61 # that should be different too
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
62
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
63 push @memopts1, '-X', '8091';
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
64 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
65 if ($memhelp =~ /-U/) {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
66 # UDP ports no longer off by default in memcached 1.2.7+
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
67
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
68 push @memopts1, '-U', '0';
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
69 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
70
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
71 $t->run_daemon('memcached', '-l', '127.0.0.1', '-p', '8081', @memopts1);
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
72
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
73 $t->run();
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
74
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
75 $t->waitforsocket('127.0.0.1:8081')
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
76 or die "Unable to start memcached";
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
77
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
78 ###############################################################################
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
79
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
80 my $memd1 = Cache::Memcached->new(servers => [ '127.0.0.1:8081' ]);
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
81
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
82 # It's possible that stale events occur, i.e. read event handler called
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
83 # for just saved upstream connection without any data available for
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
84 # read. We shouldn't close upstream connection in such situation.
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
85 #
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
86 # This happens due to reading from upstream connection on downstream write
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
87 # events. More likely to happen with multiple workers due to use of posted
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
88 # events.
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
89 #
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
90 # Stale event may only happen if reading response from upstream requires
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
91 # entering event loop, i.e. response should be big enough. On the other
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
92 # hand, it is less likely to occur with full client's connection output
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
93 # buffer.
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
94 #
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
95 # We use here 2 workers, 20k response and set output buffer on clients
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
96 # connection to 32k. This allows more or less reliably reproduce stale
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
97 # events at least on FreeBSD testbed here.
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
98
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
99 $memd1->set('/big', 'X' x 20480);
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
100
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
101 my $total = $memd1->stats()->{total}->{total_connections};
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
102
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
103 for (1 .. 100) {
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
104 http_get('/big');
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
105 }
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
106
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
107 cmp_ok($memd1->stats()->{total}->{total_connections}, '<=', $total + 2,
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
108 'only one connection per worker used');
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
109
9a4ee6fe1c6d Keepalive: avoid closing upstream connection on stale events.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
110 ###############################################################################