Mercurial > hg > nginx-tests
annotate http_resolver_cleanup.t @ 1752:ba6e24e38f03
Tests: improved stop_daemons() to send signal again.
As was observed, it's possible that a signal to complete a uwsgi daemon
can be ignored while it is starting up, which results in tests hang due
to eternal waiting on child processes termination. Notably, it is seen
when running tests with a high number of prove jobs on a low-profile VM
against nginx with broken modules and/or configuration. To reproduce:
$ TEST_NGINX_GLOBALS=ERROR prove -j16 uwsgi*.t
Inspecting uwsgi under ktrace on FreeBSD confirms that a SIGTERM signal
is ignored at the very beginning of uwsgi startup. It is then replaced
with a default action after listen(), thus waiting until uwsgi is ready
to accept new TCP connections doesn't completely solve the hang window.
The fix is to retry sending a signal some time after waitpid(WNOHANG)
continuously demonstrated no progress with reaping a signaled process.
It is modelled after f13ead27f89c that improved stop() for nginx.
author | Sergey Kandaurov <pluknet@nginx.com> |
---|---|
date | Wed, 29 Dec 2021 22:29:23 +0300 |
parents | 766bcbb632ee |
children |
rev | line source |
---|---|
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
1 #!/usr/bin/perl |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
2 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
3 # (C) Sergey Kandaurov |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
4 # (C) Nginx, Inc. |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
5 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
6 # Tests for http resolver, worker process termination. |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
7 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
8 ############################################################################### |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
9 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
10 use warnings; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
11 use strict; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
12 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
13 use Test::More; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
14 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
15 BEGIN { use FindBin; chdir($FindBin::Bin); } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
16 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
17 use lib 'lib'; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
18 use Test::Nginx; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
19 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
20 ############################################################################### |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
21 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
22 select STDERR; $| = 1; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
23 select STDOUT; $| = 1; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
24 |
1101
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
25 my $t = Test::Nginx->new()->has(qw/http proxy/); |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
26 |
1101
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
27 plan(skip_all => 'win32') if $^O eq 'MSWin32'; |
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
28 |
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
29 $t->write_file_expand('nginx.conf', <<'EOF')->plan(1); |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
30 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
31 %%TEST_GLOBALS%% |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
32 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
33 daemon off; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
34 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
35 events { |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
36 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
37 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
38 http { |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
39 %%TEST_GLOBALS_HTTP%% |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
40 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
41 server { |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
42 listen 127.0.0.1:8080; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
43 server_name localhost; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
44 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
45 location / { |
1237
e4974af3fb12
Tests: adjusted udp ports to match allocated ports range.
Andrey Zelenkov <zelenkov@nginx.com>
parents:
1101
diff
changeset
|
46 resolver 127.0.0.1:%%PORT_8981_UDP%%; |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
47 proxy_pass http://example.net/$args; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
48 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
49 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
50 location /pid { |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
51 add_header X-Pid $pid always; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
52 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
53 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
54 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
55 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
56 EOF |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
57 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
58 $t->run_daemon(\&dns_daemon, $t); |
1237
e4974af3fb12
Tests: adjusted udp ports to match allocated ports range.
Andrey Zelenkov <zelenkov@nginx.com>
parents:
1101
diff
changeset
|
59 $t->run()->waitforfile($t->testdir . '/' . port(8981)); |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
60 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
61 ############################################################################### |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
62 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
63 # truncated UDP response, no response over TCP |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
64 |
1101
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
65 my $s = http_get('/', start => 1); |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
66 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
67 pass('request'); |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
68 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
69 sleep 1; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
70 |
1101
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
71 # retrasmission timer wasn't removed during resolver cleanup, |
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
72 # while the event memory was freed, resulting in use-after-free |
78cb8e66b6bc
Tests: skipped the whole http_resolver_cleanup.t where appropriate.
Sergey Kandaurov <pluknet@nginx.com>
parents:
1098
diff
changeset
|
73 # when later removing timer in TCP connection |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
74 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
75 http_get('/pid') =~ qr/X-Pid: (\d+)/; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
76 kill 'TERM', $1; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
77 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
78 ############################################################################### |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
79 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
80 sub dns_daemon { |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
81 my ($t) = @_; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
82 my ($data); |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
83 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
84 my $socket = IO::Socket::INET->new( |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
85 LocalAddr => '127.0.0.1', |
1237
e4974af3fb12
Tests: adjusted udp ports to match allocated ports range.
Andrey Zelenkov <zelenkov@nginx.com>
parents:
1101
diff
changeset
|
86 LocalPort => port(8981), |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
87 Proto => 'udp', |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
88 ) |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
89 or die "Can't create UDP socket: $!\n"; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
90 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
91 # signal we are ready |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
92 |
1237
e4974af3fb12
Tests: adjusted udp ports to match allocated ports range.
Andrey Zelenkov <zelenkov@nginx.com>
parents:
1101
diff
changeset
|
93 open my $fh, '>', $t->testdir() . '/' . port(8981); |
1098
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
94 close $fh; |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
95 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
96 while (1) { |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
97 $socket->recv($data, 65536); |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
98 # truncation bit set |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
99 $data |= pack("n2", 0, 0x8380); |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
100 $socket->send($data); |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
101 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
102 } |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
103 |
bb1974010d0a
Tests: resolver cleanup tests.
Sergey Kandaurov <pluknet@nginx.com>
parents:
diff
changeset
|
104 ############################################################################### |