annotate proxy_chunked.t @ 1961:fe6f22da53ec default tip

Tests: tests for usage of discarded body. The client_max_body_size limit should be ignored when the request body is already discarded. In HTTP/1.x, this is done by checking the r->discard_body flag when the body is being discarded, and because r->headers_in.content_length_n is 0 when it's already discarded. This, however, does not happen with HTTP/2 and HTTP/3, and therefore "error_page 413" does not work without relaxing the limit. Further, with proxy_pass, r->headers_in.content_length_n is used to determine length of the request body, and therefore is not correct if discarding of the request body isn't yet complete. While discarding the request body, r->headers_in.content_length_n contains the rest of the body to discard (or, in case of chunked request body, the rest of the current chunk to discard). Similarly, the $content_length variable uses r->headers_in.content_length if available, and also incorrect. The $content_length variable is used when proxying with fastcgi_pass, grpc_pass, and uwsgi_pass (scgi_pass uses the value calculated based on the actual request body buffers, and therefore works correctly).
author Maxim Dounin <mdounin@mdounin.ru>
date Sat, 27 Apr 2024 18:55:50 +0300
parents 882267679006
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
1 #!/usr/bin/perl
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
2
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
3 # (C) Maxim Dounin
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
4
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
5 # Test for http backend returning response with Transfer-Encoding: chunked.
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
6
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
7 # Since nginx uses HTTP/1.0 in requests to backend it's backend bug, but we
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
8 # want to handle this gracefully. And anyway chunked support will be required
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
9 # for HTTP/1.1 backend connections.
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
10
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
11 ###############################################################################
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
12
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
13 use warnings;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
14 use strict;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
15
121
8ac1faaddd2c Tests: better handle various configure arguments.
Maxim Dounin <mdounin@mdounin.ru>
parents: 107
diff changeset
16 use Test::More;
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
17
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
18 BEGIN { use FindBin; chdir($FindBin::Bin); }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
19
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
20 use lib 'lib';
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
21 use Test::Nginx;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
22
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
23 ###############################################################################
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
24
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
25 select STDERR; $| = 1;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
26 select STDOUT; $| = 1;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
27
121
8ac1faaddd2c Tests: better handle various configure arguments.
Maxim Dounin <mdounin@mdounin.ru>
parents: 107
diff changeset
28 my $t = Test::Nginx->new()->has(qw/http proxy ssi/)->plan(3);
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
29
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
30 $t->write_file_expand('nginx.conf', <<'EOF');
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
31
107
1c0ec30614c6 Tests: add TEST_GLOBALS and TEST_GLOBALS_HTTP config chunks.
Maxim Dounin <mdounin@mdounin.ru>
parents: 94
diff changeset
32 %%TEST_GLOBALS%%
1c0ec30614c6 Tests: add TEST_GLOBALS and TEST_GLOBALS_HTTP config chunks.
Maxim Dounin <mdounin@mdounin.ru>
parents: 94
diff changeset
33
249
6a0d934950bc Tests: remove extra spaces in "daemon off".
Maxim Dounin <mdounin@mdounin.ru>
parents: 224
diff changeset
34 daemon off;
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
35
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
36 events {
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
37 }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
38
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
39 http {
107
1c0ec30614c6 Tests: add TEST_GLOBALS and TEST_GLOBALS_HTTP config chunks.
Maxim Dounin <mdounin@mdounin.ru>
parents: 94
diff changeset
40 %%TEST_GLOBALS_HTTP%%
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
41
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
42 server {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
43 listen 127.0.0.1:8080;
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
44 server_name localhost;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
45
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
46 location / {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
47 proxy_pass http://127.0.0.1:8081;
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
48 proxy_read_timeout 1s;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
49 }
94
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
50 location /nobuffering {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
51 proxy_pass http://127.0.0.1:8081;
94
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
52 proxy_read_timeout 1s;
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
53 proxy_buffering off;
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
54 }
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
55 location /inmemory.html {
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
56 ssi on;
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
57 }
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
58 }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
59 }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
60
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
61 EOF
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
62
94
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
63 $t->write_file('inmemory.html',
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
64 '<!--#include virtual="/" set="one" --><!--#echo var="one" -->');
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
65
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
66 $t->run_daemon(\&http_chunked_daemon);
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
67 $t->run()->waitforsocket('127.0.0.1:' . port(8081));
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
68
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
69 ###############################################################################
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
70
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
71 like(http_get('/'), qr/\x0d\x0aSEE-THIS$/s, 'chunked');
94
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
72 like(http_get('/nobuffering'), qr/\x0d\x0aSEE-THIS$/s, 'chunked nobuffering');
9ab3762332b9 Tests: chunked tests for unbuffered and inmemory proxy.
Maxim Dounin <mdounin@mdounin.ru>
parents: 85
diff changeset
73 like(http_get('/inmemory.html'), qr/\x0d\x0aSEE-THIS$/s, 'chunked inmemory');
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
74
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
75 ###############################################################################
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
76
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
77 sub http_chunked_daemon {
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
78 my $server = IO::Socket::INET->new(
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
79 Proto => 'tcp',
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
80 LocalAddr => '127.0.0.1:' . port(8081),
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
81 Listen => 5,
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
82 Reuse => 1
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
83 )
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
84 or die "Can't create listening socket: $!\n";
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
85
295
6fe0459b6668 Tests: eliminate startup races in proxy and fastcgi tests.
Maxim Dounin <mdounin@mdounin.ru>
parents: 249
diff changeset
86 local $SIG{PIPE} = 'IGNORE';
6fe0459b6668 Tests: eliminate startup races in proxy and fastcgi tests.
Maxim Dounin <mdounin@mdounin.ru>
parents: 249
diff changeset
87
85
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
88 while (my $client = $server->accept()) {
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
89 $client->autoflush(1);
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
90
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
91 while (<$client>) {
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
92 last if (/^\x0d?\x0a?$/);
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
93 }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
94
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
95 print $client <<'EOF';
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
96 HTTP/1.1 200 OK
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
97 Connection: close
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
98 Transfer-Encoding: chunked
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
99
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
100 9
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
101 SEE-THIS
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
102
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
103 0
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
104
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
105 EOF
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
106
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
107 close $client;
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
108 }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
109 }
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
110
1bf5eca1c4a9 Tests: add test for chunked transfer-coding from backend.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
111 ###############################################################################