annotate gunzip.t @ 1961:fe6f22da53ec default tip

Tests: tests for usage of discarded body. The client_max_body_size limit should be ignored when the request body is already discarded. In HTTP/1.x, this is done by checking the r->discard_body flag when the body is being discarded, and because r->headers_in.content_length_n is 0 when it's already discarded. This, however, does not happen with HTTP/2 and HTTP/3, and therefore "error_page 413" does not work without relaxing the limit. Further, with proxy_pass, r->headers_in.content_length_n is used to determine length of the request body, and therefore is not correct if discarding of the request body isn't yet complete. While discarding the request body, r->headers_in.content_length_n contains the rest of the body to discard (or, in case of chunked request body, the rest of the current chunk to discard). Similarly, the $content_length variable uses r->headers_in.content_length if available, and also incorrect. The $content_length variable is used when proxying with fastcgi_pass, grpc_pass, and uwsgi_pass (scgi_pass uses the value calculated based on the actual request body buffers, and therefore works correctly).
author Maxim Dounin <mdounin@mdounin.ru>
date Sat, 27 Apr 2024 18:55:50 +0300
parents 882267679006
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
1 #!/usr/bin/perl
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
2
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
3 # (C) Maxim Dounin
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
4
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
5 # Tests for gunzip filter module.
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
6
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
7 ###############################################################################
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
8
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
9 use warnings;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
10 use strict;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
11
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
12 use Test::More;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
13
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
14 BEGIN { use FindBin; chdir($FindBin::Bin); }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
15
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
16 use lib 'lib';
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
17 use Test::Nginx qw/ :DEFAULT :gzip /;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
18
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
19 ###############################################################################
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
20
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
21 select STDERR; $| = 1;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
22 select STDOUT; $| = 1;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
23
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
24 eval { require IO::Compress::Gzip; };
773
c877f91a71b4 Tests: style cleanup.
Sergey Kandaurov <pluknet@nginx.com>
parents: 302
diff changeset
25 plan(skip_all => "IO::Compress::Gzip not found") if $@;
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
26
302
07c600de2f90 Tests: added missing rewrite prerequisites.
Sergey Kandaurov <pluknet@nginx.com>
parents: 249
diff changeset
27 my $t = Test::Nginx->new()->has(qw/http gunzip proxy gzip_static rewrite/)
07c600de2f90 Tests: added missing rewrite prerequisites.
Sergey Kandaurov <pluknet@nginx.com>
parents: 249
diff changeset
28 ->plan(13);
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
29
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
30 $t->write_file_expand('nginx.conf', <<'EOF');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
31
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
32 %%TEST_GLOBALS%%
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
33
249
6a0d934950bc Tests: remove extra spaces in "daemon off".
Maxim Dounin <mdounin@mdounin.ru>
parents: 231
diff changeset
34 daemon off;
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
35
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
36 events {
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
37 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
38
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
39 http {
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
40 %%TEST_GLOBALS_HTTP%%
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
41
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
42 server {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
43 listen 127.0.0.1:8080;
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
44 server_name localhost;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
45 location / {
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
46 gunzip on;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
47 gzip_vary on;
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
48 proxy_pass http://127.0.0.1:8081/;
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
49 proxy_set_header Accept-Encoding gzip;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
50 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
51 location /error {
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
52 error_page 500 /t1;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
53 return 500;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
54 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
55 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
56
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
57 server {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
58 listen 127.0.0.1:8081;
231
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
59 server_name localhost;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
60
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
61 location / {
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
62 default_type text/plain;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
63 gzip_static on;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
64 gzip_http_version 1.0;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
65 gzip_types text/plain;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
66 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
67 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
68 }
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
69
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
70 EOF
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
71
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
72 my $in = join('', map { sprintf "X%03dXXXXXX", $_ } (0 .. 99));
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
73 my $out;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
74
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
75 IO::Compress::Gzip::gzip(\$in => \$out);
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
76
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
77 $t->write_file('t1.gz', $out);
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
78 $t->write_file('t2.gz', $out . $out);
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
79 $t->write_file('t3', 'not compressed');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
80
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
81 my $emptyin = '';
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
82 my $emptyout;
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
83 IO::Compress::Gzip::gzip(\$emptyin => \$emptyout);
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
84
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
85 $t->write_file('empty.gz', $emptyout);
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
86
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
87 $t->run();
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
88
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
89 ###############################################################################
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
90
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
91 pass('runs');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
92
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
93 my $r = http_get('/t1');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
94 unlike($r, qr/Content-Encoding/, 'no content encoding');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
95 like($r, qr/^(X\d\d\dXXXXXX){100}$/m, 'correct gunzipped response');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
96
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
97 $r = http_gzip_request('/t1');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
98 like($r, qr/Content-Encoding: gzip/, 'gzip still works - encoding');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
99 like($r, qr/\Q$out\E/, 'gzip still works - content');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
100
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
101 like(http_get('/t2'), qr/^(X\d\d\dXXXXXX){200}$/m, 'multiple gzip members');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
102
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
103 like(http_get('/error'), qr/^(X\d\d\dXXXXXX){100}$/m, 'errors gunzipped');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
104
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
105 unlike(http_head('/t1'), qr/Content-Encoding/, 'head - no content encoding');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
106
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
107 like(http_get('/t1'), qr/Vary/, 'get vary');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
108 like(http_head('/t1'), qr/Vary/, 'head vary');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
109 unlike(http_get('/t3'), qr/Vary/, 'no vary on non-gzipped get');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
110 unlike(http_head('/t3'), qr/Vary/, 'no vary on non-gzipped head');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
111
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
112 like(http_get('/empty'), qr/ 200 /, 'gunzip empty');
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
113
bc1861122d0c Tests: gunzip filter tests import.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
114 ###############################################################################