annotate proxy_cache_range.t @ 1961:fe6f22da53ec default tip

Tests: tests for usage of discarded body. The client_max_body_size limit should be ignored when the request body is already discarded. In HTTP/1.x, this is done by checking the r->discard_body flag when the body is being discarded, and because r->headers_in.content_length_n is 0 when it's already discarded. This, however, does not happen with HTTP/2 and HTTP/3, and therefore "error_page 413" does not work without relaxing the limit. Further, with proxy_pass, r->headers_in.content_length_n is used to determine length of the request body, and therefore is not correct if discarding of the request body isn't yet complete. While discarding the request body, r->headers_in.content_length_n contains the rest of the body to discard (or, in case of chunked request body, the rest of the current chunk to discard). Similarly, the $content_length variable uses r->headers_in.content_length if available, and also incorrect. The $content_length variable is used when proxying with fastcgi_pass, grpc_pass, and uwsgi_pass (scgi_pass uses the value calculated based on the actual request body buffers, and therefore works correctly).
author Maxim Dounin <mdounin@mdounin.ru>
date Sat, 27 Apr 2024 18:55:50 +0300
parents a05ba24a462b
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
1 #!/usr/bin/perl
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
2
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
3 # (C) Maxim Dounin
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
4
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
5 # Tests for http proxy cache and range filter.
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
6
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
7 ###############################################################################
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
8
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
9 use warnings;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
10 use strict;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
11
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
12 use Test::More;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
13
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
14 BEGIN { use FindBin; chdir($FindBin::Bin); }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
15
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
16 use lib 'lib';
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
17 use Test::Nginx;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
18
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
19 ###############################################################################
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
20
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
21 select STDERR; $| = 1;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
22 select STDOUT; $| = 1;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
23
1020
196d33c2bb45 Tests: removed TODO and try_run() checks for legacy versions.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 1008
diff changeset
24 my $t = Test::Nginx->new()->has(qw/http proxy cache/)->plan(7)
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
25 ->write_file_expand('nginx.conf', <<'EOF');
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
26
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
27 %%TEST_GLOBALS%%
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
28
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
29 daemon off;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
30
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
31 events {
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
32 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
33
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
34 http {
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
35 %%TEST_GLOBALS_HTTP%%
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
36
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
37 proxy_cache_path %%TESTDIR%%/cache levels=1:2
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
38 keys_zone=NAME:1m;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
39
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
40 server {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
41 listen 127.0.0.1:8080;
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
42 server_name localhost;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
43
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
44 location / {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
45 proxy_pass http://127.0.0.1:8081;
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
46 proxy_cache NAME;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
47 proxy_cache_valid 200 1m;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
48 }
1214
a05ba24a462b Tests: removed rewrite dependency in proxy_cache_range.t.
Maxim Dounin <mdounin@mdounin.ru>
parents: 1070
diff changeset
49
500
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
50 location /min_uses {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
51 proxy_pass http://127.0.0.1:8081/;
500
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
52 proxy_cache NAME;
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
53 proxy_cache_valid 200 1m;
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
54 proxy_cache_min_uses 2;
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
55 }
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
56 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
57
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
58 server {
974
882267679006 Tests: simplified parallel modifications in tests.
Andrey Zelenkov <zelenkov@nginx.com>
parents: 952
diff changeset
59 listen 127.0.0.1:8081;
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
60 server_name localhost;
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
61
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
62 location / {
1214
a05ba24a462b Tests: removed rewrite dependency in proxy_cache_range.t.
Maxim Dounin <mdounin@mdounin.ru>
parents: 1070
diff changeset
63 }
a05ba24a462b Tests: removed rewrite dependency in proxy_cache_range.t.
Maxim Dounin <mdounin@mdounin.ru>
parents: 1070
diff changeset
64
a05ba24a462b Tests: removed rewrite dependency in proxy_cache_range.t.
Maxim Dounin <mdounin@mdounin.ru>
parents: 1070
diff changeset
65 location /tbig.html {
a05ba24a462b Tests: removed rewrite dependency in proxy_cache_range.t.
Maxim Dounin <mdounin@mdounin.ru>
parents: 1070
diff changeset
66 limit_rate 50k;
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
67 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
68 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
69 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
70
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
71 EOF
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
72
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
73 $t->write_file('t.html', 'SEE-THIS');
1008
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
74
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
75 # should not fit in a single proxy buffer
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
76
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
77 $t->write_file('tbig.html',
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
78 join('', map { sprintf "XX%06dXX", $_ } (1 .. 7000)));
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
79
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
80 $t->run();
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
81
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
82 ###############################################################################
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
83
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
84 like(http_get_range('/t.html?1', 'Range: bytes=4-'), qr/^THIS/m,
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
85 'range on first request');
382
30364c578416 Tests: adjusted TODO for single range not-yet-cached requests.
Sergey Kandaurov <pluknet@nginx.com>
parents: 372
diff changeset
86
30364c578416 Tests: adjusted TODO for single range not-yet-cached requests.
Sergey Kandaurov <pluknet@nginx.com>
parents: 372
diff changeset
87 {
30364c578416 Tests: adjusted TODO for single range not-yet-cached requests.
Sergey Kandaurov <pluknet@nginx.com>
parents: 372
diff changeset
88 local $TODO = 'not yet';
30364c578416 Tests: adjusted TODO for single range not-yet-cached requests.
Sergey Kandaurov <pluknet@nginx.com>
parents: 372
diff changeset
89
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
90 like(http_get_range('/t.html?2', 'Range: bytes=0-2,4-'), qr/^SEE.*^THIS/ms,
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
91 'multipart range on first request');
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
92 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
93
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
94 like(http_get_range('/t.html?1', 'Range: bytes=4-'), qr/^THIS/m,
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
95 'cached range');
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
96 like(http_get_range('/t.html?1', 'Range: bytes=0-2,4-'), qr/^SEE.*^THIS/ms,
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
97 'cached multipart range');
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
98
500
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
99 like(http_get_range('/min_uses/t.html?3', 'Range: bytes=4-'),
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
100 qr/^THIS/m, 'range below min_uses');
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
101
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
102 like(http_get_range('/min_uses/t.html?4', 'Range: bytes=0-2,4-'),
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
103 qr/^SEE.*^THIS/ms, 'multipart range below min_uses');
b4d657ba1a62 Tests: tests for range requests below proxy_cache_min_uses.
Sergey Kandaurov <pluknet@nginx.com>
parents: 397
diff changeset
104
1214
a05ba24a462b Tests: removed rewrite dependency in proxy_cache_range.t.
Maxim Dounin <mdounin@mdounin.ru>
parents: 1070
diff changeset
105 like(http_get_range('/tbig.html', 'Range: bytes=0-19'),
1008
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
106 qr/^XX000001XXXX000002XX$/ms, 'range of response received in parts');
bab7b53a156f Tests: proxy cache range with sendfile in threads.
Sergey Kandaurov <pluknet@nginx.com>
parents: 974
diff changeset
107
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
108 ###############################################################################
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
109
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
110 sub http_get_range {
372
1d6abf0db011 Tests: whitespace fix.
Maxim Dounin <mdounin@mdounin.ru>
parents: 371
diff changeset
111 my ($url, $extra) = @_;
1d6abf0db011 Tests: whitespace fix.
Maxim Dounin <mdounin@mdounin.ru>
parents: 371
diff changeset
112 return http(<<EOF);
371
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
113 GET $url HTTP/1.1
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
114 Host: localhost
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
115 Connection: close
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
116 $extra
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
117
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
118 EOF
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
119 }
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
120
6fb6fea36560 Tests: proxy cache and range filter tests.
Maxim Dounin <mdounin@mdounin.ru>
parents:
diff changeset
121 ###############################################################################