Mercurial > hg > nginx-quic
annotate README @ 8695:41caf5410110 quic
QUIC: reject streams which we could not create.
The reasons why a stream may not be created by server currently include hitting
worker_connections limit and memory allocation error. Previously in these
cases the entire QUIC connection was closed and all its streams were shut down.
Now the new stream is rejected and existing streams continue working.
To reject an HTTP/3 request stream, RESET_STREAM and STOP_SENDING with
H3_REQUEST_REJECTED error code are sent to client. HTTP/3 uni streams and
Stream streams are not rejected.
author | Roman Arutyunyan <arut@nginx.com> |
---|---|
date | Thu, 11 Nov 2021 19:07:00 +0300 |
parents | 4d871baeacd2 |
children | be08b858086a |
rev | line source |
---|---|
7843 | 1 Experimental QUIC support for nginx |
2 ----------------------------------- | |
3 | |
4 1. Introduction | |
5 2. Installing | |
6 3. Configuration | |
7 4. Clients | |
8 5. Troubleshooting | |
7887
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
9 6. Contributing |
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
10 7. Links |
7843 | 11 |
12 1. Introduction | |
13 | |
14 This is an experimental QUIC [1] / HTTP/3 [2] support for nginx. | |
15 | |
16 The code is developed in a separate "quic" branch available | |
17 at https://hg.nginx.org/nginx-quic. Currently it is based | |
8488
8422570f6af4
README: updated after QUIC RFC publication, nginx 1.21 rebase.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8423
diff
changeset
|
18 on nginx mainline 1.21.x. We merge new nginx releases into |
8160
dd8e50e11bfc
QUIC: updated README.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8085
diff
changeset
|
19 this branch regularly. |
7843 | 20 |
21 The project code base is under the same BSD license as nginx. | |
22 | |
8160
dd8e50e11bfc
QUIC: updated README.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8085
diff
changeset
|
23 The code is currently at a beta level of quality and should not |
7843 | 24 be used in production. |
25 | |
26 We are working on improving HTTP/3 support with the goal of | |
27 integrating it to the main NGINX codebase. Expect frequent | |
28 updates of this code and don't rely on it for whatever purpose. | |
29 | |
30 We'll be grateful for any feedback and code submissions however | |
31 we don't bear any responsibilities for any issues with this code. | |
32 | |
33 You can always contact us via nginx-devel mailing list [3]. | |
34 | |
35 What works now: | |
36 | |
8488
8422570f6af4
README: updated after QUIC RFC publication, nginx 1.21 rebase.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8423
diff
changeset
|
37 Currently we support IETF-QUIC draft-29 through final RFC documents. |
7944
3c32717d7bb2
README: documented draft-28, draft-29 support.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7907
diff
changeset
|
38 Earlier drafts are NOT supported as they have incompatible wire format. |
7843 | 39 |
8160
dd8e50e11bfc
QUIC: updated README.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8085
diff
changeset
|
40 nginx should be able to respond to HTTP/3 requests over QUIC and |
7843 | 41 it should be possible to upload and download big files without errors. |
42 | |
43 + The handshake completes successfully | |
44 + One endpoint can update keys and its peer responds correctly | |
7867 | 45 + 0-RTT data is being received and acted on |
7843 | 46 + Connection is established using TLS Resume Ticket |
7866
2b580ac17a47
README: Retry support, protocol error messages implemented.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7850
diff
changeset
|
47 + A handshake that includes a Retry packet completes successfully |
7843 | 48 + Stream data is being exchanged and ACK'ed |
49 + An H3 transaction succeeded | |
50 + One or both endpoints insert entries into dynamic table and | |
51 subsequently reference them from header blocks | |
8065 | 52 + Version Negotiation packet is sent to client with unknown version |
53 + Lost packets are detected and retransmitted properly | |
8423
4117aa7fa38e
QUIC: connection migration.
Vladimir Homutov <vl@nginx.com>
parents:
8407
diff
changeset
|
54 + Clients may migrate to new address |
7843 | 55 |
56 Not (yet) supported features: | |
57 | |
8065 | 58 - Explicit Congestion Notification (ECN) as specified in quic-recovery [5] |
7843 | 59 - A connection with the spin bit succeeds and the bit is spinning |
60 - Structured Logging | |
61 | |
62 Since the code is experimental and still under development, | |
63 a lot of things may not work as expected, for example: | |
64 | |
65 - Flow control mechanism is basic and intended to avoid CPU hog and make | |
66 simple interactions possible | |
67 | |
8488
8422570f6af4
README: updated after QUIC RFC publication, nginx 1.21 rebase.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8423
diff
changeset
|
68 - Not all protocol requirements are strictly followed; some of checks are |
7843 | 69 omitted for the sake of simplicity of initial implementation |
70 | |
71 2. Installing | |
72 | |
73 You will need a BoringSSL [4] library that provides QUIC support | |
74 | |
7850
796b5b6c43cd
Mention quic branch in README.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7849
diff
changeset
|
75 $ hg clone -b quic https://hg.nginx.org/nginx-quic |
7843 | 76 $ cd nginx-quic |
7849
0e6528551f26
Configure: unbreak with old OpenSSL, --with-http_v3_module added.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7843
diff
changeset
|
77 $ ./auto/configure --with-debug --with-http_v3_module \ |
0e6528551f26
Configure: unbreak with old OpenSSL, --with-http_v3_module added.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7843
diff
changeset
|
78 --with-cc-opt="-I../boringssl/include" \ |
0e6528551f26
Configure: unbreak with old OpenSSL, --with-http_v3_module added.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7843
diff
changeset
|
79 --with-ld-opt="-L../boringssl/build/ssl \ |
0e6528551f26
Configure: unbreak with old OpenSSL, --with-http_v3_module added.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7843
diff
changeset
|
80 -L../boringssl/build/crypto" |
7843 | 81 $ make |
82 | |
8005
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
83 When configuring nginx, you can enable QUIC and HTTP/3 using the |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
84 following new configuration options: |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
85 |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
86 --with-http_v3_module - enable QUIC and HTTP/3 |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
87 --with-http_quic_module - enable QUIC for older HTTP versions |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
88 --with-stream_quic_module - enable QUIC in Stream |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
89 |
7843 | 90 3. Configuration |
91 | |
8005
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
92 The HTTP "listen" directive got two new options: "http3" and "quic". |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
93 The "http3" option enables HTTP/3 over QUIC on the specified port. |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
94 The "quic" option enables QUIC for older HTTP versions on this port. |
7843 | 95 |
8005
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
96 The Stream "listen" directive got a new option "quic" which enables |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
97 QUIC as client transport protocol instead of TCP or plain UDP. |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
98 |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
99 Along with "http3" or "quic", you also have to specify "reuseport" |
6e84524886d4
QUIC: updated README to mention "quic" listen parameter.
Roman Arutyunyan <arut@nginx.com>
parents:
7944
diff
changeset
|
100 option [6] to make it work properly with multiple workers. |
7843 | 101 |
102 A number of directives were added that specify transport parameter values: | |
103 | |
104 quic_max_idle_timeout | |
105 quic_max_ack_delay | |
8277
27bd6dc24426
README: reflect renaming of several transport parameter directives.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8241
diff
changeset
|
106 quic_max_udp_payload_size |
7843 | 107 quic_initial_max_data |
108 quic_initial_max_stream_data_bidi_local | |
109 quic_initial_max_stream_data_bidi_remote | |
110 quic_initial_max_stream_data_uni | |
111 quic_initial_max_streams_bidi | |
112 quic_initial_max_streams_uni | |
113 quic_ack_delay_exponent | |
8277
27bd6dc24426
README: reflect renaming of several transport parameter directives.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8241
diff
changeset
|
114 quic_disable_active_migration |
7843 | 115 quic_active_connection_id_limit |
116 | |
7879
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
117 To enable address validation: |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
118 |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
119 quic_retry on; |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
120 |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
121 To enable 0-RTT: |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
122 |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
123 ssl_early_data on; |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
124 |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
125 Make sure that TLS 1.3 is configured which is required for QUIC: |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
126 |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
127 ssl_protocols TLSv1.3; |
af22b60a905b
README: documented Retry, 0-RTT, TLSv1.3 configuration.
Sergey Kandaurov <pluknet@nginx.com>
parents:
7873
diff
changeset
|
128 |
8544
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
129 To enable GSO (Generic Segmentation Offloading): |
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
130 |
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
131 quic_gso on; |
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
132 |
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
133 By default this Linux-specific optimization [8] is disabled. |
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
134 Enable if your network interface is configured to support GSO. |
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
135 |
8016
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
136 A number of directives were added that configure HTTP/3: |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
137 |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
138 http3_max_table_capacity |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
139 http3_max_blocked_streams |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
140 http3_max_concurrent_pushes |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
141 http3_push |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
142 http3_push_preload |
affb0245e291
QUIC: added HTTP/3 directives list to README.
Roman Arutyunyan <arut@nginx.com>
parents:
8005
diff
changeset
|
143 |
8489
f0882db8c8d4
HTTP/3: removed $http3 that served its purpose.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8488
diff
changeset
|
144 An additional variable is available: $quic. |
7843 | 145 The value of $quic is "quic" if QUIC connection is used, |
8489
f0882db8c8d4
HTTP/3: removed $http3 that served its purpose.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8488
diff
changeset
|
146 or an empty string otherwise. |
7843 | 147 |
148 Example configuration: | |
149 | |
150 http { | |
151 log_format quic '$remote_addr - $remote_user [$time_local] ' | |
152 '"$request" $status $body_bytes_sent ' | |
8489
f0882db8c8d4
HTTP/3: removed $http3 that served its purpose.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8488
diff
changeset
|
153 '"$http_referer" "$http_user_agent" "$quic"'; |
7843 | 154 |
155 access_log logs/access.log quic; | |
156 | |
157 server { | |
158 # for better compatibility it's recommended | |
159 # to use the same port for quic and https | |
160 listen 8443 http3 reuseport; | |
161 listen 8443 ssl; | |
162 | |
163 ssl_certificate certs/example.com.crt; | |
164 ssl_certificate_key certs/example.com.key; | |
165 ssl_protocols TLSv1.3; | |
166 | |
167 location / { | |
168 # required for browsers to direct them into quic port | |
8489
f0882db8c8d4
HTTP/3: removed $http3 that served its purpose.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8488
diff
changeset
|
169 add_header Alt-Svc 'h3=":8443"; ma=86400'; |
7843 | 170 } |
171 } | |
172 } | |
173 | |
174 4. Clients | |
175 | |
176 * Browsers | |
177 | |
8324
6da4b045ec34
README: bump browsers' version after 81bb3a690c10 (old drafts rip).
Sergey Kandaurov <pluknet@nginx.com>
parents:
8323
diff
changeset
|
178 Known to work: Firefox 80+ and Chrome 85+ (QUIC draft 29+) |
7843 | 179 |
180 Beware of strange issues: sometimes browser may decide to ignore QUIC | |
181 Cache clearing/restart might help. Always check access.log and | |
182 error.log to make sure you are using HTTP/3 and not TCP https. | |
183 | |
184 + to enable QUIC in Firefox, set the following in 'about:config': | |
185 network.http.http3.enabled = true | |
186 | |
187 + to enable QUIC in Chrome, enable it on command line and force it | |
188 on your site: | |
189 | |
8085
57e5393e5d40
QUIC: switched to draft 29 by default.
Vladimir Homutov <vl@nginx.com>
parents:
8065
diff
changeset
|
190 $ ./chrome --enable-quic --quic-version=h3-29 \ |
7843 | 191 --origin-to-force-quic-on=example.com:8443 |
192 | |
193 * Console clients | |
194 | |
195 Known to work: ngtcp2, firefox's neqo and chromium's console clients: | |
196 | |
197 $ examples/client 127.0.0.1 8443 https://example.com:8443/index.html | |
198 | |
199 $ ./neqo-client https://127.0.0.1:8443/ | |
200 | |
201 $ chromium-build/out/my_build/quic_client http://example.com:8443 \ | |
8085
57e5393e5d40
QUIC: switched to draft 29 by default.
Vladimir Homutov <vl@nginx.com>
parents:
8065
diff
changeset
|
202 --quic_version=h3-29 \ |
7843 | 203 --allow_unknown_root_cert \ |
204 --disable_certificate_verification | |
205 | |
206 | |
207 If you've got it right, in the access log you should see something like: | |
208 | |
209 127.0.0.1 - - [24/Apr/2020:11:27:29 +0300] "GET / HTTP/3" 200 805 "-" | |
8489
f0882db8c8d4
HTTP/3: removed $http3 that served its purpose.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8488
diff
changeset
|
210 "nghttp3/ngtcp2 client" "quic" |
7843 | 211 |
212 | |
213 5. Troubleshooting | |
214 | |
215 Here are some tips that may help you to identify problems: | |
216 | |
8160
dd8e50e11bfc
QUIC: updated README.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8085
diff
changeset
|
217 + Ensure you are building with proper SSL library that supports QUIC |
7843 | 218 |
219 + Ensure you are using the proper SSL library in runtime | |
220 (`nginx -V` will show you what you are using) | |
221 | |
222 + Ensure your client is actually sending QUIC requests | |
223 (see "Clients" section about browsers and cache) | |
224 | |
225 We recommend to start with simple console client like ngtcp2 | |
226 to ensure you've got server configured properly before trying | |
7872 | 227 with real browsers that may be very picky with certificates, |
7843 | 228 for example. |
229 | |
230 + Build nginx with debug support [7] and check your debug log. | |
231 It should contain all details about connection and why it | |
232 failed. All related messages contain "quic " prefix and can | |
233 be easily filtered out. | |
234 | |
235 + If you want to investigate deeper, you may want to enable | |
8505
d56c7c4b66fd
README: updated path after moving QUIC sources.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8489
diff
changeset
|
236 additional debugging in src/event/quic/ngx_event_quic_connection.h: |
7843 | 237 |
238 #define NGX_QUIC_DEBUG_PACKETS | |
239 #define NGX_QUIC_DEBUG_FRAMES | |
8241
2dfc5ef29973
QUIC: introduced QUIC buffers.
Roman Arutyunyan <arut@nginx.com>
parents:
8160
diff
changeset
|
240 #define NGX_QUIC_DEBUG_ALLOC |
7843 | 241 #define NGX_QUIC_DEBUG_CRYPTO |
242 | |
7887
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
243 6. Contributing |
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
244 |
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
245 If you are willing to contribute, please refer to |
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
246 http://nginx.org/en/docs/contributing_changes.html |
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
247 |
c7d1b500bd0a
Updated README with "Contributing" section and draft details.
Vladimir Homutov <vl@nginx.com>
parents:
7879
diff
changeset
|
248 7. Links |
7843 | 249 |
8488
8422570f6af4
README: updated after QUIC RFC publication, nginx 1.21 rebase.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8423
diff
changeset
|
250 [1] https://datatracker.ietf.org/doc/html/rfc9000 |
8422570f6af4
README: updated after QUIC RFC publication, nginx 1.21 rebase.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8423
diff
changeset
|
251 [2] https://datatracker.ietf.org/doc/html/draft-ietf-quic-http |
7843 | 252 [3] https://mailman.nginx.org/mailman/listinfo/nginx-devel |
253 [4] https://boringssl.googlesource.com/boringssl/ | |
8488
8422570f6af4
README: updated after QUIC RFC publication, nginx 1.21 rebase.
Sergey Kandaurov <pluknet@nginx.com>
parents:
8423
diff
changeset
|
254 [5] https://datatracker.ietf.org/doc/html/rfc9002 |
7843 | 255 [6] https://nginx.org/en/docs/http/ngx_http_core_module.html#listen |
256 [7] https://nginx.org/en/docs/debugging_log.html | |
8544
d0ef43a53a51
QUIC: updated README with GSO details.
Vladimir Homutov <vl@nginx.com>
parents:
8505
diff
changeset
|
257 [8] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-paper-DRAFT-1.pdf |