From nobody Tue Feb 01 14:57:34 2022 X-Original-To: dev-commits-ports-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0598619ABD35; Tue, 1 Feb 2022 14:57:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jp7ML4kHCz4p7X; Tue, 1 Feb 2022 14:57:34 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643727454; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ia4wdN/Z9do5xv4CoOe1flfyHEPId8E4QOiqz6ZeyiQ=; b=AW3raDFVV05+oYN5OKu6W0PvMIduivIx8NZuqy4JGngPCm5LTcdcWchNY3NkJrd7Fqx/iu F82nENIr6bSn3cN4cckVTE3tq6ABskx8BEQc3udfAhAhOADvkFBT0nYujWs14A9wPGcd3n B9oEVDKqOC3BqF3E3This0dJ9ISAPyRZBhERcGMRKuSsKUtEZivRSvZ1c28LP2imwX7M2h wtvw3Q8SNRZqcyAmO/su5vheBL8slqRx8AJihl+37FjjUxlRyOclFr3J5eM9UYYNjj5nlo 35p+lnuXi/6sdfWp6fqy0/6LlztlcPnvihQH4nCiTFwDAJfCU7HElnZ6N+tosw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7B72412DBE; Tue, 1 Feb 2022 14:57:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 211EvY1M010859; Tue, 1 Feb 2022 14:57:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 211EvYlb010858; Tue, 1 Feb 2022 14:57:34 GMT (envelope-from git) Date: Tue, 1 Feb 2022 14:57:34 GMT Message-Id: <202202011457.211EvYlb010858@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: "Sergey A. Osokin" Subject: git: 117d256c411a - main - www/nginx-devel: update HTTPv3/QUIC patch to the recent commit List-Id: Commits to the main branch of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-main@freebsd.org X-BeenThere: dev-commits-ports-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: osa X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 117d256c411ae24dc0e1c0868a1825a59300ca4e Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643727454; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ia4wdN/Z9do5xv4CoOe1flfyHEPId8E4QOiqz6ZeyiQ=; b=fs+0zXsQpDVYyXnmMzfV5vt0dkaSkMjdD4LMoHv4qXpk2+rTSUacScD7/nBGz9s5pPXlu9 r7avzpkjJ/rsMlDcR+19BTtIf/DgeQvrXC63eXTomR+dDcTS87/eOzBvG2HjbRO5jI0plF 7ijIAaCFXuD5b2UyBCuagN7YzXrVuiaS6rf1apejZs92hd4Kf4dUCc2WXBCEMldKdihX1a zM9Ue5Bk8bzv9Ai8tNJv9lYRoQR+uwTbHaFzMojCO5XhlSXb8w5BdIgLIPJYUUu/MNI5TO 6R3biT8mYyMaXgXP9vqFI2Y7LGYCjTqQj1qmuGE5EM2G80sUF7XW0iTGYtCIaw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643727454; a=rsa-sha256; cv=none; b=c0kAobD8d9pxfGAAplOhhMjYVmMMz9LoHUwZ7moF3NBylwioTPzoAzgH7deLyT3S+a9p9j JBL3wUcLClCwOHZXidOI/w5oGddXNAGaoelrjZXk3HmyuUN6wMgeuF44gxOIPnxi6T3615 M9garm9U0DpNW9cMVO4tWy1f6AVZje5ilwvlbqQWewpBrHRSQGegc7TQVwOAqJKpvSo9+r zutiQa2Jzp01Jp98LSsUSnYhPVWyohMpZz0B9QdmR1+mgmLpJ4HIIXLpLp6HwMTqcuDlCK M2g933D3++LoirmW7uWBV8JB79uml0rzXn+KRZMJJJsYvkP2yuXUY+6/0EBO0g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by osa: URL: https://cgit.FreeBSD.org/ports/commit/?id=117d256c411ae24dc0e1c0868a1825a59300ca4e commit 117d256c411ae24dc0e1c0868a1825a59300ca4e Author: Sergey A. Osokin AuthorDate: 2022-02-01 14:57:06 +0000 Commit: Sergey A. Osokin CommitDate: 2022-02-01 14:57:26 +0000 www/nginx-devel: update HTTPv3/QUIC patch to the recent commit --- www/nginx-devel/Makefile | 2 +- www/nginx-devel/files/extra-patch-httpv3 | 785 +++++++++++++++---------------- 2 files changed, 393 insertions(+), 394 deletions(-) diff --git a/www/nginx-devel/Makefile b/www/nginx-devel/Makefile index 332014908b34..381dee0d872e 100644 --- a/www/nginx-devel/Makefile +++ b/www/nginx-devel/Makefile @@ -2,7 +2,7 @@ PORTNAME?= nginx PORTVERSION= 1.21.6 -PORTREVISION= 3 +PORTREVISION= 4 CATEGORIES= www MASTER_SITES= https://nginx.org/download/ \ LOCAL/osa diff --git a/www/nginx-devel/files/extra-patch-httpv3 b/www/nginx-devel/files/extra-patch-httpv3 index 9f0ab11e7c7c..84104bfbf152 100644 --- a/www/nginx-devel/files/extra-patch-httpv3 +++ b/www/nginx-devel/files/extra-patch-httpv3 @@ -2,7 +2,7 @@ diff --git a/README b/README new file mode 100644 --- /dev/null +++ b/README -@@ -0,0 +1,261 @@ +@@ -0,0 +1,233 @@ +Experimental QUIC support for nginx +----------------------------------- + @@ -39,8 +39,7 @@ new file mode 100644 + + What works now: + -+ Currently we support IETF-QUIC draft-29 through final RFC documents. -+ Earlier drafts are NOT supported as they have incompatible wire format. ++ We support IETF QUIC version 1. Internet drafts are no longer supported. + + nginx should be able to respond to HTTP/3 requests over QUIC and + it should be possible to upload and download big files without errors. @@ -58,21 +57,6 @@ new file mode 100644 + + Lost packets are detected and retransmitted properly + + Clients may migrate to new address + -+ Not (yet) supported features: -+ -+ - Explicit Congestion Notification (ECN) as specified in quic-recovery [5] -+ - A connection with the spin bit succeeds and the bit is spinning -+ - Structured Logging -+ -+ Since the code is experimental and still under development, -+ a lot of things may not work as expected, for example: -+ -+ - Flow control mechanism is basic and intended to avoid CPU hog and make -+ simple interactions possible -+ -+ - Not all protocol requirements are strictly followed; some of checks are -+ omitted for the sake of simplicity of initial implementation -+ +2. Installing + + You will need a BoringSSL [4] library that provides QUIC support @@ -183,21 +167,12 @@ new file mode 100644 + + * Browsers + -+ Known to work: Firefox 80+ and Chrome 85+ (QUIC draft 29+) ++ Known to work: Firefox 90+ and Chrome 92+ (QUIC version 1) + + Beware of strange issues: sometimes browser may decide to ignore QUIC + Cache clearing/restart might help. Always check access.log and + error.log to make sure you are using HTTP/3 and not TCP https. + -+ + to enable QUIC in Firefox, set the following in 'about:config': -+ network.http.http3.enabled = true -+ -+ + to enable QUIC in Chrome, enable it on command line and force it -+ on your site: -+ -+ $ ./chrome --enable-quic --quic-version=h3-29 \ -+ --origin-to-force-quic-on=example.com:8443 -+ + * Console clients + + Known to work: ngtcp2, firefox's neqo and chromium's console clients: @@ -206,10 +181,7 @@ new file mode 100644 + + $ ./neqo-client https://127.0.0.1:8443/ + -+ $ chromium-build/out/my_build/quic_client http://example.com:8443 \ -+ --quic_version=h3-29 \ -+ --allow_unknown_root_cert \ -+ --disable_certificate_verification ++ $ chromium-build/out/my_build/quic_client http://example.com:8443 + + + If you've got it right, in the access log you should see something like: @@ -227,7 +199,7 @@ new file mode 100644 + + Ensure you are using the proper SSL library in runtime + (`nginx -V` will show you what you are using) + -+ + Ensure your client is actually sending QUIC requests ++ + Ensure your client is actually sending requests over QUIC + (see "Clients" section about browsers and cache) + + We recommend to start with simple console client like ngtcp2 @@ -257,7 +229,7 @@ new file mode 100644 + + [1] https://datatracker.ietf.org/doc/html/rfc9000 + [2] https://datatracker.ietf.org/doc/html/draft-ietf-quic-http -+ [3] https://mailman.nginx.org/mailman/listinfo/nginx-devel ++ [3] https://mailman.nginx.org/mailman3/lists/nginx-devel.nginx.org/ + [4] https://boringssl.googlesource.com/boringssl/ + [5] https://datatracker.ietf.org/doc/html/rfc9002 + [6] https://nginx.org/en/docs/http/ngx_http_core_module.html#listen @@ -962,7 +934,7 @@ diff --git a/src/core/ngx_core.h b/src/core/ngx_core.h diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c --- a/src/event/ngx_event.c +++ b/src/event/ngx_event.c -@@ -266,6 +266,18 @@ ngx_process_events_and_timers(ngx_cycle_ +@@ -267,6 +267,18 @@ ngx_process_events_and_timers(ngx_cycle_ ngx_int_t ngx_handle_read_event(ngx_event_t *rev, ngx_uint_t flags) { @@ -981,7 +953,7 @@ diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c if (ngx_event_flags & NGX_USE_CLEAR_EVENT) { /* kqueue, epoll */ -@@ -336,9 +348,15 @@ ngx_handle_write_event(ngx_event_t *wev, +@@ -337,9 +349,15 @@ ngx_handle_write_event(ngx_event_t *wev, { ngx_connection_t *c; @@ -1002,7 +974,7 @@ diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c diff --git a/src/event/ngx_event.h b/src/event/ngx_event.h --- a/src/event/ngx_event.h +++ b/src/event/ngx_event.h -@@ -493,12 +493,6 @@ extern ngx_module_t ngx_event_ +@@ -494,12 +494,6 @@ extern ngx_module_t ngx_event_ void ngx_event_accept(ngx_event_t *ev); @@ -1015,7 +987,7 @@ diff --git a/src/event/ngx_event.h b/src/event/ngx_event.h ngx_int_t ngx_trylock_accept_mutex(ngx_cycle_t *cycle); ngx_int_t ngx_enable_accept_events(ngx_cycle_t *cycle); u_char *ngx_accept_log_error(ngx_log_t *log, u_char *buf, size_t len); -@@ -528,6 +522,7 @@ ngx_int_t ngx_send_lowat(ngx_connection_ +@@ -529,6 +523,7 @@ ngx_int_t ngx_send_lowat(ngx_connection_ #include #include @@ -1026,7 +998,7 @@ diff --git a/src/event/ngx_event.h b/src/event/ngx_event.h diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c -@@ -3146,6 +3146,13 @@ ngx_ssl_shutdown(ngx_connection_t *c) +@@ -3149,6 +3149,13 @@ ngx_ssl_shutdown(ngx_connection_t *c) ngx_err_t err; ngx_uint_t tries; @@ -1929,7 +1901,7 @@ diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c new file mode 100644 --- /dev/null +++ b/src/event/quic/ngx_event_quic.c -@@ -0,0 +1,1491 @@ +@@ -0,0 +1,1457 @@ + +/* + * Copyright (C) Nginx, Inc. @@ -1948,7 +1920,6 @@ new file mode 100644 + ngx_quic_header_t *pkt); +static void ngx_quic_input_handler(ngx_event_t *rev); + -+static ngx_int_t ngx_quic_close_quic(ngx_connection_t *c, ngx_int_t rc); +static void ngx_quic_close_timer_handler(ngx_event_t *ev); + +static ngx_int_t ngx_quic_handle_datagram(ngx_connection_t *c, ngx_buf_t *b, @@ -2142,17 +2113,13 @@ new file mode 100644 + + rc = ngx_quic_handle_datagram(c, c->buffer, conf); + if (rc != NGX_OK) { -+ ngx_quic_close_connection(c, rc == NGX_DECLINED ? NGX_DONE : NGX_ERROR); ++ ngx_quic_close_connection(c, rc); + return; + } + ++ /* quic connection is now created */ + qc = ngx_quic_get_connection(c); + -+ if (qc == NULL) { -+ ngx_quic_close_connection(c, NGX_DONE); -+ return; -+ } -+ + ngx_add_timer(c->read, qc->tp.max_idle_timeout); + ngx_quic_connstate_dbg(c); + @@ -2261,8 +2228,7 @@ new file mode 100644 + } + } + -+ if (ngx_quic_keys_set_initial_secret(c->pool, qc->keys, &pkt->dcid, -+ qc->version) ++ if (ngx_quic_keys_set_initial_secret(c->pool, qc->keys, &pkt->dcid) + != NGX_OK) + { + return NULL; @@ -2376,7 +2342,7 @@ new file mode 100644 + return; + } + -+ if (rc == NGX_DECLINED) { ++ if (rc == NGX_DONE) { + return; + } + @@ -2392,54 +2358,22 @@ new file mode 100644 +void +ngx_quic_close_connection(ngx_connection_t *c, ngx_int_t rc) +{ ++ ngx_uint_t i; + ngx_pool_t *pool; ++ ngx_quic_send_ctx_t *ctx; + ngx_quic_connection_t *qc; + -+ ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic ngx_quic_close_connection rc:%i", rc); -+ + qc = ngx_quic_get_connection(c); + + if (qc == NULL) { -+ if (rc == NGX_ERROR) { -+ ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic close connection early error"); -+ } -+ -+ } else if (ngx_quic_close_quic(c, rc) == NGX_AGAIN) { -+ return; -+ } -+ -+ if (c->ssl) { -+ (void) ngx_ssl_shutdown(c); -+ } -+ -+ if (c->read->timer_set) { -+ ngx_del_timer(c->read); ++ ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, ++ "quic packet rejected rc:%i, cleanup connection", rc); ++ goto quic_done; + } + -+#if (NGX_STAT_STUB) -+ (void) ngx_atomic_fetch_add(ngx_stat_active, -1); -+#endif -+ -+ c->destroyed = 1; -+ -+ pool = c->pool; -+ -+ ngx_close_connection(c); -+ -+ ngx_destroy_pool(pool); -+} -+ -+ -+static ngx_int_t -+ngx_quic_close_quic(ngx_connection_t *c, ngx_int_t rc) -+{ -+ ngx_uint_t i; -+ ngx_quic_send_ctx_t *ctx; -+ ngx_quic_connection_t *qc; -+ -+ qc = ngx_quic_get_connection(c); ++ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, ++ "quic close %s rc:%i", ++ qc->closing ? "resumed": "initiated", rc); + + if (!qc->closing) { + @@ -2458,10 +2392,11 @@ new file mode 100644 + * closed and its state is discarded when it remains idle + */ + -+ ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic closing %s connection", -+ qc->draining ? "drained" : "idle"); ++ /* this case also handles some errors from ngx_quic_run() */ + ++ ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, ++ "quic close silent drain:%d timedout:%d", ++ qc->draining, c->read->timedout); + } else { + + /* @@ -2476,7 +2411,7 @@ new file mode 100644 + + if (rc == NGX_OK) { + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic immediate close drain:%d", ++ "quic close immediate drain:%d", + qc->draining); + + qc->close.log = c->log; @@ -2496,7 +2431,7 @@ new file mode 100644 + } + + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic immediate close due to %s error: %ui %s", ++ "quic close immediate due to %serror: %ui %s", + qc->error_app ? "app " : "", qc->error, + qc->error_reason ? qc->error_reason : ""); + } @@ -2519,7 +2454,7 @@ new file mode 100644 + } + + if (ngx_quic_close_streams(c, qc) == NGX_AGAIN) { -+ return NGX_AGAIN; ++ return; + } + + if (qc->push.timer_set) { @@ -2539,18 +2474,37 @@ new file mode 100644 + } + + if (qc->close.timer_set) { -+ return NGX_AGAIN; ++ return; + } + + ngx_quic_close_sockets(c); + -+ ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic part of connection is terminated"); ++ ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "quic close completed"); + + /* may be tested from SSL callback during SSL shutdown */ + c->udp = NULL; + -+ return NGX_OK; ++quic_done: ++ ++ if (c->ssl) { ++ (void) ngx_ssl_shutdown(c); ++ } ++ ++ if (c->read->timer_set) { ++ ngx_del_timer(c->read); ++ } ++ ++#if (NGX_STAT_STUB) ++ (void) ngx_atomic_fetch_add(ngx_stat_active, -1); ++#endif ++ ++ c->destroyed = 1; ++ ++ pool = c->pool; ++ ++ ngx_close_connection(c); ++ ++ ngx_destroy_pool(pool); +} + + @@ -2633,22 +2587,18 @@ new file mode 100644 +#if (NGX_DEBUG) + if (pkt.parsed) { + ngx_log_debug5(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic packet %s done decr:%d pn:%L perr:%ui rc:%i", -+ ngx_quic_level_name(pkt.level), pkt.decrypted, -+ pkt.pn, pkt.error, rc); ++ "quic packet done rc:%i level:%s" ++ " decr:%d pn:%L perr:%ui", ++ rc, ngx_quic_level_name(pkt.level), ++ pkt.decrypted, pkt.pn, pkt.error); + } else { + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic packet done parse failed rc:%i", rc); ++ "quic packet done rc:%i parse failed", rc); + } +#endif + -+ if (rc == NGX_ERROR) { -+ return NGX_ERROR; -+ } -+ -+ if (rc == NGX_DONE) { -+ /* stop further processing */ -+ return NGX_DECLINED; ++ if (rc == NGX_ERROR || rc == NGX_DONE) { ++ return rc; + } + + if (rc == NGX_OK) { @@ -2683,7 +2633,7 @@ new file mode 100644 + } + + if (!good) { -+ return NGX_DECLINED; ++ return NGX_DONE; + } + + qc = ngx_quic_get_connection(c); @@ -2717,13 +2667,13 @@ new file mode 100644 + + rc = ngx_quic_parse_packet(pkt); + -+ if (rc == NGX_DECLINED || rc == NGX_ERROR) { -+ return rc; ++ if (rc == NGX_ERROR) { ++ return NGX_DECLINED; + } + + pkt->parsed = 1; + -+ c->log->action = "processing quic packet"; ++ c->log->action = "handling quic packet"; + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, + "quic packet rx dcid len:%uz %xV", @@ -2808,10 +2758,12 @@ new file mode 100644 + } + + if (pkt->level != ssl_encryption_initial) { ++ ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, ++ "quic expected initial, got handshake"); + return NGX_ERROR; + } + -+ c->log->action = "processing initial packet"; ++ c->log->action = "handling initial packet"; + + if (pkt->dcid.len < NGX_QUIC_CID_LEN_MIN) { + /* RFC 9000, 7.2. Negotiating Connection IDs */ @@ -3363,7 +3315,7 @@ new file mode 100644 +{ + ngx_connection_t *c; + -+ ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ev->log, 0, "quic push timer"); ++ ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ev->log, 0, "quic push handler"); + + c = ev->data; + @@ -3407,25 +3359,11 @@ new file mode 100644 + + ngx_quic_finalize_connection(c, qc->shutdown_code, qc->shutdown_reason); +} -+ -+ -+uint32_t -+ngx_quic_version(ngx_connection_t *c) -+{ -+ uint32_t version; -+ ngx_quic_connection_t *qc; -+ -+ qc = ngx_quic_get_connection(c); -+ -+ version = qc->version; -+ -+ return (version & 0xff000000) == 0xff000000 ? version & 0xff : version; -+} diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h new file mode 100644 --- /dev/null +++ b/src/event/quic/ngx_event_quic.h -@@ -0,0 +1,88 @@ +@@ -0,0 +1,109 @@ + +/* + * Copyright (C) Nginx, Inc. @@ -3456,44 +3394,66 @@ new file mode 100644 +#define NGX_QUIC_STREAM_UNIDIRECTIONAL 0x02 + + -+typedef struct { -+ ngx_ssl_t *ssl; ++typedef enum { ++ NGX_QUIC_STREAM_SEND_READY = 0, ++ NGX_QUIC_STREAM_SEND_SEND, ++ NGX_QUIC_STREAM_SEND_DATA_SENT, ++ NGX_QUIC_STREAM_SEND_DATA_RECVD, ++ NGX_QUIC_STREAM_SEND_RESET_SENT, ++ NGX_QUIC_STREAM_SEND_RESET_RECVD ++} ngx_quic_stream_send_state_e; ++ ++ ++typedef enum { ++ NGX_QUIC_STREAM_RECV_RECV = 0, ++ NGX_QUIC_STREAM_RECV_SIZE_KNOWN, ++ NGX_QUIC_STREAM_RECV_DATA_RECVD, ++ NGX_QUIC_STREAM_RECV_DATA_READ, ++ NGX_QUIC_STREAM_RECV_RESET_RECVD, ++ NGX_QUIC_STREAM_RECV_RESET_READ ++} ngx_quic_stream_recv_state_e; + -+ ngx_flag_t retry; -+ ngx_flag_t gso_enabled; -+ ngx_flag_t disable_active_migration; -+ ngx_msec_t timeout; -+ ngx_str_t host_key; -+ size_t mtu; -+ size_t stream_buffer_size; -+ ngx_uint_t max_concurrent_streams_bidi; -+ ngx_uint_t max_concurrent_streams_uni; -+ ngx_uint_t active_connection_id_limit; -+ ngx_int_t stream_close_code; -+ ngx_int_t stream_reject_code_uni; -+ ngx_int_t stream_reject_code_bidi; + -+ u_char av_token_key[NGX_QUIC_AV_KEY_LEN]; -+ u_char sr_token_key[NGX_QUIC_SR_KEY_LEN]; ++typedef struct { ++ ngx_ssl_t *ssl; ++ ++ ngx_flag_t retry; ++ ngx_flag_t gso_enabled; ++ ngx_flag_t disable_active_migration; ++ ngx_msec_t timeout; ++ ngx_str_t host_key; ++ size_t mtu; ++ size_t stream_buffer_size; ++ ngx_uint_t max_concurrent_streams_bidi; ++ ngx_uint_t max_concurrent_streams_uni; ++ ngx_uint_t active_connection_id_limit; ++ ngx_int_t stream_close_code; ++ ngx_int_t stream_reject_code_uni; ++ ngx_int_t stream_reject_code_bidi; ++ ++ u_char av_token_key[NGX_QUIC_AV_KEY_LEN]; ++ u_char sr_token_key[NGX_QUIC_SR_KEY_LEN]; +} ngx_quic_conf_t; + + +struct ngx_quic_stream_s { -+ ngx_rbtree_node_t node; -+ ngx_queue_t queue; -+ ngx_connection_t *parent; -+ ngx_connection_t *connection; -+ uint64_t id; -+ uint64_t acked; -+ uint64_t send_max_data; -+ uint64_t recv_max_data; -+ uint64_t recv_offset; -+ uint64_t recv_window; -+ uint64_t recv_last; -+ uint64_t final_size; -+ ngx_chain_t *in; -+ ngx_chain_t *out; -+ ngx_uint_t cancelable; /* unsigned cancelable:1; */ ++ ngx_rbtree_node_t node; ++ ngx_queue_t queue; ++ ngx_connection_t *parent; ++ ngx_connection_t *connection; ++ uint64_t id; ++ uint64_t acked; ++ uint64_t send_max_data; ++ uint64_t recv_max_data; ++ uint64_t recv_offset; ++ uint64_t recv_window; ++ uint64_t recv_last; ++ uint64_t final_size; ++ ngx_chain_t *in; ++ ngx_chain_t *out; ++ ngx_uint_t cancelable; /* unsigned cancelable:1; */ ++ ngx_quic_stream_send_state_e send_state; ++ ngx_quic_stream_recv_state_e recv_state; +}; + + @@ -3505,7 +3465,6 @@ new file mode 100644 + const char *reason); +ngx_int_t ngx_quic_reset_stream(ngx_connection_t *c, ngx_uint_t err); +ngx_int_t ngx_quic_shutdown_stream(ngx_connection_t *c, int how); -+uint32_t ngx_quic_version(ngx_connection_t *c); +ngx_int_t ngx_quic_handle_read_event(ngx_event_t *rev, ngx_uint_t flags); +ngx_int_t ngx_quic_handle_write_event(ngx_event_t *wev, size_t lowat); +ngx_int_t ngx_quic_get_packet_dcid(ngx_log_t *log, u_char *data, size_t len, @@ -3518,7 +3477,7 @@ diff --git a/src/event/quic/ngx_event_quic_ack.c b/src/event/quic/ngx_event_quic new file mode 100644 --- /dev/null +++ b/src/event/quic/ngx_event_quic_ack.c -@@ -0,0 +1,1190 @@ +@@ -0,0 +1,1193 @@ + +/* + * Copyright (C) Nginx, Inc. @@ -4138,10 +4097,13 @@ new file mode 100644 + case NGX_QUIC_FT_STREAM: + qs = ngx_quic_find_stream(&qc->streams.tree, f->u.stream.stream_id); + -+ if (qs && qs->connection->write->error) { -+ /* RESET_STREAM was sent */ -+ ngx_quic_free_frame(c, f); -+ break; ++ if (qs) { ++ if (qs->send_state == NGX_QUIC_STREAM_SEND_RESET_SENT ++ || qs->send_state == NGX_QUIC_STREAM_SEND_RESET_RECVD) ++ { ++ ngx_quic_free_frame(c, f); ++ break; ++ } + } + + /* fall through */ @@ -6091,7 +6053,7 @@ new file mode 100644 + } + + ngx_log_debug5(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic cid #%uL received id:%uz:%xV:%*xs", ++ "quic cid seq:%uL received id:%uz:%xV:%*xs", + cid->seqnum, id->len, id, + (size_t) NGX_QUIC_SR_TOKEN_LEN, cid->sr_token); + @@ -6171,7 +6133,7 @@ new file mode 100644 + } + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic socket #%uL is retired", qsock->sid.seqnum); ++ "quic socket seq:%uL is retired", qsock->sid.seqnum); + + ngx_quic_close_socket(c, qsock); + @@ -7349,7 +7311,7 @@ new file mode 100644 + } + + ngx_log_error(NGX_LOG_INFO, c->log, 0, -+ "quic path #%uL addr:%V successfully validated", ++ "quic path seq:%uL addr:%V successfully validated", + path->seqnum, &path->addr_text); + + ngx_quic_path_dbg(c, "is validated", path); @@ -7407,7 +7369,7 @@ new file mode 100644 + NGX_SOCKADDR_STRLEN, 1); + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic path #%uL created addr:%V", ++ "quic path seq:%uL created addr:%V", + path->seqnum, &path->addr_text); + return path; +} @@ -7535,8 +7497,8 @@ new file mode 100644 + path->received += len; + + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic packet len:%O via sock#%uL path#%uL", -+ len, qsock->sid.seqnum, path->seqnum); ++ "quic packet len:%O via sock seq:%L path seq:%uL", ++ len, (int64_t) qsock->sid.seqnum, path->seqnum); + ngx_quic_path_dbg(c, "status", path); + + return NGX_OK; @@ -7564,7 +7526,7 @@ new file mode 100644 + } + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic path #%uL addr:%V retired", ++ "quic path seq:%uL addr:%V retired", + path->seqnum, &path->addr_text); + + return NGX_OK; @@ -7587,7 +7549,7 @@ new file mode 100644 + } + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic send path set to #%uL addr:%V", ++ "quic send path set to seq:%uL addr:%V", + path->seqnum, &path->addr_text); +} + @@ -7664,7 +7626,7 @@ new file mode 100644 + } + + ngx_log_error(NGX_LOG_INFO, c->log, 0, -+ "quic migrated to path#%uL addr:%V", ++ "quic migrated to path seq:%uL addr:%V", + qc->path->seqnum, &qc->path->addr_text); + + ngx_quic_path_dbg(c, "is now active", qc->path); @@ -7683,7 +7645,7 @@ new file mode 100644 + qc = ngx_quic_get_connection(c); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic initiated validation of path #%uL", path->seqnum); ++ "quic initiated validation of path seq:%uL", path->seqnum); + + path->validating = 1; + @@ -7719,7 +7681,7 @@ new file mode 100644 + ngx_quic_frame_t frame; + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic path #%uL send path_challenge tries:%ui", ++ "quic path seq:%uL send path_challenge tries:%ui", + path->seqnum, path->tries); + + ngx_memzero(&frame, sizeof(ngx_quic_frame_t)); @@ -7809,7 +7771,7 @@ new file mode 100644 + } + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ev->log, 0, -+ "quic path #%uL validation failed", path->seqnum); ++ "quic path seq:%uL validation failed", path->seqnum); + + /* found expired path */ + @@ -7843,7 +7805,7 @@ new file mode 100644 + ngx_quic_set_connection_path(c, qc->path); + + ngx_log_error(NGX_LOG_INFO, c->log, 0, -+ "quic path #%uL addr:%V is restored from backup", ++ "quic path seq:%uL addr:%V is restored from backup", + qc->path->seqnum, &qc->path->addr_text); + + ngx_quic_path_dbg(c, "is active", qc->path); @@ -7885,7 +7847,7 @@ new file mode 100644 + +#define ngx_quic_path_dbg(c, msg, path) \ + ngx_log_debug7(NGX_LOG_DEBUG_EVENT, c->log, 0, \ -+ "quic path#%uL %s sent:%O recvd:%O state:%s%s%s", \ ++ "quic path seq:%uL %s sent:%O recvd:%O state:%s%s%s", \ + path->seqnum, msg, path->sent, path->received, \ + path->limited ? "L" : "", path->validated ? "V": "N", \ + path->validating ? "R": ""); @@ -7910,7 +7872,7 @@ diff --git a/src/event/quic/ngx_event_quic_output.c b/src/event/quic/ngx_event_q new file mode 100644 --- /dev/null +++ b/src/event/quic/ngx_event_quic_output.c -@@ -0,0 +1,1273 @@ +@@ -0,0 +1,1268 @@ + +/* + * Copyright (C) Nginx, Inc. @@ -8318,7 +8280,7 @@ new file mode 100644 + struct msghdr msg; + struct cmsghdr *cmsg; + -+#if defined(NGX_HAVE_ADDRINFO_CMSG) ++#if (NGX_HAVE_ADDRINFO_CMSG) + char msg_control[CMSG_SPACE(sizeof(uint16_t)) + + CMSG_SPACE(sizeof(ngx_addrinfo_t))]; +#else @@ -8351,7 +8313,7 @@ new file mode 100644 + valp = (void *) CMSG_DATA(cmsg); + *valp = segment; + -+#if defined(NGX_HAVE_ADDRINFO_CMSG) ++#if (NGX_HAVE_ADDRINFO_CMSG) + if (c->listening && c->listening->wildcard && c->local_sockaddr) { + cmsg = CMSG_NXTHDR(&msg, cmsg); + clen += ngx_set_srcaddr_cmsg(cmsg, c->local_sockaddr); @@ -8586,13 +8548,10 @@ new file mode 100644 +ngx_quic_init_packet(ngx_connection_t *c, ngx_quic_send_ctx_t *ctx, + ngx_quic_header_t *pkt, ngx_quic_path_t *path) +{ -+ ngx_quic_socket_t *qsock; + ngx_quic_connection_t *qc; + + qc = ngx_quic_get_connection(c); + -+ qsock = ngx_quic_get_socket(c); -+ + ngx_memzero(pkt, sizeof(ngx_quic_header_t)); + + pkt->flags = NGX_QUIC_PKT_FIXED_BIT; @@ -8612,8 +8571,7 @@ new file mode 100644 + pkt->dcid.data = path->cid->id; + pkt->dcid.len = path->cid->len; + -+ pkt->scid.data = qsock->sid.id; -+ pkt->scid.len = qsock->sid.len; ++ pkt->scid = qc->tp.initial_scid; + + pkt->version = qc->version; + pkt->log = c->log; @@ -8632,7 +8590,7 @@ new file mode 100644 + ssize_t n; + struct iovec iov; + struct msghdr msg; -+#if defined(NGX_HAVE_ADDRINFO_CMSG) ++#if (NGX_HAVE_ADDRINFO_CMSG) + struct cmsghdr *cmsg; + char msg_control[CMSG_SPACE(sizeof(ngx_addrinfo_t))]; +#endif @@ -8648,7 +8606,7 @@ new file mode 100644 + msg.msg_name = sockaddr; + msg.msg_namelen = socklen; + -+#if defined(NGX_HAVE_ADDRINFO_CMSG) ++#if (NGX_HAVE_ADDRINFO_CMSG) + if (c->listening && c->listening->wildcard && c->local_sockaddr) { + + msg.msg_control = msg_control; @@ -8863,8 +8821,7 @@ new file mode 100644 + return NGX_ERROR; + } + -+ if (ngx_quic_keys_set_initial_secret(c->pool, pkt.keys, &inpkt->dcid, -+ inpkt->version) ++ if (ngx_quic_keys_set_initial_secret(c->pool, pkt.keys, &inpkt->dcid) + != NGX_OK) + { + return NGX_ERROR; @@ -8897,7 +8854,7 @@ new file mode 100644 + return NGX_ERROR; + } + -+ return NGX_OK; ++ return NGX_DONE; +} + + @@ -9233,7 +9190,7 @@ diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_eve new file mode 100644 --- /dev/null +++ b/src/event/quic/ngx_event_quic_protection.c -@@ -0,0 +1,1186 @@ +@@ -0,0 +1,1177 @@ + +/* + * Copyright (C) Nginx, Inc. @@ -9382,7 +9339,7 @@ new file mode 100644 + +ngx_int_t +ngx_quic_keys_set_initial_secret(ngx_pool_t *pool, ngx_quic_keys_t *keys, -+ ngx_str_t *secret, uint32_t version) ++ ngx_str_t *secret) +{ + size_t is_len; + uint8_t is[SHA256_DIGEST_LENGTH]; @@ -9393,9 +9350,6 @@ new file mode 100644 + static const uint8_t salt[20] = + "\x38\x76\x2c\xf7\xf5\x59\x34\xb3\x4d\x17" + "\x9a\xe6\xa4\xc8\x0c\xad\xcc\xbb\x7f\x0a"; -+ static const uint8_t salt29[20] = -+ "\xaf\xbf\xec\x28\x99\x93\xd2\x4c\x9e\x97" -+ "\x86\xf1\x9c\x61\x11\xe0\x43\x90\xa8\x99"; + + client = &keys->secrets[ssl_encryption_initial].client; + server = &keys->secrets[ssl_encryption_initial].server; @@ -9411,7 +9365,7 @@ new file mode 100644 + is_len = SHA256_DIGEST_LENGTH; + + if (ngx_hkdf_extract(is, &is_len, digest, secret->data, secret->len, -+ (version & 0xff000000) ? salt29 : salt, sizeof(salt)) ++ salt, sizeof(salt)) + != NGX_OK) + { + return NGX_ERROR; @@ -10128,12 +10082,8 @@ new file mode 100644 + /* 5.8. Retry Packet Integrity */ + static u_char key[16] = + "\xbe\x0c\x69\x0b\x9f\x66\x57\x5a\x1d\x76\x6b\x54\xe3\x68\xc8\x4e"; -+ static u_char key29[16] = -+ "\xcc\xce\x18\x7e\xd0\x9a\x09\xd0\x57\x28\x15\x5a\x6c\xb9\x6b\xe1"; + static u_char nonce[NGX_QUIC_IV_LEN] = + "\x46\x15\x99\xd3\x5d\x63\x2b\xf2\x23\x98\x25\xbb"; -+ static u_char nonce29[NGX_QUIC_IV_LEN] = -+ "\xe5\x49\x30\xf9\x7f\x21\x36\xf0\x53\x0a\x8c\x1c"; + static ngx_str_t in = ngx_string(""); + + ad.data = res->data; @@ -10152,12 +10102,10 @@ new file mode 100644 + } + + secret.key.len = sizeof(key); -+ secret.key.data = (pkt->version & 0xff000000) ? key29 : key; ++ secret.key.data = key; + secret.iv.len = NGX_QUIC_IV_LEN; + -+ if (ngx_quic_tls_seal(ciphers.c, &secret, &itag, -+ (pkt->version & 0xff000000) ? nonce29 : nonce, -+ &in, &ad, pkt->log) ++ if (ngx_quic_tls_seal(ciphers.c, &secret, &itag, nonce, &in, &ad, pkt->log) + != NGX_OK) + { + return NGX_ERROR; @@ -10446,7 +10394,7 @@ new file mode 100644 + +ngx_quic_keys_t *ngx_quic_keys_new(ngx_pool_t *pool); +ngx_int_t ngx_quic_keys_set_initial_secret(ngx_pool_t *pool, -+ ngx_quic_keys_t *keys, ngx_str_t *secret, uint32_t version); ++ ngx_quic_keys_t *keys, ngx_str_t *secret); +ngx_int_t ngx_quic_keys_set_encryption_secret(ngx_pool_t *pool, + ngx_uint_t is_write, ngx_quic_keys_t *keys, + enum ssl_encryption_level_t level, const SSL_CIPHER *cipher, @@ -10629,7 +10577,7 @@ new file mode 100644 + qc->nsockets--; + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic socket #%L closed nsock:%ui", ++ "quic socket seq:%L closed nsock:%ui", + (int64_t) qsock->sid.seqnum, qc->nsockets); +} + @@ -10654,7 +10602,7 @@ new file mode 100644 + qsock->quic = qc; + + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, -+ "quic socket #%L listening at sid:%xV nsock:%ui", ++ "quic socket seq:%L listening at sid:%xV nsock:%ui", + (int64_t) sid->seqnum, &id, qc->nsockets); + + return NGX_OK; @@ -11307,8 +11255,8 @@ new file mode 100644 + } +#endif + -+#if BORINGSSL_API_VERSION >= 13 -+ SSL_set_quic_use_legacy_codepoint(ssl_conn, qc->version != 1); ++#if (BORINGSSL_API_VERSION >= 13 && BORINGSSL_API_VERSION < 15) ++ SSL_set_quic_use_legacy_codepoint(ssl_conn, 0); +#endif + + qsock = ngx_quic_get_socket(c); @@ -11384,7 +11332,7 @@ diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_ new file mode 100644 --- /dev/null +++ b/src/event/quic/ngx_event_quic_streams.c -@@ -0,0 +1,1599 @@ +@@ -0,0 +1,1586 @@ + +/* + * Copyright (C) Nginx, Inc. @@ -11421,17 +11369,19 @@ new file mode 100644 +static ngx_int_t ngx_quic_update_flow(ngx_connection_t *c, uint64_t last); +static ngx_int_t ngx_quic_update_max_stream_data(ngx_connection_t *c); +static ngx_int_t ngx_quic_update_max_data(ngx_connection_t *c); ++static void ngx_quic_set_event(ngx_event_t *ev); + + +ngx_connection_t * +ngx_quic_open_stream(ngx_connection_t *c, ngx_uint_t bidi) +{ + uint64_t id; -+ ngx_quic_stream_t *qs, *nqs; ++ ngx_connection_t *pc; ++ ngx_quic_stream_t *nqs; + ngx_quic_connection_t *qc; + -+ qs = c->quic; -+ qc = ngx_quic_get_connection(qs->parent); ++ pc = c->quic ? c->quic->parent : c; ++ qc = ngx_quic_get_connection(pc); + + if (bidi) { + if (qc->streams.server_streams_bidi @@ -11477,7 +11427,7 @@ new file mode 100644 + qc->streams.server_streams_uni++; + } + -+ nqs = ngx_quic_create_stream(qs->parent, id); ++ nqs = ngx_quic_create_stream(pc, id); + if (nqs == NULL) { + return NULL; + } @@ -11542,7 +11492,6 @@ new file mode 100644 +{ + ngx_pool_t *pool; + ngx_queue_t *q; -+ ngx_event_t *rev, *wev; + ngx_rbtree_t *tree; + ngx_rbtree_node_t *node; + ngx_quic_stream_t *qs; @@ -11578,19 +11527,11 @@ new file mode 100644 + { + qs = (ngx_quic_stream_t *) node; + -+ rev = qs->connection->read; -+ rev->error = 1; -+ rev->ready = 1; -+ -+ wev = qs->connection->write; -+ wev->error = 1; -+ wev->ready = 1; ++ qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD; ++ qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT; *** 947 LINES SKIPPED ***