From nobody Fri Oct 29 23:58:30 2021 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2899E181D8D7; Fri, 29 Oct 2021 23:58:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HgzsQ2gS5z4Vh7; Fri, 29 Oct 2021 23:58:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4DA7B205B0; Fri, 29 Oct 2021 23:58:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 19TNwUvN003852; Fri, 29 Oct 2021 23:58:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 19TNwUY7003851; Fri, 29 Oct 2021 23:58:30 GMT (envelope-from git) Date: Fri, 29 Oct 2021 23:58:30 GMT Message-Id: <202110292358.19TNwUY7003851@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: e508e93f36f4 - stable/13 - iSCSI: Add support for segmentation offload for hardware offloads. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e508e93f36f4a1c7506e927407943001089a28c3 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=e508e93f36f4a1c7506e927407943001089a28c3 commit e508e93f36f4a1c7506e927407943001089a28c3 Author: John Baldwin AuthorDate: 2021-08-06 21:03:00 +0000 Commit: John Baldwin CommitDate: 2021-10-29 23:20:35 +0000 iSCSI: Add support for segmentation offload for hardware offloads. Similar to TSO, iSCSI segmentation offload permits the upper layers to submit a "large" virtual PDU which is split up into multiple segments (PDUs) on the wire. Similar to how the TCP/IP headers are used as templates for TSO, the BHS at the start of a large PDU is used as a template to construct the specific BHS at the start of each PDU. In particular, the DataSN is incremented for each subsequent PDU, and the 'F' flag is only set on the last PDU. struct icl_conn has a new 'ic_hw_isomax' field which defaults to 0, but can be set to the largest virtual PDU a backend supports. If this value is non-zero, the iSCSI target and initiator use this size instead of 'ic_max_send_data_segment_length' to determine the maximum size for SCSI Data-In and SCSI Data-Out PDUs. Note that since PDUs can be constructed from multiple buffers before being dispatched, the target and initiator must wait for the PDU to be fully constructed before determining the number of DataSN values were consumed (and thus updating the per-transfer DataSN value used for the start of the next PDU). The target generates large PDUs for SCSI Data-In PDUs in cfiscsi_datamove_in(). The initiator generates large PDUs for SCSI Data-Out PDUs generated in response to an R2T. Reviewed by: mav Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D31222 (cherry picked from commit f0594f52f6fdabecee134dd5700bf936283959ad) --- sys/cam/ctl/ctl_frontend_iscsi.c | 29 +++++++++++++++++------------ sys/dev/iscsi/icl.h | 1 + sys/dev/iscsi/iscsi.c | 15 +++++++++++---- 3 files changed, 29 insertions(+), 16 deletions(-) diff --git a/sys/cam/ctl/ctl_frontend_iscsi.c b/sys/cam/ctl/ctl_frontend_iscsi.c index b3cd8ab79d76..b8fafcea69ed 100644 --- a/sys/cam/ctl/ctl_frontend_iscsi.c +++ b/sys/cam/ctl/ctl_frontend_iscsi.c @@ -2447,6 +2447,7 @@ cfiscsi_datamove_in(union ctl_io *io) struct iscsi_bhs_data_in *bhsdi; struct ctl_sg_entry ctl_sg_entry, *ctl_sglist; size_t len, expected_len, sg_len, buffer_offset; + size_t max_send_data_segment_length; const char *sg_addr; icl_pdu_cb cb; int ctl_sg_count, error, i; @@ -2504,6 +2505,11 @@ cfiscsi_datamove_in(union ctl_io *io) sg_len = 0; response = NULL; bhsdi = NULL; + if (cs->cs_conn->ic_hw_isomax != 0) + max_send_data_segment_length = cs->cs_conn->ic_hw_isomax; + else + max_send_data_segment_length = + cs->cs_conn->ic_max_send_data_segment_length; for (;;) { if (response == NULL) { response = cfiscsi_pdu_new_response(request, M_NOWAIT); @@ -2520,7 +2526,7 @@ cfiscsi_datamove_in(union ctl_io *io) bhsdi->bhsdi_initiator_task_tag = bhssc->bhssc_initiator_task_tag; bhsdi->bhsdi_target_transfer_tag = 0xffffffff; - bhsdi->bhsdi_datasn = htonl(PRIV_EXPDATASN(io)++); + bhsdi->bhsdi_datasn = htonl(PRIV_EXPDATASN(io)); bhsdi->bhsdi_buffer_offset = htonl(buffer_offset); } @@ -2536,15 +2542,11 @@ cfiscsi_datamove_in(union ctl_io *io) /* * Truncate to maximum data segment length. */ - KASSERT(response->ip_data_len < - cs->cs_conn->ic_max_send_data_segment_length, - ("ip_data_len %zd >= max_send_data_segment_length %d", - response->ip_data_len, - cs->cs_conn->ic_max_send_data_segment_length)); - if (response->ip_data_len + len > - cs->cs_conn->ic_max_send_data_segment_length) { - len = cs->cs_conn->ic_max_send_data_segment_length - - response->ip_data_len; + KASSERT(response->ip_data_len < max_send_data_segment_length, + ("ip_data_len %zd >= max_send_data_segment_length %zd", + response->ip_data_len, max_send_data_segment_length)); + if (response->ip_data_len + len > max_send_data_segment_length) { + len = max_send_data_segment_length - response->ip_data_len; KASSERT(len <= sg_len, ("len %zd > sg_len %zd", len, sg_len)); } @@ -2603,8 +2605,7 @@ cfiscsi_datamove_in(union ctl_io *io) i++; } - if (response->ip_data_len == - cs->cs_conn->ic_max_send_data_segment_length) { + if (response->ip_data_len == max_send_data_segment_length) { /* * Can't stuff more data into the current PDU; * queue it. Note that's not enough to check @@ -2619,6 +2620,8 @@ cfiscsi_datamove_in(union ctl_io *io) buffer_offset -= response->ip_data_len; break; } + PRIV_EXPDATASN(io) += howmany(response->ip_data_len, + cs->cs_conn->ic_max_send_data_segment_length); if (cb != NULL) { response->ip_prv0 = io->scsiio.kern_data_ref; response->ip_prv1 = io->scsiio.kern_data_arg; @@ -2654,6 +2657,8 @@ cfiscsi_datamove_in(union ctl_io *io) } } KASSERT(response->ip_data_len > 0, ("sending empty Data-In")); + PRIV_EXPDATASN(io) += howmany(response->ip_data_len, + cs->cs_conn->ic_max_send_data_segment_length); if (cb != NULL) { response->ip_prv0 = io->scsiio.kern_data_ref; response->ip_prv1 = io->scsiio.kern_data_arg; diff --git a/sys/dev/iscsi/icl.h b/sys/dev/iscsi/icl.h index adcd580f299b..bd12569a8957 100644 --- a/sys/dev/iscsi/icl.h +++ b/sys/dev/iscsi/icl.h @@ -93,6 +93,7 @@ struct icl_conn { #endif uint32_t ic_max_recv_data_segment_length; uint32_t ic_max_send_data_segment_length; + size_t ic_hw_isomax; size_t ic_maxtags; bool ic_header_crc32c; bool ic_data_crc32c; diff --git a/sys/dev/iscsi/iscsi.c b/sys/dev/iscsi/iscsi.c index 7ddb5a9ce1ec..f60904bb3de0 100644 --- a/sys/dev/iscsi/iscsi.c +++ b/sys/dev/iscsi/iscsi.c @@ -1154,7 +1154,7 @@ iscsi_pdu_handle_r2t(struct icl_pdu *response) struct iscsi_bhs_data_out *bhsdo; struct iscsi_outstanding *io; struct ccb_scsiio *csio; - size_t off, len, total_len; + size_t off, len, max_send_data_segment_length, total_len; int error; uint32_t datasn = 0; @@ -1203,11 +1203,16 @@ iscsi_pdu_handle_r2t(struct icl_pdu *response) //ISCSI_SESSION_DEBUG(is, "r2t; off %zd, len %zd", off, total_len); + if (is->is_conn->ic_hw_isomax != 0) + max_send_data_segment_length = is->is_conn->ic_hw_isomax; + else + max_send_data_segment_length = + is->is_conn->ic_max_send_data_segment_length; for (;;) { len = total_len; - if (len > is->is_conn->ic_max_send_data_segment_length) - len = is->is_conn->ic_max_send_data_segment_length; + if (len > max_send_data_segment_length) + len = max_send_data_segment_length; if (off + len > csio->dxfer_len) { ISCSI_SESSION_WARN(is, "target requested invalid " @@ -1232,7 +1237,7 @@ iscsi_pdu_handle_r2t(struct icl_pdu *response) bhsr2t->bhsr2t_initiator_task_tag; bhsdo->bhsdo_target_transfer_tag = bhsr2t->bhsr2t_target_transfer_tag; - bhsdo->bhsdo_datasn = htonl(datasn++); + bhsdo->bhsdo_datasn = htonl(datasn); bhsdo->bhsdo_buffer_offset = htonl(off); error = icl_pdu_append_data(request, csio->data_ptr + off, len, M_NOWAIT); @@ -1245,6 +1250,8 @@ iscsi_pdu_handle_r2t(struct icl_pdu *response) return; } + datasn += howmany(len, + is->is_conn->ic_max_send_data_segment_length); off += len; total_len -= len;