From nobody Sat Dec 03 15:38:20 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NPYqc5dNlz4jg40; Sat, 3 Dec 2022 15:38:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NPYqc56nNz3pW5; Sat, 3 Dec 2022 15:38:20 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670081900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ha9zQx5I/hjt5dyr/Po+q4XBKc3ES2hhINSJ9zhdqbg=; b=YsG9mgIG/J5S7QorZIu8zLme5AV8tpvZBqip+2ZQ/CmVvPDh+uQbR4nHOOokKp3tqxB22o 9tiZFzBOuxzJsxvoQN/AqxVSnCmFklt1NZbyWIoeAqhIp/qXjmj2OvXZXZh2JmKZvrMxsS TUHY4lhIVgiGKx0aYSY7bJbQOa3f6+ehH/Wu6kuYyW2I7mIJsn0aTGKObEd9f/qZ8nqHx2 H6ROaDdQAgcjoVllWPqvR96mZuUWfuz5nmiHSHQh/09mkIKGbx7doQijndUFfhP8rfTiWM 5G5hnl7/syZ3I3ovWCg2ewSXMArjlPNxecH0S3VThyB8Rp4DxuCJLSjBtip/TQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670081900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ha9zQx5I/hjt5dyr/Po+q4XBKc3ES2hhINSJ9zhdqbg=; b=LF2vnsbhB8SCHdb01VUIVVAXdOedkz3qEbzLLZELJm0ScYjsR7Wxzrn3kkdRNhkEheQ158 hOzPzbEJuhHSzJ4Tv2N/ErNbP+1AvEBaS0VR4XhgGaiUEJZk6KTXOUkzLRN506hx0IFUWo DgHwf4ktKlPTKgO+BdrmjQLROsNDl7JyJw9sHczm/6ZlE+Egq5OImApUnm6z7+I+I1gaxN KYSd0DSPLzHexg5EXRO9wQdm3ENUtX0J4NpJL/hyf8T2GTNckFKPIrrT3Lg2qtgwR8+6x6 u+G+8tKQKHHMXtsdqfBG9onw+22JHDZbDTpoFLOXc5XXj5sfkcyPjczw122Oog== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1670081900; a=rsa-sha256; cv=none; b=PPSpGs8paafy1Ggx6W7egy5hEIaVzZmQq+vuc+Jk+05rR+Oy99RBVNDhZYoI0PV9TBLsCv Gnc2m+psAA9Yda/ZVU87cTydm6C4IHDlMxSNd7RIP3761amKJ9bfu2komnCLdvCQgTM+JU HEdraESnlbxxEUq+Zu1jt9rhY/+q5riDrgTKS+unGrDHEwgCBvyCK28fgcJvNUspRJ35V0 T9xsBKqfZPSXNbxeDWeLcMD4x03xzvbglG5yearYVdveNTRFHthbAbUfTNu63ixC96OdOc 8NzgTYmcT8W7R8qxJrZnhEH/HwSWAdMBE+AYglRP2oSk4nTH0cprdr4WqUBEEg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NPYqc43gQz10Nt; Sat, 3 Dec 2022 15:38:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2B3FcKeX065515; Sat, 3 Dec 2022 15:38:20 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2B3FcKlQ065514; Sat, 3 Dec 2022 15:38:20 GMT (envelope-from git) Date: Sat, 3 Dec 2022 15:38:20 GMT Message-Id: <202212031538.2B3FcKlQ065514@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alexander Motin Subject: git: 0acc026dda9e - main - CTL: Increase maximum SCSI tag size from 32 to 64 bits. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0acc026dda9e7405f7a88993d1b51e5fc846cbc5 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=0acc026dda9e7405f7a88993d1b51e5fc846cbc5 commit 0acc026dda9e7405f7a88993d1b51e5fc846cbc5 Author: Alexander Motin AuthorDate: 2022-12-03 15:23:29 +0000 Commit: Alexander Motin CommitDate: 2022-12-03 15:23:29 +0000 CTL: Increase maximum SCSI tag size from 32 to 64 bits. SAM-5 specification states maximum size of command identifier (tag), defined by specific transports, should not be larger than 64 bits. While most of supported transports use 32 bits or less, it was reported that virtio-scsi uses 64 bits. Truncation to 32 bits in bhyve code caused false tag conflict errors reported and possibly other issues. This changes CTL ABI and HA protocol, so CTL_HA_VERSION is bumped. While we make HA protocol incompatible, increase default maximum number of ports in CTL from 256 to 1024, matching number of LUNs. There are many reports from people who need many iSCSI targets with only one LUN each. Increased memory consumption should be less of a problem these days. PR: 267539 --- share/man/man4/ctl.4 | 2 +- sys/cam/ctl/ctl.c | 13 +++++++------ sys/cam/ctl/ctl_frontend_ioctl.c | 4 ++-- sys/cam/ctl/ctl_io.h | 11 +++++------ sys/cam/ctl/ctl_ioctl.h | 3 ++- sys/cam/ctl/ctl_tpc_local.c | 2 +- sys/cam/ctl/ctl_util.c | 6 +++--- sys/cam/ctl/scsi_ctl.c | 2 +- usr.sbin/bhyve/pci_virtio_scsi.c | 4 ++-- 9 files changed, 24 insertions(+), 23 deletions(-) diff --git a/share/man/man4/ctl.4 b/share/man/man4/ctl.4 index e150d70665d2..18005ae3c7a0 100644 --- a/share/man/man4/ctl.4 +++ b/share/man/man4/ctl.4 @@ -201,7 +201,7 @@ Specifies the maximum number of LUNs we support, must be a power of 2. The default value is 1024. .It Va kern.cam.ctl.max_ports Specifies the maximum number of ports we support, must be a power of 2. -The default value is 256. +The default value is 1024. .El .Sh SEE ALSO .Xr cfiscsi 4 , diff --git a/sys/cam/ctl/ctl.c b/sys/cam/ctl/ctl.c index 07ebade1e44a..a9551c875f7a 100644 --- a/sys/cam/ctl/ctl.c +++ b/sys/cam/ctl/ctl.c @@ -435,7 +435,7 @@ SYSCTL_INT(_kern_cam_ctl, OID_AUTO, max_luns, CTLFLAG_RDTUN, /* * Maximum number of ports registered at one time. */ -#define CTL_DEFAULT_MAX_PORTS 256 +#define CTL_DEFAULT_MAX_PORTS 1024 static int ctl_max_ports = CTL_DEFAULT_MAX_PORTS; TUNABLE_INT("kern.cam.ctl.max_ports", &ctl_max_ports); SYSCTL_INT(_kern_cam_ctl, OID_AUTO, max_ports, CTLFLAG_RDTUN, @@ -2443,7 +2443,7 @@ ctl_serialize_other_sc_cmd(struct ctl_scsiio *ctsio) case CTL_ACTION_OVERLAP_TAG: LIST_REMOVE(&ctsio->io_hdr, ooa_links); mtx_unlock(&lun->lun_lock); - ctl_set_overlapped_tag(ctsio, ctsio->tag_num); + ctl_set_overlapped_tag(ctsio, ctsio->tag_num & 0xff); badjuju: ctl_copy_sense_data_back((union ctl_io *)ctsio, &msg_info); msg_info.hdr.original_sc = ctsio->io_hdr.remote_io; @@ -2491,6 +2491,7 @@ ctl_ioctl_fill_ooa(struct ctl_lun *lun, uint32_t *cur_fill_num, entry = &kern_entries[*cur_fill_num]; entry->tag_num = io->scsiio.tag_num; + entry->tag_type = io->scsiio.tag_type; entry->lun_num = lun->lun; #ifdef CTL_TIME_IO entry->start_bt = io->io_hdr.start_bt; @@ -12594,7 +12595,7 @@ ctl_datamove(union ctl_io *io) * the data move. */ if (io->io_hdr.flags & CTL_FLAG_ABORT) { - printf("ctl_datamove: tag 0x%04x on (%u:%u:%u) aborted\n", + printf("ctl_datamove: tag 0x%jx on (%u:%u:%u) aborted\n", io->scsiio.tag_num, io->io_hdr.nexus.initid, io->io_hdr.nexus.targ_port, io->io_hdr.nexus.targ_lun); @@ -12996,7 +12997,7 @@ ctl_datamove_remote(union ctl_io *io) * have been done if need be on the other controller. */ if (io->io_hdr.flags & CTL_FLAG_ABORT) { - printf("%s: tag 0x%04x on (%u:%u:%u) aborted\n", __func__, + printf("%s: tag 0x%jx on (%u:%u:%u) aborted\n", __func__, io->scsiio.tag_num, io->io_hdr.nexus.initid, io->io_hdr.nexus.targ_port, io->io_hdr.nexus.targ_lun); @@ -13042,12 +13043,12 @@ ctl_process_done(union ctl_io *io) ctl_scsi_command_string(&io->scsiio, NULL, &sb); sbuf_printf(&sb, "\n"); sbuf_cat(&sb, path_str); - sbuf_printf(&sb, "Tag: 0x%04x/%d, Prio: %d\n", + sbuf_printf(&sb, "Tag: 0x%jx/%d, Prio: %d\n", io->scsiio.tag_num, io->scsiio.tag_type, io->scsiio.priority); break; case CTL_IO_TASK: - sbuf_printf(&sb, "Task Action: %d Tag: 0x%04x/%d\n", + sbuf_printf(&sb, "Task Action: %d Tag: 0x%jx/%d\n", io->taskio.task_action, io->taskio.tag_num, io->taskio.tag_type); break; diff --git a/sys/cam/ctl/ctl_frontend_ioctl.c b/sys/cam/ctl/ctl_frontend_ioctl.c index f326100cb013..f53dae694867 100644 --- a/sys/cam/ctl/ctl_frontend_ioctl.c +++ b/sys/cam/ctl/ctl_frontend_ioctl.c @@ -75,7 +75,7 @@ struct ctl_fe_ioctl_params { struct cfi_port { TAILQ_ENTRY(cfi_port) link; - uint32_t cur_tag_num; + u_int cur_tag_num; struct cdev * dev; struct ctl_port port; }; @@ -634,7 +634,7 @@ ctl_ioctl_io(struct cdev *dev, u_long cmd, caddr_t addr, int flag, io->io_hdr.flags |= CTL_FLAG_USER_REQ; if ((io->io_hdr.io_type == CTL_IO_SCSI) && (io->scsiio.tag_type != CTL_TAG_UNTAGGED)) - io->scsiio.tag_num = cfi->cur_tag_num++; + io->scsiio.tag_num = atomic_fetchadd_int(&cfi->cur_tag_num, 1); retval = cfi_submit_wait(io); if (retval == 0) diff --git a/sys/cam/ctl/ctl_io.h b/sys/cam/ctl/ctl_io.h index de312608da8e..9f22898e3ee5 100644 --- a/sys/cam/ctl/ctl_io.h +++ b/sys/cam/ctl/ctl_io.h @@ -328,8 +328,7 @@ struct ctl_scsiio { uint8_t scsi_status; /* SCSI status byte */ uint8_t seridx; /* Serialization index. */ uint8_t priority; /* Command priority */ - uint32_t residual; /* Unused */ - uint32_t tag_num; /* tag number */ + uint64_t tag_num; /* tag number */ ctl_tag_type tag_type; /* simple, ordered, head of queue,etc.*/ uint8_t cdb_len; /* CDB length */ uint8_t cdb[CTL_MAX_CDBLEN]; /* CDB */ @@ -373,7 +372,7 @@ typedef enum { struct ctl_taskio { struct ctl_io_hdr io_hdr; /* common to all I/O types */ ctl_task_type task_action; /* Target Reset, Abort, etc. */ - uint32_t tag_num; /* tag number */ + uint64_t tag_num; /* tag number */ ctl_tag_type tag_type; /* simple, ordered, etc. */ uint8_t task_status; /* Complete, Succeeded, etc. */ uint8_t task_resp[3];/* Response information */ @@ -382,7 +381,7 @@ struct ctl_taskio { /* * HA link messages. */ -#define CTL_HA_VERSION 3 +#define CTL_HA_VERSION 4 /* * Used for CTL_MSG_LOGIN. @@ -483,7 +482,7 @@ struct ctl_ha_msg_dt { */ struct ctl_ha_msg_scsi { struct ctl_ha_msg_hdr hdr; - uint32_t tag_num; /* tag number */ + uint64_t tag_num; /* tag number */ ctl_tag_type tag_type; /* simple, ordered, etc. */ uint8_t cdb[CTL_MAX_CDBLEN]; /* CDB */ uint8_t cdb_len; /* CDB length */ @@ -502,7 +501,7 @@ struct ctl_ha_msg_scsi { struct ctl_ha_msg_task { struct ctl_ha_msg_hdr hdr; ctl_task_type task_action; /* Target Reset, Abort, etc. */ - uint32_t tag_num; /* tag number */ + uint64_t tag_num; /* tag number */ ctl_tag_type tag_type; /* simple, ordered, etc. */ }; diff --git a/sys/cam/ctl/ctl_ioctl.h b/sys/cam/ctl/ctl_ioctl.h index 543f97a65d30..a5d237f2b65e 100644 --- a/sys/cam/ctl/ctl_ioctl.h +++ b/sys/cam/ctl/ctl_ioctl.h @@ -256,7 +256,8 @@ struct ctl_ooa_entry { ctl_ooa_cmd_flags cmd_flags; uint8_t cdb[CTL_MAX_CDBLEN]; uint8_t cdb_len; - uint32_t tag_num; + uint64_t tag_num; + ctl_tag_type tag_type; uint32_t lun_num; struct bintime start_bt; }; diff --git a/sys/cam/ctl/ctl_tpc_local.c b/sys/cam/ctl/ctl_tpc_local.c index ba6deada86a3..e661cde27234 100644 --- a/sys/cam/ctl/ctl_tpc_local.c +++ b/sys/cam/ctl/ctl_tpc_local.c @@ -61,7 +61,7 @@ __FBSDID("$FreeBSD$"); struct tpcl_softc { struct ctl_port port; - int cur_tag_num; + u_int cur_tag_num; }; static struct tpcl_softc tpcl_softc; diff --git a/sys/cam/ctl/ctl_util.c b/sys/cam/ctl/ctl_util.c index 245c3e89dfb9..0406388ce4bf 100644 --- a/sys/cam/ctl/ctl_util.c +++ b/sys/cam/ctl/ctl_util.c @@ -740,7 +740,7 @@ ctl_io_sbuf(union ctl_io *io, struct sbuf *sb) case CTL_IO_SCSI: sbuf_cat(sb, path_str); ctl_scsi_command_string(&io->scsiio, NULL, sb); - sbuf_printf(sb, " Tag: %#x/%d, Prio: %d\n", + sbuf_printf(sb, " Tag: %#jx/%d, Prio: %d\n", io->scsiio.tag_num, io->scsiio.tag_type, io->scsiio.priority); break; @@ -754,7 +754,7 @@ ctl_io_sbuf(union ctl_io *io, struct sbuf *sb) sbuf_printf(sb, "Task Action: %s", task_desc); switch (io->taskio.task_action) { case CTL_TASK_ABORT_TASK: - sbuf_printf(sb, " Tag: %#x/%d\n", + sbuf_printf(sb, " Tag: %#jx/%d\n", io->taskio.tag_num, io->taskio.tag_type); break; default: @@ -869,7 +869,7 @@ ctl_data_print(union ctl_io *io) for (i = 0; i < len; ) { sbuf_new(&sb, str, sizeof(str), SBUF_FIXEDLEN); sbuf_cat(&sb, path_str); - sbuf_printf(&sb, " %#6x:%04x:", io->scsiio.tag_num, i); + sbuf_printf(&sb, " %#jx:%04x:", io->scsiio.tag_num, i); for (j = 0; j < 16 && i < len; i++, j++) { if (j == 8) sbuf_cat(&sb, " "); diff --git a/sys/cam/ctl/scsi_ctl.c b/sys/cam/ctl/scsi_ctl.c index 8c141a88e155..b54ae89a53ca 100644 --- a/sys/cam/ctl/scsi_ctl.c +++ b/sys/cam/ctl/scsi_ctl.c @@ -1185,7 +1185,7 @@ ctlfedone(struct cam_periph *periph, union ccb *done_ccb) bcopy(atio_cdb_ptr(atio), io->scsiio.cdb, io->scsiio.cdb_len); #ifdef CTLFEDEBUG - printf("%s: %u:%u:%u: tag %04x CDB %02x\n", __func__, + printf("%s: %u:%u:%u: tag %jx CDB %02x\n", __func__, io->io_hdr.nexus.initid, io->io_hdr.nexus.targ_port, io->io_hdr.nexus.targ_lun, diff --git a/usr.sbin/bhyve/pci_virtio_scsi.c b/usr.sbin/bhyve/pci_virtio_scsi.c index bad95e4e85b0..20f08b149af8 100644 --- a/usr.sbin/bhyve/pci_virtio_scsi.c +++ b/usr.sbin/bhyve/pci_virtio_scsi.c @@ -405,7 +405,7 @@ pci_vtscsi_tmf_handle(struct pci_vtscsi_softc *sc, io->io_hdr.nexus.initid = sc->vss_iid; io->io_hdr.nexus.targ_lun = pci_vtscsi_get_lun(tmf->lun); io->taskio.tag_type = CTL_TAG_SIMPLE; - io->taskio.tag_num = (uint32_t)tmf->id; + io->taskio.tag_num = tmf->id; switch (tmf->subtype) { case VIRTIO_SCSI_T_TMF_ABORT_TASK: @@ -519,7 +519,7 @@ pci_vtscsi_request_handle(struct pci_vtscsi_queue *q, struct iovec *iov_in, } io->scsiio.sense_len = sc->vss_config.sense_size; - io->scsiio.tag_num = (uint32_t)cmd_rd->id; + io->scsiio.tag_num = cmd_rd->id; switch (cmd_rd->task_attr) { case VIRTIO_SCSI_S_ORDERED: io->scsiio.tag_type = CTL_TAG_ORDERED;