From nobody Mon Nov 16 21:00:35 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jT0d004HMz70wSr for ; Mon, 16 Nov 2026 21:00:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jT0cz1MDvz4CN5 for ; Mon, 16 Nov 2026 21:00:35 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1794862835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ts2PcjEEfu1gnIXSAscwApR1pOaccX10crrl9EysQSE=; b=ti0EXpoqBLbnZ+tByyA0c9nAqQ2WJa+/DtHUeMGXhOEW0PHn/9rLI/6h9p4Q/gCCmVrn5G 5Z1NgoEEwkw/rbs134g+DjCz8IsGTy9EmSXNZwvENv8xUInqE+2ft6yltah/C6j7F5xekM S+8lXcA4f6GZ11JYwikwLUih/lWdv4pg3jKDxux++t3aieZs1mECkCJcgId7FOmRHI8bUH l/nnMczJZiaqXbpOWF8ecqGEAZmTOfgjcCAGYsmEIl6Oj/qZHrDkZloBvTl6G/Z7pZ6HAX T8ct3LsAboMg/ZZUT9T98n7G6j4pyNxzoi6FrzZP5hCE18UpyfTawgcTGma5Rw== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1794862835; b=h0LRfIGjjk2akQnpN79l7OdDrJS8bAyfsDO5YfPonyYCSKyO7sPbKk8UwU6WFi5sEY4CGj GXvoxkEvcD569PHG+rwNxB6PkV57yFCWohXWlwD8DrsiHOMX2QHZNsPyDm1aGByl7Drzf2 UFK+WYl7lIVcdIGyTdgGltAt9vonw3vm1/oOyo2bun9BclM0OIDj30zjuCykcWeXte1dZX yDO/3kVsSmRki48285KMaztFiqtkvwx5os0N068bcD+jl9fk2qBvE1YlSxLnnHR+Gw1whw 1goWY3oUZ68cU5C9q7suF5ic9E1BVr1W/n5V6+1zC/3om+DNEARcC3ZldobHpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1794862835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ts2PcjEEfu1gnIXSAscwApR1pOaccX10crrl9EysQSE=; b=Dmn9oCaUlktFoAM8DifOVwZqgCmu4QJE3FE8Iy3L+lxUNYkK3oDOcaUyfc8OydPhnUm7Wv 9FrdivogrIkqyAaUPgHXzexbDdKN8pL1bUXwdB1/pDWVuSbae80VEw0Wc3K9mUuC59MwdF KOuBFVtErCNFAeYq6c7z61l5gmq1Ww/mZoAeTiz2YflfLx9qDC6hT6k9/4CH3QnjxPKIAy G4cUcU+//fKUmbXD6L14uypPPIJzgRCC+MfRJ7mwdCZVZ68K2kEwTKE0cykUt8ejCkonCi nguofttcEGs+LvtfqBfRALX3wQMuLf/Ia9olx9F+bupNcBsMUCEcLSS6j1pIkg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jT0cz0xJsz19XP for ; Mon, 16 Nov 2026 21:00:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 8f8f by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 16 Nov 2026 21:00:35 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kenneth D. Merry Subject: git: 89c1a891dbb2 - stable/15 - camcontrol: Support NVMe reservations via the persist subcommand List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ken X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 89c1a891dbb2fb62a06b0083c0378641194f57f7 Auto-Submitted: auto-generated Date: Mon, 16 Nov 2026 21:00:35 +0000 Message-Id: <6afb6ef3.8f8f.3ad040a0@gitrepo.freebsd.org> The branch stable/15 has been updated by ken: URL: https://cgit.FreeBSD.org/src/commit/?id=89c1a891dbb2fb62a06b0083c0378641194f57f7 commit 89c1a891dbb2fb62a06b0083c0378641194f57f7 Author: Kenneth D. Merry AuthorDate: 2026-11-07 01:05:55 +0000 Commit: Kenneth D. Merry CommitDate: 2026-11-16 16:12:24 +0000 camcontrol: Support NVMe reservations via the persist subcommand The NVMe reservation model is closely patterned after SCSI Persistent Reservations, so support NVMe devices through the existing persist subcommand rather than adding a transport-specific subcommand. The persist subcommand now dispatches on the device protocol, in the same way the identify subcommand does. The SCSI syntax is unchanged. For NVMe devices, the -i modes all map to the Reservation Report command (read_full_status requests the extended data structure, which contains 128-bit Host Identifiers), and the -o service actions map to the corresponding NVMe reservation operations: register, register_ignore, reserve, preempt, preempt_abort, release and clear, plus two NVMe-only actions, unregister and replace. The reservation type names for -T are shared between SCSI and NVMe. A new -c option gives explicit control over the NVMe Persist Through Power Loss State; -p is equivalent to -c enable. SCSI-only options return an error for NVMe devices. Tested against a PASCARI X200 series SSD (reservation capabilities: WR_EX, PTPL, IEKEY13): register, reserve, release, replace, unregister, -c enable/disable (Persist Through Power Loss State change verified in the report output), reservation report, and the error paths. register_ignore is rejected by drives that implement NVMe 1.3 IEKEY semantics, as expected. SCSI persist against a da(4) device is unchanged. sbin/camcontrol/nvres.c: New file, the NVMe reservation backend for the persist subcommand. sbin/camcontrol/camcontrol.c: Dispatch the persist subcommand to scsipersist() or nvmepersist() based on the device protocol. sbin/camcontrol/camcontrol.h: Add the nvmepersist() prototype. sbin/camcontrol/Makefile: Add nvres.c to the build. sbin/camcontrol/camcontrol.8: Document NVMe support in the persist subcommand section, and note the differences from SCSI where they occur. Co-authored-by: Reid Linnemann Co-authored-by: Claude Fable 5 Reviewed by: imp MFC after: 1 week Sponsored by: Spectra Logic Differential Revision: https://reviews.freebsd.org/D59534 (cherry picked from commit ba75cf05727e7f3130e6e6884f4dc7085b7aec3a) --- sbin/camcontrol/Makefile | 2 +- sbin/camcontrol/camcontrol.8 | 114 +++++++- sbin/camcontrol/camcontrol.c | 32 +- sbin/camcontrol/camcontrol.h | 3 + sbin/camcontrol/nvres.c | 674 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 808 insertions(+), 17 deletions(-) diff --git a/sbin/camcontrol/Makefile b/sbin/camcontrol/Makefile index 2b0c2c9dbc14..28817436f887 100644 --- a/sbin/camcontrol/Makefile +++ b/sbin/camcontrol/Makefile @@ -18,7 +18,7 @@ CFLAGS+= -I${SRCTOP}/sbin/nvmecontrol SRCS+= identify_ext.c SRCS+= nc_util.c .PATH: ${SRCTOP}/sys/dev/nvme -SRCS+= nvme_util.c +SRCS+= nvme_util.c nvres.c # This is verboten .if ${MACHINE_CPUARCH} == "arm" WARNS?= 3 diff --git a/sbin/camcontrol/camcontrol.8 b/sbin/camcontrol/camcontrol.8 index 0a6d69fada5b..4efb44cb0b21 100644 --- a/sbin/camcontrol/camcontrol.8 +++ b/sbin/camcontrol/camcontrol.8 @@ -312,6 +312,7 @@ .Op generic args .Aq Fl i Ar action | Fl o Ar action .Op Fl a +.Op Fl c Ar state .Op Fl I Ar trans_id .Op Fl k Ar key .Op Fl K Ar sa_key @@ -1801,12 +1802,23 @@ register values that would be used for the firmware download command. Do not ask for confirmation. .El .It Ic persist -Persistent reservation support. +Persistent reservation support for +.Tn SCSI +and +.Tn NVMe +devices. Persistent reservations are a way to reserve a particular .Tn SCSI -LUN for use by one or more +LUN or +.Tn NVMe +namespace for use by one or more initiators (hosts). +The +.Tn NVMe +reservation model is closely patterned after .Tn SCSI -initiators. +Persistent Reservations, so the two share this interface; +.Nm +issues the commands appropriate for the device given. If the .Fl i option is specified, @@ -1814,7 +1826,9 @@ option is specified, will issue the .Tn SCSI PERSISTENT RESERVE IN -command using the requested service action. +command using the requested service action, or the +.Tn NVMe +Reservation Report command. If the .Fl o option is specified, @@ -1822,18 +1836,25 @@ option is specified, will issue the .Tn SCSI PERSISTENT RESERVE OUT -command using the requested service action. +command using the requested service action, or the corresponding +.Tn NVMe +reservation command. One of those two options is required. .Pp Persistent reservations are complex, and fully explaining them is outside the scope of this manual. Please visit https://www.t10.org -and download the latest SPC spec for a full explanation of persistent -reservations. +and download the latest SPC spec, or visit +https://www.nvmexpress.org +and download the latest NVMe Base Specification, for a full explanation of +persistent reservations. .Bl -tag -width 8n .It Fl i Ar mode Specify the service action for the PERSISTENT RESERVE IN command. +For +.Tn NVMe +devices, all of the supported modes issue the Reservation Report command. Supported service actions: .Bl -tag -width 19n .It read_keys @@ -1843,11 +1864,24 @@ registered keys. Report the persistent reservation, if any. .It report_capabilities Report the persistent reservation capabilities of the LUN. +Not supported for +.Tn NVMe +devices; the reservation capabilities of a namespace are in the Identify +Namespace data. .It read_full_status Report the full status of persistent reservations on the LUN. +For +.Tn NVMe +devices, this requests the extended data structure, which contains 128-bit +Host Identifiers. .El .It Fl o Ar mode Specify the service action for the PERSISTENT RESERVE OUT command. +For +.Tn NVMe +devices, each service action maps to the corresponding +.Tn NVMe +reservation operation. For service actions like register that are components of other service action names, the entire name must be specified. Otherwise, enough of the service action name must be specified to @@ -1867,6 +1901,19 @@ This is similar to the register subcommand, except that the Reservation Key is ignored. The Service Action Reservation Key will overwrite any previous key registered for the initiator. +.It unregister +Unregister the key specified as the Reservation Key. +This action is only supported for +.Tn NVMe +devices; on +.Tn SCSI +devices a key is unregistered with the register action. +.It replace +Replace the key specified as the Reservation Key with the key specified as +the Service Action Reservation Key. +This action is only supported for +.Tn NVMe +devices. .It reserve Create a reservation. A key must be registered with the LUN before the LUN can be reserved, and @@ -1894,16 +1941,39 @@ operation being performed. Register another initiator with the LUN, and establish a reservation on the LUN for that initiator. The Reservation Key and Service Action Reservation Key must be specified. +Not supported for +.Tn NVMe +devices. .It replace_lost Replace Lost Reservation information. +Not supported for +.Tn NVMe +devices. .El .It Fl a Set the All Target Ports (ALL_TG_PT) bit. This requests that the key registration be applied to all target ports and not just the particular target port that receives the command. This only applies to the register and register_ignore actions. +Not supported for +.Tn NVMe +devices. +.It Fl c Ar state +Change the Persist Through Power Loss State for the namespace. +The state may be set to +.Dq enable +or +.Dq disable . +By default, no change is made to the current setting. +This option is only supported for +.Tn NVMe +devices, and only applies to the register, register_ignore, unregister and +replace actions. .It Fl I Ar tid Specify a Transport ID. +Not supported for +.Tn NVMe +devices. This only applies to the Register and Register and Move service actions for Persistent Reserve Out. Multiple Transport IDs may be specified with multiple @@ -1993,12 +2063,25 @@ Specify the Service Action Reservation Key. This may be in decimal, octal or hexadecimal format. The value is zero by default if not otherwise specified. The value must be between 0 and 2^64 - 1, inclusive. +For +.Tn NVMe +devices, this is the new reservation key for the register, register_ignore +and replace actions, and the key to preempt for the preempt and +preempt_abort actions. .It Fl p Enable the Activate Persist Through Power Loss bit. This is only used for the register and register_ignore actions. This requests that the reservation persist across power loss events. +For +.Tn NVMe +devices, this enables Persist Through Power Loss State, and is equivalent +to specifying +.Fl c Cm enable . .It Fl s Ar scope Specify the scope of the reservation. +Not supported for +.Tn NVMe +devices. The scope may be specified by name or by number. The scope is ignored for register, register_ignore and clear. If the desired scope isn't available by name, you may specify the number. @@ -2018,6 +2101,9 @@ Reserve Out command. .It Fl S Enable the SPEC_I_PT bit. This only applies to the Register service action of Persistent Reserve Out. +Not supported for +.Tn NVMe +devices. You must also specify at least one Transport ID with .Fl I if this option is set. @@ -2029,10 +2115,18 @@ Specify the reservation type. The reservation type may be specified by name or by number. If the desired reservation type isn't available by name, you may specify the number. +The same names are used for +.Tn SCSI +and +.Tn NVMe +devices. Supported reservation type names: .Bl -tag -width 11n .It read_shared Read Shared mode. +Not supported for +.Tn NVMe +devices. .It wr_ex Write Exclusive mode. May also be specified as @@ -2041,6 +2135,9 @@ May also be specified as Read Exclusive mode. May also be specified as .Dq read_exclusive . +Not supported for +.Tn NVMe +devices. .It ex_ac Exclusive access mode. May also be specified as @@ -2069,6 +2166,9 @@ By default, the target will not unregister the initiator that sends the Register and Move request. This option only applies to the Register and Move service action of the Persistent Reserve Out command. +Not supported for +.Tn NVMe +devices. .El .It Ic attrib Issue the diff --git a/sbin/camcontrol/camcontrol.c b/sbin/camcontrol/camcontrol.c index bec8756324e6..97210634648c 100644 --- a/sbin/camcontrol/camcontrol.c +++ b/sbin/camcontrol/camcontrol.c @@ -228,7 +228,7 @@ static struct camcontrol_opts option_table[] = { {"security", CAM_CMD_SECURITY, CAM_ARG_NONE, "d:e:fh:k:l:qs:T:U:y"}, {"hpa", CAM_CMD_HPA, CAM_ARG_NONE, "Pflp:qs:U:y"}, {"ama", CAM_CMD_AMA, CAM_ARG_NONE, "fqs:"}, - {"persist", CAM_CMD_PERSIST, CAM_ARG_NONE, "ai:I:k:K:o:ps:ST:U"}, + {"persist", CAM_CMD_PERSIST, CAM_ARG_NONE, "ac:i:I:k:K:o:ps:ST:U"}, {"attrib", CAM_CMD_ATTRIB, CAM_ARG_NONE, "a:ce:F:p:r:s:T:w:V:"}, {"opcodes", CAM_CMD_OPCODES, CAM_ARG_NONE, "No:s:T"}, {"zone", CAM_CMD_ZONE, CAM_ARG_NONE, "ac:l:No:P:"}, @@ -10144,8 +10144,8 @@ usage(int printlong) " [-q] [-s max_sectors] [-U pwd] [-y]\n" " camcontrol ama [dev_id][generic args] [-f] [-q] [-s max_sectors]\n" " camcontrol persist [dev_id][generic args] <-i action|-o action>\n" -" [-a][-I tid][-k key][-K sa_key][-p][-R rtp]\n" -" [-s scope][-S][-T type][-U]\n" +" [-a][-c state][-I tid][-k key][-K sa_key][-p]\n" +" [-R rtp][-s scope][-S][-T type][-U]\n" " camcontrol attrib [dev_id][generic args] <-r action|-w attr>\n" " [-a attr_num][-c][-e elem][-F form1,form1]\n" " [-p part][-s start][-T type][-V vol]\n" @@ -10207,7 +10207,7 @@ usage(int printlong) "powermode send the ATA CHECK POWER MODE command to the named device\n" "fwdownload program firmware of the named device with the given image\n" "security report or send ATA security commands to the named device\n" -"persist send the SCSI PERSISTENT RESERVE IN or OUT commands\n" +"persist send SCSI PERSISTENT RESERVE or NVMe reservation commands\n" "attrib send the SCSI READ or WRITE ATTRIBUTE commands\n" "opcodes send the SCSI REPORT SUPPORTED OPCODES command\n" "zone manage Zoned Block (Shingled) devices\n" @@ -10829,12 +10829,26 @@ main(int argc, char **argv) error = sanitize(cam_dev, argc, argv, combinedopt, task_attr, retry_count, timeout); break; - case CAM_CMD_PERSIST: - error = scsipersist(cam_dev, argc, argv, combinedopt, - task_attr, retry_count, timeout, - arglist & CAM_ARG_VERBOSE, - arglist & CAM_ARG_ERR_RECOVER); + case CAM_CMD_PERSIST: { + struct ccb_pathinq cpi; + + if (get_cpi(cam_dev, &cpi) != 0) { + warnx("couldn't get CPI"); + error = 1; + break; + } + if (cpi.protocol == PROTO_NVME) + error = nvmepersist(cam_dev, argc, argv, combinedopt, + retry_count, timeout, + arglist & CAM_ARG_VERBOSE, + arglist & CAM_ARG_ERR_RECOVER); + else + error = scsipersist(cam_dev, argc, argv, combinedopt, + task_attr, retry_count, timeout, + arglist & CAM_ARG_VERBOSE, + arglist & CAM_ARG_ERR_RECOVER); break; + } case CAM_CMD_ATTRIB: error = scsiattrib(cam_dev, argc, argv, combinedopt, task_attr, retry_count, timeout, diff --git a/sbin/camcontrol/camcontrol.h b/sbin/camcontrol/camcontrol.h index 6417fea97bfd..637f7c3b191a 100644 --- a/sbin/camcontrol/camcontrol.h +++ b/sbin/camcontrol/camcontrol.h @@ -77,6 +77,9 @@ int get_ata_status(struct cam_device *dev, union ccb *ccb, uint8_t *error, uint8_t *status); int nvme_get_data(struct cam_device *dev, int nvme_data_type, void *data, size_t data_len); +int nvmepersist(struct cam_device *device, int argc, char **argv, + char *combinedopt, int retry_count, int timeout, + int verbosemode, int err_recover); int camxferrate(struct cam_device *device); int fwdownload(struct cam_device *device, int argc, char **argv, char *combinedopt, int printerrors, int task_attr, diff --git a/sbin/camcontrol/nvres.c b/sbin/camcontrol/nvres.c new file mode 100644 index 000000000000..dc36954f1fe9 --- /dev/null +++ b/sbin/camcontrol/nvres.c @@ -0,0 +1,674 @@ +/*- + * Copyright (c) 2020-2021 Spectra Logic Corporation + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification. + * 2. Redistributions in binary form must reproduce at minimum a disclaimer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for further + * binary redistribution. + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES. + * + * Authors: Ken Merry (Spectra Logic Corporation) + * Reid Linnemann (Spectra Logic Corporation) + */ +/* + * NVMe Reservation support for camcontrol(8). + */ + +#include +#include +#include +#include +#include +#include + + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include "camcontrol.h" + +/* + * PERSISTENT RESERVE OUT style action names, translated to the NVMe + * reservation opcode, the action field within that command, and whether + * to set the Ignore Existing Key bit. The names match the camcontrol + * persist -o actions where SCSI and NVMe have equivalent operations; + * unregister and replace are NVMe-only. + */ +struct nvres_out_action { + uint8_t opc; /* NVMe reservation opcode */ + int action; /* action field within the command */ + int iekey; /* set Ignore Existing Key */ +}; + +static struct nvres_out_action nvres_out_actions[] = { + { NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_REGISTER, 0 }, + { NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_REGISTER, 1 }, + { NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_UNREGISTER, 0 }, + { NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_REPLACE, 0 }, + { NVME_OPC_RESERVATION_ACQUIRE, NVME_RESV_ACQUIRE_ACQUIRE, 0 }, + { NVME_OPC_RESERVATION_ACQUIRE, NVME_RESV_ACQUIRE_PREEMPT, 0 }, + { NVME_OPC_RESERVATION_ACQUIRE, NVME_RESV_ACQUIRE_PREEMPT_ABT, 0 }, + { NVME_OPC_RESERVATION_RELEASE, NVME_RESV_REL_RELEASE, 0 }, + { NVME_OPC_RESERVATION_RELEASE, NVME_RESV_REL_CLEAR, 0 } +}; + +/* Values are indices into nvres_out_actions[]. */ +static struct scsi_nv nvres_out_action_names[] = { + { "register", 0 }, + { "register_ignore", 1 }, + { "unregister", 2 }, + { "replace", 3 }, + { "reserve", 4 }, + { "preempt", 5 }, + { "preempt_abort", 6 }, + { "release", 7 }, + { "clear", 8 } +}; + +/* + * PERSISTENT RESERVE IN style mode names. All of them map to the NVMe + * Reservation Report command; read_full_status additionally requests the + * extended data structure (128-bit host IDs). Values are the eds flag. + */ +static struct scsi_nv nvres_in_modes[] = { + { "read_keys", 0 }, + { "read_reservation", 0 }, + { "read_full_status", 1 } +}; + +/* Change Persist Through Power Loss State values for -c. */ +static struct scsi_nv nvres_cptpl_vals[] = { + { "disable", NVME_RESV_CPTPL_0 }, + { "enable", NVME_RESV_CPTPL_1 } +}; + +static struct scsi_nv nvres_res_types[] = { + { "write_exclusive", NVME_RESV_TYPE_WR_EX }, + { "wr_ex", NVME_RESV_TYPE_WR_EX }, + { "exclusive_access", NVME_RESV_TYPE_EX_AC }, + { "ex_ac", NVME_RESV_TYPE_EX_AC }, + { "write_exclusive_reg_only", NVME_RESV_TYPE_WR_EX_RO }, + { "wr_ex_ro", NVME_RESV_TYPE_WR_EX_RO }, + { "exclusive_access_reg_only", NVME_RESV_TYPE_EX_AC_RO }, + { "ex_ac_ro", NVME_RESV_TYPE_EX_AC_RO }, + { "write_exclusive_all_regs", NVME_RESV_TYPE_WR_EX_AR }, + { "wr_ex_ar", NVME_RESV_TYPE_WR_EX_AR }, + { "exclusive_access_all_regs", NVME_RESV_TYPE_EX_AC_AR }, + { "ex_ac_ar", NVME_RESV_TYPE_EX_AC_AR } +}; + +static int +nvmeres_acquire(struct cam_device *device, int verbosemode, int err_recover, + int rtype, int iekey, int racqa, uint64_t crkey, uint64_t prkey, + int retry_count, int timeout) +{ + union ccb *ccb = NULL; + uint64_t data[2]; + int error = 0; + + ccb = cam_getccb(device); + if (ccb == NULL) { + warnx("%s: error allocating CCB", __func__); + error = 1; + goto bailout; + } + CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio); + + data[0] = htole64(crkey); + data[1] = htole64(prkey); + + nvme_resv_acquire(/*nvmeio*/ &ccb->nvmeio, + /*retries*/ retry_count, + /*cbfcnp*/ NULL, + /*nsid*/ device->target_lun, + /*rtype*/ rtype, + /*iekey*/ iekey, + /*rrega*/ racqa, + /*data_ptr*/ (uint8_t *)data, + /*dxfer_len*/ sizeof(data), + /*timeout*/ timeout ? timeout : 30000); + + if (err_recover != 0) + ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER; + + if (cam_send_ccb(device, ccb) < 0) { + warn("error sending NVMe Reservation Acquire"); + error = 1; + goto bailout; + } + + if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) { + warnx("NVMe Reservation Acquire failed"); + if (verbosemode != 0) { + cam_error_print(device, ccb, CAM_ESF_ALL, CAM_EPF_ALL, + stderr); + } + error = 1; + goto bailout; + } + +bailout: + if (ccb != NULL) { + cam_freeccb(ccb); + } + return (error); +} + +static int +nvmeres_register(struct cam_device *device, int verbosemode, int err_recover, + int cptpl, int iekey, int rrega, uint64_t curkey, + uint64_t newkey, int retry_count, int timeout) +{ + union ccb *ccb = NULL; + uint64_t data[2]; + int error = 0; + + ccb = cam_getccb(device); + if (ccb == NULL) { + warnx("%s: error allocating CCB", __func__); + error = 1; + goto bailout; + } + CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio); + + data[0] = htole64(curkey); + data[1] = htole64(newkey); + + nvme_resv_register(/*nvmeio*/ &ccb->nvmeio, + /*retries*/ retry_count, + /*cbfcnp*/ NULL, + /*nsid*/ device->target_lun, + /*cptpl*/ cptpl, + /*iekey*/ iekey, + /*rrega*/ rrega, + /*data_ptr*/ (uint8_t*)data, + /*dxfer_len*/ sizeof(data), + /*timeout*/ timeout ? timeout: 30000); + + if (err_recover != 0) + ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER; + + if (cam_send_ccb(device, ccb) < 0) { + warn("error sending NVMe Reservation Registration"); + error = 1; + goto bailout; + } + + if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) { + warnx("NVMe Reservation Register failed"); + if (verbosemode != 0) { + cam_error_print(device, ccb, CAM_ESF_ALL, + CAM_EPF_ALL, stderr); + } + error = 1; + goto bailout; + } + + +bailout: + if (ccb != NULL) { + cam_freeccb(ccb); + } + return (error); +} + +static int +nvmeres_release(struct cam_device *device, int verbosemode, int err_recover, + int rtype, int iekey, int rrela, uint64_t curkey, + int retry_count, int timeout) +{ + union ccb *ccb = NULL; + uint64_t data[1]; + int error = 0; + + ccb = cam_getccb(device); + if (ccb == NULL) { + warnx("%s: error allocating CCB", __func__); + error = 1; + goto bailout; + } + CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio); + + data[0] = htole64(curkey); + + nvme_resv_release(/*nvmeio*/ &ccb->nvmeio, + /*retries*/ retry_count, + /*cbfcnp*/ NULL, + /*nsid*/ device->target_lun, + /*rtype*/ rtype, + /*iekey*/ iekey, + /*rrela*/ rrela, + /*data_ptr*/ (uint8_t*)data, + /*dxfer_len*/ sizeof(data), + /*timeout*/ timeout ? timeout: 30000); + + if (err_recover != 0) + ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER; + + if (cam_send_ccb(device, ccb) < 0) { + warn("error sending NVMe Reservation Release"); + error = 1; + goto bailout; + } + + if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) { + warnx("NVMe Reservation Release failed"); + if (verbosemode != 0) { + cam_error_print(device, ccb, CAM_ESF_ALL, + CAM_EPF_ALL, stderr); + } + error = 1; + goto bailout; + } + + +bailout: + if (ccb != NULL) { + cam_freeccb(ccb); + } + return (error); +} + +static int +nvmeres_report(struct cam_device *device, int verbosemode, int err_recover, + int eds, int retry_count, int timeout) +{ + uint8_t report_data[8192]; + union ccb *ccb = NULL; + struct nvme_resv_status *rstatus; + struct nvme_resv_status_ext *extstatus; + struct sbuf *hostid_sbuf = NULL; + const char *rtype_str = NULL; + int num_ents = 0; + int error = 0; + int i; + + ccb = cam_getccb(device); + if (ccb == NULL) { + warnx("%s: error allocating CCB", __func__); + error = 1; + goto bailout; + } + CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio); + + bzero(report_data, sizeof(report_data)); + + nvme_resv_report(/*nvmeio*/ &ccb->nvmeio, + /*retries*/ retry_count, + /*cbfcnp*/ NULL, + /*nsid*/ device->target_lun, + /*report_type*/ eds, + /*data_ptr*/ report_data, + /*dxfer_len */ sizeof(report_data), + /*timeout*/ timeout ? timeout : 30000); + + if (err_recover != 0) + ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER; + + if (cam_send_ccb(device, ccb) < 0) { + warn("error sending NVMe Reservation Report"); + error = 1; + goto bailout; + } + + if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) { + warnx("NVMe Reservation Report failed"); + if (verbosemode != 0) { + cam_error_print(device, ccb, CAM_ESF_ALL, + CAM_EPF_ALL, stderr); + } + error = 1; + goto bailout; + } + + hostid_sbuf = sbuf_new(NULL, NULL, 64, SBUF_FIXEDLEN); + if (hostid_sbuf == NULL) { + warn("error allocating sbuf"); + error = 1; + goto bailout; + } + rstatus = (struct nvme_resv_status *)report_data; + /* + * XXX KDM evidently there is no concept of a residual in NVMe. + */ + if (eds == 0) + nvme_resv_status_swapbytes(rstatus,sizeof(report_data)); + else + nvme_resv_status_ext_swapbytes((struct nvme_resv_status_ext *) + report_data,sizeof(report_data)); + + num_ents = (rstatus->regctl[1] << 8) | rstatus->regctl[0]; + printf("Generation: %u\n", rstatus->gen); + printf("Reservation type: %u", rstatus->rtype); + if (rstatus->rtype > 0) { + rtype_str = scsi_nv_to_str(nvres_res_types, + sizeof(nvres_res_types) / sizeof(nvres_res_types[0]), + rstatus->rtype); + printf(" (%s)", rtype_str?rtype_str:"invalid"); + } + printf("\n"); + printf("Number of Registered Controllers: %u\n", num_ents); + printf("Persist Through Power Loss State: %u\n", rstatus->ptpls); + + /* + * The number of entries comes from the device. Constrain it to + * the number that actually fit in our data buffer, in case the + * device reports more. + */ + if (eds == 1) { + extstatus = (struct nvme_resv_status_ext *)report_data; + if (num_ents > (int)((sizeof(report_data) - + sizeof(*extstatus)) / sizeof(extstatus->ctrlr[0]))) + num_ents = (sizeof(report_data) - + sizeof(*extstatus)) / sizeof(extstatus->ctrlr[0]); + for (i = 0; i < num_ents; i++) { + sbuf_clear(hostid_sbuf); + nvme_hostid_sbuf(hostid_sbuf, + (uint8_t*)(extstatus->ctrlr[i].hostid), 1); + sbuf_finish(hostid_sbuf); + printf("Controller ID: 0x%04x\n", + extstatus->ctrlr[i].ctrlr_id); + printf("Reservation Status: %u\n", + extstatus->ctrlr[i].rcsts); + printf("Host Identifier: %s\n", sbuf_data(hostid_sbuf)); + printf("Reservation Key: 0x%016jx\n", + (uintmax_t)extstatus->ctrlr[i].rkey); + } + } else { + if (num_ents > (int)((sizeof(report_data) - + sizeof(*rstatus)) / sizeof(rstatus->ctrlr[0]))) + num_ents = (sizeof(report_data) - + sizeof(*rstatus)) / sizeof(rstatus->ctrlr[0]); + for (i = 0; i < num_ents; i++) { + sbuf_clear(hostid_sbuf); + nvme_hostid_sbuf(hostid_sbuf, + (uint8_t*)&(rstatus->ctrlr[i].hostid), 0); + sbuf_finish(hostid_sbuf); + printf("Controller ID: 0x%04x\n", + rstatus->ctrlr[i].ctrlr_id); + printf("Reservation Status: %u\n", + rstatus->ctrlr[i].rcsts); + printf("Host Identifier: %s\n", sbuf_data(hostid_sbuf)); + printf("Reservation Key: 0x%016jx\n", + (uintmax_t)rstatus->ctrlr[i].rkey); + } + } +bailout: + if (ccb != NULL) { + cam_freeccb(ccb); + } + if (hostid_sbuf != NULL) { + sbuf_delete(hostid_sbuf); + } + return (error); +} + +int +nvmepersist(struct cam_device *device, int argc, char **argv, char *combinedopt, + int retry_count, int timeout, int verbosemode, int err_recover) +{ + uint64_t curkey = 0, other_key = 0; + int eds = 0, cptpl = 0, iekey = 0; + int error = 0; + int in = 0, out = 0; + int opc = -1, op_action = -1, res_type = 0; + int c; + + while ((c = getopt(argc, argv, combinedopt)) != -1) { + switch (c) { + case 'i': { + scsi_nv_status status; + int table_entry = 0; + + status = scsi_get_nv(nvres_in_modes, + sizeof(nvres_in_modes) / sizeof(nvres_in_modes[0]), + optarg, &table_entry, SCSI_NV_FLAG_IG_CASE); + if (status == SCSI_NV_FOUND) { + in = 1; + opc = NVME_OPC_RESERVATION_REPORT; + eds = nvres_in_modes[table_entry].value; + } else { + warnx("%s: %s in mode %s for NVMe devices", + __func__, (status == SCSI_NV_AMBIGUOUS) ? + "ambiguous" : "invalid", optarg); + error = 1; + goto bailout; + } + break; + } + case 'o': { + scsi_nv_status status; + struct nvres_out_action *oa; + int table_entry = 0; + + status = scsi_get_nv(nvres_out_action_names, + sizeof(nvres_out_action_names) / + sizeof(nvres_out_action_names[0]), + optarg, &table_entry, SCSI_NV_FLAG_IG_CASE); + if (status == SCSI_NV_FOUND) { + out = 1; + oa = &nvres_out_actions[ + nvres_out_action_names[table_entry].value]; + opc = oa->opc; + op_action = oa->action; + iekey = oa->iekey; + } else { + warnx("%s: %s out action %s for NVMe devices", + __func__, (status == SCSI_NV_AMBIGUOUS) ? + "ambiguous" : "invalid", optarg); + error = 1; + goto bailout; + } + break; + } + case 'k': + case 'K': { + char *endptr; + uint64_t tmpval; + + tmpval = strtoumax(optarg, &endptr, 0); + if (*endptr != '\0') { + warnx("%s: invalid key argument %s", __func__, + optarg); + error = 1; + goto bailout; + } + if (c == 'k') + curkey = tmpval; + else + other_key = tmpval; + break; + } + case 'c': { + scsi_nv_status status; + int table_entry = 0; + + status = scsi_get_nv(nvres_cptpl_vals, + sizeof(nvres_cptpl_vals) / + sizeof(nvres_cptpl_vals[0]), + optarg, &table_entry, SCSI_NV_FLAG_IG_CASE); + if (status == SCSI_NV_FOUND) + cptpl = nvres_cptpl_vals[table_entry].value; + else { + warnx("%s: %s persist through power loss " + "value %s", __func__, + (status == SCSI_NV_AMBIGUOUS) ? + "ambiguous" : "invalid", optarg); + error = 1; + goto bailout; + } + break; + } + case 'p': + /* + * SCSI APTPL is approximately NVMe Change Persist + * Through Power Loss State, enabled. The default + * is to make no change to the current setting; + * -c gives explicit control. + */ + cptpl = NVME_RESV_CPTPL_1; + break; + case 'T': { + scsi_nv_status status; + int table_entry = 0; + + status = scsi_get_nv(nvres_res_types, + sizeof(nvres_res_types) /sizeof(nvres_res_types[0]), + optarg, &table_entry, SCSI_NV_FLAG_IG_CASE); + if (status == SCSI_NV_FOUND) + res_type = nvres_res_types[table_entry].value; + else { + warnx("%s: %s reservation type %s", __func__, + (status == SCSI_NV_AMBIGUOUS) ? + "ambiguous" : "invalid", optarg); + error = 1; + goto bailout; + } + + break; + } + case 'a': + case 'I': + case 's': + case 'S': + case 'U': + warnx("%s: option -%c is not supported for NVMe " + "devices", __func__, c); + error = 1; + goto bailout; + } + } + if ((in + out) != 1) { + warnx("%s: you must specify one of -i or -o", __func__); + error = 1; + goto bailout; *** 95 LINES SKIPPED ***