Resizing a zpool as a VMware ESXi guest ...
mgrooms at shrew.net
Tue May 10 20:46:31 UTC 2016
On 11/28/2015 10:03 PM, Matthew Grooms wrote:
> On 11/28/2015 6:10 PM, Matthew Grooms wrote:
>> On 11/27/2015 7:44 PM, Matthew Grooms wrote:
>>> I spent the day looking over the FreeBSD cam and scsi_da source code.
>>> After sprinkling a bunch of printf's around to see what code paths
>>> were being called, It's obvious that Edward was correct in assuming
>>> that ESXi doesn't return any 'Unit Attention' sense information in
>>> response to a 'Read Capacity' request. This kinda makes sense as ESXi
>>> emulates SCSI-2 disk devices and, as far as I can tell, the 0x2A/0x09
>>> ASC/ASCQ sense code that denotes 'Capacity Data Has Changed' wasn't
>>> defined until the SCSCI-3 spec. It's frustrating that the only way to
>>> get the scsci_da code to call reprobe() is by receiving a command
>>> from the device. Would something like this work? ...
>>> 1) Register a callback using xpt_register_async( daasync,
>>> AC_REPROBE_DEVICE, path ) that calls reprobe()
>>> 2) Implement a new IOCTL in cam_xpt that camcontrol can call with the
>>> bus:target:lun as the argument
>>> 3) have cam_xpt capture the IOCTL request and call xpt_async(
>>> AC_REPROBE_DEVICE, path ) as a result
>>> This way users would have the option of manually asking cam to
>>> communicate the new size to geom. The only option now is one or more
>>> reboots to gain access to the increased disk capacity. If this sounds
>>> like a reasonable approach, I'll take a stab at implementing it.
>> Here is a proof of concept patch. I'm a complete noob when it comes to
>> cam, scsi or freebsd kernel development for that matter, so I'm sure
>> it could have been done a better way. In any case, I added a new
>> command to camcontrol that allows you to specify a bus, target and lun
>> as an argument. For example ...
>> # camcontrol readcap da1 -h
>> Device Size: 32 G, Block Length: 512 bytes
>> # gpart show da1
>> => 40 58720176 da1 GPT (28G)
>> 40 58720176 1 freebsd-ufs (28G)
>> Note, I resized the VMDK disk in ESXi. The camcontrol output shows the
>> size as 32G but geom thinks its 28G.
>> # camcontrol devlist
>> <NECVMWar VMware IDE CDR10 1.00> at scbus1 target 0 lun 0 (cd0,pass0)
>> <VMware Virtual disk 1.0> at scbus2 target 0 lun 0 (pass1,da0)
>> <VMware Virtual disk 1.0> at scbus2 target 1 lun 0 (pass2,da1)
>> <FREEBSD CTLDISK 0001> at scbus3 target 0 lun 0 (da2,pass3)
>> # camcontrol reprobe 2:1:0
>> This generates an event that is captured by the scsci da device to
>> forces a reprobe. The kernel output looks almost identical to when the
>> 'Unit Attention' sense data is received ...
>> Nov 28 17:46:13 iscsi-i kernel: (da1:mpt0:0:1:0): Re-probe requested
>> Nov 28 17:46:13 iscsi-i kernel: GEOM_PART: da1 was automatically resized.
>> Nov 28 17:46:13 iscsi-i kernel: Use `gpart commit da1` to save changes
>> or `gpart undo da1` to revert them.
>> Now that geom knows about the increased disk capacity, I can increase
>> the partition size and grow the fs ...
>> [root at iscsi-i /home/mgrooms]# gpart show da1
>> => 40 67108784 da1 GPT (32G)
>> 40 58720176 1 freebsd-ufs (28G)
>> 58720216 8388608 - free - (4.0G)
>> # gpart resize -i 1 da1
>> da1p1 resized
>> # growfs da1p1
>> Device is mounted read-write; resizing will result in temporary write
>> suspension for /var/data1.
>> It's strongly recommended to make a backup before growing the file
>> OK to grow filesystem on /dev/da1p1, mounted on /var/data1, from 28GB
>> to 32GB? [Yes/No] Yes
>> super-block backups (for fsck_ffs -b #) at:
>> 58983232, 60265472, 61547712, 62829952, 64112192, 65394432, 66676672
>> # df -h
>> Filesystem Size Used Avail Capacity Mounted on
>> /dev/da0p3 18G 5.3G 12G 31% /
>> devfs 1.0K 1.0K 0B 100% /dev
>> /dev/da1p1 31G 32M 28G 0% /var/data1
>> /dev/da2p1 15G 32M 14G 0% /var/data2
>> Sure would be nice to have something like this in the tree. It's
>> really a drag to have to reboot production VMs to increase disk
>> capacity when it could be easily avoided. I'm not sure what the
>> correct IOCTL should look like. Maybe CAMIOCOMMAND is a better way to
>> go? If someone with some experience with the cam/scsi subsystems was
>> willing to give me some direction I'd be willing to try and rewrite
>> the patch in a way that would be commit worthy. I just need some
> Ok, last post until I get some feedback. Here's a new version of the
> patch complete with man page updates. It communicates via CAMIOCOMMAND
> instead of introducing a new ioctl value. I tried to model it after the
> device reset option, hopefully with some degree of success. Functionally
> it should be the same as the first patch.
The PR 204901 filed for this can be closed now that the author (ahem)
has committed support for the camcontrol reprobe command ...
More information about the freebsd-current