Re: sym vs ncr driver for emulated LSI 53C895A
- In reply to: Miroslav Lachman : "Re: sym vs ncr driver for emulated LSI 53C895A"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Sat, 18 Feb 2023 18:15:47 UTC
Am 15.02.23 um 20:26 schrieb Miroslav Lachman: > On 10/02/2023 22:22, Stefan Esser wrote: >> Am 09.02.23 um 20:04 schrieb Miroslav Lachman: >>> I have FreeBSD 12.4 virtual machine installed inside KVM. This machine has 2 >>> disks. One is 30GB connected as VirtIO vtbd0. There is installed system. The >>> second is 20TB iSCSI connected to KVM and should be available inside VM as >>> SCSI disk da0 but it is not. >> >> This is an emulated SCSI controller in KVM instead of a physical >> controller on a PCI bus? > > As far as I know it is emulated by KVM. I search the net and found this for > example: > https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines#qm_hard_disk_bus > > "the SCSI controller, designed in 1985, is commonly found on server grade > hardware, and can connect up to 14 storage devices. Proxmox VE emulates by > default a LSI 53C895A controller. The issue with this emulation is that the 53c8xx family of devices is not an "intelligent" controller that gets a command written into some register and then executes the transfer, instead it is a very simple CPU that executes a limited and domain specific command set. The early NCR 53x8xx devices did not contain any indirect address modes, you had to store the address of a calculated memory access into the next instruction to fetch from a variable address (i.e. it depended on self modifying code). And if KVM emulates the NCR 53c895 (which btw. has indirect addressing modes), then any trivial operation requires the emulation of hundreds of simple NCR instructions, at a cost of thousands or tens of thousands CPU instructions. > A SCSI controller of type VirtIO SCSI is the recommended setting if you aim for > performance and is automatically selected for newly created Linux VMs since > Proxmox VE 4.3. Linux distributions have support for this controller since > 2012, and FreeBSD since 2014." If possible use VirtIO! These "virtual" devices are implemented in a way that makes them very efficient. The parameters of a request and the data to be transfered are just transfered to the hypervisor via shared memory areas, much like a system call into the kernel. >> The device emulation may be incomplete and may therefore violate >> assumptions made in the driver. The 53c8xx devices were complex >> and the driver contains a "firmware" blob that has to be executed >> using main CPU instructions by the emulated Symbios device. This >> is extremely inefficient compared to other emulstions (it takes >> hundreds of emulated controller instructions to execute trivial >> SCSI transfers). >> >> Why can't you use the iSCSI driver to connect to an iscsi device? > > Because my VM does not have access to network where the iSCSI target is (for > security reason) But VirtIO would be offered? >> Regarding the history of the ncr and sym drivers: >> >> I'm the co-author of the ncr driver, which originally supported only >> the NCR53c810 chip. Later additions of this device family added WIDE >> SCSI support and support for faster synchronous transfers, and their >> support was added to the ncr driver. > > [..] > >> The ncr driver has been removed from later FreeBSD releases, since sym >> covers all devices (since it is an extension of the ncr driver). > > Oh, thank you for the clarification, I thought ncr is newer than sym. So > building ncr into kernel does not have any sense. No, definitely not. The NCR driver used only the initial command set, without the indirect memory access modes offered by later NCR chips. >> The sym driver attached correctly, it should work. It has been more than >> 10 years since I last used the driver and had a machine with SCSI drives. >> >> In the meantime Marius Strobl made a few changes to the driver, but I do >> not know, whether he still has access to a system with that controller. >> >> The error messages: >> >>> (probe1:sym0:0:1:0): INQUIRY. CDB: 12 00 00 00 24 00 >>> (probe1:sym0:0:1:0): CAM status: Command timeout >>> (probe1:sym0:0:1:0): Retrying command, 3 more tries remain >>> sym0:1: message c sent on bad reselection. >>> sym0:1:control msgout: 80 6. >> >> The device should execute the INQUIRY command, but not reply was received >> (the device did not reselect the controlled). Several attempts are made, >> but the retries are exausted and the device is ignored. >> >> It seems that the selection/reselection and sending of SCSI messages is >> not perfectly emulated. >> >> You may want to contact the author of the 53c895a emulation, since you'll >> need debug output from that emulation in order to understand what's wrong. > > The administrator of KVM hypervisor changed some settings on the translation > layer iSCSI / LSI emul. / VirtIO. Even if he told me I do not understand it > well but now I see the disk correctly: > > # diskinfo -v da0 > da0 > 512 # sectorsize > 23747947921408 # mediasize in bytes (22T) > 46382710784 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > 2887190 # Cylinders according to firmware. > 255 # Heads according to firmware. > 63 # Sectors according to firmware. > QNAP iSCSI Storage # Disk descr. > 194d2dd6-683a-42e1-9985-acd5580b119d # Disk ident. > vtscsi0 # Attachment > No # TRIM/UNMAP support > Unknown # Rotation rate in RPM > Not_Zoned # Zone Mode > > > # pciconf -l -v > > sym0@pci0:0:7:0: class=0x010000 rev=0x00 hdr=0x00 vendor=0x1000 > device=0x0012 subvendor=0x0000 subdevice=0x1000 > vendor = 'Broadcom / LSI' > device = '53c895a' > class = mass storage > subclass = SCSI > virtio_pci2@pci0:0:8:0: class=0x010000 rev=0x00 hdr=0x00 vendor=0x1af4 > device=0x1004 subvendor=0x1af4 subdevice=0x0008 > vendor = 'Red Hat, Inc.' > device = 'Virtio SCSI' > class = mass storage > subclass = SCSI Yes, that's the device you want to use! > virtio_pci3@pci0:0:9:0: class=0x010000 rev=0x00 hdr=0x00 vendor=0x1af4 > device=0x1001 subvendor=0x1af4 subdevice=0x0002 > vendor = 'Red Hat, Inc.' > device = 'Virtio block device' > class = mass storage > subclass = SCSI > > virtio_pci0: <VirtIO PCI (legacy) Block adapter> port 0xc100-0xc17f mem > 0xfc0b5000-0xfc0b5fff, 0xfebf0000-0xfebf3fff irq 10 at device 5.0 on pci0 > vtblk0: <VirtIO Block Adapter> on virtio_pci0 > vtblk0: 30720MB (62914560 512 byte sectors) > virtio_pci1: <VirtIO PCI (legacy) Balloon adapter> port 0xc240-0xc27f mem > 0xfebf4000-0xfebf7fff irq 10 at device 6.0 on pci0 > vtballoon0: <VirtIO Balloon Adapter> on virtio_pci1 > sym0: <895a> port 0xc000-0xc0ff mem 0xfc0b6000-0xfc0b63ff,0xfc0b2000-0xfc0b3fff > irq 11 at device 7.0 on pci0 > sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking > virtio_pci2: <VirtIO PCI (legacy) SCSI adapter> port 0xc280-0xc2bf mem > 0xfc0b7000-0xfc0b7fff,0xfebf8000-0xfebfbfff irq 11 at device 8.0 on pci0 > vtscsi0: <VirtIO SCSI Adapter> on virtio_pci2 > virtio_pci3: <VirtIO PCI (legacy) Block adapter> port 0xc180-0xc1ff mem > 0xfc0b8000-0xfc0b8fff,0xfebfc000-0xfebfffff irq 10 at device 9.0 on pci0 > vtblk1: <VirtIO Block Adapter> on virtio_pci3 > vtblk1: 102400MB (209715200 512 byte sectors) > > da0 at vtscsi0 bus 0 scbus3 target 0 lun 0 > da0: <QNAP iSCSI Storage 4.0> Fixed Direct Access SPC-3 SCSI device > da0: Serial Number 194d2dd6-683a-42e1-9985-acd5580b119d > da0: 300.000MB/s transfers > da0: Command Queueing enabled > da0: 22647808MB (46382710784 512 byte sectors) > > So I assume the iSCSI device is now connected as VirtIO SCSI device on vtscsi0 > / virtio_pci2. Yes, "da0 at vtscsi0" is exactly what you wanted ... You can remove the sym driver from your kernel or disable it. It will only consume (some) memory, but is of no use in your VM. > The important thing is that it works! And it does not only work this way - this is the most efficient way to connect to a device offered by the hypervisor. > I greatly appreciated your insightful reply and of course your work on ncr / sym. Thanks, and I'm glad my knowledge about this long obsolete (for lack of physical PCI slots in current systems, and due to the limit of at most 40 MB/s of the fastest parallel SCSI transfer mode) was still of use ... Regards, STefan