Re: sym vs ncr driver for emulated LSI 53C895A

Am 09.02.23 um 20:04 schrieb Miroslav Lachman:
> I have FreeBSD 12.4 virtual machine installed inside KVM. This machine has 2 
> disks. One is 30GB connected as VirtIO vtbd0. There is installed system. The 
> second is 20TB iSCSI connected to KVM and should be available inside VM as SCSI 
> disk da0 but it is not.

This is an emulated SCSI controller in KVM instead of a physical
controller on a PCI bus?

The device emulation may be incomplete and may therefore violate
assumptions made in the driver. The 53c8xx devices were complex
and the driver contains a "firmware" blob that has to be executed
using main CPU instructions by the emulated Symbios device. This
is extremely inefficient compared to other emulstions (it takes
hundreds of emulated controller instructions to execute trivial
SCSI transfers).

Why can't you use the iSCSI driver to connect to an iscsi device?


Regarding the history of the ncr and sym drivers:

I'm the co-author of the ncr driver, which originally supported only
the NCR53c810 chip. Later additions of this device family added WIDE
SCSI support and support for faster synchronous transfers, and their
support was added to the ncr driver.

Linux had after several years of development still no working driver
for that chip.

A Linux developer decided to port the FreeBSD ncr driver to Linux and
improved it by adding support for NCR/Symbios chips with an extended
command set. He preserved the FreeBSD specific code and this version
was imported into FreeBSD as the sym driver (since the NCR controller
business had been sold to Hyundai Electronics and named Symbios at
that time).

The Symbios driver got two sets of "Firmware", one with the limited
instruction set of the original device and one using the "advanced"
instructions of the later devices (53c825 and later, IIRC).

I haven't had a system with SCSI drives (or even a PCI bus) for some
time, and therefore can no longer test the driver.

> I see following errors in a log
> 
> sym0:1: message c sent on bad reselection.
> sym0:1:control msgout: 80 6.
> (probe1:sym0:0:1:0): INQUIRY. CDB: 12 00 00 00 24 00
> (probe1:sym0:0:1:0): CAM status: Command timeout
> (probe1:sym0:0:1:0): Retrying command, 3 more tries remain
> sym0:1: message c sent on bad reselection.
> sym0:1:control msgout: 80 6.
[...]
> (probe1:sym0:0:1:0): INQUIRY. CDB: 12 00 00 00 24 00
> (probe1:sym0:0:1:0): CAM status: Command timeout
> (probe1:sym0:0:1:0): Retrying command, 0 more tries remain
> sym0:1: message c sent on bad reselection.
> sym0:1:control msgout: 80 6.
> (probe1:sym0:0:1:0): INQUIRY. CDB: 12 00 00 00 24 00
> (probe1:sym0:0:1:0): CAM status: Command timeout
> (probe1:sym0:0:1:0): Error 5, Retries exhausted
> 
> Relevant part of the dmesg:
> 
> virtio_pci0: <VirtIO PCI Block adapter> port 0xc200-0xc27f mem 
> 0xfc0b7000-0xfc0b7fff,0xfebf8000-0xfebfbfff irq 10 at device 5.0 on pci0
> vtblk0: <VirtIO Block Adapter> on virtio_pci0
> vtblk0: 30720MB (62914560 512 byte sectors)
> virtio_pci1: <VirtIO PCI Balloon adapter> port 0xc2c0-0xc2ff mem 
> 0xfebfc000-0xfebfffff irq 10 at device 6.0 on pci0
> vtballoon0: <VirtIO Balloon Adapter> on virtio_pci1
> sym0: <895a> port 0xc000-0xc0ff mem 0xfc0b8000-0xfc0b83ff,0xfc0b2000-0xfc0b3fff 
> irq 11 at device 7.0 on pci0
> sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking
> sym1: <895a> port 0xc100-0xc1ff mem 0xfc0b9000-0xfc0b93ff,0xfc0b4000-0xfc0b5fff 
> irq 11 at device 8.0 on pci0
> sym1: No NVRAM, ID 7, Fast-40, LVD, parity checking
> ahci0: <Intel ICH9 AHCI SATA controller> port 0xc380-0xc39f mem 
> 0xfc0ba000-0xfc0bafff irq 10 at device 9.0 on pci0
[...]
> The controller is listed by pciconf
> 
> # pciconf -lv | grep -A4 sym
> sym0@pci0:0:7:0:        class=0x010000 card=0x10000000 chip=0x00121000 rev=0x00 
> hdr=0x00
>      vendor     = 'Broadcom / LSI'
>      device     = '53c895a'
>      class      = mass storage
>      subclass   = SCSI
> sym1@pci0:0:8:0:        class=0x010000 card=0x10000000 chip=0x00121000 rev=0x00 
> hdr=0x00
>      vendor     = 'Broadcom / LSI'
>      device     = '53c895a'
>      class      = mass storage
>      subclass   = SCSI
> 
> I tried to search what should be the right driver for LSI 53c895a but I am a 
> bit confused. sym(4) is attached on boot, but it is also listed on hardware 
> notes and man page for ncr(4) driver
> 
> https://www.freebsd.org/releases/12.4R/hardware/

The ncr driver has been removed from later FreeBSD releases, since sym
covers all devices (since it is an extension of the ncr driver).

> I tried to kldload ncr but can't load ncr: No such file or directory.
> 
> Should ncr be built separately? (the system is running 12.4 with GENERIC kernel)

No, ncr will not solve your problems and is not optimized for the 895a
chip you use.

> And how can I configure it to use ncr instead of sym?
> Or are there any other way to make FreeBSD guest happy in KVM host? (I am not 
> controlling the KVM host, I don't know the exact configuration of it)

The sym driver attached correctly, it should work. It has been more than
10 years since I last used the driver and had a machine with SCSI drives.

In the meantime Marius Strobl made a few changes to the driver, but I do
not know, whether he still has access to a system with that controller.

The error messages:

> (probe1:sym0:0:1:0): INQUIRY. CDB: 12 00 00 00 24 00
> (probe1:sym0:0:1:0): CAM status: Command timeout
> (probe1:sym0:0:1:0): Retrying command, 3 more tries remain
> sym0:1: message c sent on bad reselection.
> sym0:1:control msgout: 80 6.

The device should execute the INQUIRY command, but not reply was received
(the device did not reselect the controlled). Several attempts are made,
but the retries are exausted and the device is ignored.

It seems that the selection/reselection and sending of SCSI messages is
not perfectly emulated.

You may want to contact the author of the 53c895a emulation, since you'll
need debug output from that emulation in order to understand what's wrong.

The driver has been used with all kinds of NCR and Symbios hostadapters when
PCI and SCSI were relevant, and I do not know of any functional issues with
real hardware.

Regards, STefan