Chelsio 10GB PCI-e Opt Card PCI-E 110-1088-30 is a T320 supported via cxgb(4)

Wed Jun 22 14:28:13 UTC 2016

For the Community Documentation on the inter-webs and Google searches -

For about a month or two, I was trying to figure out exactly which platform the Chelsio 10GB Opt Card, part number 110-1088-30 was built on, and if it was supported under FreeBSD.

I suspected it was an N320, but could not confirm it. Chelsio's site didn’t even have any cross reference for this part number.

There are various eBay auctions for these cards, with some PCB variants, running at the $25-35 range, which would seem to be a steal for a dual ported 10Gbps ethernet card.

So, I broken down and purchased one of these part number 110-1088-30 PCIe 10Gb “Opt Cards”.
 - http://www.ebay.com/itm/351719918339

In fact, this does appears to be the Terminator 3 ASIC platform (T3).  I assume this was later rebranded/rev’ed  by Chelsio to the T3 Unified Wire collection under product name “N320”.  But can’t find any references or documentation to confirm this.

I don’t know how to probe FreeBSD to check the PCIe sync up.  However, I believe it is a PCIe 1.1 x8 device.  This means the PCIe x8 bus maxes out at 16 Gbps.  However, it appears that the T3 version on this card maxes out at about 11 Gbps (~5.5 Gbps each port with iperf when lighting up both ports at the same time).

This card - even as the N320, the marketing material lists this as a failover/HA card, not intended for a 20 Gbps LAG.

I also found some new, Finisar SFP+ SR optics on eBay for about $18-20/each. Combined with some fiber from mono price. For about $50, this feels like a pretty good and cheap solution for cheap 10Gbps connectivity for home labs/NASes - with a really good and well supported brand/card.

(** This should work in FreeNAS, at least by the kernel, too - the cxgb(4) support has been around for a long time ! *** )

Hopefully someone at some point down the road, finds this info useful.

=== pciconf -lv ===

cxgbc0 at pci0:8:0:0:	class=0x020000 card=0x00011425 chip=0x00311425 rev=0x00 hdr=0x00
    vendor     = 'Chelsio Communications Inc'
    device     = 'T320 10GbE Dual Port Adapter'
    class      = network
    subclass   = ethernet

== dmesg, verbose boot ===
pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pcib0: allocated type 3 (0xd8300000-0xd83fffff) for rid 20 of pcib8
pcib8:   domain            0
pcib8:   secondary bus     8
pcib8:   subordinate bus   8
pcib8:   memory decode     0xd8300000-0xd83fffff
pcib8:   special decode    ISA
pci8: <ACPI PCI bus> on pcib8
pcib8: allocated bus range (8-8) for rid 0 of pci8
pci8: domain=0, physical bus=8
found-> vendor=0x1425, dev=0x0031, revid=0x00
        domain=0, bus=8, slot=0, func=0
        class=02-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0147, statreg=0x0010, cachelnsz=8 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=7
        powerspec 3  supports D0 D3  current D0
        MSI supports 32 messages, 64 bit
        MSI-X supports 32 messages in map 0x20
        map[10]: type Memory, range 64, base 0xd8301000, size 12, enabled
pcib8: allocated memory range (0xd8301000-0xd8301fff) for rid 10 of pci0:8:0:0
        map[20]: type Memory, range 64, base 0xd8300000, size 12, enabled
pcib8: allocated memory range (0xd8300000-0xd8300fff) for rid 20 of pci0:8:0:0
pcib8: matched entry for 8.0.INTA
pcib8: slot 0 INTA hardwired to IRQ 16
cxgbc0: <Chelsio T320, 2 ports> mem 0xd8301000-0xd8301fff,0xd8300000-0xd8300fff irq 16 at device 0.0 on pci8
cxgbc0: attempting to allocate 9 MSI-X vectors (32 supported)
msi: routing MSI-X IRQ 258 to local APIC 0 vector 54
msi: routing MSI-X IRQ 259 to local APIC 0 vector 55
msi: routing MSI-X IRQ 260 to local APIC 0 vector 56
msi: routing MSI-X IRQ 261 to local APIC 0 vector 57
msi: routing MSI-X IRQ 262 to local APIC 0 vector 58
msi: routing MSI-X IRQ 263 to local APIC 0 vector 59
msi: routing MSI-X IRQ 264 to local APIC 0 vector 60
msi: routing MSI-X IRQ 265 to local APIC 0 vector 61
msi: routing MSI-X IRQ 266 to local APIC 0 vector 62
cxgbc0: using IRQs 258-266 for MSI-X
cxgbc0: using MSI-X interrupts (9 vectors)
cxgb0: <Port 0 10GBASE-R> on cxgbc0
cxgb0: Using defaults for TSO: 65518/35/2048
cxgb0: bpf attached
cxgb0: Ethernet address: 00:07:43:0a:a0:84
cxgb1: <Port 1 10GBASE-R> on cxgbc0
cxgb1: Using defaults for TSO: 65518/35/2048
cxgb1: bpf attached
cxgb1: Ethernet address: 00:07:43:0a:a0:85
cxgbc0: Firmware Version 7.11.0

========

-
David P. Discher
http://davidpdischer.com/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20160622/4c72d1d3/attachment.sig>