kern/126688: 1.4.7 ixgbe driver panic with 4GB and PAE

[Petr Lampa]

>Number:         126688
>Category:       kern
>Synopsis:       1.4.7 ixgbe driver panic with 4GB and PAE
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Aug 20 16:40:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Petr Lampa
>Release:        HEAD
>Organization:
FIT BUT
>Environment:
FreeBSD rtboz.fit.vutbr.cz 8.0-CURRENT FreeBSD 8.0-CURRENT #10: Wed Aug 20 17:58:29 CEST 2008     root at rtboz.fit.vutbr.cz:/usr/src/sys/i386/compile/RADKA  i386

>Description:
Initialization of ixgbe buffers in ixgbe_get_buf() fails with errno=ENOMEM from bus_dmamap_load_mbuf_sg() and this error is ignored in ixgbe_setup_receive_ring(). When this happens, some buffer pointers are null and later system panics in ixgbe_rxeof():

        mp = rxr->rx_buffers[i].m_head;    /* m_head is NULL!!! */
        mp->m_len = mp->m_pkthdr.len =     /* <--- PANIC HERE */
                (rxr->rx_buffers[i].bigbuf ? MJUMPAGESIZE:MCLBYTES);



>How-To-Repeat:

>Fix:
Change this line:
    if (ixgbe_get_buf(rxr, j) == ENOBUFS) {
to
    if (ixgbe_get_buf(rxr, j) != 0) {

I've also noticed some useless assignments and one incorrect comment, so the full patch contains some other small fixes:

*** ixgbe.c     Wed Jul 30 20:15:18 2008
--- ixgbe.c.new Wed Aug 20 18:34:44 2008
***************
*** 2106,2112 ****
                error = ENOMEM;
                goto fail;
        }
-       txr = adapter->tx_rings;

        /* Next allocate the RX */
        if (!(adapter->rx_rings =
--- 2106,2111 ----
***************
*** 2116,2122 ****
                error = ENOMEM;
                goto rx_fail;
        }
-       rxr = adapter->rx_rings;

        /* For the ring itself */
        tsize = roundup2(adapter->num_tx_desc *
--- 2115,2120 ----
***************
*** 2169,2175 ****
                rxr->adapter = adapter;
                rxr->me = i;

!               /* Initialize the TX side lock */
                snprintf(name_string, sizeof(name_string), "%s:rx(%d)",
                    device_get_nameunit(dev), rxr->me);
                mtx_init(&rxr->rx_mtx, name_string, NULL, MTX_DEF);
--- 2167,2173 ----
                rxr->adapter = adapter;
                rxr->me = i;

!               /* Initialize the RX side lock */
                snprintf(name_string, sizeof(name_string), "%s:rx(%d)",
                    device_get_nameunit(dev), rxr->me);
                mtx_init(&rxr->rx_mtx, name_string, NULL, MTX_DEF);
***************
*** 2918,2924 ****
                goto fail;
        }

!       for (i = 0; i < adapter->num_rx_desc; i++, rxbuf++) {
                rxbuf = &rxr->rx_buffers[i];
                error = bus_dmamap_create(rxr->rxtag[0],
                    BUS_DMA_NOWAIT, &rxbuf->map[0]);
--- 2916,2922 ----
                goto fail;
        }

!       for (i = 0; i < adapter->num_rx_desc; i++) {
                rxbuf = &rxr->rx_buffers[i];
                error = bus_dmamap_create(rxr->rxtag[0],
                    BUS_DMA_NOWAIT, &rxbuf->map[0]);
***************
*** 2981,2987 ****
        }

        for (j = 0; j < adapter->num_rx_desc; j++) {
!               if (ixgbe_get_buf(rxr, j) == ENOBUFS) {
                    rxr->rx_buffers[j].m_head = NULL;
                    rxr->rx_base[j].read.pkt_addr = 0;
                    /* If we fail some may have change size */
--- 2979,2985 ----
        }

        for (j = 0; j < adapter->num_rx_desc; j++) {
!               if (ixgbe_get_buf(rxr, j) != 0) {
                    rxr->rx_buffers[j].m_head = NULL;
                    rxr->rx_base[j].read.pkt_addr = 0;
                    /* If we fail some may have change size */

With this patch it correctly prints to syslog:
   ix0: Could not setup receive structures
and doesn't panic. However this patch doesn't handle the primary problem, why bus_dmamap_load_mbuf_sg() fails with ENOMEM (probably 256*4 bounce buffers is too much or something like that). 

>Release-Note:
>Audit-Trail:
>Unformatted: