CAMBRIA and more than one atheros card

[John Hay]

On Thu, Jul 10, 2014 at 06:02:27AM +0200, John Hay wrote:
> On Wed, Jul 09, 2014 at 06:46:12AM +0200, John Hay wrote:
> > Hi Guys,
> > 
> > The problem is back / still there. I initially saw the problem during
> > boot, with the interface configs in rc.conf, but because it is so
> > mixed with the rest, I took it out and put it in the script, then
> > after multi-user boot was finished, I did a login and ran the script,
> > with the output I showed in the initial post.
> > 
> > So I put the interface configs back into rc.conf and I'm seeing the
> > same problem, here is a cut during boot:
> > 
> > ###############
> > Starting file system checks:
> > /dev/ad0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS
> > /dev/ad0s1a: clean, 93385 free (33 frags, 11669 blocks, 0.0% fragmentation)
> > Mounting local file systems:.
> > Writing entropy file:.
> > Setting hostname: tst-cambria-11.
> > wlan0: Ethernet address: 00:21:a4:35:70:42
> > wlan1: Ethernet address: 00:21:a4:35:6c:96
> > ath1: unable to start recv logic
> > wlan2: Ethernet address: 00:21:a4:32:38:c2
> > ath2: unable to start recv logic
> > ###############
> > 
> > Looking at the vmstat -z output the 256 Bucket fail is much higher than
> > if I let it boot to multiuser and then configured the interfaces. It
> > was in the 6000, while now it is much higher:
> > 
> > Without wlan configs in rc.conf, but configured afterwards:
> > 16 Bucket:               64,      0,      15,     300,    3139,  16,   0
> > 256 Bucket:            1024,      0,      31,       1,     592,6062,   0
> > vmem btag:               28,      0,    4496,     256,    4496,  32,   0
> > 
> > With wlan configs in rc.conf:
> > 16 Bucket:               64,      0,      16,     299,    8611,  16,   0
> > 256 Bucket:            1024,      0,      26,       6,     773,16928,   0
> > vmem btag:               28,      0,    4405,      59,    4405,  30,   0
> > 
> > Both of them boot from a ro compact-flash with md /etc and /var, but
> > they are small 4.3M and 2.1M. I have hacked in the use of tmpfs, but
> > that did not make a difference.
> > 

Friday I power cycled the board and it came up without an error. All 3
atheros cards configured in rc.conf. So I left it on for the weekend
and by this morning there was still no errors, so I rebooted it and
again saw the "ath1: unable to start recv logic" message for ath1 and
ath2. I power cycled it, but still get the error, so Friday was just
lucky with some timing thing, maybe if the card receive while it is
still configuring? Also if I leave the board booted in this state,
ath0 also start to give problems:

#################
ath0: ath_rx_proc: no mbuf!
ath0: ath_rx_proc: no mbuf!
...
ath0: device timeout
ath0: ath_reset: unable to start recv logic
...
#################

In anycase it seems that memory allocation problem. How do I figure out
where it is? "netstat -m" does not seem to point to an error. The mbuf
info in vmstat -z also look ok. It is only "256 Bucket" in vmstat -z
that point to an alloc failure. How do I figure out where that is and
how do I fix it or work around it? 

###################
tst-11-arm:~ # netstat -m
128/382/510 mbufs in use (current/cache/total)
127/129/256/7654 mbuf clusters in use (current/cache/total/max)
127/126 mbuf+clusters out of packet secondary zone in use (current/cache)
0/0/0/3827 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/1134 9k jumbo clusters in use (current/cache/total/max)
0/0/0/637 16k jumbo clusters in use (current/cache/total/max)
286K/353K/639K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/3/1488 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
tst-11-arm:~ # vmstat -z | grep mbuf
mbuf_packet:            256,  48990,     127,     126,    2294,   0,   0
mbuf:                   256,  48990,       1,     256,     920,   0,   0
mbuf_cluster:          2048,   7654,     253,       3,     253,   0,   0
mbuf_jumbo_page:       4096,   3827,       0,       0,       0,   0,   0
mbuf_jumbo_9k:         9216,   1134,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,    637,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,     504,       1,   0,   0
tst-11-arm:~ # vmstat -z | grep Bucket
4 Bucket:                16,      0,       7,     497,    1565,   0,   0
6 Bucket:                24,      0,       0,       0,       0,   0,   0
8 Bucket:                32,      0,       3,     375,      83,   0,   0
12 Bucket:               48,      0,       2,     334,       5,   0,   0
16 Bucket:               64,      0,      14,     301,   43687,  16,   0
32 Bucket:              128,      0,       7,     148,     196,   0,   0
64 Bucket:              256,      0,      19,      56,      77,   0,   0
128 Bucket:             512,      0,      16,      16,      48,   0,   0
256 Bucket:            1024,      0,      29,       3,    1521,86738,   0
tst-11-arm:~ #
###################

Regards

John
--
John Hay -- jhay at meraka.csir.co.za / jhay at meraka.org.za / jhay at FreeBSD.org