msk driver issues [was: Re: vlan issues with 7.0-RC3]

[Christopher Cowart]

On Fri, Feb 29, 2008 at 03:03:53PM +0900, Pyun YongHyeon wrote:
> On Wed, Feb 27, 2008 at 06:38:40PM -0800, Christopher Cowart wrote:
>>On Tue, Feb 26, 2008 at 04:43:55PM +0900, Pyun YongHyeon wrote:
>>>On Mon, Feb 25, 2008 at 01:17:12AM -0800, Christopher Cowart wrote:
>>>> I have a mac mini running 7.0-RC3, which I'm trying to turn it into a
>>>> router. I have a Linksys SRW2008 "fully managed" (via an IE only web
>>>> interface, ick) switch. 
>>>> 
>>>> Switch:
>>>>   Port 1 - Trunk vlans 10,60,98 - FreeBSD Box
>>>>   Port 7 - Access vlan 98 - Existing LAN (192.168.1.0/24)
>>>> 
>>>> OpenWRT (192.168.1.1):
>>>>   WRT54G box on the Existing LAN
>>>> 
>>>> FreeBSD Box:
>>>>   ifconfig msk0 up
>>>>   ifconfig vlan98 create vlan 98 vlandev msk0 inet 192.168.1.67/24
>>>> 
>>>> With this configuration, I can ping hosts on the other lan segment (Port
>>>> 7). Arp and icmp seem to be quite happy. Unfortunately, I'm not having
>>>> any luck with tcp and udp. Any attempt to ssh to OpenWRT or dig
>>>> @OpenWRT hangs indefinitely. If I do a tcpdump, I see the SYN or A?
>>>> leaving and absolutely no response returning. If I run a tcpdump on
>>>> OpenWRT, I see no incoming traffic.
>>>> 
>>>> When I try to connect *to* the FreeBSD box from the other lan segment, I
>>>> continue to have problems. tcpdump shows the SYNs arriving via vlan98
>>>> and the FreeBSD box responding with SYN-ACK. OpenWRT receives the SYNACK.
>>>> 
>>>> I disabled ipfw just to be sure (sysctl -w net.inet.ip.fw.enable=0), but
>>>> it had no effect on the problem. If I connect the FreeBSD box to a vlan
>>>> 98 access port and assign the address to msk0, my connectivity problems
>>>> go away. This leads me to believe that the firewall on OpenWRT is not
>>>> the problem and the problem is related to vlans.
>>>> 
>>>> Thinking it was a problem with the not-so-cheap Linksys POS (bitterness
>>>> about the IE web interface again), I plugged my MacBook (running
>>>> Leopard, not FreeBSD) into the trunk port. Running the ifconfig commands
>>>> above (s/msk0/en0/), I got up and running without any problems. This
>>>> causes me to suspect the FreeBSD box.
>>>> 
>>>> Does anyone have any idea what's going on here? Any suggestions for
>>>> further troubleshooting?
>>>
>>> Try disabling hardware features one by one in msk(4) and see how
>>> it goes.
>>>  o Disable TSO.
>>>  o Disable Tx checksum offload.
>>>  o Disable VLAN hardware tagging.
>>
>>Works great after `sudo ifconfig msk0 -txcsum'. 
>>
>>Is this a known bug, or should I file a PR? Let me know if there are any
>>other details I can provide to help somebody squash it.
> 
> Would you capture broken TCP/UDP frames with tcpdump on receiving side
> and show it to me?

Thanks for your help. I will e-mail you the corresponding tracefile
off-list. I wanted to discuss my initial findings here.

When the FreeBSD box sends tcp or udp traffic to elsewhere on the
network, it is not seen at the receiving side, not even with bad
checksums. I tried setting up the Linksys device with a port mirror, but
apparently that's only supported on native vlan access ports. Great.

So, I decided to cross-connect the FreeBSD box with my MacBook. I set up
a vlan interface and was able to reproduce the behavior. I did a traffic
dump on the parent interface (not the vlan interface) on the MacBook,
and noticed that the ethernet frames originating from the FreeBSD box
lacked 802.1q tags. Is it possible that the interface is not performing
the hardware tagging when txcsum is enabled? 

While I have your attention, I am also suffering from a problem that was
reported to -questions here[1]. About 3 times a day, I'll see the
watchdog timeout (missed Tx interrupts) message get logged, after which
point the NIC is useless until I reboot. Any ideas?

[1] http://lists.freebsd.org/pipermail/freebsd-questions/2008-February/169633.html

-- 
Chris Cowart
Network Technical Lead
Network & Infrastructure Services, RSSP-IT
UC Berkeley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 825 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20080303/2f95fd67/attachment.pgp