Packet loss when using multiple IP addresses

[Frank Bartels]

Hi freebsd-net,

I'm trying to run a root server using FreeBSD using four different
IP addresses. Everything works fine with one IP address, but if I
add more addresses I notice a packet loss of about 10% after some
minutes, in rare cases after three hours. Sometimes the packet loss
raises to 50% and somtimes up to 95%. But never 100%.

Incoming connections are affected. I cannot type in my ssh connection.
Outgoing connections do not seem to be affected, mtr has 0.0% loss
after thousands of packets. Running mtr on the host seems to "help" (*).
There are no error messages in messages and console.log.

The second IP address is used for a jail and the third IP address
for a VM running VirtualBox.

The final configuration is with native IPv6 (dual stack).

I've tried all of the following without success:

- FreeBSD 8.1-RELEASE and 8.1-STABLE (20100729)
- re(4) and em(4)
- with or without the changes to /sys/dev/re/if_re.c from r207977
- with or without jail (alias IP address)
- with or without vbox
- with or without IPv6
- with or without powerd
- with ifconfig_${INTERFACE}="DHCP" and with static configuration
- with or without rxcsum,txcsum
- the motherboard was changed already (mainly because of problems
  with ahci enabled)

If I use tcpdump in order to trace the ICMP packets on the ethernet
interface, I do not see the incoming ICMP requests.

The motherboard is a MSI-X58Pro-E. kernel is GENERIC. I do not use
ipfw(8). I've recompiled virtualbox after switching from 8.1-RELEASE
to 8.1-STABLE. Ports are updated daily.

I have a very similar setup running in the office without problems.

I'm very frustrated because I have absolutely no idea what's going
on here.

(*) I can repoduce reducing the packet loss a lot (to less than 1%)
by running mtr from the host using the remote console. If I start
the mtr, the packet loss goes down and I can login using ssh.

If you have any idea what I can do in order to find the source of
my problem, please answer. Maybe a single keyword is enough. :)

Thanks,
Knarf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4042 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20100730/bf4a913b/smime.bin