[solved/workaround?] Re: em very slow, shared irq... on 6.3p8

[Jack Vogel]

On Wed, Feb 27, 2008 at 11:50 AM, Holger Kipp <hk at alogis.com> wrote:
> On Wed, Feb 27, 2008 at 09:50:16AM -0500, Mike Tancsa wrote:
>
>  more details below. as it currently is, polling seems to do
>  the trick, however handling several em-interfaces with the
>  same irq (mind you, it is pci) shouldn't cause delays of
>  up to 1.5 seconds for a simple ping... Therefore I consider
>  using polling for a nearly idle system more a workaround
>  than a solution to this problem :-(
>
>  > At 05:49 AM 2/27/2008, Holger Kipp wrote:
>  >
>  > >I therefore assume that the problem is between receiving the
>  > >irq from em<x> and getting the data from the interface on the firewall
>  > >itself.
>  >
>  > I would try upgrading to 6.3R (there are several em driver bug fixes)
>  done. system is now 6.3-RELEASE-p1 which also gave me -c option for pciconf
>  and msi syscontrols (were missing in the old 6.2).
>
>
>  > and then try the box with
>  > % cat /boot/loader.conf
>  > hw.pci.enable_msi=1
>  >
>  > ...if the cards support msi.
>  >
>  > I think pciconf -lvc should tell you if the cards and slots support it or
>  > not.
>
>  pciconf -lvc says for all em<x>:
>  cap 05[d0] = MSI supports 1 message, 64 bit
>  so I assume they do support MSI.
>
>  with msi disabled I get
>
>  38 packets transmitted, 38 packets received, 0% packet loss
>  round-trip min/avg/max/stddev = 4.833/228.022/1539.337/339.768 ms
>
>  with msi enabled (via sysctl) I get
>
>  33 packets transmitted, 33 packets received, 0% packet loss
>  round-trip min/avg/max/stddev = 1.865/156.421/1339.841/239.375 ms
>
>  so looks equally bad (I don't consider 30-40 packets a meaningful sample).
>  I don't know if it makes any differences if switched on directly in
>  loader.conf, though.
>
>  enabling polling (withous MSI) gives
>
>  30 packets transmitted, 30 packets received, 0% packet loss
>  round-trip min/avg/max/stddev = 0.366/0.790/1.339/0.290 ms
>
>  (maybe I should have used HZ=2000 to keep it below 0.6ms ;-)
>
>  > Also, if you dont need IPV6, use FAST_IPSEC. It does not need
>  > mpsafe.  If you do need IPSEC and IPV6, 7.0R got rid of that restriction.
>
>  I think this enough changes in one go for a production system ;-)

Hmmm, something is really broken here that POLL just is just bypassing,
what was the adapter type exactly (pciconf -l). Sorry, but I must have
missed this email earlier.

Jack