FreeBSD nfe driver and IPMI cards

[Robert Wojciechowski]

>  > I'm the FreeBSD nfe driver from
>  >
http://www.f.csce.kyushu-u.ac.jp/~shigeaki/software/freebsd-nfe.html
>  > with FreeBSD 6-stable with good results for the most part. The only
>  > issue I've experienced is that during a detach/shutdown of if_nfe,
> the
>  > IPMI IP address I have set on my servers ceases to respond as well
> as
>  > the ability to manage the servers.
>  >
>  >
>  >
>  > I traced the problem down to nfe_stop() and the fact that it
> completely
>  > disables the Rx and Tx on the NIC. I have patched the driver to not
>  > disable the Rx/Tx and IPMI continues to work after a 'ifconfig nfe0
>  > down', 'shutdown -p now', etc.
>  >
>  >
>  >
>  > Does anyone have any comments on this change I've made and any
> possible
>  > side effects? Can this be included in the mainstream distribution
of
> the
> 
> Because MAC is still alive if's possible to recieve a packet. All DMA
> maps are unloaded and buffers are already freed in nfe_stop so it
> would cause panic I guess. But I'm not familiar with IPMI so I'm not
> sure.

Interesting, that is an issue that was also resolved by the forcedeth
driver in Linux by resetting in nv_close to prevent DMA into freed
memory.

> 
>  > nfe drivers (and updated in 7-CURRENT) without causing any adverse
>  > problems?
>  >
> 
> I have no experience on IPMI but the change you've made would not
> completely solve the issue. I guess supporting IPMI needs lots of
> more work including:
>  o Autodetect IPMI capability.
>  o Autodetect active IPMI session in device attach and don't blindly
>    reset MAC/PHY.
>  o Don't blindly stop Tx/Rx on device detach.
> Given that lack of publicly available datasheet for the hardware
> supporing IPMI would be severly limited. Fortunately Linux seems to
> have basic IPMI support in their forcedeth driver. Their code doesn't
> easy to read but you may see what should be done in driver. However
> I have no idea what we can do when active IPMI session is present in
> driver attach phase. Normally PHY driver would reset PHY hardware
> itself in driver attach which in turn would result in losing the IPMI
> connection.
> 

Since we have no idea how to auto-detect IPMI, I added a device sysctl
(enable_ipmi) to control the behavior I'm testing in-house right now.

If the enable_ipmi sysctl is enabled, it'll:

  * resets after disabling the Rx/Tx and before disabling interrupts
  * re-enables the Rx/Tx after the ring buffers are freed, just like
forcedeth but also enabling the Tx for IPMI (Linux just enabled the Rx
for WOL)

I have attached my new patch to this email that does the above, guarded
by the sysctl.

I have no idea how to handle the second case you mentioned, during
attach. It does indeed cause a disruption in IPMI, but only for a few
seconds. Where is the MAC and PHY reset and if it wasn't reset when
using IPMI, what problems could it cause?

-- Robert
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nfe-ipmi-2.patch
Type: application/octet-stream
Size: 4558 bytes
Desc: nfe-ipmi-2.patch
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20070912/5fa2257c/nfe-ipmi-2.obj