crashing problem I cant figure related to IF_ADDR_LOCK, BSD 8.0

[Ricky Charlet]

Howdy,

                FreeBSD 8.0-RELEASE  running on an 8 core amd64

                I'm writing a packet filter hook. It is an outbound hook attached with :
                pfil_add_hook(chkoutput, NULL, PFIL_OUT | PFIL_WAITOK, pfh_inet);

                Inside the hook (chkoutput) I have the following code snipit (where I happen to know that ifp already points to an interface I specifically don't want to process packets for):

                IF_ADDR_LOCK(ifp);
                                TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
                                if (ifa->ifa_addr->sa_family == AF_INET) {
                                                struct sockaddr_in *sa = (struct sockaddr_in*)ifa->ifa_addr;
                                                if(sa->sin_addr.s_addr == ip->ip_src.s_addr) {
                                                                /* nevermind */
                                                                IF_ADDR_UNLOCK(ifp);
                                                                return 0;
                                                }
                                }
                }
                IF_ADDR_UNLOCK(ifp);


                Well, it runs fine / logically sound / does exactly what I want. However, in later processing, on packets I am receiving (*not* traversing the output hook) I crash with various stack traces but all culminating in sbdrop_internal. In that function, I have  a pointer to an mbuf which is garbage (unreferencable)  memory.


-          If I take the above code snipit out of my output hook, the system remains stable. (though, of course, the hook is not doing all I want)

-          If I remove the LOCK and UNLOCK macros, the same crash happens.

-          If I take IFNET_RLOCK_NOSLEEP or IFNET_RLOCK, same crash happens.


I'm fairly convinced that my output hook at the IP layer is somehow corrupting the receive socket layer. But I see no relationship. Even if I were running beyond loop bounds here, I'm not really writing any memory. On the other hand, I don't truly know my way around dealing with kernel locks and I'm just mimicking code I saw in ip_input  (the "Check for broadcast addresses" bits).  Any Ideas?

Thanks in advance
---
Ricky Charlet
Adara Networks<http://www.adaranet.com/welcome.html>
USA 408-433-4942


PS Some kgdb output here:


#10 0xffffffff80860183 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:224
#11 0xffffffff805ec873 in sbdrop_internal (sb=0xffffff0001b976d0, len=0)
    at /usr/src/sys/kern/uipc_sockbuf.c:891
#12 0xffffffff806ff187 in tcp_do_segment (m=0xffffff000185f200,
    th=0xffffff00018f0024, so=0xffffff0001b97550, tp=0xffffff0001b24a50,
    drop_hdrlen=40, tlen=0, iptos=0 '\0', ti_locked=2)
    at /usr/src/sys/netinet/tcp_input.c:2357
#13 0xffffffff80700f72 in tcp_input (m=0xffffff000185f200, off0=Variable "off0" is not available.
)
    at /usr/src/sys/netinet/tcp_input.c:1020
#14 0xffffffff806984ba in ip_input (m=0xffffff000185f200)
---Type <return> to continue, or q <return> to quit---
    at /usr/src/sys/netinet/ip_input.c:775
#15 0xffffffff806423ee in netisr_dispatch_src (proto=1, source=Variable "source" is not available.
)
    at /usr/src/sys/net/netisr.c:917
#16 0xffffffff8063ab2d in ether_demux (ifp=0xffffff0001579000, m=0xffffff000185f200)
    at /usr/src/sys/net/if_ethersubr.c:895

(kgdb) frame 11
#11 0xffffffff805ec873 in sbdrop_internal (sb=0xffffff0001b976d0, len=0)
    at /usr/src/sys/kern/uipc_sockbuf.c:891
891             if (m == NULL) {
(kgdb) print *sb
$1 = {sb_sel = {si_tdlist = {tqh_first = 0x0, tqh_last = 0x0}, si_note = {kl_list = {
        slh_first = 0x0}, kl_lock = 0xffffffff8055bf00 <knlist_mtx_lock>,
      kl_unlock = 0xffffffff8055bed0 <knlist_mtx_unlock>,
      kl_assert_locked = 0xffffffff80559220 <knlist_mtx_assert_locked>,
      kl_assert_unlocked = 0xffffffff80559230 <knlist_mtx_assert_unlocked>,
      kl_lockarg = 0xffffff0001b97718}, si_mtx = 0x0}, sb_mtx = {lock_object = {
      lo_name = 0xffffffff8096f9d5 "so_snd", lo_flags = 16973824, lo_data = 0,
      lo_witness = 0x0}, mtx_lock = 18446742974221313824}, sb_sx = {lock_object = {
      lo_name = 0xffffffff8096ff95 "so_snd_sx", lo_flags = 36896768, lo_data = 0,
      lo_witness = 0x0}, sx_lock = 1}, sb_state = 0, sb_mb = 0x8c4600000000,
  sb_mbtail = 0xffffff0001901900, sb_lastrecord = 0xffffff0001901900,
  sb_sndptr = 0x0, sb_sndptroff = 0, sb_cc = 0, sb_hiwat = 33580, sb_mbcnt = 0,
  sb_mcnt = 0, sb_ccnt = 0, sb_mbmax = 262144, sb_ctl = 0, sb_lowat = 2048,
  sb_timeo = 0, sb_flags = 2048, sb_upcall = 0, sb_upcallarg = 0x0}
(kgdb) print *sb->sb_mb
Cannot access memory at address 0x8c4600000000
(kgdb)