[HEADS UP] merging projects/pf into head

Ian FREISLICH ianf at clue.co.za
Wed Sep 12 10:42:07 UTC 2012


Gleb Smirnoff wrote:
>   [announce goes both to net@ and pf@, but any discussion should
>    go on on pf at FreeBSD.org only, please]
> 
>   As you already may now, last half a year I've been working on
> making pf SMP-scalable and faster in general. More info can be
> found here:

I've had your code running in production for the last few days.
Sadly, HEAD is a little unstable and the system panics after about
1 hour of use.

Fatal trap 12: page fault while in kernel mode
cpuid = 9; apic id = 09
fault virtual address   = 0x28
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff802d9ff1
stack pointer           = 0x28:0xffffff84626540b0
frame pointer           = 0x28:0xffffff8462654110
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 11 (irq257: bce1)
trap number             = 12
panic: page fault
cpuid = 9
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
panic() at panic+0x1ce
trap_fatal() at trap_fatal+0x290
trap_pfault() at trap_pfault+0x210
trap() at trap+0x2b4
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff802d9ff1, rsp = 0xffffff84626540b0, rbp = 0xffffff
8462654110 ---
pf_anchor_node_RB_NEXT() at pf_anchor_node_RB_NEXT+0x1
pf_test_rule() at pf_test_rule+0x4d7
pf_test() at pf_test+0x2b28
pf_check_in() at pf_check_in+0x26
pfil_run_hooks() at pfil_run_hooks+0x9e
ip_fastforward() at ip_fastforward+0x1b9
ether_demux() at ether_demux+0x17e
ether_nh_input() at ether_nh_input+0x24b
netisr_dispatch_src() at netisr_dispatch_src+0x212
ether_demux() at ether_demux+0x6c
ether_nh_input() at ether_nh_input+0x24b
netisr_dispatch_src() at netisr_dispatch_src+0x212
bce_intr() at bce_intr+0x47a
intr_event_execute_handlers() at intr_event_execute_handlers+0xfd
ithread_loop() at ithread_loop+0x9e
fork_exit() at fork_exit+0x11e
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8462654cb0, rbp = 0 ---
Uptime: 1h26m28s
Dumping 1367 out of 16368 MB

The crashdump is useless however:

#0  0xffffffff80490882 in doadump ()
(kgdb) bt
#0  0xffffffff80490882 in doadump ()
#1  0x0000000000000004 in ?? ()
#2  0x0000000100000000 in ?? ()
#3  0xffffff8462653d00 in ?? ()
#4  0xffffffff80490dc4 in kern_reboot ()
#5  0x9cd880c7c748c3c9 in ?? ()
#6  0xe8ebffe59860e880 in ?? ()
#7  0x0f00000000801f0f in ?? ()
#8  0x485500000000801f in ?? ()
etc

I have the following tunables set:
[firewall2.jnb1] ~ # cat /boot/loader.conf 
console="comconsole"
net.isr.maxthreads="8"
net.isr.defaultqlimit="4096"
net.isr.maxqlimit="81920"
net.isr.direct="0"
net.isr.direct_force="0"
kern.ipc.nmbclusters="262144"
kern.maxusers="1024"
hw.bce.rx_pages="8"
hw.bce.tx_pages="8"

[firewall2.jnb1] ~ # cat /etc/sysctl.conf 
net.inet.tcp.blackhole=2
net.inet.udp.blackhole=1
net.inet.ip.fastforwarding=1
net.inet.carp.preempt=1
net.inet.icmp.icmplim_output=0
net.inet.icmp.icmplim=0
kern.random.sys.harvest.interrupt=0
kern.random.sys.harvest.ethernet=0
kern.random.sys.harvest.point_to_point=0
net.route.netisr_maxqlen=8192

CPU usage is down from about 17% to 5% for our traffic load.  We're
averaging about 400k states, peaking at 550k states (220Mbit/s of
pfsync traffic!!) and 426329 routes.

Ian

-- 
Ian Freislich


More information about the freebsd-pf mailing list