Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

[Paul]

I am getting this message with normal routing.

say...

em0 10.1.1.1/24

em1 10.2.2.1/24

using a box 10.1.1.2 on em0
and having another box on 10.2.2.2 on em1

I send packet from 10.1.1.2 which goes through em0 and has a route to 
10.2.2.2 out em1 of course and I get MASSIVE RTM_MISS messages but ONLY 
with this certain packets.. I don't get it?   I posted the tcpdump of 
the types of packets that generate them and the ones that don't.  
RTM_MISS is normal if the box can't get to a route, it's the 
'destination unreachable' message.

I would prefer a kernel option to disable this message to save  CPU 
cycles though as it is completely unnecessary to generate.

I even set the default gateway to loopback interface and I STILL get the 
message.. Something is wrong in the code somewhere.
Does anyone have any idea how to disable this message? It's causing 
major cpu usage on my zebra daemon which is watching the route messages 
and most likely severely limiting pps throughput :/

It generates the messages with only ip on em1 and em0 with nothing else 
in the routing table and a default gateway set.  So it has nothing to do 
with zebra.  It happens in 7-STABLE and (8) -CURRENT, I tested both.

There are no RTM_MISS message in 7-RELEASE so someone changed something 
to -STABLE :/

Paul


Steve Bertrand wrote:
> Mike Tancsa wrote:
>> At 04:04 AM 6/29/2008, Paul wrote:
>>> This is just a question but who can get more than 400k pps 
>>> forwarding performance ?
>>
>>
>> OK, I setup 2 boxes on either end of a RELENG_7 box from about May 
>> 7th just now, to see with 2 boxes blasting across it how it would 
>> work.  *However*, this is with no firewall loaded and, I must enable 
>> ip fast forwarding. Without that enabled, the box just falls over.
>>
>> even at 20Kpps, I start seeing all sorts of messages spewing to route 
>> -n monitor
>>
>>
>> got message of size 96 on Mon Jun 30 15:39:10 2008
>> RTM_MISS: Lookup failed on this address: len 96, pid: 0, seq 0, errno 
>> 0, flags:<DONE>
>> locks:  inits:
>> sockaddrs: <DST>
>>  default
>
> Mike,
>
> Is the monitor running on the 7.0 box in the middle you are testing?
>
> I set up the same configuration, and even with almost no load (< 
> 1Kpps) can replicate these error messages by making the remote IP 
> address (in your case 'default', disappear (ie: unplug the cable, DDoS 
> etc).
>
> ...to further, I can even replicate the problem at a single packet per 
> second by trying to ping an IP address that I know for fact that the 
> router can not get to.
>
> Do you see these error messages if you set up a loopback address with 
> an IP on the router, and effectively chop your test environment in 
> half? In your case, can the router in the middle actually get to a 
> default gateway for external addresses (when I perform the test, your 
> 'default' is substituted with the IP I am trying to reach, so I am 
> only assuming that 'default' is implying default gateway).
>
> Steve
>