Poor throughput with natd

Ian Smith smithi at nimnet.asn.au
Tue Nov 24 04:51:15 UTC 2009

In freebsd-questions Digest, Vol 286, Issue 4, Message 16
On Mon, 23 Nov 2009 15:28:12 -0800 James Long <list at museum.rain.com> wrote:
 > Please copy me on replies.
 > I am testing ipfw and natd on a gateway machine running FreeBSD
 > 7.2-STABLE #0: Tue Oct 27 00:12:39 PDT 2009  with the generic
 > kernel.  ipfw.ko and ipdivert.ko are loaded as modules, since
 > they're not part of the GENERIC kernel.
 > The symptom is that scp uploads from the gateway machine have
 > very poor throughput, often showing "stalled" status in the scp
 > progress output.
 > Machines on the LAN do not suffer this problem, and can upload
 > their traffic via NAT with no observed degradation in throughput.
 > That's why I haven't noticed this problem until recently, when I
 > tried rsync-ing some files outbound from the gateway to a remote
 > machine.
 > I can work around the problem, but this problem has never cropped
 > up in the past.  Is there a problem in my configuration, or in
 > recent natd?
 > Thanks for your time!

Hi Jim,

among the over-copious notes in my rc.firewall is:

  #% Julian Elischer, 22Oct06 in freebsd-net:
  # "one thing that you need to name sure of is that only the packets that
  # have potential of being on interest to natd are passed to natd.
  # i.e. be VERY specific in your natd rules..
  # ipfw add 1000 divert natd ip from any to any out recv {inner-ineterface}
  #    xmit {outer-interface}.
  # ipfw add 1001 divert natd ip from any to {inner-interface-address} in
  #    recv {outer-interface}.
  # don't waste natd's time with packets it doesn't care about."

1001 is actually not quite right, I'll get to that, but the principle is 
correct; the only packets natd can do anything useful with are these:

a) going OUT on the external interface that were received on internal 
interface, so needing source address translation to the outside address.

b) coming IN on the external interface, which MAY match previous (a) 
packets, so requiring destination address remapping to an internal IP.

In the case you outline, the scp is happening between this box itself 
and an outside host so are of no interest to natd, costing extra time.

 > All commands below were executed on the gateway machine that is
 > running natd with very basic options:
 > 15:07:37 /root# findps natd
 > root    480  0.0  0.1  3388  1252  ??  Ss   12Nov09   4:32.81 natd -n fxp1

Here I rather use -a ${ext_ip} but that probably doesn't matter.

 > Here are the ipfw rules:
 > 14:55:41 /root# ipfw show
 > 00100   949306    56746770 allow ip from any to any via lo0
 > 00200        0           0 deny ip from any to
 > 00300        0           0 deny ip from to any
 > 00400    77293     8699526 divert 8668 ip from any to any via fxp1
 > 00500 35245946 28535731864 allow ip from any to any
 > 65535        0           0 deny ip from any to any

Try, where ext_if=fxp1, int_if=$your_internal_if and ext_ip=$yours

ipfw add 400 divert natd ip from any to any out recv $int_if xmit $ext_if
ipfw add 410 divert natd ip from any to $ext_ip in recv $ext_if

Apart from not passing natd undivertable packets, use of 'via' here has
natd being called at least once and maybe twice on each packet coming or 
going on the outside interface, including those from the host itself.

 > Downloading, scp has no trouble:
 > 14:55:59 /root# scp -p remote:public_html/video/tatra1.mpg .
 > tatra1.mpg                                                              100%   85MB 559.4KB/s   02:36
 > But uploads stall.  This scp process was killed after about 60 seconds:

Might there be an MTU issue as well?  Anything in /etc/natd.conf?

Despite that the above divert rules will prevent outbound host traffic 
being diverted at all, I'm still surprised natd's impact was so severe?

 > 14:58:40 /root# scp -p tatra1.mpg remote:/tmp/
 > tatra1.mpg                                                                0%  320KB   1.8KB/s - stalled -
 > ^CKilled by signal 2.
 > Deleting the DIVERT rule eliminates the stalling:
 > 14:59:54 /root# ipfw delete 400
 > 15:00:04 /root# scp -p tatra1.mpg remote:/tmp/
 > tatra1.mpg                                                               27%   23MB 248.2KB/s   04:14 ETA
 > ^CKilled by signal 2.
 > But of course, it also eliminates NAT.
 > 15:01:14 /root# ipfw add 400 divert 8668 ip from any to any via fxp1
 > 00400 divert 8668 ip from any to any via fxp1
 > Adding this rule works around the natd throughput problem:
 > 15:01:29 /root# ipfw add 350 allow all from me to any via fxp1
 > 00350 allow ip from me to any via fxp1
 > 15:02:03 /root# scp -p tatra1.mpg remote:/tmp/
 > tatra1.mpg                                                              100%   85MB 266.9KB/s   05:27

350 has same effect as putting the selective requirements on outbound 
divert.  You still need to check inbound packets for possible NAT'ing.

cheers, Ian

More information about the freebsd-questions mailing list