network tuning and performance troubleshooting

[Ian Smith]

 > Message: 18
 > Date: Thu, 04 Jan 2007 18:05:27 -0500
 > From: Bob McIsaac <bobmc at bobmc.net>

 > Doug Hardie wrote:
 > >
 > > On Jan 3, 2007, at 22:57, Bob McIsaac wrote:
 > >
 > >> Problem: Browser status 'looking up address' for 10 seconds for any
 > >> web page clicked. Slashdot takes almost a minute to load. But FTP
 > >> performance is good and running a shoutcast stream is no problem.
 > >> Sending mail via my ISP is slow.
 > >>
 > >> Investigation: -  "sysctl -a  | grep net | less" shows a ton of 
 > >> variables
 > >> with values assigned. Ping of nameserver assigned by dhcp takes 0.5ms.
 > >> Ping of freebsd.org = 90ms. Nothing obvious in loader.conf or
 > >> rc.conf  (defaults).  /var/log/messages has only startup info.
 > >>
 > >> Question: - How to solve this thorny performance problem?  -Bob-
 > >
 > > You might want to run tcpdump and monitor one of those slow loads.  
 > > Include the timestamp in the output and see what it is doing during 
 > > that time.  I would tend to suspect DNS timeouts.
 > >
 > >
 > tcpdump confirms there is a ten second delay as seen on the browser.
 > 1. there are some UDP packets to/from the nameserver.
 > 2. nothing happens for ten seconds
 > 3. now there is a TCP connection
 > 
 > tcpdump: listening on vr0, link-type EN10MB (Ethernet),
 > 
 > 17:34:07.537419 proto: UDP (17)
 >   192.168.1.102.53032 > 192.168.1.254.domain: 
 >   45959+ A? www.google.ca. (31)

You ask 192.168.1.254 - presumably your gateway, and/or internal DNS
server? - for www.google.ca's IPv4 address. 
 
 > 17:34:07.545218 IP proto: UDP (17)
 >   192.168.1.254.domain > 192.168.1.102.53032: 
 >   45959 6/7/4 www.google.ca. CNAME[|domain]

It's a CNAME.  Not sure if you got the right IP address there, though
from the later (after delay) connect to google.com, I suppose so ..

 > 17:34:07.545500 IP proto: UDP (17)
 >   192.168.1.102.64463 > 192.168.1.254.domain: 
 >   45960+ AAAA? www.google.ca. (31)

Then you ask for www.google.ca's IPv6 address.  Do you really want that? 
You get no response on that, but maybe you're prepared to wait for it,
ie are you somehow relying on getting an IPv6 address, and if so, why? 

 > 17:34:07.868410 IP proto: UDP (17)
 >   192.168.1.102.61375 > 192.168.1.254.domain: 
 >   48085+ PTR? 254.1.168.192.in-addr.arpa. (44)

You then ask for your gateway's IP address, by name.  Hmm.  But you get
no response to that query.  Looks like you're about to wait for one ..
~4.6 seconds later you're still waiting, and you ask again .. 

 > 17:34:12.545947 IP proto: UDP (17)
 >   192.168.1.102.54649 > 192.168.1.254.domain: 
 >   45960+ AAAA? www.google.ca. (31)

.. for that IPv6 address, and then you ask again ..

 > 17:34:12.868866 IP proto: UDP (17)
 >   192.168.1.102.55840 > 192.168.1.254.domain: 
 >   48085+ PTR? 254.1.168.192.in-addr.arpa. (44)

.. for your gateway's IP address from its name.  No answer.

 > <<<<<<<< nothing happens for 10 seconds?? >>>>>>>>>>>>

.. and then you appear to contact google.com successfully.

 > 17:34:22.546051
 >   (tos 0x0, ttl  64, id 226, offset 0, flags [DF],
 >   proto: TCP (6), length: 64)
 >   192.168.1.102.52363 > qb-in-f147.google.com.http: S,
 >   cksum 0x3aa5 (correct),
 >   1762925400:1762925400(0) win 65535 <mss 1460,
 >   nop,wscale 1,
 >   nop,nop,timestamp 1758025 0,sackOK,eol>

Are you obliged to use 192.168.1.254 for DNS?  The AAAA queries aside
(which it should quickly NAK if it doesn't handle them), it seems broken
if it can't resolve it's own reverse DNS?  Can you use your upstream
provider's DNS server/s instead (ie in resolv.conf)?  Is your IP fixed
or DHCP-assigned?  If the latter, with or without auto DNS assignment? 

Cheers, Ian