Slow disk write speeds over network
Terry Lambert
tlambert2 at mindspring.com
Tue Jun 10 01:26:41 PDT 2003
Eric Anderson wrote:
> >>No, considering I would have to update about 600 hosts. Plus, those
> >>same hosts are running just fine with my other file servers at the
> >>moment.
> >
> > If your server's network buffers are ok, I would guess that the
> > "bottleneck" is on your clients machines.
> >
> > just my 2 cents...
>
> Paulo, thanks for the thoughts, however, that doesn't make a whole lot
> of sense (although I see why you are saying that), because in this case
> I have several fileservers on the same gigabit switches as this
> fileserver, and clients can transfer data quickly between all, in both
> directions, all the time, except data being sent TO this one host. I've
> even tried machines that are idle, on the same switch as this server,
> and I still receive slow transfers.
Swap cables with another box.
BTW: 4 Gigabit cards in one box, with you going to local disk...
you've got about 8 cards worth of traffic over your PCI bus.
Unless this is a PCI-X based box, you are most likely livelocked;
even if it's a PCI-X based box, you could still be livelocked.
You haven't said if you were using UDP or TCP for the mounts;
you should definitely use TCP with FreeBSD NFS servers; it's
also just generally a good idea, since UDP frags act as a fixed
non-sliding window: NFS over UDP sucks.
Also, you haven't said whether you are using aliases on your
network cards; aliases and NFS tend to interact badly.
Finally, you probably want to tweak some sysctl's, e.g.
net.inet.ip.check_interface=0
net.inet.tcp.inflight_enable=1
net.inet.tcp.inflight_debug=0
net.inet.tcp.msl=3000
net.inet.tcp.inflight_min=6100
net.isr.enable=1
Given your overloading of your bus, that last one is probably
the most important one: it enables direct dispatch.
You'll also want to enable DEVICE_POLLING in your kernel
config file (assuming you have a good ethernet card whose
driver supports it):
options DEVICE_POLLING
options HZ=2000
...and yet more sysctl's for this:
kern.polling.enable=1
kern.polling.user_frac=50 # 0..100; whatever works best
If you've got a really terrible Gigabit Ethernet card, then
you may be copying all your packets over again (e.g. m_pullup()),
and that could be eating your bus, too.
-- Terry
More information about the freebsd-performance
mailing list