100Mbit/s LAN slow, TX only ~3MB/s (esp. file transfer) -- why?

[Andreas Ntaflos]

Hello list, 

here's the situation: a small LAN with two FreeBSD machines (one
4.10-STABLE, one 5.1-RELEASE-p11), one Gentoo box, one Windows 2000
Laptop. All the machines have 100Mbit/s capable network interfaces,
configured for full duplex and auto-negotiation, and actually running 
in 100baseTX-FD mode (5.1 uses rl0, 4.10 dc0, Gentoo and Windows are
also equipped with RealTek RTL8139 NICs).

They are all connected through CAT5 UTP cables and a 5 port 100Mbit/s
switch. All machines except the FreeBSD 5.1-RELEASE box are quite
capable (800MHz--1GHz, 128MB--640MB RAM, (U)DMA mode harddisks, etc).
The 5.1 box is an old 200MHz Pentium-MMX, 64MB RAM, slow disks.

Here's the problem: Using FTP to transfer a 600MB file (ISO image) from
one FreeBSD machine to the other runs only as fast as 3MB/s (maximum).
Transferring the same file from the Gentoo box to one of the FreeBSD
machines runs equally slow. Transferring the file from the Gentoo box to
the Windows 2000 laptop however runs fine with about 10MB/s.
Transferring from FreeBSD to Windows 2000 is slow again (2--5MB/s).

Using SCP gives me a maximum of 1MB/s tranfer rate, on all possible
connections and client pairs. I suppose this could be SCPs fault since 
it needs a fast CPU to encrypt and decrypt the data, but 800MHz and 1GHz
ought to be enough so that this shouldn't be a bottleneck, hm?

I've also tried connecting the machines directly via a crossover cable, 
to no avail. 4.10 <--> 5.1 is still 3MB/s, Gentoo <--> Windows 2000 runs
even faster, and Gentoo <--> FreeBSD is also about 2MB/s.

I remember that about a year ago with the same setup (except for the 5.1
box, which I didn't have back then) FTP speed was most excellent,
transferring a 1GB file took about 3 minutes. Now it seems to take
forever to copy just 50MB.

So why could this be? I am mostly interested in the two FreeBSD
machines, why would that be such a slow connection? Because the 5.1 box
is quite old? Could the RealTek card have a problem? The switch is
almost certainly not faulty. Also why would a transfer between the 4.10
box and the Gentoo machine take so long? As I mentioned, a year ago I
was enjoying TX rates of almost 11MB/s between the two of them.

I believe at least part of the problem lies with the FreeBSD machines,
so I am asking for ideas on what this issue could be, how to
investigate further to track down the possible sources of the problem
and how to solve it. 

Sorry for this lengthy post, I tried to give as much information as
possible and make clear that I have already invested a good amount of
time and effort to debug and solve this myself, but I am running out 
of ideas. I also posted on two other boards but nobody could suggest 
anything actually useful.

I refuse to believe that this is the expected mode of operation for FTP
(and even SCP) :-)

Any help would be greatly appreciated!

Thanks in advance!

Andreas 
-- 
Andreas "daff" Ntaflos | "A cynic is a man who knows the price of
daff AT dword DOT org  | everything, and the value of nothing."
Vienna, AUSTRIA        |                              Oscar Wilde