i386/68719: [usb] USB 2.0 mobil rack+ fat32 performance problem

Fri May 27 05:27:44 PDT 2005

(Posted to freebsd-fs as the PR is assigned to freebsd-usb@, but it seems to 
be more related to the msdos filesystem than the USB system so perhaps it 
should be reassigned?)

I've been evaluating the performance of some usb2 hard discs with FreeBSD and 
I found this PR (68719). The submitter is correct that performance with 
msdosfs is severely limited.

I tested a 'LaCie' USB2 disc:

da0 at umass-sim0 bus 0 target 0 lun 0
da0: <Maxtor 7Y250P0 YAR4> Fixed Direct Access SCSI-2 device 
da0: 40.000MB/s transfers
da0: 239372MB (490234752 512 byte sectors: 255H 63S/T 30515C)

egg# diskinfo -t da0
 ...
Seek times:
        Full stroke:      250 iter in   5.271879 sec =   21.088 msec
        Half stroke:      250 iter in   4.055049 sec =   16.220 msec
        Quarter stroke:   500 iter in   6.696545 sec =   13.393 msec
        Short forward:    400 iter in   2.316910 sec =    5.792 msec
        Short backward:   400 iter in   2.052681 sec =    5.132 msec
        Seq outer:       2048 iter in   1.574044 sec =    0.769 msec
        Seq inner:       2048 iter in   1.576574 sec =    0.770 msec
Transfer rates:
        outside:       102400 kbytes in   3.445316 sec =    29722 kbytes/sec
        middle:        102400 kbytes in   3.441593 sec =    29754 kbytes/sec
        inside:        102400 kbytes in   3.435809 sec =    29804 kbytes/sec

I used 10GB chunks of data to test the USB disc. Each test used a different 
10GB of data to avoid caching distorting results. I made the following 
measurements with both UFS2 and FAT32.

1. Local disc copy from a new SATA-150 disc.
2. Ftp copy over a local 100Mbit network from a server with a SATA-150 disc. 
3. Create a zero-file using dd to test simple write performance.

Client with attached USB disc: P4 2.6Ghz 768MB DDR, if_fxp, 1x ATA-100 disc
Server used for FTP: Celeron 2.4GHz 1.5GB DDR, if_em, 4x SATA-150 discs.

Both the client and server are running FreeBSD 5.4-STABLE built at
Thu May 26 22:52:15 BST 2005.

In test 1 I could not achieve any better than 5.1MB/s on an msdosfs 
filesystem. Using UFS2 and softupdates a transfer rate of 22~25MB/s was 
possible. Both test data sets were copied from the systems ATA-100 disc. In 
both tests at these peaks gstat reports the device is 100% busy.

A snapshot from gstat(8) during test 1. da0s1 is the fat32 filesystem.

dT: 0.501  flag_I 500000us  sizeof 240  i -1
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    2     90     90   3363    5.2      0      0    0.0   38.2| ad0
    2     90     90   3363    5.2      0      0    0.0   38.4| ad0s1
      ...
    2     90     90   3363    5.3      0      0    0.0   39.0| ad0s1g
   96   1295      2      8  163.9   1293   5170  141.6   99.8| da0
   96   1295      2      8  163.9   1293   5170  143.1   99.9| da0s1

In test 2 again the msdosfs filesystem could not achieve higher than 5MB/s. 
With UFS2 the limit of the network was reached before the limit of the USB2 
bus so the transfer was limited to 10.5MB/s average. During this period gstat 
reported about 35-45% activity on the device which matches up as I would have 
expected.

I managed to improve the performance in these results a little by upping 
MAXPHYS to 256, and then to 512 on the client. Going from 128 to 256 improved 
the diskinto -t transfer rates by about 3MB/s increasing it to 512 seemed to 
have no further benefit. Enabling polling for the fxp interface helped as 
well by reducing the interrupt rate from ~8k/s to 2k/s during the second 
test.

Finally, I used dd to test just the filesystem-write.

ufs2:

egg# dd if=/dev/zero of=/mnt/file.test bs=64k count=10000
10000+0 records in
10000+0 records out
655360000 bytes transferred in 25.093943 secs (26116262 bytes/sec)

And from gstat during the `dd':

dT: 0.501  flag_I 500000us  sizeof 240  i -1
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
      ...
    2     50      2     32   47.8     48  24510   45.7   97.8| da0
    2     50      2     32   47.9     48  24510   45.7   97.8| da0s1

msdosfs:

egg# dd if=/dev/zero of=/mnt/file2.test bs=64k count=10000
10000+0 records in
10000+0 records out
655360000 bytes transferred in 123.332992 secs (5313744 bytes/sec)

gstat:

dT: 0.501  flag_I 500000us  sizeof 240  i -1
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
      ...
  163   1314      0      0    0.0   1314   5258  145.4  100.0| da0
  163   1314      0      0    0.0   1314   5258  146.5  100.0| da0s1

The ECHI controller is:

ehci0: <EHCI (generic) USB 2.0 controller> mem 0xffa80800-0xffa80bff irq 23 at 
device 29.7 on pci0
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <EHCI (generic) USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub4: 8 ports with 8 removable, self powered

I have not made any tests of read performance but from looking at the results 
I do not expect that it will be significantly better than write performance. 
I may do some when I get more time to investigate and follow up if the 
results are unexpected.

Hopefully this will generate some interest in the problem, it is beyond my 
time and expertise but it would be very nice to be able to access MS-DOS 
formatted filesystems at a reasonable speed!

Thank you,
-- 
Dominic
GoodforBusiness.co.uk
I.T. Services for SMEs in the UK.