i386/68719: [usb] USB 2.0 mobil rack+ fat32 performance problem

Fri May 27 05:30:07 PDT 2005

The following reply was made to PR i386/68719; it has been noted by GNATS.

From: Dominic Marks <dom at goodforbusiness.co.uk>
To: bug-followup at FreeBSD.org,
 banhalmi at field.hu,
 freebsd-fs at freebsd.org
Cc:  
Subject: Re: i386/68719: [usb] USB 2.0 mobil rack+ fat32 performance problem
Date: Fri, 27 May 2005 13:28:57 +0100

 (Posted to freebsd-fs as the PR is assigned to freebsd-usb@, but it seems to 
 be more related to the msdos filesystem than the USB system so perhaps it 
 should be reassigned?)

 I've been evaluating the performance of some usb2 hard discs with FreeBSD and 
 I found this PR (68719). The submitter is correct that performance with 
 msdosfs is severely limited.

 I tested a 'LaCie' USB2 disc:

 da0 at umass-sim0 bus 0 target 0 lun 0
 da0: <Maxtor 7Y250P0 YAR4> Fixed Direct Access SCSI-2 device 
 da0: 40.000MB/s transfers
 da0: 239372MB (490234752 512 byte sectors: 255H 63S/T 30515C)

 egg# diskinfo -t da0
  ...
 Seek times:
         Full stroke:      250 iter in   5.271879 sec =   21.088 msec
         Half stroke:      250 iter in   4.055049 sec =   16.220 msec
         Quarter stroke:   500 iter in   6.696545 sec =   13.393 msec
         Short forward:    400 iter in   2.316910 sec =    5.792 msec
         Short backward:   400 iter in   2.052681 sec =    5.132 msec
         Seq outer:       2048 iter in   1.574044 sec =    0.769 msec
         Seq inner:       2048 iter in   1.576574 sec =    0.770 msec
 Transfer rates:
         outside:       102400 kbytes in   3.445316 sec =    29722 kbytes/sec
         middle:        102400 kbytes in   3.441593 sec =    29754 kbytes/sec
         inside:        102400 kbytes in   3.435809 sec =    29804 kbytes/sec

 I used 10GB chunks of data to test the USB disc. Each test used a different 
 10GB of data to avoid caching distorting results. I made the following 
 measurements with both UFS2 and FAT32.

 1. Local disc copy from a new SATA-150 disc.
 2. Ftp copy over a local 100Mbit network from a server with a SATA-150 disc. 
 3. Create a zero-file using dd to test simple write performance.

 Client with attached USB disc: P4 2.6Ghz 768MB DDR, if_fxp, 1x ATA-100 disc
 Server used for FTP: Celeron 2.4GHz 1.5GB DDR, if_em, 4x SATA-150 discs.

 Both the client and server are running FreeBSD 5.4-STABLE built at
 Thu May 26 22:52:15 BST 2005.

 In test 1 I could not achieve any better than 5.1MB/s on an msdosfs 
 filesystem. Using UFS2 and softupdates a transfer rate of 22~25MB/s was 
 possible. Both test data sets were copied from the systems ATA-100 disc. In 
 both tests at these peaks gstat reports the device is 100% busy.

 A snapshot from gstat(8) during test 1. da0s1 is the fat32 filesystem.

 dT: 0.501  flag_I 500000us  sizeof 240  i -1
  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
     2     90     90   3363    5.2      0      0    0.0   38.2| ad0
     2     90     90   3363    5.2      0      0    0.0   38.4| ad0s1
       ...
     2     90     90   3363    5.3      0      0    0.0   39.0| ad0s1g
    96   1295      2      8  163.9   1293   5170  141.6   99.8| da0
    96   1295      2      8  163.9   1293   5170  143.1   99.9| da0s1

 In test 2 again the msdosfs filesystem could not achieve higher than 5MB/s. 
 With UFS2 the limit of the network was reached before the limit of the USB2 
 bus so the transfer was limited to 10.5MB/s average. During this period gstat 
 reported about 35-45% activity on the device which matches up as I would have 
 expected.

 I managed to improve the performance in these results a little by upping 
 MAXPHYS to 256, and then to 512 on the client. Going from 128 to 256 improved 
 the diskinto -t transfer rates by about 3MB/s increasing it to 512 seemed to 
 have no further benefit. Enabling polling for the fxp interface helped as 
 well by reducing the interrupt rate from ~8k/s to 2k/s during the second 
 test.

 Finally, I used dd to test just the filesystem-write.

 ufs2:

 egg# dd if=/dev/zero of=/mnt/file.test bs=64k count=10000
 10000+0 records in
 10000+0 records out
 655360000 bytes transferred in 25.093943 secs (26116262 bytes/sec)

 And from gstat during the `dd':

 dT: 0.501  flag_I 500000us  sizeof 240  i -1
  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
       ...
     2     50      2     32   47.8     48  24510   45.7   97.8| da0
     2     50      2     32   47.9     48  24510   45.7   97.8| da0s1

 msdosfs:

 egg# dd if=/dev/zero of=/mnt/file2.test bs=64k count=10000
 10000+0 records in
 10000+0 records out
 655360000 bytes transferred in 123.332992 secs (5313744 bytes/sec)

 gstat:

 dT: 0.501  flag_I 500000us  sizeof 240  i -1
  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
       ...
   163   1314      0      0    0.0   1314   5258  145.4  100.0| da0
   163   1314      0      0    0.0   1314   5258  146.5  100.0| da0s1

 The ECHI controller is:

 ehci0: <EHCI (generic) USB 2.0 controller> mem 0xffa80800-0xffa80bff irq 23 at 
 device 29.7 on pci0
 usb4: EHCI version 1.0
 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
 usb4: <EHCI (generic) USB 2.0 controller> on ehci0
 usb4: USB revision 2.0
 uhub4: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
 uhub4: 8 ports with 8 removable, self powered

 I have not made any tests of read performance but from looking at the results 
 I do not expect that it will be significantly better than write performance. 
 I may do some when I get more time to investigate and follow up if the 
 results are unexpected.

 Hopefully this will generate some interest in the problem, it is beyond my 
 time and expertise but it would be very nice to be able to access MS-DOS 
 formatted filesystems at a reasonable speed!

 Thank you,
 -- 
 Dominic
 GoodforBusiness.co.uk
 I.T. Services for SMEs in the UK.