Very slow writes on flash + msdosfs

[Bruce Evans]

On Fri, 5 Oct 2007, Dmitry Marakasov wrote:

> I have USB flash:
> ...
> But newfs_msdos ran on it is very slow:
>
> % time newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1
> /dev/da0s1: 4072456 sectors in 509057 FAT32 clusters (4096 bytes/cluster)
> bps=512 spc=8 res=32 nft=2 mid=0xf0 spt=63 hds=255 hid=0 bsec=4080447 bspf=3978 rdcl=2 infs=1 bkbs=0xffff
> newfs_msdos -F32 -LAMDmi3 -k 0xffff /dev/da0s1  0,02s user 0,21s system 0% cpu 2:54,37 total

Try 512 bytes/cluster for real slowness.

> ...
> Writing a 1.0Mb directory with 340 files takes 1.5 minutes (up to 300k/sec
> writes => seems like much more data is actually written that it's
> needed).
>
> Larger files behave somewhat better (up to 3 MB/s).
>
> Btw, dd if=/dev/zero of=/dev/da0 bs=512 show the same 23k/s speed as
> newfs_msdosfs.
>
> So where is the problem? Why's there no caching and why's there 1 sector
> writes?

Old versions of msdosfs don't implement clustering.

> PS. I use 6.1, has the situation changed in -CURRENT?

Yes.

However, clustering won't help much for small files, due to BSD's
fundamental design error of per-vnode buffering.  With 340 files in a
1.0MB directory, the average file size is about 3K.  This is smaller
than the block size of 4K, so for most files clustering will have no
effect, and there will be about 340 write accesses for the data alone,
even though the data is written asynchronously (except in the sync-mount
case where everythiung is written synchronously -- this will be much
slower).  There will be at least another 340 write accesses for writing
the directory entry synchronously on creation of the files.  There
will be a few more accesses for writes to the FAT and writes to the
the directory entry for completion of writes to the files.  Another
340 or so accesses may be required after bugs in synchronous update
of metadata a fixed (currently, all FAT updates are asynchronous, but
many should be synchronous unless the file system is mounted sync).

This gives a minimum of 2 or 3 writes per file so if the write speed
is 23k/sec for 512-blocks = 46 transactions/sec, then writing 340 files
will take a minimum of 15-22 seconds.  I don't know why it would take
1.5 minutes with 4K-clusters, but it would take about that long with
512-clusters.

Async mounts would reduce the minimum number of writes per file to
about 1 (for the data block).  msdosfs doesn't implement them yet.

Bruce