Writing contigiously to UFS2?

[Bruce Evans]

On Wed, 26 Sep 2007, I wrote:

> ... Even doubling the [block] size wouldn't make much
> difference.  I see differences of at most 25% going the other way and
> halving the block size twice, which halves the cg size 4 times: on ffs1:
>
>    4K blocks, 512-frags -e 512  (broken default):     40MB/S
>    4K blocks, 512-frags -e 1024 (broken default):     44MB/S
>    4K blocks, 512-frags -e 2048 (best), kernel fixes: 47MB/S
>    4K blocks, 512-frags -e 8192 (try too hard), kernel fixes
>       (kernel fixes are not complete enough to handle this case;
>       defaults and -e values which are < the cg size work best except
>       possibly when the fixes are complete):          45MB/S

[Max possible is 52 MB/S.  1MB = 10^6 bytes.]

All of these must have been with some kernel fixes.  Retesting with -current
gives 33MB/S with -e 512 and a max of 42MB/S with each of -e 1024, 2048 and
3072.  Reducing the -e parameter below 512 gives surprisingly (if you don't
remember how slow seeks can be) large further losses.  E.g., -e 128 gives
21MB/S and the following bad layout:

% fs_bsize = 4096
% fs_fsize = 512
% [bpg = 3240, maxbpg = 128]
% 4:	lbn 0-11 blkno 624-719
% 	lbn [<1>indir]12-1035 blkno 608-615

The first indirect block is laid out discontiguously.  This is a standard
bug in reallocblks.

% 	lbn 12-127 blkno 720-1647

The blocks pointed to by the first indirect block are laid out contigously
with the last direct block.  Reallocblks does extra work to move the
indirect block out of the way so that only the data blocks are contiguous.

% 	lbn 128-255 blkno 1744-2767
% 	lbn 256-383 blkno 2864-3887

There is a bug after every maxbpg = 128 blocks. Due to a hack,
ffs_blkpref_ufs1() handles the blocks pointed to by the first indirect
block specially (maxbpg doesn't work for them), and due to a bug
somewhere, it leaves a gap of 2864-2768 = 96 blkno's (frags) after
every maxbpg blocks.

% 	lbn 384-511 blkno 3984-5007
% 	lbn 512-639 blkno 5104-6127
% 	lbn 640-767 blkno 6224-7247
% 	lbn 768-895 blkno 7344-8367
% 	lbn 896-1035 blkno 8464-9583

It keeps leaving gaps of 96 blkno's until the end of the indirect block.

% 	lbn [<2>indir]1036-1049611 blkno 207368-207375
% 	lbn [<1>indir]1036-2059 blkno 207376-207383
% 	lbn 1036-1163 blkno 207824-208847

Now ffs_blkpref_ffs1() skips 7 cg's (1 cg = 3240 blocks = 8*3240 =
25920 frags) because it doesn't know the current cg and its best guess
is off by a factor of 8 due to our maxbpg being weird by a factor of
8.  Normally it only skips 1 cg due to the default maxbpg being wrong
by a factor of 2.

% 	lbn 1164-1291 blkno 259664-260687
% 	lbn 1292-1419 blkno 311504-312527
% 	lbn 1420-1547 blkno 363344-364367
% 	lbn 1548-1675 blkno 415184-416207
% 	lbn 1676-1803 blkno 467024-468047
% 	lbn 1804-1931 blkno 518864-519887
% 	lbn 1932-2059 blkno 570704-571727

Indirect blocks after the first are not handled specially, so a new
cg is preferred after every maxbpg blocks, and some other bug causes
1 cg to be skipped for every maxbpg blocks.

% 	...

The pattern continues for subsequent indirect blocks.  The layout is only
unusual for the first one, so the pessimizations for the first one have
little effect for large files -- for large files, the speed is dominated
by seeking every 128 blocks as requested by -e 128.

Bruce