Two questions --- SSD block sizes and buffering
Polytropon
freebsd at edvax.de
Fri Apr 13 23:06:49 UTC 2018
On Fri, 13 Apr 2018 08:27:14 +0100, Arthur Chance wrote:
> On 12/04/2018 20:06, Polytropon wrote:
> > On Wed, 11 Apr 2018 20:03:07 -0700, Ronald F. Guilmette wrote:
> >>
> >> Two rather simple questions:
> >>
> >> 1) I don't know much, generally speaking, but I have certainly read that
> >> the underyling hardware of flash memory products (which I guess
> >> includes both USB sticks and also SSDs) all effectively have
> >> "physical" block sizes on the order of 128 KiB.
> >>
> >> So anyway, I'm just curious to know to what extent, if any, FreeBSD,
> >> when running with, on, or from any such (flash-memory based) mass
> >> storage device does things specifically with these larger physical
> >> block sizes (of flash memory) in mind.
> >
> > No, it just works(TM). :-)
> >
> >
> >
> >> Does FreeBSD automagically sense that it is dealing with an SSD, and
> >> does it then adjust the way it operates on the relevant filesystem(s)
> >> accordingly, for maximal performance? Or must the system administrator
> >> tweek something explicitly (e.g. tunefs parameters, or perhaps newfs
> >> parameters) in order to get optimal performance on/with an SSD?
> >
> > The fragment size is to be applied by the system administrator
> > when initializing the disk, depending on the block size of the
> > device. For example, newfs allows you to do so:
> >
> > # newfs -m 0 -i 16384 -b 16384 -f 4069 -L somelabel \
> > -t enable -n enable -U /dev/ada0a
> >
> > Note the -f flag; from "man newfs":
> >
> > -f frag-size
> > The fragment size of the file system in bytes. It must be a
> > power of two ranging in value between blocksize/8 and blocksize.
> > The default is 2048 bytes.
>
> Which revision of FBSD are you running? I'm on 10 (must upgrade) and man
> newfs says
>
> -f frag-size
> The fragment size of the file system in bytes. It must be a
> power of two ranging in value between blocksize/8 and blocksize.
> The default is 4096 bytes.
>
> so it's been fixed for 4k disks.
Yes, this has been taken from a 8.x system I was just logged in to.
You know, one of those systems that run and run and run and run and
run and run... obeying the "never touch a running system" rule. :-)
You are correct, 4k became the default frag size, so adjusting it
manually is only needed for specific cases now.
> > But as mentioned in many cases: Deviating from the default
> > values should be done for good and educated reasons, and
> > depending on actual requirements.
> >
> > The example above uses a 4k frag size as typically used
> > with newer hard disks. Additionally, slices, partitions
> > and labels should be adjusted to match block size multiples
> > for better performance.
>
> It's worth noting that by default the first GPT partition (which is
> usually the boot partition) starts at block 34, which isn't a multiple
> of 4k. I always set it to start at block 40 to align with 4k disks and
> start the second partition at block 2048 so it's 1 MB aligned.
Fully agree. Using 1M as factor makes it easy to align the
partitions.
> >> 2) I have written some modest C programs that output lots and lots of
> >> very short little tidbits of data, one per line. In some cases these
> >> programs output millions of lines.
> >>
> >> For reasons that I can explain, these programs explicitly call setvbuf()
> >> on stdout during startup, and they set the buffering type for stdout
> >> to _IOLBF (i.e. line buffering).
> >>
> >> My question is just this: Assme that one of these programs is called
> >> "xyz". Now, if I run the program thusly:
> >>
> >> xyz > xyz.output
> >>
> >> i.e. so that stdout is redirected to a file, will there be one actual
> >> write to disk for each and every line that is written to stdout by xyz?
> >> In other words, will my act of explicitly setting line buffering (for
> >> stdout) in a case like this cause the xyz program to beat the living
> >> hell out of my disk drive?
> >
> > Probably not. The actual write operationg are being issued
> > somewhere "down the line" through the file system driver
> > down to the disk driver. Even a "sync" command issued by
> > the OS will not _immediately_ cause the drive to act.
>
> write(2) calls, which underlie stdio, add the data to the disk block
> image in the VM cache. Unless your machine is under extreme memory
> pressure or you call fsync or sync the buffers eventually get written
> out by a kernel task. See man syncer for details.
>
> >> I hope not, but I'd like to know if it will, or know why it won't, if
> >> it won't.
> >
> > Isn't all output buffered, per default?
>
> There is a magic open flag that prevents it, but that's for those doing
> raw block io to devices.
Interesting pointer, thanks! I think I found it in "man 2 open". :-)
--
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...
More information about the freebsd-questions
mailing list