Is maximum swap usage tunable?

[Warner Losh]

On Sun, Mar 4, 2018 at 11:46 AM, Ian Lepore <ian at freebsd.org> wrote:

> On Sun, 2018-03-04 at 11:35 -0700, Warner Losh wrote:
> > On Sun, Mar 4, 2018 at 11:28 AM, bob prohaska <fbsd at www.zefox.net>
> wrote:
> >
> > >
> > > On Sat, Mar 03, 2018 at 08:26:05AM -0800, bob prohaska wrote:
> > > >
> > > >
> > > > Is there some sort of experiment which can distinguish hardware
> delays
> > > > from software delays? For example, would logging gstat output shed
> any
> > > > light?
> > > >
> > > For lack of any better ideas, I tried running
> > > make -j2 -DNO_CLEAN buildworld > buildworld.log && make -j2 -DNO_CLEAN
> > > KERNCONF=
> > > ZEFOX buildkernel > buildkernel.log
> > >
> > > while also running
> > > gstat -a -B -I 10s > j2_gstat.log & in another ssh session
> > >
> > > In due course the console reported
> > >
> > > FreeBSD/arm64 (www.zefox.org) (ttyu0)
> > >
> > > login: Mar  4 09:28:30 www kernel: pid 9310 (c++), uid 0, was killed:
> out
> > > of swap space
> > >
> > > as expected.
> > >
> > > However, a grep of j2_gstat  revealed a maximum write delay of 30ms/w
> > > for swap on microSD.
> > >
> > > Swap on USB flash is slower, but still generally under 100 ms.
> > > Only a handful of widely spaced delays exceeded 200 ms/w.
> > >
> > gstat doesn't tell you the worst-case. It tells you the average of the
> > requests. This can (and does) hide very long outlier behavior which
> drives
> > the crazy delayed swap messages.
> >
> > The worst-case events were
> > >
> > > dT: 10.002s  w: 10.000s
> > >  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
> > >
> > >     0      6      0      0    0.0      6    113   14.6    3.3  da0b
> > >     0      4      0      0    0.0      4     48   29.0    3.1  da0b
> > >     0      9      5     79    3.0      5     47    7.9    2.6  da0b
> > >     4      8      0      0    0.0      8     99   67.5   32.5  da0b
> > >     0      1      0     13    5.6      1     28   5674   88.3  da0b
> > >     0      0      0      0    0.0      0     38   18.6    0.3  da0b
> > >     0      1      1      9    2.8      0      0    0.0    0.2  da0b
> > >     0      1      1     26    5.1      0      0    0.0    0.4  da0b
> > >     0      0      0      3    2.6      0      0    0.0    0.1  da0b
> > >     0      1      1      9  161.8      0      0    0.0   14.3  da0b
> > >
> > > No "indefinite delay" warnings were presented on the console.
> > > uname -a reports r329893, sources are at 330383.
> > >
> > > I hope this is useful information,
> > >
> > Despite my quibble over what you're measuring (I look at this stuff all
> the
> > time for Netflix), I think this is quite useful.... Thanks!
> >
>
> I think it would be more useful with a -I1 instead of 10.  Including
> the -d flag might also be useful, since deleting is what's so slow on
> flash-based devices (I'm not sure if swapping triggers BIO_DELETE
> stuff, but other filesystem activity does).
>

BIO_DELETE is generated only by filesystems. Nobody has thought through the
best strategies to swap to flash. The swap pager doesn't issue BIO_DELETE
at all.

Warner