Pi3 out of swap at < 50%

[Ian Lepore]

On Sat, 2018-02-17 at 09:34 -0800, bob prohaska wrote:
> On Sat, Feb 17, 2018 at 09:43:21AM -0700, Ian Lepore wrote:
> > 
> > On Sat, 2018-02-17 at 08:27 -0800, bob prohaska wrote:
> > > 
> > > Running make -j4 buildworld on a Pi3 at r329360 tends to result
> > > in??messages similar to:
> > > 
> > > pid 33492 (llvm-tblgen), uid 0, was killed: out of swap space
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 11764, size: 28672
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 16080, size: 4096
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 15721, size: 20480
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28139, size: 65536
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 40544, size: 4096
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 58384, size: 65536
> > > pid 49735 (c++), uid 0, was killed: out of swap space
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28470, size: 4096
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 24889, size: 8192
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28736, size: 4096
> > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 29736, size: 4096
> > > 
> > > even though swap usage appears to be less than one GB out of two available.
> > > 
> > > So far it's been possible to use the -DNO_CLEAN option with make to pick
> > > up where things left off and I've backed down to -j3 to see if that helps.
> > > 
> > > There are two 1 GB swap partitions, one on USB flash and one on the microSD
> > > card and both appear to be in use according to swapinfo. There are no warnings
> > > in the boot messages and no explicit changes have been made to configuration
> > > files apart from adding the swap entries to /etc/fstab.
> > > 
> > > World and kernel are about two weeks out of sync, so top isn't perfectly
> > > up-to-date with this kernel; could that account for the mismatch between
> > > apparent swap usage and the "out of swap" messages?
> > > 
> > > Thanks for reading, and any ideas!
> > > 
> > > bob prohaska
> > I suspect your swap devices are too slow to keep up with instantaneous
> > demands. ?An sdcard can easily have read and write latencies as big as
> > 30 seconds; they make pretty poor swap devices (have gstat running in
> > another window during the build to see what I mean, look at the ms/r
> > and ms/w columns). ?USB flash drives may not be much better.
> > 
> I'm watching gstat now. With make -j3 running and three cc processes in
> the top of top the usual value of ms/r and ms/w is less than 5 ms. An
> eyeball average is probably less than 10 ms. Before cc got going (make
> dominant in top) there were very brief excursions to 2000 ms, as you posit,
> but they didn't last and didn't put any errors on the console. 
> 
> The experiment ended when make stopped with:
> 
> /usr/src/lib/libc/rpc/auth_time.c:115:10: error: unknown type name 'endpoint'
> free_eps(endpoint eps[], int num)
>          ^
> /usr/src/lib/libc/rpc/auth_time.c:143:8: error: unknown type name 'nis_server'
> static nis_server *
>        ^
> /usr/src/lib/libc/rpc/auth_time.c:144:49: error: unknown type name 'nis_server'
> get_server(struct sockaddr_in *sin, char *host, nis_server *srv,
>                                                 ^
> /usr/src/lib/libc/rpc/auth_time.c:145:5: error: unknown type name 'endpoint'
>     endpoint eps[], int maxep) 
> 
> among others.
> 
> I'll update sources and restart buildworld with -j4, watching gstat more
> carefully. 
> 
> Thanks very much for the hint!
> 
> bob prohaska

Part of the problem is clang-tblgen, which requires HUGE amounts of
ram; if you've gotten beyond that in the build, you may not see any
more problems.

-- Ian