RPI3 swap experiments

[bob prohaska]

On Sun, Jul 01, 2018 at 09:42:15AM +1000, Trev wrote:
> bob prohaska wrote on 30/06/2018 01:51:
> > On Fri, Jun 29, 2018 at 10:28:25PM +1000, Trev wrote:
> >>
> >> Jun 29 20:20:35 rpi3 kernel: pid 66844 (c++), uid 0, was killed: out of
> >> swap space
> >>
> >> At the time of "out of swap" only 157M of the 4G swap was in use. The
> >> 4GB swap partition is on a 4GB USB2 memory key (da1p1), everything else
> >> is on the sdcard. There are no excessively long ms/w being recorded.
> >> There are no USB or other related errors.
> >>
> > 
> > That's a little surprising, and disturbing. By my experience your setup
> > should have run to completion.
> > 
> > Do you see a "too much swap" complaint? I don't know if it's a problem,
> > but 4 GB of swap triggers one on my RPi3. 3 GB does not. 2 GB is enough
> > to finish -j4 buildworld.
> 
> Jun 30 23:32:12 rpi3 kernel: pid 60735 (c++), uid 0, was killed: out of 
> swap space
> 
> No go with 2GB (I deleted the da0p1 swap partition; destroyed da0; then 
> remade it it with a 2GB p1; then rebooted - no "too much swap" 
> complaints). Out of swap with 56M of 2GB used. And this was with make 
> -j2 buildworld.
> 
It just crossed my mind that the successful -j4 buildworlds that used
USB swap (both mechanical and flash) separate from /usr and /var placed
the swap  on a powered USB hub.  It would be exceeding strange if that 
proved helpful, but as things are going....

The other successful -j4 buildworlds put swap on the root microSD card.
I'd rate that a harder experiment, even if there is free space on the card.
My talents with gpart leave much to be desired. If you're properly gifted
it might also be worth trying.

> Here's the top and gstat stats leading up to, at the time of out of swap 
> and just after:
> 
> last pid: 61290;  load averages:  2.42,  2.27,  2.19  up 0+12:23:41 
> 23:32:07
> 33 processes:  2 running, 31 sleeping
> 
> Mem: 609M Active, 296K Inact, 95M Laundry, 176M Wired, 97M Buf, 20M Free
> Swap: 2048M Total, 47M Used, 2001M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      4     68      0      0    0.0     68   1487   20.9   36.7  da0p1
> 
> last pid: 61294;  load averages:  2.42,  2.27,  2.19  up 0+12:23:42 
> 23:32:08
> 33 processes:  2 running, 31 sleeping
> 
> Mem: 612M Active, 180K Inact, 94M Laundry, 176M Wired, 97M Buf, 19M Free
> Swap: 2048M Total, 49M Used, 1999M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      4    110      0      0    0.0    110   3653   24.3   77.8  da0p1
> 
> last pid: 61298;  load averages:  2.42,  2.27,  2.19  up 0+12:23:43 
> 23:32:09
> 33 processes:  2 running, 31 sleeping
> 
> Mem: 614M Active, 248K Inact, 90M Laundry, 176M Wired, 97M Buf, 20M Free
> Swap: 2048M Total, 53M Used, 1995M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      0     28      0      0    0.0     28   2737  157.7  113.7  da0p1
> 
> last pid: 61302;  load averages:  2.42,  2.27,  2.19  up 0+12:23:45 
> 23:32:11
> 33 processes:  2 running, 31 sleeping
> 
> Mem: 322M Active, 297M Inact, 88M Laundry, 176M Wired, 97M Buf, 17M Free
> Swap: 2048M Total, 55M Used, 1993M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      5      0      0      0    0.0      0      0    0.0    0.0  da0p1
> 
> last pid: 61306;  load averages:  2.42,  2.27,  2.19  up 0+12:23:46 
> 23:32:12
> 33 processes:  1 running, 32 sleeping
> 
> Mem: 326M Active, 164K Inact, 379M Laundry, 176M Wired, 97M Buf, 19M Free
> Swap: 2048M Total, 56M Used, 1992M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      6      0      0      0    0.0      0      0    0.0    0.0  da0p1
> 
> last pid: 61310;  load averages:  2.39,  2.27,  2.19  up 0+12:23:47 
> 23:32:13
> 33 processes:  1 running, 32 sleeping
> 
> Mem: 331M Active, 184K Inact, 379M Laundry, 176M Wired, 97M Buf, 14M Free
> Swap: 2048M Total, 56M Used, 1992M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      6      0      0      0    0.0      0      0    0.0    0.0  da0p1
> 
> [OUT OF SWAP at 23:32:14 captured immediately below]
> 
> last pid: 61315;  load averages:  2.39,  2.27,  2.19  up 0+12:23:48 
> 23:32:14
> 33 processes:  1 running, 32 sleeping
> 
> Mem: 78M Active, 176K Inact, 315M Laundry, 174M Wired, 97M Buf, 332M Free
> Swap: 2048M Total, 56M Used, 1992M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      0     48     37    710  126.2     11   1186  341.9   26.4  da0p1
> 
> last pid: 61320;  load averages:  2.39,  2.27,  2.19  up 0+12:23:49 
> 23:32:15
> 33 processes:  2 running, 31 sleeping
> 
> Mem: 55M Active, 848K Inact, 1644K Laundry, 174M Wired, 97M Buf, 669M Free
> Swap: 2048M Total, 48M Used, 2000M Free, 2% Inuse
>      0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
>      0      0      0      0    0.0      0      0    0.0    0.0  da0p1

The delays look much better than I've seen on successful -j4 buildworlds.