RPI3 swap experiments

[Trev]

Jun 29 20:20:35 rpi3 kernel: pid 66844 (c++), uid 0, was killed: out of 
swap space

At the time of "out of swap" only 157M of the 4G swap was in use. The 
4GB swap partition is on a 4GB USB2 memory key (da1p1), everything else 
is on the sdcard. There are no excessively long ms/w being recorded. 
There are no USB or other related errors.

top and gstat stats leading up to the process being killed are below.

last pid: 66931;  load averages:  3.23,  3.14,  3.25  up 0+03:18:04 
20:20:16
47 processes:  3 running, 44 sleeping

Mem: 647M Active, 25M Inact, 22M Laundry, 188M Wired, 97M Buf, 19M Free
Swap: 4044M Total, 136M Used, 3908M Free, 3% Inuse
     1     15      0      0    0.0     15    607   65.0   97.4  mmcsd0s2a
     4     17      7     56    2.0     10    120    8.9    4.2  da1p1

last pid: 66936;  load averages:  3.29,  3.16,  3.25  up 0+03:18:08 
20:20:20
47 processes:  4 running, 43 sleeping

Mem: 664M Active, 5732K Inact, 27M Laundry, 184M Wired, 97M Buf, 19M Free
Swap: 4044M Total, 138M Used, 3906M Free, 3% Inuse
     1     13      0      0    0.0     13    527   58.8   74.7  mmcsd0s2a
     4     34      2     20    2.9     32   2187   40.4   40.8  da1p1

last pid: 66941;  load averages:  3.35,  3.17,  3.25  up 0+03:18:13 
20:20:25
47 processes:  3 running, 44 sleeping

Mem: 644M Active, 524K Inact, 58M Laundry, 183M Wired, 97M Buf, 15M Free
Swap: 4044M Total, 140M Used, 3904M Free, 3% Inuse
     0      3      0      0    0.0      3     96   98.0   29.1  mmcsd0s2a
     5     42      0      0    0.0     42   2205   88.9   99.8  da1p1

last pid: 66946;  load averages:  3.32,  3.17,  3.25  up 0+03:18:17 
20:20:29
47 processes:  3 running, 44 sleeping

Mem: 601M Active, 420K Inact, 96M Laundry, 184M Wired, 97M Buf, 20M Free
Swap: 4044M Total, 150M Used, 3893M Free, 3% Inuse
     1      1      0      0    0.0      1     96  169.5   16.9  mmcsd0s2a
     0    102      5     32  158.3     97   1357   38.2   93.8  da1p1

last pid: 66951;  load averages:  3.38,  3.18,  3.26  up 0+03:18:21 
20:20:33
47 processes:  3 running, 44 sleeping

Mem: 271M Active, 339M Inact, 89M Laundry, 184M Wired, 97M Buf, 18M Free
Swap: 4044M Total, 157M Used, 3887M Free, 3% Inuse
     0      7      0      0    0.0      7    223   79.4   55.4  mmcsd0s2a
     5      0      0      0    0.0      0      0    0.0    0.0  da1p1

[OUT OF SWAP OCCURS HERE at 20:20:35 - 2 secs after the above and 2 secs 
before the below]

last pid: 66957;  load averages:  3.34,  3.18,  3.25  up 0+03:18:25 
20:20:37
47 processes:  3 running, 44 sleeping

Mem: 233M Active, 680K Inact, 296M Laundry, 184M Wired, 98M Buf, 187M Free
Swap: 4044M Total, 157M Used, 3887M Free, 3% Inuse
     0      0      0      0    0.0      0      0    0.0    0.0  mmcsd0s2a
     6      0      0      0    0.0      0      0    0.0    0.0  da1p1