A shell script test case

[Peter Holm]

On Thu, Mar 06, 2014 at 07:51:46AM -0800, Garrett Cooper wrote:
> On Mar 5, 2014, at 4:33 AM, Peter Holm <peter at holm.cc> wrote:
> 
> > I tried my hand with a test scenario for a current FreeBSD VM problem:
> > At first I did not understand what I did wrong when I got:
> > 
> > $ kyua test vm_pageout_test
> > vm_pageout_test:__test_cases_list__  ->  broken: Tester did not exit
> > cleanly: kyua-atf-tester: Test program list did not return success
> > [0.000s]
> > 
> > 0/1 passed (1 failed)
> > Committed action 15
> > $ 
> > 
> > but I figured it out.
> > 
> > Now to the real question: Are regression tests like this one really
> > ATF material? I would think that the long runtime could be prohibitive?
> 
> You should set the timeout variable to something other than 300 if it will take longer to execute (from atf-test-case(4)):
> 

I should have included the Makefile. I used:
"TEST_METADATA.vm_pageout_test+=     timeout="1200"

>      timeout            Type: integral.  Optional; defaults to `300'.
> 
>                         Specifies the maximum amount of time the test case can
>                         run.  This is particularly useful because some tests
>                         can stall either because they are incorrectly coded or
>                         because they trigger an anomalous behavior of the pro-
>                         gram.  It is not acceptable for these tests to stall
>                         the whole execution of the test program.
> 
>                         Can optionally be set to zero, in which case the test
>                         case has no run-time limit.  This is discouraged.
> 
> > $ kyua debug vm_pageout_test:basic
> > FAIL
> > UID  PID PPID CPU PRI NI   VSZ  RSS MWCHAN STAT TT     TIME COMMAND
> >   0  662    1   0  52  0 10268   0 pause  IWs   -  0:00.00 nfsuserd: master (nfsuserd)
> >  25  845    1   0  20  0 12796   0 pause  IWs   -  0:00.00 sendmail: Queue runner at 00:30:00 for /var/spool/clientmqueue (sendmail)
> >   0  850    1   0  20  0 10024   0 nanslp SWs   -  0:00.00 /usr/sbin/cron -s
> > 1001 4069 4068   0  20  0 11808   0 wait   IWs   0  0:00.00 -bash (bash)
> >   0 4115 4069   0  20  0 10496   0 wait   IW    0  0:00.00 su
> >   0 4116 4115   0  23  0 10772   0 pause  IW    0  0:00.00 _su (csh)
> > 1001 4354 4353   0  20  0 11808   0 wait   IWs   1  0:00.00 -bash (bash)
> >   0 4416 4354   0  20  0 10496   0 wait   IW    1  0:00.00 su
> >   0 4417 4416   0  23  0 10772   0 pause  IW    1  0:00.00 _su (csh)
> >   0 4419 4417   0  20  0 11808   0 wait   IW    1  0:00.00 bash
> >   0 6838 4419   0  52  0 13412   0 wait   IW+   1  0:00.00 kyua debug vm_pageout_test:basic
> >   0 6857 6838   0  52  0  9900   0 wait   IW    1  0:00.00 /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=false -vunprivileged-user=tests /usr/tests/sys/vm/vm_pageout_test basi
> > vm_pageout_test:basic  ->  failed: Failed
> > $ kyua test vm_pageout_test      
> > vm_pageout_test:basic  ->  failed: Failed  [367.941s]
> 
> Why this took 67 seconds (367 total) to fail beyond the standard timeout (300) is a bit puzzling/concerning. I expect there to be an additional 1-5 seconds of needed cleanup, but 67 seems a bit high...
> 

Sorry about the confusion. It's the "sort /dev/zero" that take up most
of the time; Just a simple way of generate some VM usage.

Anyway I think I have come to the conclusion that these kind of
resource hogging tests probably do not belong in ATF.
But as "my first shell script test" it was a success :)

> Thanks!
> -Garrett

-- 
Peter