A shell script test case

[Garrett Cooper]

On Mar 5, 2014, at 4:33 AM, Peter Holm <peter at holm.cc> wrote:

> I tried my hand with a test scenario for a current FreeBSD VM problem:
> At first I did not understand what I did wrong when I got:
> 
> $ kyua test vm_pageout_test
> vm_pageout_test:__test_cases_list__  ->  broken: Tester did not exit
> cleanly: kyua-atf-tester: Test program list did not return success
> [0.000s]
> 
> 0/1 passed (1 failed)
> Committed action 15
> $ 
> 
> but I figured it out.
> 
> Now to the real question: Are regression tests like this one really
> ATF material? I would think that the long runtime could be prohibitive?

You should set the timeout variable to something other than 300 if it will take longer to execute (from atf-test-case(4)):

     timeout            Type: integral.  Optional; defaults to `300'.

                        Specifies the maximum amount of time the test case can
                        run.  This is particularly useful because some tests
                        can stall either because they are incorrectly coded or
                        because they trigger an anomalous behavior of the pro-
                        gram.  It is not acceptable for these tests to stall
                        the whole execution of the test program.

                        Can optionally be set to zero, in which case the test
                        case has no run-time limit.  This is discouraged.

> $ kyua debug vm_pageout_test:basic
> FAIL
> UID  PID PPID CPU PRI NI   VSZ  RSS MWCHAN STAT TT     TIME COMMAND
>   0  662    1   0  52  0 10268   0 pause  IWs   -  0:00.00 nfsuserd: master (nfsuserd)
>  25  845    1   0  20  0 12796   0 pause  IWs   -  0:00.00 sendmail: Queue runner at 00:30:00 for /var/spool/clientmqueue (sendmail)
>   0  850    1   0  20  0 10024   0 nanslp SWs   -  0:00.00 /usr/sbin/cron -s
> 1001 4069 4068   0  20  0 11808   0 wait   IWs   0  0:00.00 -bash (bash)
>   0 4115 4069   0  20  0 10496   0 wait   IW    0  0:00.00 su
>   0 4116 4115   0  23  0 10772   0 pause  IW    0  0:00.00 _su (csh)
> 1001 4354 4353   0  20  0 11808   0 wait   IWs   1  0:00.00 -bash (bash)
>   0 4416 4354   0  20  0 10496   0 wait   IW    1  0:00.00 su
>   0 4417 4416   0  23  0 10772   0 pause  IW    1  0:00.00 _su (csh)
>   0 4419 4417   0  20  0 11808   0 wait   IW    1  0:00.00 bash
>   0 6838 4419   0  52  0 13412   0 wait   IW+   1  0:00.00 kyua debug vm_pageout_test:basic
>   0 6857 6838   0  52  0  9900   0 wait   IW    1  0:00.00 /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=false -vunprivileged-user=tests /usr/tests/sys/vm/vm_pageout_test basi
> vm_pageout_test:basic  ->  failed: Failed
> $ kyua test vm_pageout_test      
> vm_pageout_test:basic  ->  failed: Failed  [367.941s]

Why this took 67 seconds (367 total) to fail beyond the standard timeout (300) is a bit puzzling/concerning. I expect there to be an additional 1-5 seconds of needed cleanup, but 67 seems a bit high...

Thanks!
-Garrett