[Bug 300580] ministat(1): -c 99.5 uses Student's t critical values for 99.8% confidence

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=300580

            Bug ID: 300580
           Summary: ministat(1): -c 99.5 uses Student's t critical values
                    for 99.8% confidence
           Product: Base System
           Version: CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Many People
          Priority: ---
         Component: bin
          Assignee: bugs@FreeBSD.org
          Reporter: gxenos95@gmail.com

Created attachment 277068
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=277068&action=edit
patch

In usr.bin/ministat/ministat.c, the sixth confidence level is labeled 99.5
(studentpct[], line 31), but the matching column of the t table (lines
32-134) holds the two-sided 99.8% critical values. So "-c 99.5" runs a
stricter test than it reports, and can miss differences that are
significant at 99.5%. The other columns are correct.

Compare with the two-sided columns of a standard t table:
https://en.wikipedia.org/wiki/Student%27s_t-distribution#Table_of_selected_values

For xample: 

  df    ministat "99.5"   99.8%     99.5% (correct)
  1     318.313           318.309   127.321
  4     7.173             7.173     5.598
  inf   3.090             3.090     2.807

To reproduce:

  $ printf '1\n2\n3\n' > a
  $ printf '6.3\n7.3\n8.3\n' > b
  $ ministat -q -c 99.5 a b
  ...
  No difference proven at 99.5% confidence

The attached patch replaces the sixth column with t(0.9975, df), rounded to
three decimals. 

To be clear I found this using an LLM reading a stripped ministat binary from
its raw bytes, then confirmed it in the source.

The exact way it was found is described here if you are interested:
https://sgiannopoulos.github.io/blog/machine-code-programming/

-- 
You are receiving this mail because:
You are the assignee for the bug.