help on testing for FreeBSD...

John-Mark Gurney jmg at
Fri Oct 31 22:25:51 UTC 2014

Garrett Cooper wrote this message on Fri, Oct 31, 2014 at 14:52 -0700:
> On Oct 24, 2014, at 16:08, John-Mark Gurney <jmg at> wrote:
> > John-Mark Gurney wrote this message on Wed, Oct 15, 2014 at 17:52 -0700:
> >> NGie Cooper wrote this message on Wed, Oct 15, 2014 at 17:46 -0700:
> >>> On Wed, Oct 15, 2014 at 5:38 PM, John-Mark Gurney <jmg at> wrote:
> >>>> Garrett Cooper wrote this message on Wed, Oct 15, 2014 at 17:25 -0700:
> >>>>> Hey jmg@ (-testing CCed)!
> >>>> 
> >>>> Sure, np...
> >>>> 
> >>>>> On Oct 15, 2014, at 17:15, John-Mark Gurney <jmg at> wrote:
> >>>>> 
> >>>>>> Hello,
> >>>>>> 
> >>>>>> I'm working on testing parts of OpenCrypto.  I have committed some work
> >>>>>> to p4 at:
> >>>>>>
> >>>>>> 
> >>>>>> But it requires a few files to be installed...  I'm not sure where the
> >>>>>> best location to install these are and now best to install them?
> >>>>> 
> >>>>> I have a couple questions:
> >>>>> - Where would the source live in the tree?
> >>>> 
> >>>> Which source are you talking about?  The python source? or the source
> >>>> for the input data files?  If you mean the later, it's where they will
> >>>> be committed...  They are taken from NIST's CAVP program unchanged...
> >>>> I COULD possibly write a complicated rules to fetch and extract the
> >>>> files, but decided not to...
> >>> 
> >>> The bulk majority of the driver source (.c, .h, etc).
> >> 
> >> The code that I'm testing is in src/sys/opencrypto (and there are other
> >> drivers that can get tested too, but it still goes through the opencrypto
> >> code)...  Hence why I put them in tests/sys/opencrypto...  If that's the
> >> wrong location, let me know and I can move them...
> >> 
> >>>>> - Why are they written in python?
> >>>> 
> >>>> Because I'm too lazy to write code in C...  I had already written
> >>>> framework code in Python, and C doesn't have a standard function to
> >>>> decode hex.. :)  And error handling it much easier in python...
> >>> 
> >>> I ask because we don't have a means for doing unittest level execution
> >>> right now and not being written in something that's in base or
> >>> installed automatically (perl for instance) would mean that your test
> >>> code wouldn't be run by default on jenkins runs.
> >> 
> >> Yeh...  Though as a kld will have to be loaded, and a sysctl set as
> >> root, there's other issues to having it automaticly run as part of
> >> jenkins...
> > 
> > Hello, any help w/ this?  I haven't received a response on how to
> > integrate my code..  If I don't get help w/ this soon, I will add it
> > to src/tools/regression, and call it a day...
> Sorry for not responding sooner. I?m off today, so I?ll be around the usual channels on #efnet if you need to track me down.

Basicly, need help on where to install files that aren't executables,
but are necessary for the tests to run... stimulus files and python
module files..  If you want to take a look at what I have so far, go
ahead and look at the link above to the p4 depot..

Is anyone going to be at MeetBSD that could help me with this??  Then
we could sit down and knock this out...

> Also, I spotted this in passing while trying to see if we had any arc4random testcases?
> This might be more worthwhile to integrate than pursing a custom test infrastructure/tests route as it?s a part of NetBSD?s open crypto testsuite (there?s probably going to be some API differences, but they should be relatively small).

Oh, didn't see NetBSD's, but their tests aren't very expansive.. only
a single test vector for GCM, no tests for XTS, no tests for 3DES, and
7 tests for SHA1-HMAC...  Good thing I looked at this.. our SHA1-HMAC
has a bug in that it only allows 20 byte keys instead of arbitrary
length keys..  The test vectors for NIST doesn't have any 20 byte
keys... Opps... If we fix that, we'd have 1575 tests...  So, even though
the NetBSD tests would be useful to integrate, it's clear that they
don't overlap much...

Also, OpenBSD has some tests too, but they mostly compile code directly
from the kernel instead of going through /dev/crypto...  Which makes
it harder for us to use the tests...

Thanks for the help!

  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."

More information about the freebsd-testing mailing list