svn commit: r251024 - in head: lib/msun lib/msun/ld128 lib/msun/ld80 lib/msun/src tools/regression/lib/msun

[Garrett Cooper]

On May 27, 2013, at 12:58 PM, Ryan Stone wrote:

> On Mon, May 27, 2013 at 1:23 PM, David Schultz <das at freebsd.org> wrote:
> 
>> If someone were to write or import a good test framework,
>> get all the tests to use it, and put together a test tinderbox, that
>> would have a lot of impact.
>> 
>> 
> People are working on bringing ATF into FreeBSD:
> 
> https://wiki.freebsd.org/TestingFreeBSD

Thanks for the plug!

Yes, ATF is in (it's a bit outdated so I'll have to submit a patch for ATF 0.17), build infrastructure's on its way, and the best news of all is that I've already ported a large chunk of the msun testcases from NetBSD to FreeBSD. They work with little hacking for the most part. Example:

# cd /usr/src/lib/msun/tests/
# make obj depend all check

Summary for 19 test programs:
    295 passed test cases.
    0 failed test cases.
    0 expected failed test cases.
    5 skipped test cases.

*** The verbatim output of atf-run has been saved to /usr/obj/usr/src/lib/msun/tests/atf-run.log
#

Why isn't this in FreeBSD yet (along with some of the other testsuites from NetBSD)? Well, a couple of reasons:

1. The test build infrastructure changes haven't been fully pushed in (Marcel's helping me out with the latest set of patches).
2. There are issues with porting stuff over from NetBSD because some of the macros needed to minimize porting divergence don't have a happy home yet. I gently poked several people from FreeBSD, NetBSD, and OpenBSD about getting some degree of solidarity and agreement on macros, but I haven't heard back from anyone I emailed for several weeks. So I figure that FreeBSD needs to bend by adding some [compat] macros to strategic locations in sys/sys, and then we can work on standardizing things later. I really don't want to hack all of the tests to use libnetbsd because while it works for nmtree and nmakefs, it doesn't make sense for testcases as the volume of required changes wouldn't make sense from a porting perspective.
3. I really REALLY need help porting over changes we've added that can be contributed back to NetBSD, and FreeBSD specific tests need to be cordoned off properly.

As far as the test image stuff is concerned and test tinderbox stuff is concerned, yes, it's on my radar.. I'm working on installing tests via .iso images and USB images; once that's done we can work on making the other testing stuff work on a cron basis via tinderbox or another similar mechanism.

So, what say ye to making things work better and helping me in this process :)?

Thanks!
-Garrett