Running tests as a developer prior to commit

[Julio Merino]

On Feb 26, 2015, at 22:17, Craig Rodrigues <rodrigc at freebsd.org> wrote:
> 
> On Mon, Feb 23, 2015 at 7:20 PM, Ed Maste <emaste at freebsd.org> wrote:
> 
>> 
>> This is precisely why I'd like to find a way to make it easy to run a
>> (rather small) subset of tests, if a developer desires to do so. The
>> alternative is that they decide it's too awkward to run the tests, and
>> just don't bother. Even in that case we're still better off today
>> (since the Jenkins task will catch it) than we were some time ago
>> before consistent test infrastructure existed. But it seems we ought
>> to avoid the latency, mailing list churn, and masking effect of a
>> broken test run if we can facilitate the developer running a set of
>> tests that they've identified.
>> 
> 
> If I could do:
>   cd /usr/src/bin/ls/
>   make test
> 
> and have it run the unit tests associated with /bin/ls, that would be a
> good start.

That already exists but is apparently broken as Garrett mentions.  Should be fixed.  I may take a look if I have time.

> It wouldn't be perfect, because if a change to /bin/ls, affects some other
> test
> in /usr/src/usr.bin/foobar/ , this wouldn't catch it.  You would have to do
> something
> more complicated by building an expert system which can compute the
> dependencies
> in the tree and run the appropriate tests.  But having that is better than
> nothing.

Right, and this is why the existing test target outputs a fat warning before startup.  But as you say, having this working in some form would be sufficient to validate most changes quickly.

> 
> As far as I understand things, the way the unit tests are integrated in the
> Makefile
> infrastructure, the tests need to be staged to /usr/tests/ in order to be
> run.

Not really.  The programs _and_ their corresponding Kyuafile need to be built, but once built, the majority of them can be run from the object directory (be it separate from the source directory or not).