CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds

[Jia-Shiun Li]

Updated patch works fine for me surviving -j64 buildworld on
quad-core. Compile time did not differ much, ~35 mins. I 'd guess
that's because build has already been well paralleled inside each
subdirs for ~4 jobs.

It would be even better if someone can stress it on multicore system,
say 12+ cores, to explore higher hw concurrency to see the difference.

Looks other targets using .for loops can be translated to this finer
dependency rules as well? e.g. _bootstrap-tools, probably already on
your todo list?


-Jia-Shiun.


On Wed, May 28, 2014 at 8:27 PM, Ian Lepore <ian at freebsd.org> wrote:
> On Wed, 2014-05-28 at 12:58 +0800, Jia-Shiun Li wrote:
>> It failed cleandir at libmb with -j4. Test script attached and log
>> snippet below.
>>
>> Tested with:
>> - HW: i5-3450 CPU w/ 8GB memory
>> - /usr/obj & src mounted on tmpfs.
>>
>> src uses ~1GB without .svn dir. /usr/obj uses another ~2GB for
>> buildworld, not including buildkernel. If memory is not constrained I
>> think it is easier to use tmpfs to uncover parallel timing/race issues
>> hidden by slower I/O.
>>
>>
>> -Jia-shiun.
>
> Doh!  There was a typo, libmb should have been libmd.  More importantly,
> it shows that my testing didn't test anything at all.  I think I applied
> the patch in one sandbox and then ran make universe in a different one.
> Here's an updated patch (this time I'll try to test it correctly myself
> too).
>
> Thanks for testing.
>
> -- Ian
>