[TEST] make -j patch [take 2]
Ruslan Ermilov
ru at freebsd.org
Fri Nov 12 10:57:35 PST 2004
On Fri, Nov 12, 2004 at 03:52:37PM +0100, Poul-Henning Kamp wrote:
> In message <20041112130548.GA86162 at ip.net.ua>, Ruslan Ermilov writes:
>
> >> At the very least, do not commit your patch until you have managed
> >> to come up with at least one instance of real world data where it
> >> is a good idea.
> >>
> >I gave you one example already, and Harti gave you another.
> >
> >"make universe" builds 6 worlds today. By starting it with
> >"make -j12", you build 6 worlds in parallel, and another 6
> >jobs are made available for building these worlds. When one
> >world finishes, 7 jobs will be available for building 5 other
> >worlds, etc. This is fine as long as you're interested in
> >the final result of "make universe".
>
> You have not actually _tried_ running make universe with
> my patch, have you ?
>
> I thought so :-(
>
> make -j 12 universe will make 12 jobs available. The main
> make will grab six of these to start the buildworlds, and
> those six buildworlds compete for the remaining 6 tokens.
>
> The result is that those six buildworlds progress _in parallel_
> each of them using as many tokens as they can lay their hands on
> at any one time.
>
This is exactly what I said above. I well understand that
six buildworlds will compete for six remaining slots, then
five buildworld will compete for seven slots, and so on.
> If you want them to have more slots, say "-j 18" instead
> and they will have 12 slots to compete for.
>
> This is, as far as I can tell, just the thing you want to happen,
> only it doesn't suffer from pointless compartementalization
> of resources such as in your proposal.
>
No, this is not what I want. :-)
> The crucial point is that if at one particular instance the
> i386.buildworld needs only one token, it will only claim
> one token, and the other jobs can get more tokens and start
> more things. Once the i386 needs to run more things in
> parallel, it will compete for tokens again and run as much
> in parallel as it can get tokens for.
>
I well understand this.
> In your scenario, the tokens not used by i386 cannot be
> used by the other buildworlds and they are wasted.
>
Not wasted, they just don't share the same pool of jobs. ;)
> >If you're by chance familiar with how you can ask Oracle to
> >optimize your queries, you can either optimize for ALL ROWS or
> >for FIRST ROWS, and it's up to you to decide on this. The
> >"rows" in "make universe" are results of doing buildworld
> >and buildkernel, and you have control over this.
>
> This is not a database.
>
> This is not even a cartesian problem space.
>
I'm interested in completing i386 buildworld as fast as
possible. In "make -j12 universe", the i386 world will
be built in approximately (TOTAL) seconds (let's assume
for the moment that we don't build kernels). If I use
"make universe JFLAG=-j12", i386 world will complete in
(TOTAL / 6) seconds. If I use "make -j2 universe
JFLAG=-j6", i386 and pc98 worlds will complete in
(TOTAL / 6 * 2) seconds. Can you see a difference?
I'm interested in "first rows". And I'm not saying
I'm always interested in them, just want to be able
to test my patches as quickly as possible. ;)
> >Do you disagree with anything of the above?
>
> Yes, I disagree with pretty much all of it. It is clear that
> you have not actually tried running with the code which is in
> the tree.
>
Why? It's clear that I understand how it works, and
you fail to see _my_ point. ;)
> >Can you agree to the below patch? It doesn't change anything
> >except when JFLAG is set explicitly to request another level
> >of parallelism for building individual worlds and kernels?
>
> The patch is pointless, but harmless once people learn not to set
> JFLAG.
>
> But I will not agree to this patch _until_ you show me _actual_ _real
> _life_ data that shows it performs better than just running without
> a JFLAG at all.
>
You disagree that the i386 part of "make universe" will
complete faster in my scenario, while still providing an
overall limit of 12 jobs? I understand the drawback of
my approach -- that the total time for "make universe"
will be longer in my case.
I now feel like everything that should have been said
was said, and this thread should die. ;)
Cheers,
--
Ruslan Ermilov
ru at FreeBSD.org
FreeBSD committer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-current/attachments/20041112/c60cf625/attachment.bin
More information about the freebsd-current
mailing list