DESTDIR implementation [Was:: ATTENTION: is the way DESTDIR was introduced completely wrong?]

[Gábor Kövesdán]

John E Hein wrote:
> Gábor Kövesdán wrote at 21:44 +0200 on Aug 23, 2006:
>  > Kris Kennaway wrote:
>  > > mount_nullfs ${PORTSDIR} ${DESTDIR}${PORTSDIR}
>  > > mount_nullfs ${WRKDIR} ${DESTDIR}${WRKDIR}
>  > > mount_devfs foo ${DESTDIR}/dev
>  > > chroot ${DESTDIR} cd ${.CURDIR} && make install
>  > >
>  > > A suitable version of the above should allow all ports to be installed
>  > > into a jail-ready filesystem hierarchy, while requiring 0 port
>  > > changes.
>  >
>  > Ok, I think it's time to follow this way, but have to make some parts 
>  > clearer. For your last line, it should be
>  > 
>  > chroot ${DESTDIR} cd ${.CURDIR} && make ${.TARGETDIR}
>
> I assume you mean .TARGET
>   
Sure, TARGETDIR is just stuck in my mind. :)
>
>  > since we want to run all targets chrooted. We could put this part into 
>  > an .if defined(DESTDIR) block before the targets, but I don't know how 
>  > to prevent running the further code, since we don't want to reach 
>  > do-install in the host environment, but only in the chroot. I think of 
>  > exit 0, if that's correct, or what else is better.
>  > So, what I mean:
>  > 
>  > .if defined(DESTDIR)
>  > .BEGIN   # We need this if not in a target
>  >     ${MOUNT_NULLFS} ${PORTSDIR} ${DESTDIR}${PORTSDIR}
>  >     ${MOUNT_NULLFS} ${WRKDIR} ${DESTDIR}${WRKDIR}
>  >     ${MOUNT_DEVFS} foo ${DESTDIR}/dev
>  >     ${CHROOT} ${DESTDIR} cd ${.CURDIR} && ${MAKE} ${.TARGETDIR}
>  >     exit 0
>  > .endif
>
> Each line with a command in a makefile is it's own shell.
> So 'exit 0' will start a shell and exit that shell.
> It will not prevent make from continuing.
>
> 'exit 1' (or anything that returns a non-zero code, like false) will
> stop make (unless -k or -i or similar is specified), but that's not
> what you want.
>
>
> make(1) is not a very good tool for this kind of thing (a script would
> probably be better), but here is one way to do this kind of two stage
> make (untested, but illustrates the point):
>
> bsd.port.mk would have something like this:
>
> STAGE?=     1
> .if ${STAGE} == 1 && defined(DESTDIR) && !empty(DESTDIR)
>
> .MAKEFLAGS: STAGE=2
>
> .BEGIN:
> 	${MOUNT_NULLFS} ${PORTSDIR} ${DESTDIR}${PORTSDIR}
> 	${MOUNT_NULLFS} ${WRKDIR} ${DESTDIR}${WRKDIR}
> 	${MOUNT_DEVFS} foo ${DESTDIR}/dev
> 	${CHROOT} ${DESTDIR} cd ${.CURDIR} && ${MAKE} ${.TARGET}
>
> .else
>  .
>  .
>  rest of bsd.port.mk
>  .
>  .
> .endif
>
> You could put the first part into a bsd.destdir.mk and .include it
> to make it look cleaner.
>
> Note there are other complications like unmounting the nullfs and
> devfs.
>   
What about renaming bsd.port.mk to bsd.build.mk and creating a 
bsd.destdir.mk. At the same time, we could make a "lightweight" 
bsd.index.mk, which would only contain things that are really necessary 
for index building. It could make the index building faster, couldn't 
it? The new bsd.port.mk would be something like this then:

STAGE?=     1
.if target(index)
.include <bsd.index.mk>
.elif ${STAGE} == 1 && defined(DESTDIR) && !empty(DESTDIR)
.include <bsd.destdir.mk>
.else
.include <bsd.build.mk>
.endif

This method would offer better modularization, as I mentioned the index 
building example. The current bsd.port.mk file is huge and it is 
included every time by port Makefiles, but we doesn't actually need 
this, e.g. index building requires only snippets from bsd.port.mk.

Comments?
I'm interested in the portmgr standpoint about this as well, before I 
start to work on it. I did some useless work in the last round, so I 
want to discuss things better this time.

>
>  > The another issue I find is how we can pass variables to the chrooted 
>  > make. E.g. if we want to set WITH_FOO in command line or in make.conf. 
>  > And note, that we can't just pass everything, since DESTDIR should be 
>  > unset in the chroot, otherwise we would run into infinite loop and it 
>  > would fail due to the non-existent directories.
>
> Really, for these and other reasons, a script would be better.  It
> could be done with make(1) and bsd.port.mk, but it's probably not
> worth it.
>
>   
I think it would be quite complicated with scripts as well. But anyway, 
I'm unsure if we really have to pass such variables. Maybe it is not 
used so often, and can be put into the jail's make.conf as well if 
really necessary. I doubt if it's worth to spam the code with ugly 
workarounds for this.
> Again, if we do decide to go this route, we should remove all mention
> of DESTDIR from Mk/*.mk (rather than pretend to support it, but
> incorrectly).
>   
Agreed. The current progress has to be completely reverted for this, 
only little pieces should be kept. E.g. somebody told that he likes my 
OSVERSION change. It is more rational that the old one.
> Others have mentioned cross building.  Neither proper DESTDIR support
> in bsd.port.mk nor a scripted mount_nullfs & a chroot is going to be
> the one thing to solve that problem.  Some ports will build okay if
> you give them access to cross tools (like ppc-gcc) and tell them what
> target architecture to produce.  Other ports will need to be taught.
> Some ports might need to run intermediate native architecture tools
> produced at build time in order to produce the final cross arch
> product.  It's not as simple as just being able to install to an
> alternate location.  And I don't think it matters much how ports/Mk
> handles DESTDIR.  There are bigger issues.
>   
Cross-building is a hard nut to crack at all...

-- 
Cheers,

Gabor