poudriere: build on on server, rsync to another

[Dan Langille]

On Fri, Sep 27, 2019, at 10:44 AM, Matthew Seaman wrote:
> On 27/09/2019 15:26, Dan Langille wrote:
> > Hello,
> > 
> > I am working on putting a new poudriere build server into production at work.
> > 
> > At present, we build packages on one server & deploy them from another server.
> > 
> > The package build server mounts an NFS share from /usr/local/poudriere/data/packages
> > 
> > The package distribution server mounts that same NFS share in RO-mode. All hosts install from this server.
> > 
> > I want to change this. The goal:
> > 
> > * poudriere builds to local disk
> > * /usr/local/poudriere/data/packages will be rsync'd later to NFS
> > * the build & rsync process will not be serial/linked (i.e. a build will not trigger an rsync)
> > 
> > My concern: will this work? Will the package distribution server get incomplete sets of packages?
> > 
> > Thank you.
> > 
> 
> If you enable the ATOMIC_PACKAGE_REPOSITORY setting in poudriere.conf, 
> then you won't see inconsistencies because you're rysnc'ing while 
> poudriere is in the middle of re-building the repository.
> 
> You might see inconsistencies because someone is trying to install 
> packages while you're in the middle of rsyncing an updated repository. 
> However, the atomic package repository mechanism works by juggling 
> symbolic links at the top of the /usr/local/poudriere/data/packages 
> directory tree -- if you arrange things to rsync any subdirectories 
> first, and then finally synch over the symbolic link bits, you should be 
> golden.

This is pretty much what we came up with on Twitter, but not so quickly. ;)

> Although, just sticking a webserver onto your poudriere build box would 
> likely be easier overall and less effort to maintain.

Yes, I think so. We do have about 1TB of free disk space on this host.

Thinking further on this: one reason we install from another host is 
avoid blocking package maintenance on other hosts when the build server
undergoes maintenance.  This is something we can think about.

Thank you.


-- 
  Dan Langille
  dan at langille.org