svn commit: r318250 - in head: etc etc/newsyslog.conf.d etc/syslog.d tools/build/mk

[Julian Elischer]

On 16/5/17 4:11 am, Ian Lepore wrote:
> On Mon, 2017-05-15 at 12:22 -0700, John Baldwin wrote:
>> On Monday, May 15, 2017 11:43:21 AM Ian Lepore wrote:
>>> On Mon, 2017-05-15 at 10:13 -0700, John Baldwin wrote:
>>>> On Saturday, May 13, 2017 10:39:15 AM Warner Losh wrote:
>>>>>
>>>>>>
>>>>>>>
>>>>>>> - It's really easy to screw up a mergemaster call if you
>>>>>>> edit
>>>>>>> the files, and install the stock version which removes the
>>>>>>> edits.
>>>>>> Also, programmatically removing the entries means you have to
>>>>>> bake the metadata into etc/Makefile, which is already
>>>>>> complicated
>>>>>> enough as-is.
>>>>> Why do you care about removing them at all? They are no-ops if
>>>>> the
>>>>> files don't exist. Why not just always install all these files
>>>>> is
>>>>> where I'm going with this...
>>>> I think this is actually the bigger question.  I think it is
>>>> perfectly
>>>> sensible to support conf.d/* files for ports to use and as a way
>>>> to
>>>> manage logs for application logs on an appliance, etc.  However,
>>>> this
>>>> shuffling is a bit of a merge nightmare for anyone using
>>>> mergemaster
>>>> or etcupdate, and the biggest cost is that newsyslog will create
>>>> a
>>>> one-line file in /var/log for entries with 'C'.
>>>>
>>> That's only a good argument for keeping the lines in the monolithic
>>> file if those lines will be ignored when a file in the .conf.d
>>> directory provides conflicting config.  Otherwise my embedded
>>> product
>>> that drops different rules for rotating /var/log/messages into
>>> .conf.d
>>> STILL has to programmatically edit the monolithic file to remove
>>> the
>>> standard rule(s).
>> Now you have to programmatically edit the file in
>> conf.d/foo.  However,
>> by this argument the monolithic conf file shouldn't even exist.  The
>> current approach is a half-way mix with the worst of both models it
>> seems.
>>
> Programmatically editing a single file containing only config for a
> single component typically means just rewriting the entire file with
> your new contents.  In particular, you don't need to attempt to
> preserve other information, the format of which you may not even know,
> including free-form comments and who knows what-else.
>
>> Also, _you_ could just splat an empty /etc/newsyslog.conf file on
>> your
>> appliance and create a bunch of conf.d/foo files if that is easier
>> for
>> you to use on an appliance.  The files we ship in a release aren't
>> really
>> tailored for an appliance (I've yet to see an appliance that doesn't
>> use
>> a FooBSD with local patches).  OTOH, the existing setup is probably
>> simpler to manage for an out-of-the-box install.
>>
>> I'm also suprised you don't manage the newsyslog.conf file yourself
>> rather than trying to edit and merge in upstream changes?  That is, I
>> can see a few approaches:
>>
> You seem to be picturing some sort of etcupdate kind of thing.  I'm
> more talking about a GUI or other config-management tool within an
> embedded product that has to edit or rewrite configuration on the fly
> based on user choices.
>
> Of course, separate files does also simplify the update process, for
> the most part.  If a new subsystem is added in a new freebsd release, I
> have zero work to do to upgrade a system in the field if that new
> subsystem just drops a new file into a .conf.d directory.  If it has
> new entries in a monolithic file, then I do have to do some sort of
> merge/edit operation.
>
>> 1) Keep your real newsyslog.conf / syslogd.conf files in your
>> FooBSD's
>>     VCS and when newsyslog.conf changes upstream you merge that in the
>>     way you normally merge changes.
>>
>> 2) Move the "vendor" newsyslog.conf out entirely and install your own
>>     versions of these files either as a monolithic assembled by config
>>     management rules or a bunch of conf.d/foo files (here I would
>> probably
>>     opt for separate files).
>>
>> However, your approach doesn't seem to describe either of these since
>> this commit doesn't impact those work flows (if 1), you would have
>> already
>> made any local changes you need and if anything merging this commit
>> gives
>> you the kind of merge conflicts people will get on the next
>> mergemaster /
>> etcupdate for non-appliance boxes, or if 2) you ignore these files)
>>
> This seems to be an argument for everyone doing for themselves the
> operation of splitting the distributed monolithic file into finer
> grained files, and re-performing that operation (or at least the
> analysis part of it) on every update.
We (Panzura) use the directory /etc/newsyslog.conf.d and have a separate
panzura.conf in there that we write out in one hit with all our log 
files in it.
we rewrite it all (from our internal metadata) if it needs to change.
We also have individual files for optional components that do their 
own logging,
so it is definitely important to us to have the ability to do the 
small files but I don't
really see a need to have a bunch of files for standard log files.
I think files should be grouped in "installable unit" atomicity.
and at this time the installable unit for the base system is the 
entire base unit..

Now when 'pkgbase' hist and we install our system from N separate pkg 
units the
argument will be different. It also needs to be decided what happens 
it multipl modules
use a shared file, or a single pkg has different options that require 
different log files..
> In general a lot of this feels like "I only needed 6 big config files
> to control my whole system in 1988, and so I should only need those
> same 6 files now."  Sure, all us old-timers have the finger memory for
> editing rc.conf and syslog.conf and so on, but how often do you crack
> open syslog.conf with the plan of editing 12 different lines in it at
> once?  Because the main objection to .conf.d directories seems to be
> that there are more files to edit, and that just doesn't feel like a
> big problem in actual daily use.
>
> -- Ian
>
>