[Testers wanted] /dev/console cleanups

[Nate Eldredge]

On Thu, 20 Nov 2008, Jeremy Chadwick wrote:

> On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote:
>> On Wed, 19 Nov 2008, Jeremy Chadwick wrote:
>>
>>> On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote:
>>
>>>> I hope that never gets committed - it will make debugging kernel
>>>> problems much harder.  There is already a kern.msgbuf_clear sysctl and
>>>> maybe people who are concerned about msgbuf leakage need to learn to
>>>> use it.
>>>
>>> And this sysctl is only usable *after* the kernel loads, which means
>>> you lose all of the messages shown from the time the kernel loads to
>>> the time the sysctl is set (e.g. hardware detected/configured).  This is
>>> even less acceptable, IMHO.
>>
>> But surely you can arrange that the contents are written out to
>> /var/log/messages first?
>>
>> E.g. a sequence like
>>
>> - mount /var
>> - write buffer contents via syslogd
>> - clear buffer via sysctl
>> - allow user logins
>
> This has two problems, but I'm probably missing something:
>
> 1) See my original post, re: users of our systems use "dmesg" to find
> out what the status of the system is.  By "status" I don't mean "from
> the point the kernel finished to now", I literally mean they *expect*
> to see the kernel device messages and all that jazz.  No, I'm not
> making this up, nor am I arguing just to hear myself talk (despite
> popular belief).  I can bring these users into the discussion if people
> feel it would be useful.

I forgot about that point.  I can sympathize with those users; I 
feel the same way.  It's a good way to learn about a system as a 
mere user (since usually sysadmins don't remember or bother to 
disable it).

However, in my experience dmesg isn't really the best thing for that 
purpose; the kernel message buffer tends to get wiped out once the system 
has been up for a while.  (It fills with ipfw logs, ethernet link state 
changes, etc.)

Maybe a better approach would be to point them to /var/log/messages 
or whichever log file stores them permanently.  Or, better yet, do some 
syslogd magic to make a logfile that can be appropriately readable but 
doesn't have any overly sensitive messages directed there (e.g. kernel 
yes, sshd no).

> 2) I don't understand how this would work (meaning, technically and
> literally: I do not understand).  How do messages like "CPU: Intel(R)
> Core(TM)2 Duo CPU E8400 @ 3.00GHz (2992.52-MHz K8-class CPU)" get
> written to syslog when syslogd isn't even running (or any filesystems)
> mounted at that time?  There must be some magic involved there (since
> syslog == libc, not syscall) when syslogd starts, but I don't know
> how it works.

I think you're conflating a couple of things, and I also explained my idea 
poorly.

As I understand it (from memory, which is a little vague), syslogd gets 
messages from two places: from the kernel via /dev/klog, and from other 
processes via a Unix domain socket in /var/run.  These messages then get 
sent to the appropriate log files.  The syslog(3) function of libc just 
connects and writes the message to the Unix domain socket.  If syslogd 
isn't running to listen on that socket, syslog(3) won't work very well.

Now /dev/klog should be a direct line to the kernel's message buffer.  So 
when syslogd starts and reads /dev/klog for the first time, it will get 
everything that's accumulated so far, including the earliest boot 
messages.  It should then write them to the appropriate log files.  This 
already works, which is why /var/log/messages contains the kernel 
copyright message, etc.

My idea is, after syslogd does this, but before the system goes 
multi-user, you should clear the kernel buffer.  Early boot messages are 
already in the log files, so they won't be lost.  Maybe the best thing 
would be to build this functionality into syslogd itself, to minimize the 
possibility of losing messages due to a race.

>> This way the buffer is cleared before any unprivileged users get to do
>> anything.  No kernel changes needed, just a little tweaking of the init
>> scripts at most.
>>
>> If you should have a crash and suspect there is useful data in the
>> buffer, you can boot to single-user mode (avoiding the clear) and
>> retrieve it manually.
>>
>> Seems like this should make everyone happy.
>
> What I'm not understanding is the resistance towards Rink's patch,
> assuming the tunable defaults to disabled/off.

It seems reasonable to me.  The only catch I can see is that if you have a 
crash and you want to look at the message buffer after rebooting, you'll 
have to remember to stop at the loader prompt and turn off that tunable. 
Which might be easy to forget in the heat of the moment.

-- 

Nate Eldredge
neldredge at math.ucsd.edu