Accounting changes

[Diomidis Spinellis]

Robert Watson wrote:
> 
> On Mon, 9 Apr 2007, Diomidis Spinellis wrote:
> 
>> I'd like your permission to perform a change during the slush period.
>>
>> Following some gentle prodding by bde, I'm working on an overhaul of 
>> the accounting system.  The reason for the change is that time values 
>> with AHZ == 64 (1/64 second) are very often zero on modern machines.  
>> The change involves storing acct(5) time values as proper IEEE-487 
>> floats with AHZ being 1000000 (i.e storing \mu s).  For the change 
>> I'll touch the following files/directories:
>>
>> sys/sys/acct.h
>> sys/kern/kern_acct.c
>> usr.sbin/sa
>> usr.bin/lastcomm
>> share/man/man5/acct.5
>> UPDATING
>>
>> The change will render old daily accounting files incompatible with 
>> the new commands and formats.  Because the summary databases don't 
>> store any version information, the old versions of these will also 
>> become unusable. Therefore users will need to clear 
>> /var/account{acct*,savacct,usracct}. Apparently, this problem has 
>> occurred multiple times in the past, when we changed the size of 
>> elements like uid_t and dev_t.
> 
> What do you think of the idea of changing the file format a little to 
> include a short file header at the front, and that the first field of 
> that head is zero-filled u_int32_t, and the second a version number?  
> Right now, the first field of the acct structure is the name of the 
> command, which will always be a non-nul string, so always have a first 
> character non-nul.  If we see non-nul data in the file header's first 
> field, we use the old structure layout, and otherwise we check the 
> version number and use the new layout?  This would provide backwards 
> compatibility for reading old accounting data, which I would think would 
> generally be desirable, and allow us to explicitly version the file in 
> the future.

There are three types of accounting files: acct(5), which contains 
records written by the kernel, and two others containing sa(8) 
aggregated results of acct per user and per command.  I'd prefer to keep 
acct(5) free of version data.  This 1) simplifies the kernel, which 
would otherwise have to write version data each time the file is turned 
over, 2) is consistent with current practice for storing such files, and 
3) simplifies the processing programs, which can process data at 
arbritrary locations in the file. (lastcomm reads records backwards, and 
recently I updated it to also work on the output of tail -f). 
Typically, these files are aggregated and turned over daily; IMO doing 
this also over an OS upgrade isn't a big deal.

The aggregated files are different.  They are dbm-based, and their 
structure is opaque.  I could therefore easily add a special "file 
version" record to them.  My concern is how to handle multiple file 
versions over a long time period.  Hard-coding legacy file formats in 
the code is inelegant.  I'd prefer to add a text-based import/export 
capability, so that administrators can migrate data by exporting it 
before an OS upgrade and importing it afterwords.

Even that could be an overkill.  Based on the problems Bruce and I have 
seen in the accounting system, my impression is that not many people use 
it nowadays.  Once/if we get dtrace in FreeBSD, I'd prever to scrap t 
existing accounting infrastructure and re-implement it on top of that.

Diomidis - http://www.spinellis.gr