Audit pipe "interest" expression

[Robert Watson]

FreeBSD 7.x's audit pipe facility currently provides a direct pipe of audit 
record data as generated; the set of records received is limited by three 
factors:

- Global audit selection settings in /etc/security.

- Ability for the audit pipe reader to keep up with audit record flow (the
   queue is lossy).

- The audit pipe reader providing sufficiently large buffers to hold complete
   audit records (a requirement which may change).

This is quite useful, but raises a specific problem: if intrusion detection 
interests associated with audit pipe consumers don't align with global audit 
requirements, you either end up with insufficient information being fed to the 
intrusion detection application, or unnecessary information going into the 
global audit trail.  The obvious solution is to allow audit pipe consumers to 
receive records selected using different criteria than the global trail.  This 
would add an additional criteria to the above list:

- The audit record has been selected based on per-pipe selection criteria.

Then the obvious question becomes: what facility for criteria specification 
should the audit pipe facility provide?  The global selection facility, in the 
style of Sun's BSM, allows selection of records based on audit identity, event 
class, and success/failure status of the event.  Events are mapped to event 
classes in order to provide a level of indirection and avoid administrators 
from having to futz with individual event types, which are often opaque to 
everyone but application, and possibly system, developers.

One possibility is to extend this facility to allow each audit pipe to specify 
its own independent criteria along the same lines: for example, allowing IDS 
tools to request mappings of {class mask, success/failure, audit ID} which 
will match events.  This would rely on the existing globally defined classes, 
and have fairly minimal implementation requirements.

Another more complex possibility is to allow audit pipes to each have their 
own mask context, or to allow pipe consumers to additionally specify direct 
event requirements.  This would require significantly more state management 
and a more complex configuration mechanism.

My leaning, for now, is to implement only the first.  Record pre-selection 
points that currently exist would be enhanced to:

- Examine audit pipe requirements in addition to current global requirements.

- Tag accepted records with information on which consumer (trail, pipes, etc)
   had selected the record, so that the record is only delivered to requesting
   consumers.

This raises an additional issue: drop behavior under overload.  The existing 
audit trail facility will cause system stalls on overload, maintaining a 
maximum queue size for delivery to the file system.  This maintains audit 
trail reliability in the presence of load.  Intrusion detection consumers may 
not be interested in this behavior, and may prefer that records be dropped. 
The current audit pipe facility takes the approach of providing a fixed (but 
configurable) maximum queue depth for each pipe; when the pipe buffer is 
filled, new records are dropped.  It has been suggested that allowing pipe 
consumers to specify priorities would permit a more informed drop decision: 
i.e., keep login/logout records in preference to stat() system calls.  This 
makes a lot of sense for audit pipe consumers who track system events and 
maintain state associated with the running system -- for example, 
back-tracking socket communications to the executed command that lead to the 
communications.  One possible approach here would be to extend the selection 
facility to allow priorities to be assigned to records, not just a simple 
boolean match.

Comments on the above thoughts would be most welcome; I hope to spend some 
time working on adding an audit pipe selection facility in the next few weeks. 
In particular, I'd welcome information on intrusion detection consumers of 
audit data and their requirements -- confirmation that the lossy approach is 
good, thoughts on a precedence mechanism, and thoughts on what selection 
criteria are important to application writers.  The current BSM selection 
mechanism is derived from the evaluation requirements of the Orange Book and 
CAPP.  These don't necessarily match current real-world application 
requirements for IDS and monitoring, so advice along those lines would be 
great!

I'd especially like to thank Martin Fong at SRI for his input and feedback 
relating to audit pipes, as his interest has driven current thinking about 
audit pipe selection facilities.

Robert N M Watson