sensors framework continued (architecture)

[Alexander Leidinger]

Quoting "Poul-Henning Kamp" <phk at phk.freebsd.dk> (Sat, 10 Nov 2007 22:54:41 +0000):

> In message <20071110203229.64021d85 at deskjail>, Alexander Leidinger writes:
> >Quoting "Poul-Henning Kamp" <phk at phk.freebsd.dk> (Sat, 10 Nov 2007 10:04:22 +0000):
> >
> >> >This
> >> > implies that we have to move the polling intervall code for non-event driv
> >> >en sensors from the userland to the kernel.
> >> 
> >> No it does not.
> >> 
> >> It would be prefectly sensible to have an ioctl(2) (or write(2)!)
> >> that says "poll this sensor now" to keep control in userland if
> >> that is desired.
> >>
> >> Some sensors however, are hardware timed, and for those it makes
> >> more sense to be able to tell them "we want one report every second"
> >> and not worry more about it.
> >
> >Two kinds of behavior come into my mind when I read this.
> >
> >First behavior ("poll this sensor now"): If the userland application
> >wants to poll the non-event driven sensors (or sensors which are not
> >hardware timed), it has to issue 2 syscalls for each sensor, one to let
> >it poll (ioctl), and one to read the data. The polling interval is
> >handled by the userland.
> >
> >What I like here is, that the polling interval handling is in the
> >userland for non-event driven and non-hardware-timed sensors (let's
> >call those "simple" sensors). What I don't like is that we have to issue
> >more than one syscall.
> 
> But you forget that sensors may have considerable "conversion" time.
> It is a benefit for us to be able to start the sensor and not block
> on the syscall waiting for it to do its thing.

How do you know from the approach you propose when to read out the
newly polled data without blocking in some syscall? How many such
sensors per system does it require to make this a problem? How do you
quantify "a problem"? Can this problem be circumvented by userland code
instead (maybe some tunable amount of threads or some other way)?
I don't say your proposal is bad, currently I still think you are in
the premature optimization territory with this part of your proposals
(moving this part of the polling responsibility into the kernel), and
we haven't seen hard data or good arguments against the premature
optimization argument yet.

> >> >So to sum it up it looks like the architecture looks as follows:
> >> > - using sysctls for centralized exporting of sensor data from kernel
> >> >   to userland (polling interface for userland polling code)
> >> 
> >> I think you are jumping to conclusions here.
> >
> >As written, nobody came up with strong arguments (actually there where
> >no replies at all) to Roberts mail.
> 
> You are reading waaay too much into Roberts email and overlooking
> that a fd based kernel interface can also be MIB based.

I don't overlook the MIB based part. I see that we get the MIB part via
sysctls for free, and that we have to write a filesystem with all the
bells and whistles VFS needs to work, to get this MIB based part via
the fd approach.

> >> sysctl is by definition limited to polling, which, for any significant
> >> number of sensors, becomes very expensive compared the ability of
> >> an fd-based approach to queue and batch data.
> >
> >See my caching argument above.
> 
> Which just adds to the mess, because then you have to also pass
> along a timestamp to tell how old the returned value is.

For events it doesn't matter, it's the current state. If you haven't
got the initial state transition, you didn't start the monitoring early
enough. For smart sensors the max time offset is known from the
configuration and it also represents the "current" value. If you want
the "real" current value, you need to tell the smart sensor to forget
his hardware timed polling and do the poll "now". In typical monitoring
software (be it open source or commercial) you don't give a acquired
data value a different date stamp than "now". If you have the
requirement for this, you are not in a generic monitoring environment
anymore, but in a specialized application specific situation, which
deserves its own special monitoring program, which should acquire the
sensor data specific to the service delivery requirements on its own.
We don't try to solve a specific application/serve problem here (and
get rid of all specialized 3rd party software which is specific to a
special service delivery problem like, e.g., monitoring a nuclear power
plant), we try to make it more easy to monitor your every day computing
environment. I see benefits for your proposals for special
environments (if this is your main target for the kernel sensors
framework), but it complicates the system in 99% of the real world
application use of this framework. I don't think it is a good idea to
put that much code into the kernel for just 1% (estimated, probably
it's much less) of the use cases, when you can write specialized
software in userland for this 1% (remember, the single-system sensors
framework is supposed to handle userland stuff, and this estimated 1%
of smart sensors fits nicely into this part instead of into the kernel
sensors framework).

> There are a lot of downsides to sysctls which you do not seem
> to be aware of, locking, access time, overhead and several other
> issues makes sysctl very undesirable for anything which isn't
> "ad-hoc" or really seldomly used.

You need locking for the filesystem approach too. For the access time
part I proposed another way of handling it in userland instead of the
kernel (which also works for a fd based approach). For the overhead
part the simple sysctl approach looks more lightweight than the
writting a filesystem approach, feel free to show it isn't. What are
the other issues?

Bye,
Alexander.

-- 
http://www.Leidinger.net  Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org     netchild @ FreeBSD.org  : PGP ID = 72077137