locking in a device driver

[Scott Long]

M. Warner Losh wrote:
> In message: <4360DD7B.20900 at samsco.org>
>             Scott Long <scottl at samsco.org> writes:
> : Dinesh Nair wrote:
> : > 
> : > carrying on this discussion, what would be a good locking mechanism to 
> : > use to protect tsleep() and other sensitive areas in a driver in freebsd 
> : > 4.x ?
> : > 
> : > the current code for the driver in 5.x uses mtx_lock and mtx_unlock with 
> : > some parts even being protected by mtx_lock(&Giant).
> : > 
> : > would the use of simple_lock() or s_lock() do, given that 
> : > SIMPLELOCK_DEBUG was defined in the 4.x kernel ?
> : >
> : > the mechanism is actually a pseudo device driver which communicates with 
> : > the real device driver. the pseudo device driver creates a bunch of 
> : > /dev/ devices which the userland reads/writes to, and the pseudo device 
> : > driver then places data in a few buffers.
> : > 
> : > the real device driver then reads these buffers and uses busdma to send 
> : > the data to the device. reading is done by using busdma to read from the 
> : > device and then placing the data in these buffers for the pseudo device 
> : > to return to the userland process.
> : > 
> : > locking in the real device driver uses splhigh/splx, but what locking 
> : > should be used in the pseudo device driver ?
> : > 
> : 
> : If you need to protect your pseudodriver from being interrupted by the
> : real driver then you'll need to use the same spl() as the driver.  Note
> : that you shouldn't be using splhigh() unless you really know what you
> : are doing.  Other than that, there likely isn't anything that you need
> : to do for 'locking' in 4.x.  The kernel is non-reentrant there, so you
> : don't need to worry about synchronizing multiple threads.
> 
> One thing to also bear in mind is that in 4.x spl locking is a code
> lock.  It keeps multiple 'threads' of execution from entering a block
> of code.  mutexes in -current are data locks.  While usually one can
> think of the two the same, it can trip the unweary up.
> 
> Locking in 4.x is indeed much simpler.
> 
> Warner

I wouldn't characterize spls that way.  An spl keeps top-half code from 
being preempted by an interrupt that would cause bottom half code to 
run.  It's more of a special critical section that a code serializer.
It's big advantage is that it doesn't mask out all interrupts, just
the ones that you want, and it's much more light weight on x86 that
doing explicit cli/sti instructions.  It's the BGL spinlock that keeps
multiple processes from executing the top half at the same time, and
there is no control over that; it's just 'there'.  The synchronization
guarantees that you have in the 4.x kernel are:

1.  Only one process will be executing in the kernel at a time. 
Multiple processes might be blocked at the same time, but only one
will be executing, regardless of the number of CPUs.

2.  Only one interrupt handler will execute at a time, and while it is
executing there will not be any top half code executing on any other CPUs.

3.  Interrupt handlers can preempt a process executing in the kernel 
unless the appropriate spl mask/level is set.

Scott