m_pkthdr.rcvif dangling pointer problem

[Gleb Smirnoff]

On Sun, Jul 24, 2011 at 09:43:59AM +0100, Robert N. M. Watson wrote:
R> Instead, I think we should go for a more radical notion, which is a bit harder to implement in our stack: the network stack needs a race-free way to "drain" all mbufs referring to a particular ifnet, which does not cause existing processing to become more expensive. This is easy in some subsystems, but more complex in others -- and the composition of subsystems makes it all much harder since we need to know that (to be 100% correct) packets aren't getting passed between subsystems (and hence belong to neither) in a way that races with a sweep through the subsystems. It may be possible to get this 99.9% right simply by providing a series of callbacks into subsystems that cause queues to be walked and drained of packets matching the doomed ifnet. It may also be quite cheap to have subsystems that "hold" packets outside of explicit queues for some period (i.e., in a thread-local pointer out of the stack) add explicit invalidation tests  (i.e., for IFF_DYING) before handing off to prevent those packets from traversing into other subsystems -- which can be done synchronisation-free, but still wouldn't 100% prevent the race
R> 
R> Just to give an example: netisr should offer a method for netisr_drain_ifnet(struct ifnet *) that causes netisr to walk all of its queues to find matching packets and free them. Due to direct dispatch and thread-local queues during processing, netisr should also check IFF_DYING before handing off.
R> 
R> If we do that, I wonder how robust the system then becomes...? This may not be too hard to test. But I'd rather we penalise ifnet removal than, say, the IP input path when it needs to check a source interface property.

What if some thread (e.g. netisr) have taken an mbuf off the queue, stored pointer
to it on stack, and got rescheduled. Then an ifnet has departured, all callbacks
were called, all queues cleaned up from pointers. Then a thread with that mbuf on 
stack continues to process. ?

-- 
Totus tuus, Glebius.