Giant-free polling [PATCH]

[Pawel Jakub Dawidek]

On Tue, Mar 01, 2005 at 04:29:49PM -0800, Luigi Rizzo wrote:
+> [cc-ing net at freebsd.org to get more opinions]
+> 
+> On Tue, Mar 01, 2005 at 04:42:48PM -0500, John Baldwin wrote:
+> > On Thursday 17 February 2005 07:10 pm, Luigi Rizzo wrote:
+> > > i am no expert about the locking but i see places where
+> > > you grab polling_lock followed by ifnet_lock, and others where
+> > > you use the opposite order. This seems prone to deadlock...
+> > 
+> > Yes, it basically seems that the polling_lock should be removed and just the 
+> > ifnet_lock used because the the ifnet_lock is already always held when the 
+> > polling_lock is locked.

Yeah, but we cannot grap ifnet_lock in polling code, because this is
internal mutex, not visible from outside.
I was thinking about moving ifnet_lock into ifnet structure...

+> this said, if the lock requests are blocking, you basically end
+> up with the polling loops always contending for the locks, with only one
+> doing actual work and the other one always busy-waiting.
+> 
+> Assuming the primitive exists, I think the correct way to implement
+> SMP polling is to use non-blocking locks, something like this
+> (in pseudocode)
+> 
+>     poll_loop() {
+> 	have_polling_lock = try_get_lock(polling_lock)
+> 	foreach(ifp in polled_interfaces) {
+> 	    if (have_polling_lock)
+> 		have_ifp_lock = get_lock(ifp) // returns true
+> 	    else
+> 		have_ifp_lock = try_get_lock(ifp)
+> 	    if (have_ifp_lock) {
+> 		ifp->poll();
+> 		release_lock(ifp)
+> 	    }
+> 	}
+> 	if (have_polling_lock)
+> 	    release_lock(polling_lock);
+>     }
+> 
+> so additional polling processes after the first one will not
+> block on busy interfaces and move forward instead.
+> This should remove a bit of contention, and let separate processes actually
+> work on different interfaces.

I think we should just implement per-interface idlepoll threads, so we can
run polling code on many CPUs for many interfaces.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20050304/e540503f/attachment.bin