UMA alloc with max items < potential items in per processor buckets

[Sebastian Huber]

Hello,

I don't use FreeBSD directly, but instead a port of the FreeBSD network 
stack to RTEMS. So, this problem may not apply to FreeBSD. I observed 
the following problem during an UDP socket create. They are allocated 
from the following zone:

void
udp_init(void)
{

     in_pcbinfo_init(&V_udbinfo, "udp", &V_udb, UDBHASHSIZE, UDBHASHSIZE,
         "udp_inpcb", udp_inpcb_init, NULL, UMA_ZONE_NOFREE,
         IPI_HASHFIELDS_2TUPLE);
     V_udpcb_zone = uma_zcreate("udpcb", sizeof(struct udpcb),
         NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE);
     uma_zone_set_max(V_udpcb_zone, maxsockets);
     EVENTHANDLER_REGISTER(maxsockets_change, udp_zone_change, NULL,
         EVENTHANDLER_PRI_ANY);
}

In my setup maxsockets is 32. This is probably artificially small 
compared to a real FreeBSD machine. The system has 24 processors, so we 
need 128 * 24 items for the per processor cache buckets. This is 
considerably lager than the single keg of the zone can deliver. Thus in 
case a processor without a per processor bucket tries to do a 
uma_zalloc(V_udpcb_zone, M_NOWAIT | M_ZERO), then it will get no item if 
other processors already consumed all items for their per processor 
cache buckets. I adjusted the uma_zone_set_max() like this

diff --git a/freebsd/sys/vm/uma_core.c b/freebsd/sys/vm/uma_core.c
index 845c433..f88d3d3 100644
--- a/freebsd/sys/vm/uma_core.c
+++ b/freebsd/sys/vm/uma_core.c
@@ -2903,6 +2903,10 @@ uma_zone_set_max(uma_zone_t zone, int nitems)
         uma_keg_t keg;

         ZONE_LOCK(zone);
+#ifdef SMP
+       if (nitems < zone->uz_count * mp_maxid)
+               nitems = zone->uz_count * mp_maxid;
+#endif
         keg = zone_first_keg(zone);
         keg->uk_maxpages = (nitems / keg->uk_ipers) * keg->uk_ppera;
         if (keg->uk_maxpages * keg->uk_ipers < nitems)

and this seems to fix the problem. Is there an implicit assumption in 
FreeBSD that the zone max is always large enough to provide items for 
the per processor cache buckets?

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.