Re: git: ca2a7262df5e - main - Free UMA zones when a pass(4) instance goes away.

From: Kenneth D. Merry <ken_at_FreeBSD.ORG>
Date: Thu, 13 Jan 2022 20:00:14 UTC
On Thu, Jan 13, 2022 at 10:49:08 -0800, Gleb Smirnoff wrote:
>   Kenneth,
> 
> On Thu, Jan 13, 2022 at 03:56:39PM +0000, Kenneth D. Merry wrote:
> K> commit ca2a7262df5ec5fd07d4ac61738947f48c9cd7f2
> K> Author:     Kenneth D. Merry <ken@FreeBSD.org>
> K> AuthorDate: 2022-01-13 15:50:40 +0000
> K> Commit:     Kenneth D. Merry <ken@FreeBSD.org>
> K> CommitDate: 2022-01-13 15:54:56 +0000
> K> 
> K>     Free UMA zones when a pass(4) instance goes away.
> K>     
> K>     If the UMA zones are not freed, we get warnings about re-using the
> K>     sysctl variables associated with the UMA zones, and we're leaking
> K>     the other memory associated with the zone structures.  e.g.:
> K>     
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.size)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.flags)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.bucket_size)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.bucket_size_max)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.name)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.rsize)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.ppera)!
> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.ipers)!
> K>     
> K>     Also, correctly clear the PASS_FLAG_ZONE_INPROG flag in
> K>     passcreatezone().  The way it was previously done, it would have
> K>     had set the flag and cleared all other flags that were set at
> K>     that point.
> 
> Definitely not my area, but I wonder why would we create a zone per
> device? Why not a global zone in the driver?

Good question.  The primary reason is just that I didn't think about doing
it that way.  Most everything else in the CAM peripheral drivers is done on
a per-instance basis.

That said, having separate zones does make it a little easier/clearer to
track down memory leaks.  The zones are created on the first call to
the CAMIOQUEUE ioctl for that instance.  99% of users won't have the zone
created, because they haven't used an application that would touch it.  The
only application in the tree that uses it is camdd(8), when you use the
pass I/O method.

BTW, the original reason for the asynchronous pass(4) interface was an
application at Spectra that sends SCSI passthrough commands to all of
the disks in a system (you could have hundreds or perhaps a thousand)
simultaneously.  Rather than using an application thread per drive, or a
thread pool, this allows using one thread, and using kqueue(2)/kevent(2)
to get I/O completion notifications.

We could change it to use a global zone for I/O requests and another for
request buffer memory, but I don't see a pressing reason to do it right
now.

Ken
-- 
Kenneth Merry
ken@FreeBSD.ORG