Re: git: ca2a7262df5e - main - Free UMA zones when a pass(4) instance goes away.

On 13.01.2022 15:00, Kenneth D. Merry wrote:
> On Thu, Jan 13, 2022 at 10:49:08 -0800, Gleb Smirnoff wrote:
>>    Kenneth,
>>
>> On Thu, Jan 13, 2022 at 03:56:39PM +0000, Kenneth D. Merry wrote:
>> K> commit ca2a7262df5ec5fd07d4ac61738947f48c9cd7f2
>> K> Author:     Kenneth D. Merry <ken@FreeBSD.org>
>> K> AuthorDate: 2022-01-13 15:50:40 +0000
>> K> Commit:     Kenneth D. Merry <ken@FreeBSD.org>
>> K> CommitDate: 2022-01-13 15:54:56 +0000
>> K>
>> K>     Free UMA zones when a pass(4) instance goes away.
>> K>
>> K>     If the UMA zones are not freed, we get warnings about re-using the
>> K>     sysctl variables associated with the UMA zones, and we're leaking
>> K>     the other memory associated with the zone structures.  e.g.:
>> K>
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.size)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.flags)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.bucket_size)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.bucket_size_max)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.name)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.rsize)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.ppera)!
>> K>     sysctl_warn_reuse: can't re-use a leaf (vm.uma.pass44.keg.ipers)!
>> K>
>> K>     Also, correctly clear the PASS_FLAG_ZONE_INPROG flag in
>> K>     passcreatezone().  The way it was previously done, it would have
>> K>     had set the flag and cleared all other flags that were set at
>> K>     that point.
>>
>> Definitely not my area, but I wonder why would we create a zone per
>> device? Why not a global zone in the driver?
> 
> Good question.  The primary reason is just that I didn't think about doing
> it that way.  Most everything else in the CAM peripheral drivers is done on
> a per-instance basis.
> 
> That said, having separate zones does make it a little easier/clearer to
> track down memory leaks.  The zones are created on the first call to
> the CAMIOQUEUE ioctl for that instance.  99% of users won't have the zone
> created, because they haven't used an application that would touch it.  The
> only application in the tree that uses it is camdd(8), when you use the
> pass I/O method.
> 
> BTW, the original reason for the asynchronous pass(4) interface was an
> application at Spectra that sends SCSI passthrough commands to all of
> the disks in a system (you could have hundreds or perhaps a thousand)
> simultaneously.  Rather than using an application thread per drive, or a
> thread pool, this allows using one thread, and using kqueue(2)/kevent(2)
> to get I/O completion notifications.
> 
> We could change it to use a global zone for I/O requests and another for
> request buffer memory, but I don't see a pressing reason to do it right
> now.
There is obviously no rush, but single zone for multiple devices would 
be much more efficient from per-CPU caches side on large SMP systems.

-- 
Alexander Motin