git: 3394d4239b85 - main - cam: allocate CCBs from UMA for SCSI and ATA IO
Shawn Webb
shawn.webb at hardenedbsd.org
Sat May 15 23:24:41 UTC 2021
On Sat, May 15, 2021 at 11:05:49AM +0000, Edward Tomasz Napierala wrote:
> The branch main has been updated by trasz:
>
> URL: https://cgit.FreeBSD.org/src/commit/?id=3394d4239b85b5577845d9e6de4e97b18d3dba58
>
> commit 3394d4239b85b5577845d9e6de4e97b18d3dba58
> Author: Edward Tomasz Napierala <trasz at FreeBSD.org>
> AuthorDate: 2021-05-15 10:17:22 +0000
> Commit: Edward Tomasz Napierala <trasz at FreeBSD.org>
> CommitDate: 2021-05-15 11:03:49 +0000
>
> cam: allocate CCBs from UMA for SCSI and ATA IO
>
> This patch makes it possible for CAM to use small CCBs allocated
> from an periph-specific UMA zone instead of the usual, huge ones.
> The end result is that CCBs issued via da(4) take 544B (size of
> ccb_scsiio) instead of the usual 2kB (size of 'union ccb', ~1.5kB,
> rounded up by malloc(9)). For ATA it's 272B. We waste less
> memory, we avoid zeroing the unused 1kB, and it should be easier
> to allocate those CCBs in low memory conditions. It should also
> be possible to use uma_zone_reserve(9) to improve behaviour
> in low memory conditions even further.
>
> Note that this does not change the size, or the layout, of CCBs
> as such. CCBs get allocated in various different ways, in particular
> on the stack, and I don't want to redo all that. Instead, this
> provides an opt-in mechanism for the periph to declare "my start()
> callback is fine with receiving a CCB allocated from this UMA zone".
> In other words, most of the code works exactly as it used to; the
> change only happens to IOs issued by xpt_run_allockq(), which
> is - conveniently - pretty much all that matters for performance.
>
> The reason for doing it this way is that it's pretty small, localized
> change, and can be implemented gradually and iteratively: take a
> periph, make sure its start() callback only casts the CCBs it takes
> to a particular type of CCB, for example ccb_scsiio, and that it only
> casts CCBs returned by cam_periph_getccb() to that type, then add UMA
> zone for that size, and declare it safe to XPT.
>
> This is disabled by default. Set 'kern.cam.ada.enable_uma_ccbs=1'
> and 'kern.cam.da.enable_uma_ccbs=1' tunables to enable it. Testing
> is welcome; I will flip the default to enable in two weeks from now.
>
> Reviewed By: imp
> Sponsored by: NetApp, Inc.
> Sponsored by: Klara, Inc.
> Differential Revision: https://reviews.freebsd.org/D28674
> ---
> sys/cam/ata/ata_da.c | 18 ++++++++++++++++++
> sys/cam/ata/ata_xpt.c | 7 +++++++
> sys/cam/cam_ccb.h | 14 +++++++++++++-
> sys/cam/cam_periph.h | 3 +++
> sys/cam/cam_xpt.c | 32 +++++++++++++++++++++++++++++---
> sys/cam/scsi/scsi_da.c | 18 ++++++++++++++++++
> sys/cam/scsi/scsi_xpt.c | 7 +++++++
> 7 files changed, 95 insertions(+), 4 deletions(-)
Hey Edward,
I suspect this may be the cause of a kernel panic I just had when
booting my Softiron Overdrive 1000 arm64 system:
panic: ata_action: ccb 0xffff000040388828, func_code 0x15 should not be allocated from UMA zone
cpuid = 1
time = 1
__HardenedBSD_version = 1400000 __FreeBSD_version = 1400013
version = FreeBSD 14.0-CURRENT-HBSD #1 hardened/current/master-n190374-a541810e5fa: Sat May 15 17:05:42 EDT 2021
shawn at arm64-02:/usr/obj/usr/src/arm64.aarch64/sys/HARDENEDBSD
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x30
vpanic() at vpanic+0x188
panic() at panic+0x44
ata_action() at ata_action+0x340
_ata_announce_periph() at _ata_announce_periph+0x74
ata_announce_periph_sbuf() at ata_announce_periph_sbuf+0x24
xpt_announce_periph_sbuf() at xpt_announce_periph_sbuf+0x1a8
adaregister() at adaregister+0x420
cam_periph_alloc() at cam_periph_alloc+0x528
adaasync() at adaasync+0xa8
xptsetasyncfunc() at xptsetasyncfunc+0x100
xptdevicetraverse() at xptdevicetraverse+0x9c
xpttargettraverse() at xpttargettraverse+0x78
xptbustraverse() at xptbustraverse+0x80
xpt_register_async() at xpt_register_async+0x1fc
adainit() at adainit+0x4c
periphdriver_init() at periphdriver_init+0x54
xpt_finishconfig_task() at xpt_finishconfig_task+0x18
taskqueue_run_locked() at taskqueue_run_locked+0x188
taskqueue_thread_loop() at taskqueue_thread_loop+0x9c
fork_exit() at fork_exit+0x74
fork_trampoline() at fork_trampoline+0x14
KDB: enter: panic
[ thread pid 0 tid 100023 ]
Stopped at kdb_enter+0x44: undefined f904411f
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/dev-commits-src-main/attachments/20210515/ac840bf1/attachment.sig>
More information about the dev-commits-src-main
mailing list