[Bug 231038] memguard is broken with options NUMA

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231038

            Bug ID: 231038
           Summary: memguard is broken with options NUMA
           Product: Base System
           Version: CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: markj at FreeBSD.org

With INVARIANTS enabled I get:

# sysctl vm.memguard.desc=mbuf
panic: vm_reserv_extend: Domain mismatch from reservation.                      
cpuid = 4                                                                       
time = 1535660704                                                               
Uptime: 44s                                                                     
Dumping 1295 out of 32617 MB:..2%..12%..22%..31%..41%..51%..61%..71%..81%..91%  

__curthread () at ./machine/pcpu.h:230                                       
230             __asm("movq %%gs:%1,%0" : "=r" (td)                             
(kgdb) bt                                                                       
#0  __curthread () at ./machine/pcpu.h:230                                      
#1  doadump (textdump=1) at
/usr/home/markj/src/freebsd-dev/sys/kern/kern_shutdown.c:366                    
#2  0xffffffff80768ab0 in kern_reboot (howto=260) at
/usr/home/markj/src/freebsd-dev/sys/kern/kern_shutdown.c:446                    
#3  0xffffffff80768f03 in vpanic (fmt=<optimized out>, ap=0xfffffe0f812a1570)
at /usr/home/markj/src/freebsd-dev/sys/kern/kern_shutdown.c:872                
#4  0xffffffff80768c63 in panic (fmt=<unavailable>) at
/usr/home/markj/src/freebsd-dev/sys/kern/kern_shutdown.c:799                    
#5  0xffffffff80ad729c in vm_reserv_extend (req=546, object=0xffffffff81312c78
<kernel_object_store>, pindex=2053, domain=0, mpred=<optimized out>)          
    at /usr/home/markj/src/freebsd-dev/sys/vm/vm_reserv.c:879                   
#6  0xffffffff80ac86fe in vm_page_alloc_domain_after (object=<optimized out>,
pindex=2053, domain=<optimized out>, req=546, mpred=0xfffff8086998d9e8)
    at /usr/home/markj/src/freebsd-dev/sys/vm/vm_page.c:1835                    
#7  0xffffffff80ab5958 in kmem_back_domain (domain=0, object=<optimized out>,
addr=<optimized out>, size=4096, flags=<optimized out>)                        
    at /usr/home/markj/src/freebsd-dev/sys/vm/vm_kern.c:440                     
#8  0xffffffff80ab5c49 in kmem_back (object=<optimized out>,
addr=18446741874694705152, size=4096, flags=<unavailable>)
    at /usr/home/markj/src/freebsd-dev/sys/vm/vm_kern.c:487                     
#9  0xffffffff80aafe72 in memguard_alloc (req_size=256, flags=1) at
/usr/home/markj/src/freebsd-dev/sys/vm/memguard.c:351
#10 0xffffffff80aaa4be in uma_zalloc_arg (zone=0xfffffe101af84000,
udata=0xfffffe0f812a1870, flags=1)
    at /usr/home/markj/src/freebsd-dev/sys/vm/uma_core.c:2385                   
#11 0xffffffff80890d58 in m_gethdr (how=1, type=255) at
/usr/home/markj/src/freebsd-dev/sys/sys/mbuf.h:790                              
#12 _iflib_fl_refill (ctx=0xfffff8000b676800, fl=<optimized out>,
count=<optimized out>) at /usr/home/markj/src/freebsd-dev/sys/net/iflib.c:1974  
#13 0xffffffff8089054b in __iflib_fl_refill_lt (ctx=<optimized out>, max=24,
fl=<optimized out>) at /usr/home/markj/src/freebsd-dev/sys/net/iflib.c:2071
#14 iflib_rxeof (rxq=<optimized out>, budget=24) at
/usr/home/markj/src/freebsd-dev/sys/net/iflib.c:2723
#15 0xffffffff8088c0f9 in _task_fn_rx (context=0xfffff80480637580) at
/usr/home/markj/src/freebsd-dev/sys/net/iflib.c:3834
#16 0xffffffff807af459 in gtaskqueue_run_locked (queue=0xfffff8000b1dca00) at
/usr/home/markj/src/freebsd-dev/sys/kern/subr_gtaskqueue.c:332
#17 0xffffffff807af218 in gtaskqueue_thread_loop (arg=<optimized out>) at
/usr/home/markj/src/freebsd-dev/sys/kern/subr_gtaskqueue.c:507
#18 0xffffffff80728e24 in fork_exit (callout=0xffffffff807af190
<gtaskqueue_thread_loop>, arg=0xfffffe101af9a068, frame=0xfffffe0f812a1ac0)
    at /usr/home/markj/src/freebsd-dev/sys/kern/kern_fork.c:1057
#19 <signal handler called>

memguard is the sole remaining consumer of the kmem_back() API, which takes a
KVA range and backs it with pages from a domain selected using an iterator. 
For normal allocations with kmem_malloc(), the iterator is used to select the
vmem arena as well.  The per-domain vmem arenas have a quantum equal to the
superpage size, which ensures that we consistently use the same physical memory
domain to back regions in a superpage-sized KVA range.  memguard just uses a
single global arena with a 4KB quantum, and basically picks a random domain
which may not have the same "colour" as the 4KB KVA range.  This mismatch
triggers the panic.

-- 
You are receiving this mail because:
You are the assignee for the bug.