mmap performance and memory use

[Alan Cox]

On 10/26/2011 06:23, Svatopluk Kraus wrote:
> Hi,
>
> well, I'm working on new port (arm11 mpcore) and pmap_enter_object()
> is what I'm debugging rigth now. And I did not find any way in
> userland how to force kernel to call pmap_enter_object() which makes
> SUPERPAGE mapping without promotion. I tried to call mmap() with
> MAP_PREFAULT_READ without success. I tried to call madvise() with
> MADV_WILLNEED without success too.
>

mmap() should call pmap_enter_object() if MAP_PREFAULT_READ was 
specified.  I'm surprised to hear that it's not happening for you.

> To make SUPERPAGE mapping, it's obvious that all physical pages under
> SUPERPAGE must be allocated in vm_object. And SUPERPAGE mapping must
> be done before first access to them, otherwise a promotion is on the
> way. MAP_PREFAULT_READ does nothing with it. If madvice() is used,
> vm_object_madvise() is called but only cached pages are allocated in
> advance. Of coarse, an allocation of all physical memory behind
> virtual address space in advance is not preferred in most situations.
>
> For example, I want to do some computation on 4M memory space (I know
> that each byte will be accessed) and want to utilize SUPERPAGE mapping
> without promotion, so save 4K page table (i386 machine). However,
> malloc() leads to promotion, mmap() with MAP_PREFAULT_READ doesn't do
> nothing so SUPERPAGE mapping is promoted, and madvice() with
> MADV_WILLNEED calls vm_object_madvise() but because the pages are not
> cached (how can be on anonymous memory), it is not work without
> promotion too.
>
> So, SUPERPAGE mapping without promotions is fine, but it can be done
> only if physical memory being mapped is already allocated. Is it
> really possible to force that in userland?
>

To force the allocation of the physical memory?  Right now, the only way 
is for your program to touch the pages.

> Moreover, the SUPERPAGE mapping is made readonly firstly. So, even if
> I have SUPERPAGE mapping without promotion, the mapping is demoted
> after first write, and promoted again after all underlying pages are
> accessed by write. There is 4K page table saving no longer.
>

Yes, that is all true.  It is possible to change things so that the page 
table pages are reclaimed after a time, and not kept around 
indefinitely.  However, this not high on my personal priority list.  
Before that, it is more likely that I will add an option to avoid the 
demotion on write, if we don't have to copy the entire superpage to do so.

Alan