Superpages on amd64 FreeBSD 7.2-STABLE

[Bernd Walter]

On Wed, Dec 09, 2009 at 09:07:33AM -0500, John Baldwin wrote:
> On Thursday 26 November 2009 10:14:20 am Linda Messerschmidt wrote:
> > It's not clear to me if this might be a problem with the superpages
> > implementation, or if squid does something particularly horrible to
> > its memory when it forks to cause this, but I wanted to ask about it
> > on the list in case somebody who understands it better might know
> > whats going on. :-)
> 
> I talked with Alan Cox some about this off-list and there is a case that can 
> cause this behavior if the parent squid process takes write faults on a 
> superpage before the child process has called exec() then it can result in 
> superpages being fragmented and never reassembled.  Using vfork() should 
> prevent this from happening.  It is a known issue, but it will probably be 
> some time before it is addressed.  There is lower hanging fruit in other areas 
> in the VM that will probably be worked on first.

For me the whole threads puzzles me.
Especially because vfork is often called a solution.

Scenario A
Parent with super page
fork/exec
This problem can happen because there is a race.
The parent now has it's super pages fragmented permanently!?
the child throws away his pages because of the exec!?

Scenario B
Parent with super page
vfork/exec
This problem won't happen because the child has no pseudo copy of the
parents memory and then starts with a completely new map.

Scenario C
Parent with super page
fork/ no exec
The problem can happen because the child shares the same memory over
it's complete lifetime.
The parent can get it's super pages fragmented over time.

I don't see a use case for scenario A, because vfork is there since
over 16 years.
I use fork myself, because it is easier sometimes, but people writing
big programms such as squid should know better.
If squid doesn't use vfork they likely have a reason.
With scenario C I don't see how vfork can help, since this is not a legal
case for vfork.
I use quid myself, but don't know how it handles it's childs.
But isn't the whole story about such slave childs that they share memory
with the master? - How can vfork be solution for this case?
How can fragmentation of super pages be avoided at all?

I obviously don't have enough clue about this to understand those details.
Hope that someone can enlighten me.

-- 
B.Walter <bernd at bwct.de> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.