Behavior of madvise(MADV_FREE)

[Alan Cox]

On 10/12/2012 12:13, Poul-Henning Kamp wrote:
> --------
> In message<9FEBC10C-C453-41BE-8829-34E830585E90 at xcllnt.net>, Marcel Moolenaar writes:
>
>> Now on to the questions:
>> 1.  madvise(MADV_FREE) marks the pages as clean and moves
>>     them to the inactive queue.
> You can blame me for MADV_FREE, I nagged dyson into adding it for
> phkmalloc back when 16meg RAM cost a fortune.
>
> The main point of MADV_FREE when was to prevent a page-out of a
> dirty page, when the actual bits in the page were discarded from
> the applications point of view.
>
> The idea was for it to be a very fast operation, which would
> (potentially) save two disk operations, and the implementation
> simply washed the dirty bits away.
>
> (I think a later modification added further VM-wizardry to retain
> the allocated paging area, while avoiding the page-in in the case
> the pages were stolen.)

It would be more accurate to say that no action is taken to reclaim any 
previously allocated swap space, which is unlikely to exist anyway.

> The mapping were left alone, because it was estimated and measured
> that there were a high frequency of mapping reuse, in particular
> in the WorldStone where multiple passes of GCC competed for physical
> memory.  The reference bit were left alone to not disturb the
> perceived/ measured VM-footprint of the process.

This last "bit" is dated.  :-)  We have long cleared the reference bit.

> I seem to recall that we experimented with a more aggressive
> version which severed the mapping and sent the page to pre-zeroing
> but the benchmarks showed too much overhead in the common case
> and too little gain in the relevant scenario.
>
> The name was clearly a mistake, but MADV_CLEAN would have been
> worse, and none of us had any other ideas at the time.  Today I
> would have called it MADV_GARBAGE or MADV_RECYCLE.
>
> Given that paging anonymous pages to disk is not a major activity
> and unlikely to ever become a performance bottleneck again, it may
> make good sense to let MADV_FREE make the pages more appetizing
> for the rest of the system.

Given the way that madvise(MADV_FREE) is still used by malloc()/free(), 
any additional overhead would still be a pessimization most of the 
time.  Over the years, I think you and I have both said that what we 
really need is the ability for the kernel to deliver an event to 
malloc()/free() when memory is trending toward scarcity.  Applications 
could also use this to reduce the size of their internal caches.

> But all things considered, I would advocate implementing MADV_RECYCLE
> for that, we need more nuance and expressability in madvice(2) and
> MADV_RECYCLE would be a good addition in its own right.

Probably so.

Alan