ZFS ARC and mmap/page cache coherency question

[Matthew Macy]

 ---- On Tue, 05 Jul 2016 10:35:12 -0700 Freddie Cash <fjwcash at gmail.com> wrote ---- 
 > On Tue, Jul 5, 2016 at 10:19 AM, Matthew Macy <mmacy at nextbsd.org> wrote:
 > 
 > >  ---- On Mon, 04 Jul 2016 19:26:06 -0700 Karl Denninger <
 > > karl at denninger.net> wrote ----
 > >  > On 7/4/2016 18:45, Matthew Macy wrote:
 > >  > >  ---- On Sun, 03 Jul 2016 08:43:19 -0700 Karl Denninger <
 > > karl at denninger.net> wrote ----
 > >  > >  >
 > >  > >  > On 7/3/2016 02:45, Matthew Macy wrote:
 > >  > >  > >
 > >  > >  > >             Cedric greatly overstates the intractability of
 > > resolving it. Nonetheless, since the initial import very little has been
 > > done to improve integration, and I don't know of anyone who is up to the
 > > task taking an interest in it. Consequently, mmap() performance is likely
 > > "doomed" for the foreseeable future.-M----
 > >  > >  >
 > >  > >  > Wellllll....
 > >  > >  >
 > >  > >  > I've done a fair bit of work here (see
 > >  > >  > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594) and the
 > >  > >  > political issues are at least as bad as the coding ones.
 > >  > >
 > >  > > Strictly speaking, the root of the problem is the ARC. Not ZFS per
 > > se. Have you ever tried disabling MFU caching to see how much worse LRU
 > > only is? I'm not really convinced the ARC's benefits justify its cost.
 > >  >
 > >  > The ARC is very useful when it gets a hit as it avoid an I/O that would
 > >  > otherwise take place.
 > >  >
 > >  > Where it sucks is when the system evicts working set to preserve ARC.
 > >  > That's always wrong in that you're trading a speculative I/O (if the
 > >  > cache is hit later) for a *guaranteed* one (to page out) and maybe *two*
 > >  > (to page back in.)
 > >
 > > The question wasn't ARC vs. no-caching. It was LRU only vs LRU + MFU.
 > > There are a lot of issues stemming from the fact that ZFS is a
 > > transactional object store with a POSIX FS on top. One is that it caches
 > > disk blocks as opposed to file blocks. However, if one could resolve that
 > > and have the page cache manage these blocks life would be much much better.
 > > However, you'd lose MFU. Hence my question.
 > >
 > 
 > ​Are you confusing terms here?
 > 
 > Pretty sure the ARC uses MRU (Most Recently Used) and MFU (Most Frequently
 > Used) caches.  Not LRU (Least Recently Used).
 > 
 > Or am I misunderstanding what you're trying to say?
 
If it caches based on MRU, by definition it evicts LRU. I did mix caching policy with eviction policy in the same sentence which is obviously not correct. Nonetheless, it should be obvious that I meant MFU+MRU caching vs MRU caching only.

Thanks.
-M