RF_CACHEABLE flag

[Konstantin Belousov]

On Sun, Feb 21, 2016 at 07:42:49PM -0600, Justin Hibbits wrote:
> The Freescale/NXP Datapath Acceleration Architecture uses both
> cache-inhibited and cache-enabled memory regions for buffer portals.
> This doesn't quite fit right into the existing framework, so I've
> added to my personal repo (on github) a RF_CACHEABLE flag to be used
> by this.  Now that I'm ready to commit the driver to head, I want to
> float this on -arch to get opinions.
> 
> I did consider another route, using bus_space_map()/bus_space_unmap(),
> and stashing sizes around, but adding a simple flag to rman would take
> care of all the details, and rman already knows all the other details
> for the region anyway.
> 
> I put the diff on phabricator, at https://reviews.freebsd.org/D5384 .
> 
> Thoughts on this?

Not making any opinion about RF_CACHEABLE, only pointing out some facts
that might be interesting to you.  On x86, some BARs need specific memory
access modes at least for performance.

E.g., for Intel GPUs, there is a combined BAR where the first 512KB or
2M maps the registers and must be uncacheable, and the rest of the BAR
maps GTT. To get a reasonable performance from the graphics hardware,
GTT should be mapped as write-combining (i.e. not cacheable but writes
may sit in the combining buffers and flushed on serialization points).

Look at dev/agp/agp_i810.c:agp_gen4_install_gatt() to see direct
pmap_change_attr(WC) call to set it up.

No flag could accomodate this mode.  OTOH, why couldn't you explicitely
add pmap_change_attr() to the driver of your device ?