Page fault on Tyan K8S Pro (S2882) board

[David O'Brien]

[ DON'T top post -- it looses context, and usually I won't reply... ]

On Wed, Dec 01, 2004 at 02:02:02AM -0800, Astrodog wrote:
> On Wed, 1 Dec 2004 01:00:24 -0800, David O'Brien <obrien at freebsd.org> wrote:
> > On Fri, Nov 26, 2004 at 09:01:03PM -0800, Astrodog wrote:
> > > Running 1 DIMM in this kind of setup is
> > > shooting yourself in the foot. If CPU #1 requests something through CPU
> > > #0, and we're using 1 bank of memory the whole setup..... performance is
> > > gonna be crap. Both CPUs will be blocked from processing instructions
> > > while the request is fulfilled.
> > 
> > That is not at all true -- CPU0 is not blocked because CPU1 is accessing
> > memory directly attached to CPU0.  The memory controller in an Opteron
> > operates independently of the CPU cache unit and central processing unit
> > ("core").  All of the the HyperTransport connections (3 of them), the
> > memory controller, cache unit (2 for dual-core) all attach together thru
> > a cross-bar switch.

..snip..

> Isn't CPU0 blocked from memory, while CPU1 makes a request (and has it
> fulfilled?)
> If not... how'd they manage that? Thats cool as hell. Heh.

Not any more than with a traditional northbridge system.  The memory
controller has a request queue.  While it is getting the data from the
memory that CPU1 requested, CPU0 can certainly add entries to the request
queue during the time the data is becoming ready to read from the DIMM's,
etc...  Think of it as a "fine-grain locked kernel".  The memory
controller doesn't have the "Big Giant Lock" such that it can't do a
single other thing as soon as CPU1 makes a memory request.

Digging deeper,
http://www.amd.com/us-en/assets/content_type/DownloadableAssets/MPF_Hammer_Presentation.PDF
slides 2628 are good pictures showing who there are actually two pieces
to what most call a "memory controller" -- the true memory controller
(with queues and scheduling (handling dependencies) etc..) and the DRAM
controller (handling the real-world electrical duties).  Also you can see
the high-level data queues and buffers and how they interconnect with
cross-bar switches.

-- 
-- David  (obrien at FreeBSD.org)