AMD64 NUMA-awareness?

[Jeff Roberson]

On Thu, 19 May 2005, Scott Long wrote:

> Jung-uk Kim wrote:
>
> > ULE scheduler paper
> > (http://www.usenix.org/publications/library/proceedings/bsdcon03/tech/roberson.html)
> > says:
> >
> > 'SMT introduces a concept of non-uniform processors into ULE which
> > could be extended to support NUMA. The concept of expressing the
> > penalty for migration through the use of separate queues could be
> > further developed to include a local and global load-balancing
> > policy. At the time of this writing, however, FreeBSD does not
> > support any true NUMA capable machines and so this is left until such
> > time that it does.'
> >
> > I am not sure about the meaning of 'true NUMA capable machines' but
> > AMD64 is ccNUMA unless I am completely mistaken, and FreeBSD/amd64 is
> > well-supported.  Even multicore processors are available now and
> > Intel is going to release dual-core processors with HTT to make
> > matters worse.
> >
>
> Even HTT by itself presents some interesting scheduling challenges that
> need to be accounted for.
>
> > Is there anybody working on this?
>
> Jeff stated recently that both ULE and 4BSD have some very primitive
> awareness of topology, but that much more work is needed.  NUMA would
> also benefit from the UMA memory allocator understanding memory topology
> and working with the scheduler to keep processes on CPUs that are
> closest to their memory.  Unfortunately, there doesn't appear to be
> much work going on, so if anyone is interested, it's a great project and
> I encourage all to help.

I have put a great deal of thought into the issues in both UMA and
schedulers for NUMA.  I have not put any code in as of yet, however.  I
think that linux has had limited success with numa support on AMD64.  The
remote penalty doesn't seem to be great enough to really warrant much more
complicated algorithms.  Interested parties are certainly welcome to
explore this more.  I believe remote accesses are in the range of 33-50%
more expensive than local.  We might benefit by making a copy of the
kernel text for each processor.  That would be a good starting point to
see if any gains are observed in the best case.

>
> Scott
>
> _______________________________________________
> freebsd-arch at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "freebsd-arch-unsubscribe at freebsd.org"
>