HEADS UP Re: cvs commit: src/sys/conf options.i386 src/sys/i386/i386 bios.c locore.s machdep.c mpboot.s pmap.c vm86bios.s vm_machdep.c src/sys/i386/include _types.h bus_at386.h param.h pmap.h

[Jake Burkholder]

Apparently, On Sat, Mar 29, 2003 at 09:24:53PM -0800,
	Jake Burkholder said words to the effect of;

> jake        2003/03/29 21:24:53 PST
> 
>   FreeBSD src repository
> 
>   Modified files:
>     sys/conf             options.i386 
>     sys/i386/i386        bios.c locore.s machdep.c mpboot.s pmap.c 
>                          vm86bios.s vm_machdep.c 
>     sys/i386/include     _types.h bus_at386.h param.h pmap.h 
>   Log:
>   - Add support for PAE and more than 4 gigs of ram on x86, dependent on the
>     kernel opition 'options PAE'.  This will only work with device drivers
>     which either use busdma, or are able to handle 64 bit physical addresses.

There are a few issues still to be resolved, but for the most part this
is complete and works.  As I mentioned this will only work with certified
device drivers.  The drivers that I was able to test this with in a machine
with more than 4G of ram are: ahc, ata, em, and the usual other drivers which
don't use dma (syscons etc).  Device drivers which use busdma, and which
create their dma tags properly should work, bounce buffers will be used if
required.  Device drivers which don't use busdma and are unable to use 64bit
pci addressing will not work and may cause massive data corruption if used
in a system with more than 4 gigs of ram.  This is serious.  Do not test
random device drivers in such machines with PAE enabled unless you know
what you are doing and can handle file system corruption.  I lost several
file systems during development.  Many many drivers will not compile with
PAE because they are not written properly to handle 64 bit vm_paddr_t or
64 bit bus_addr_t.

There is a problem with cam, in that it tries to store a physical address
in a pointer for certain types of requests.  These requests are not
implemented in cam, and are not actually used by any device drivers, but
casting a pointer to a 64 bit bus_addr_t causes them to not compile.  I
have applied a temporary fix to ahc because I know it works with 39 bit
addressing.  A better solution to this will need to be found, but in the
mean time maintainers should ensure their drivers will work properly with
64 bit physical addresses before just making them compile.

Also, you must not use kernel modules if you enable PAE.  PAE is like
KVA_PAGES in that it changes all kinds of things about the kernel address
space.  Basically any module which calls pmap_mapdev, pmap_kextract or
vtophys (actually pmap_kextract) or which uses the stack gap must be
statically compiled into the kernel.  This includes acpi.  The MAC framework
has similar problems and the only real solution is for modules to be built
with the same options headers as the kernel.  I'm told a solution to this
is being worked on.

Given these caveats, if anyone has a large memory system with a suitable
hardware configuration I would appreciate feedback.

I would like to thank Robert Watson for finding the funding for this, and
once again would like to thank Lanny Baron for his loan of a 6G machine,
without which this would not have been possible.

Have fun.

Jake