[Bug 279501] linuxulator: amd64 Global Descriptor Table ABI incompatibility

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=279501

            Bug ID: 279501
           Summary: linuxulator: amd64 Global Descriptor Table ABI
                    incompatibility
           Product: Base System
           Version: Unspecified
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: theron.tarigo@gmail.com

Created attachment 251198
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=251198&action=edit
Minimal test of executing 32bit code from 64bit process

On amd64, a program may jump between 32bit and 64bit code using far returns to
set the code segment register.  The mechanism is OS-dependent since it depends
on the layout of the Global Descriptor Table initialized by the kernel.  It is
used for example for a 64bit program to utilize 32bit assembly libraries or for
a 32bit compatibility layer implemented using 64bit libraries as with WINE's
WOW64.

Currently this mechanism is broken in the linux compatibility layer: whereas
the program expects a Linux GDT, it is the FreeBSD GDT in effect.  Also, Linux
program code reading the cs register gets different results than expected: 0x23
vs 0x33 for 32bit, 0x33 vs 0x43 for 64bit.

$ yasm -fbin -o test testgdt.asm
produces a FreeBSD executable which writes "A", then writes "B", then exits.

$ yasm -fbin -o test testgdt.asm -DLINUX -DLINUX_GDT
produces a Linux executable with the expected result only when run on Linux
x86_64.
On FreeBSD it writes "A" before dying with a bus error.

$ yasm -fbin -o test testgdt.asm -DLINUX
produces an executable which writes the "expected result" only when run on
FreeBSD amd64 with linux compatibility.
On Linux it writes "A" before dying with a segfault.

-- 
You are receiving this mail because:
You are the assignee for the bug.