amd64/169927: siginfo, si_code for fpe errors when error occurs
using the SSE math processor
Bruce Evans
brde at optusnet.com.au
Tue Jul 17 13:30:13 UTC 2012
The following reply was made to PR amd64/169927; it has been noted by GNATS.
From: Bruce Evans <brde at optusnet.com.au>
To: Ed Alley <wea at llnl.gov>
Cc: freebsd-gnats-submit at freebsd.org, freebsd-amd64 at freebsd.org
Subject: Re: amd64/169927: siginfo, si_code for fpe errors when error occurs
using the SSE math processor
Date: Tue, 17 Jul 2012 23:25:55 +1000 (EST)
On Tue, 17 Jul 2012, Ed Alley wrote:
>> Description:
> According to sigaction(2) by choosing SA_SIGINGO as one of the sa_flags
> one can catch sigfpe signals. What is returned to the signal handler
> for the sigfpe is a structure defined in siginfo(3). Within that
> structure: the si_code entry gives the error code as defined in siginfo(3)
> man page. This is useful when de-bugging a large code,
> because one can retrieve not only the actual fpe error: divide by zero,
> overflow or etc, but also the location of the error is also returned.
It's interesting that anyone even enables SIGFPE for FP exceptions
(I tried to keep them as the default for critical exceptions, but
was OBE. SIGFPE now normally means (!F)PE for integer division by 0.)
> For FPE errors using the x87 everything works fine, but when
> the SSE is used for floating point calculations the si_code that
> is returned is always zero. I have a fix for this which I have
> included as a patch for FreeBSD 8.2 release. I have been applying
> this fix since I got my 64-bit box since FreeBSD 7.x. I have not
> sent this patch in, since I had assumed that the problem would get
> fixed in later releases. However, that has not been the case so
> here is the patch (upgraded to version 8.2) that I have been using.
>
> To apply the patch, just cd into /usr/src/sys/amd64 and apply the
> patch. It will operate on two files in the directory amd64:
> trap.c and fpu.c.
i386 has the same bug.
> In trap.c a single line will be replaced as can be seen in the patch.
> This line occurs in the user trap switch for the case: T_XMMFLT. The
> line ucode = 0; is replaced with ucode = fputrap(); This then will call
> the fputrap() code similarly to the T_ARITHTRAP case.
I didn't know that it had a different exception number.
> The process fputrap() is found in file fpu.c which is where the
> rest of the patch operates. In function fputrap() I added additional
> code to access the mxcsr status bits. These are then ORed into
> the status code before the argument to the fpetable[] is calculated.
>
> Following the x87 case, before I return, I zero out the error flags
> in the mxcsr register. Let me know if this is useful, also I have
> not found an equivalent instruction to the fnclex (that zeros out
> the x87 error flags) for easily zeroing out the mxcsr error flags,
> so I have resorted to anding them out of a memory copy of the mxcsr
> that I loaded earlier and then storing it back into the register.
This shouldn't be done. The fnclex is both a bug and a feature (mostly
a bug). The corresponding clearing of the mxcsr bits is just a bug.
Reason for the fnclex: it clears the fault condition, so that buggy
SIGFPE handlers can return and have the main code not immediately
fault again. But the behaviour is still undefined. In particular,
the i387 FP stack may be corrupt (unless the SIGFPE handler actually
understands FP and has fixed up the stack, but if it understands FP
then it won't depend on the fnclex, and it won't simply return). It
is better for the fault to repeat endlessly. Most faults including
SIGFPE for integer division by 0 repeat endlessly, which tells you
that you have a buggy fault handler that returns.
History of the fnclex and of other FP bugs in signal handling:
- in FreeBSD-[1-4], the i387 exception handling was:
- save the status word in the "saved exception status word" in the PCB
- fnclex
- starting in about FreeBSD-4, encode the saved status word in the
signal code. This loses some info.
- call the signal handler with the same FP context as the normal process,
except for the saved exception status word. This was bad, but it was
easy for a SIGFPE handler to fix up the state. Fixing up the status
word was most complicated. After about FreeBSD-4, the the signal code
might have been enough (convert it back to a status word). If not,
the saved exception status word was recoverable via the sigcontext
pointer.
- return to the normal process with the same FP context as left by the
signal handler. This was OK for SIGFPE handlers (they could fix up
everything except the status word without using the complications of
the sigcontext pointer or the different unportabilities given by the
signal code and other siginfo things (siginfo was mostly unavailable
then), but very bad for non-SIGFPE signal handlers, since any FP in
the signal handler would corrupt the FP context for the normal process.
- gdb never understood the sigcontext pointer, but it used to understand
the saved exception status word after I made it do this in FreeBSD 1 or
2. It printed status for both the normal status word and the saved one.
- most of this was broken in FreeBSD-5. Now, the i387 exception handling
is (and similarly for amd64):
- save the status word in a local variable. It never reaches the PCB.
- fnclex, as before
- encode the old status word in the signal code, as before
- call the signal handler with an independent context. This is bad,
but it makes it harders for a SIGFPE handler to fix up the state.
Especially the status word, since that was clobbered after not
saving it except in the local variable. The handler can retrieve
the FP state using either the sigcontext pointer or siginfo, but
that gives the clobbered status word. Now the signal code gives
the only trace of the exception bits in the old status word.
- switch back to the normal FP context after returning from the signal
handler.
- current gdb no longer understands the saved exception status word,
on old kernels that have it. It is much further from understanding
the full switched state. In FreeBSD-[1-4], you could easily see the
normal process's FP state in a signal handler since it wasn't switched,
and you could fix it up manually by editing it. Now you have the same
complications as a SIGFPE handler that does fixups -- it is hard to
even see this state. I think you can still do it manually by looking
at in the stack. Perhaps this can be done using gdb macros. An
average signal handler doesn't declare sigcontext or siginfo pointers
(or even the signal code), so to debug it you might need to add the
declarations and recompile, or possibly fake them using macros. I
haven't actually tried this.
On why clobbering the mxcsr status is even less useful:
- for the i387, the trap happens on the next non-control FP instruction
after the one that caused the exception. Not repeating the fault on
this next (hopefully non-problematic) instruction almost makes sense.
- for SSE, the trap happens on the one that causes the exception. It's
good to repeat this fault.
> With these changes in place, my kernel now handles SIMD fpe errors
> (trap code 29) and returns the mxcsr decoded error in the si_code entry of the
> siginfo_t structure.
> diff -Naur amd64-orig/fpu.c amd64/fpu.c
> --- amd64-orig/fpu.c 2012-06-24 18:59:36.000000000 -0700
> +++ amd64/fpu.c 2012-07-16 22:07:19.000000000 -0700
> ...
> @@ -356,6 +359,7 @@
> fputrap()
> {
> u_short control, status;
> + u_int mxcsr;
>
> critical_enter();
>
> @@ -367,13 +371,18 @@
> if (PCPU_GET(fpcurthread) != curthread) {
> control = GET_FPU_CW(curthread);
> status = GET_FPU_SW(curthread);
> + mxcsr = GET_MXCSR(curthread);
> + status |= (mxcsr & 0x3f);
Lots of tab and other whitespace lossage.
This change and the one to call fputrap() for T_XMMFLT, and similarly
for i386, might be enough.
> } else {
> fnstcw(&control);
> fnstsw(&status);
> + stmxcsr(&mxcsr);
> + status |= (mxcsr & 0x3f);
> + fnclex(); /* Clear the x87 error bits */
> + mxcsr &= ~0x3f; /* Clear the mxcsr error bits */
> + ldmxcsr(&mxcsr);
Best not to touch either.
> }
>
> - if (PCPU_GET(fpcurthread) == curthread)
> - fnclex();
Try removing this too.
> critical_exit();
> return (fpetable[status & ((~control & 0x3f) | 0x40)]);
> }
Here are some of my old changes to npxtrap(). They mainly ifdef out the
fnclex:
% Index: npx.c
% ===================================================================
% RCS file: /home/ncvs/src/sys/i386/isa/npx.c,v
% retrieving revision 1.152
% diff -u -2 -r1.152 npx.c
% --- npx.c 19 Jun 2004 22:24:16 -0000 1.152
% +++ npx.c 22 Apr 2006 11:58:31 -0000
% @@ -578,5 +624,5 @@
% */
% static char fpetable[128] = {
% - 0,
% + -1, /* 0 - no unmasked exception (probably bogus IRQ13) */
% FPE_FLTINV, /* 1 - INV */
% FPE_FLTUND, /* 2 - DNML */
% @@ -642,5 +688,5 @@
% FPE_FLTDIV, /* 3E - DNML | DZ | OFL | UFL | IMP */
% FPE_FLTINV, /* 3F - INV | DNML | DZ | OFL | UFL | IMP */
% - FPE_FLTSUB, /* 40 - STK */
% + -1, /* 40 - STK, but no unmasked exception so no trap */
% FPE_FLTSUB, /* 41 - INV | STK */
% FPE_FLTUND, /* 42 - DNML | STK */
These -1's are supposed to give a unique error for cases that shouldn't happen.
% @@ -751,7 +806,16 @@
% }
%
% + /* Ignore some spurious traps. */
% + if ((status & ~control & 0x3f) == 0) {
% + intr_restore(saveintr);
% + return (-1);
% + }
IIRC, this is mainly for old IRQ13 exception handling.
% +
% +#if 0
% + /* XXX this clobbers the status. */
% if (PCPU_GET(fpcurthread) == curthread)
% fnclex();
% - intr_restore(savecrit);
% +#endif
% + intr_restore(saveintr);
Kill the fnclex. Unrelated renaming of savecrit (savecrit is a bogus name,
since there are no critical sections here).
% return (fpetable[status & ((~control & 0x3f) | 0x40)]);
% }
The only effect that I noticed from killing the fnclex was that some of
my old test programs with buggy SIGFPE handling appear to hang (they
actually fault endlessly). They were depending on the signal handler to
return and then fixed up the FP state after it returned. The signal
handler was too standards-conforming and just set a flag of type
sig_atomic_t. The quick fix was to use a FreeBSD-4-compat signal handler
so that the signal handler can corrupt the normal process state; then
just add an fnclex to it (it is now very non-standards-conforming).
Configuring to use a FreeBSD-4-compat signal handler in FreeBSD-5+ is
nontrivial, but I force this configuration anyway for portability.
With FreeBSD-5+ signal handlers, the signal handler would need an enormous
amount of code to apply the fnclex to the normal process state.
Note that with SSE, everyone has had the endless-faulting behaviour
for most FP unmasked exceptions on amd64, since T_XMMFLT wasn't
connected to fputrap() and fputrap() didn't clobber the status anwyay.
So everyone must have gotten used to this. Except, hardly anyone
unmasks FP exceptions. I only unmask them for debugging.
Bruce
More information about the freebsd-amd64
mailing list