Deltas between FreeBSD and NetBSD and POSIX conformance (was "[Bug 194828] [test] lib.libc.sys.getcontext_test.setcontext_link fails on amd64, not i386")

[Julio Merino]

On Fri, Nov 14, 2014 at 12:34 PM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
>
> On Nov 14, 2014, at 2:28, Justin Cormack <justin at specialbusservice.com> wrote:
>
>> On Fri, Nov 14, 2014 at 7:35 AM,  <bugzilla-noreply at freebsd.org> wrote:
>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194828
>>>
>>> --- Comment #12 from Garrett Cooper,425-314-3911 <ngie at FreeBSD.org> ---
>>> In a perfect world I would like for everything to be consistent between FreeBSD
>>> and NetBSD, but that's not the way it should be, and that's a silly ideal to
>>> hope for :).
>>
>> NetBSD dev here... I would rather that tests actually test behaviour
>> as defined in standards, especially for eg libc tests, and indeed I
>> have bunch of stuff to add more tests on standards compliance.  So if
>> stuff is testing implementation internals it should go away, and if
>> NetBSDs behaviour is incorrect please file an issue, or if the test
>> setup is not eg standards compliant please file an issue
>> http://www.netbsd.org/cgi-bin/sendpr.cgi?gndb=netbsd

That's a nice ideal... but in practice, the standard does sometimes
leave details up to the implementation and/or the implementation
provides custom extensions. Both need tests, and such tests will be
implementation-dependent.

So how could we deal with this? Tests for the standard should really
be unified across both codebases, but there should be a way to
supplement them with implementation-specific tests. Not sure what the
best organization scheme for this would be, nor how we'd maintain the
"common code".