git: d106f982a54c - main - qsort.c: prevent undefined behavior

From: Stefan Eßer <se_at_FreeBSD.org>
Date: Thu, 13 Jan 2022 10:25:39 UTC
The branch main has been updated by se:

URL: https://cgit.FreeBSD.org/src/commit/?id=d106f982a54cd299671ccad58bc456138a22ae7b

commit d106f982a54cd299671ccad58bc456138a22ae7b
Author:     Stefan Eßer <se@FreeBSD.org>
AuthorDate: 2022-01-13 10:09:38 +0000
Commit:     Stefan Eßer <se@FreeBSD.org>
CommitDate: 2022-01-13 10:09:38 +0000

    qsort.c: prevent undefined behavior
    
    Mark Milliard has detected a case of undefined behavior with the LLVM
    UBSAN. The mandoc program called qsort with a==NULL and n==0, which is
    allowed by the POSIX standard. The qsort() in FreeBSD did not attempt
    to perform any accesses using the passed pointer for n==0, but it did
    add an offset to the pointer value, which is undefined behavior in
    case of a NULL pointer. This operation has no adverse effects on any
    achitecture supported by FreeBSD, but could be caught in more strict
    environments.
    
    After some discussion in the freebsd-current mail list, it was
    concluded that the case of a==NULL and n!=0 should still be caught by
    UBSAN (or cause a program abort due to an illegal access) in order to
    not hide errors in programs incorrectly invoking qsort().
    
    Only the the case of a==NULL and n==0 should be fixed to not perform
    the undefined operation on a NULL pointer.
    
    This commit makes qsort() exit before reaching the point of
    potentially undefined behvior for the case n==0, but does not test
    the value of a, since the result will not depend on whether this
    pointer is NULL or an actual pointer to an array if n==0.
    
    The issue found by Mark Milliard in the whatis command has been
    reported to the upstream (OpenBSD) and has already been patched
    there.
    
    MFC after:      1 week
---
 lib/libc/stdlib/qsort.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/libc/stdlib/qsort.c b/lib/libc/stdlib/qsort.c
index 5016fff7895f..0b99c04507d3 100644
--- a/lib/libc/stdlib/qsort.c
+++ b/lib/libc/stdlib/qsort.c
@@ -108,6 +108,8 @@ local_qsort(void *a, size_t n, size_t es, cmp_t *cmp, void *thunk)
 	int cmp_result;
 	int swap_cnt;
 
+	if (__predict_false(n == 0))
+		return;
 loop:
 	swap_cnt = 0;
 	if (n < 7) {