svn commit: r278634 - head/lib/libc/gen

[Pedro F. Giffuni]

Author: pfg
Date: Thu Feb 12 21:07:42 2015
New Revision: 278634
URL: https://svnweb.freebsd.org/changeset/base/278634

Log:
  ulimit(3): Fix broken check.
  
  The existing implementation had a broken comparison that could overflow.
  Replace this with a check that avoids the overflow before it happens.
  
  Consistently return a maximum value also on the case of negative
  arguments since negative is considered an overflow and means
  infinity for our current setrlimit().
  
  Discussed with:	bde (rather extensively)
  
  CID:		1199295
  MFC after:	1 week

Modified:
  head/lib/libc/gen/ulimit.c

Modified: head/lib/libc/gen/ulimit.c
==============================================================================
--- head/lib/libc/gen/ulimit.c	Thu Feb 12 20:57:57 2015	(r278633)
+++ head/lib/libc/gen/ulimit.c	Thu Feb 12 21:07:42 2015	(r278634)
@@ -53,13 +53,13 @@ ulimit(int cmd, ...)
 		va_start(ap, cmd);
 		arg = va_arg(ap, long);
 		va_end(ap);
+		if (arg > RLIM_INFINITY / 512 || arg < 0)
+			arg = RLIM_INFINITY / 512;
 		limit.rlim_max = limit.rlim_cur = (rlim_t)arg * 512;
 
 		/* The setrlimit() function sets errno to EPERM if needed. */
 		if (setrlimit(RLIMIT_FSIZE, &limit) == -1)
 			return (-1);
-		if (arg * 512 > LONG_MAX)
-			return (LONG_MAX);
 		return (arg);
 	} else {
 		errno = EINVAL;