ports/181919: [PATCH] x11-toolkits/Xaw3d has an overflow error in geometry computation code, causing xfig to hang

[David Wolfskill]

>Number:         181919
>Category:       ports
>Synopsis:       [PATCH] x11-toolkits/Xaw3d has an overflow error in geometry computation code, causing xfig to hang
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-ports-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Sep 07 20:50:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator:     David Wolfskill
>Release:        FreeBSD 9.2-PRERELEASE i386
>Organization:
Wolfskill & Dowling Residence
>Environment:
System: FreeBSD g1-227.catwhisker.org 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #542  r255345M/255355:902503: Sat Sep  7 05:28:53 PDT 2013     root at g1-227.catwhisker.org:/usr/obj/usr/src/sys/CANARY  i386

>Description:
	See <http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=32445>
	(yes, from Jan 2006!) for additional details and the original source
	of the below patch.

	Per the above, Xaw3d uses 16-bit integers to calculate the
	geometry for Box Layout in Box.c; with a sufficiently large
	logical screen, the computations are susceptible to an overflow.

	The observed result is that xfig (for example) will start OK, but
	as soon as one selects (say) a type of object to draw, xfig
	appears to hang (indefinitely, until it is killed).  This renders
	xfig completely unusable.

	Note: it may well be that with a smaller logical screen, one might
	not see these symptoms.  Between the physical screen (1920x1200)
	and the window manager I use, the effective logical screen size I
	normally use is 5760x3600.

	Also, I credit Thomas Mueller for pointing out the NetBSD PR,
	and regret that I failed to file this PR back when he & I
	were discussing the issue.
>How-To-Repeat:
	As indicated above, start xfig.  While the NetBSD PR indicated KDE,
	I use a variant of tvtwm (piewm).
>Fix:
	The below was adapted from the patch in the NetBSD PR.

	After applying it & performing "make clean reinstall" in
	/usr/ports/x11-toolkits/Xaw3d, xfig works for me.

diff -ruN Xaw3d/files/patch-Box Xaw3d/files/patch-Box
--- Xaw3d/files/patch-Box	1969-12-31 16:00:00.000000000 -0800
+++ Xaw3d/files/patch-Box	2013-09-07 11:17:54.000000000 -0700
@@ -0,0 +1,16 @@
+--- Box.c	2013/09/07 18:16:40	1.1
++++ Box.c	2013/09/07 18:06:10
+@@ -352,8 +352,12 @@
+ 	}
+ 	else {
+ 	    width = preferred_width;
++	    if (0==width) width=1;
+ 	    do { /* find some width big enough to stay within this height */
+-		width *= 2;
++		if (width>=32768) /* overflow */
++		    width=constraint->width;
++		else
++		    width *= 2;
+ 		if (width > constraint->width) width = constraint->width;
+ 		DoLayout(w, width, 0, &preferred_width, &preferred_height, FALSE);
+ 	    } while (preferred_height > constraint->height &&
>Release-Note:
>Audit-Trail:
>Unformatted: