OpenSSL breaks factor(6)

[Steve Kargl]

On Sun, Dec 29, 2019 at 01:34:28AM -0500, Garance A Drosehn wrote:
> On 29 Dec 2019, at 0:10, Steve Kargl wrote:
> 
> > On Sat, Dec 28, 2019 at 10:46:52PM -0500, Garance A Drosehn wrote:
> >>
> >> What if the user wants to factor a hexadecimal value which does not
> >> happen to include [a...f]?
> >>
> >> How about recognizing a prefix of '0x' as a way to indicate the value
> >> is hexadecimal?
> >
> > An interested user will need to add that support.  AFAIK, factor(6)
> > has never recognized the 0x prefix, and I'm not trying to add new
> > features.  I'm simply fixing factor(6) to match its documentation,
> > and trying to ensure WITH_OPENSSL and WITHOUT_OPENSSL give the
> > same results where applicable.
> 
> Well, I'd be willing to do the work to add the new feature, and also
> make the commit.  It'd be a nice easy task for me to tackle...  :)
> 
> But I think documenting that "hex works, but only for hex values
> which have at least one A-F in the value" is not something that I'd
> want to draw attention to via documentation.
> 

You have a 17 year history to worry about as hexadecimal support
was added by "r104720 | fanf | 2002-10-09".  Compiling factor(6)
with and without OpenSSL support after 2002-10-09 gives a utility
with different inconsistent behavior.

Current code:

With OpenSSL
% factor 1abc
1: 1
% factor 1abczzzz
1: 1
% factor +125
factor: +125: illegal numeric format.

Without OpenSSL
% factor 1abc
6844: 2 2 29 59
% factor 1abczzzz
factor: 1abczzzz: illegal numeric format.
% factor +125
125: 5 5 5

Patched code:

With OpenSSL
% factor 1abc
6844: 2 2 29 59
% factor 1abczzzz
6844: 2 2 29 59
% factor +125
125: 5 5 5

Without OpenSSL
% factor 1abc
6844: 2 2 29 59
% factor 1abczzzz
6844: 2 2 29 59
% factor +125
125: 5 5 5

-- 
Steve