GNU-compatible, BSD-licensed bc

[Devin Teske]

> On Jan 9, 2019, at 12:47 PM, Gavin Howard <gavin.d.howard at gmail.com> wrote:
> 
> On Wed, Jan 9, 2019 at 12:58 PM Devin Teske <dteske at freebsd.org <mailto:dteske at freebsd.org>> wrote:
>> 
>> 
>> 
>> On Jan 7, 2019, at 11:45 PM, Gavin Howard <gavin.d.howard at gmail.com> wrote:
>> 
>> On Mon, Jan 7, 2019 at 10:57 PM Devin Teske <dteske at freebsd.org> wrote:
>> 
>> 
>> 
>> 
>> On Jan 7, 2019, at 4:42 PM, Gavin Howard <gavin.d.howard at gmail.com> wrote:
>> 
>> On Mon, Jan 7, 2019 at 5:38 PM Conrad Meyer <cem at freebsd.org> wrote:
>> 
>> 
>> On Mon, Jan 7, 2019 at 4:00 PM Devin Teske <dteske at freebsd.org> wrote:
>> 
>> How do you handle arbitrary arithmetic precision?
>> 
>> 
>> Looks like https://github.com/gavinhoward/bc/blob/master/src/num.c .
>> 
>> 
>> That is correct. Because this bc is meant to help bootstrap the Linux
>> kernel and have no dependencies other than POSIX 2008, I wrote my own.
>> 
>> 
>> Impressive. It might be worth turning this into a library itself.
>> 
>> 
>> Eh...it is completely tuned for bc. And it won't be fast,
>> unfortunately. See below.
>> 
>> Also, the POSIX bc standard mandates doing math in decimal. OpenSSL
>> would not be smart if they did that.
>> 
>> 
>> Not sure I understand what you mean here.
>> 
>> 
>> Well, for starters, OpenSSL's BIGNUM is integer only. Yes, those
>> integers can be any size, but they are only integers. That is not good
>> enough for bc; it has to allow arbitrary precision, including
>> non-integers, and its fractional part can be any size, up to a certain
>> limit, which you can get from my bc by typing "limits" at the prompt
>> and looking for the value of BC_NUM_MAX (which is actually the maximum
>> number of decimal digits, period).
>> 
>> 
>> Thanks for explaining that further.
>> 
>> [snip]
>> 
>> There are also a few
>> peculiarities with the POSIX bc standard that (more or less) require a
>> standalone implementation.
>> 
>> 
>> How hard would it be to convert a bn(3)-based library to use your code?
>> Would there be any performance impact -- I've benchmarked bn(3) to
>> be pretty fast.
>> 
>> 
>> It would not be terribly hard, but as I said above, it would not be
>> fast at all, at least compared to a well-written hardware-based,
>> binary bignum implementation. But if you want to, go ahead; I would
>> appreciate the credit (though the license does not even require that).
>> 
>> 
>> Well, unfortunately, my needs are purely whole-integer arithmetic but
>> speed is paramount.
>> 
>> My application of OpenSSL bn(3) is here:
>> https://reviews.freebsd.org/D16132
>> 
>> Though worth noting that I haven't updated the review since November.
>> Since then, I have made many changes which can be seen on GitHub:
>> https://FrauBSD.org/libcmb
>> 
>> 
>> 
>> Also, right now I am working on getting a release candidate out that
>> will enable me to make a quick port that Stefan could use as a jumping
>> off point. My build system changed between 1.0 and now, and I would
>> like to be able to test it.
>> 
>> 
>> Cool. Looking forward to it.
>> 
>> 
>> It's out. It works great. The Makefile that I sent to the mailing list
>> a few messages back does the job well enough, though I was told in a
>> private message not to use the GNU bc port's pkg-descr file, since it
>> might be under the GPL.
>> 
>> However, note that this is not the final 1.1 release; it is just for
>> testing, even though it is high quality.
>> 
>> 
>> I'm wondering why you chose to write your own configure.sh instead of
>> leveraging autotools.
> 
> I had a goal of absolutely zero dependencies (that is one reason why I
> imported and adapted an existing history implementation instead of
> just making readline or editline an optional dependency). That was
> also why I wrote a custom parser, even though it is complicated,
> rather than using Lex/Flex and YACC/Bison. Autotools would be a
> dependency, so I wrote a custom one.
> 
>> Also, it looks like you have a high number of build-time options. I also
>> notice that you're into writing tests for your software. It might be interesting
>> to apply my tool for combining all possible combinations of build options.
>> 
>> Seen here:
>> https://github.com/FrauBSD/pkgcenter/blob/master/depend/libcmb/release/Makefile <https://github.com/FrauBSD/pkgcenter/blob/master/depend/libcmb/release/Makefile>
>> 
>> It's a great way to make sure all the various build options work together.
>> --
>> Cheers,
>> Devin
> 
> Thank you. It looks interesting, though unfortunately, some of my
> build options are exclusive (for example, the `-b` and `-d` flags to
> the `configure.sh` script cannot both be used at the same time), and
> from a cursory glance, I can't tell if libcmb can handle them.
> 

You likely would not use libcmb directly in the build process but rather
the cmb utility to provide your combinations.

For example, taking into account that -b and -d are exclusive:

cmb -e -- -E -g -G -H "-k 64" "-O3" -S

Produces 128 lines, each line representing a unique combination of the above options.

You could, for each unique combination, then perform one as-is, one with "-d", one
with "-B", etc.

Latest version of cmb is here:
https://FrauBSD.org/cmb <https://fraubsd.org/cmb>

Which depends on libcmb:
https://FrauBSD.org/libcmb <https://fraubsd.org/libcmb>



> I do have a way of testing all of the (valid) build option
> combinations in `release.sh`, which is a script I run before every
> release. It tests that it compiles without error and/or warnings on
> gcc and clang, in debug, release, reldebug, and min size modes,
> running the test suite on every build.

release.sh could be enhanced to support more than 16 variants in runtestseries()

I have a similar release process for libcmb/cmb (previously linked) except
instead of just -Wall, I am a glutton for punishment by using with clang:

	-Weverything -Werror


> It also runs the test suite
> under ASan, UBSan, MSan, and Valgrind. (I think it runs 532 builds and
> does the test suite for all of them.) It takes 5 hours 20 minutes to
> run on my fast, custom-built machine.
> 

Impressive!


> And then on top of that, I have a script to generate random math
> problems (`tests/randmath.py`), which I run for about 10 million
> iterations. And then I run the afl fuzzer for at least 100 cycles on
> both calculators, fixing every single crash it finds. I am doing the
> latter two right now, and if it passes, the release will be out right
> after that.
> 

Sweet, looking forward. Excellent work.
-- 
Devin