dc(1) fails with "big number failure" on 2^64

[Michiel Boland]

On 11/08/2012 19:32, Michiel Boland wrote:
[...]
> No fix, but I see a problem in the BN_add_word function in
> /usr/src/crypto/openssl/crypto/bn/bn_word.c

Small test case:-

#include <openssl/bn.h>
#include <limits.h>

int main()
{
         BIGNUM *n;

         n = BN_new();
         BN_set_word(n, ULONG_MAX - 1);
         BN_add_word(n, 2);
         BN_free(n);
         return 0;
}


$ gcc x.c -lcrypto
$ valgrind ./a.out
==30682== Memcheck, a memory error detector
==30682== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==30682== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==30682== Command: ./a.out
==30682==
==30682== Invalid write of size 8
==30682==    at 0x1328EA8: BN_add_word (bn_word.c:158)
==30682==    by 0x40076E: main (in /usr/home/boland/a.out)
==30682==  Address 0x18fc0a8 is 0 bytes after a block of size 8 alloc'd
==30682==    at 0x100410B: malloc (in 
/usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==30682==    by 0x1331B82: CRYPTO_malloc (mem.c:328)
==30682==    by 0x1330F76: ??? (bn_lib.c:317)
==30682==    by 0x13310C7: bn_expand2 (bn_lib.c:432)
==30682==    by 0x133121C: BN_set_word (bn_lib.c:570)
==30682==    by 0x400760: main (in /usr/home/boland/a.out)