Message buffer and printf reentrancy patch

[Don Lewis]

On 17 Jun, Ian Dowse wrote:
> In message <20030616205631.F28116 at gamplex.bde.org>, Bruce Evans writes:
>>On Mon, 16 Jun 2003, Don Lewis wrote:
>>> It looks like MSGBUF_SEQNORM() could avoid the conditional code and any
>>> questions about signed remainders if it was defined like this:
>>>
>>> #define MSGBUF_SEQNORM(mbp, seq) (((seq) + (mbp)->msg_seqmod) % \
>>>     (mbp)->msg_seqmod)
>>>
>>> as long as msg_seqmod < INT_MAX/2.  MSGBUF_SEQNORM() could be simplified
>>> further if msg_seqmod was added by the caller (such as MSGBUF_SEQSUB())
>>> if the argument could be negative.
>>
>>Yes.  The negative numbers of interest seem to be limited to at most
>>differences of sequence numbers (or maybe differeces of indexes, which
>>are smaller), so they are larger than -msg_seqmod.  MSGBUF_SEQSUB()
>>shouldn't add the bias, however,  since it is used in contexts where
>>we really want to see the negative values.

Since MSGBUF_SEQSUB() calls MSGBUF_SEQNORM() on the difference between
the sequence numbers, a negative value will never be returned.  If you
want a signed result, you'll probably want to do something more like:
	tmp = MSGBUF_SEQNORM(mbp, (seq1) - (seq2) + (mbp)->seqmod);
	return (tmp < ((mbp)->seqmod / 2)) ? tmp : (tmp - (mbp)->seqmod));

and you'll have to use a slightly different function if you are
comparing indexes.

> The only minor problem I see with the above is that it is fragile
> with respect to arbitrary input sequence numbers, in that it could
> return a negative value. However, the property of guaranteeing to
> return a normalised sequence number can be achieved by forcing an
> unsigned division like in MSGBUF_SEQ_TO_POS, i.e.:
> 
> #define MSGBUF_SEQNORM(mbp, seq) ((int)((u_int)((seq) + \
>     (mbp)->msg_seqmod) % (mbp)->msg_seqmod))
>     
> This should do the right thing for the expected ranges, but also
> ensures that the macro itself can never return an out-of-range
> sequence number, whatever the input value.

Wouldn't it be better to have assertions to detect obviously bogus
sequence numbers rather than using them to generate a valid pointer to a
random location in the message buffer?