cvs commit: src/lib/libc/gen syslog.c

[Andrea Campi]

On Sat, Oct 09, 2004 at 03:07:14PM -0400, Brian Fundakowski Feldman wrote:
> On Sat, Oct 09, 2004 at 05:39:17PM +0200, Andrea Campi wrote:
> > Unless I'm missing something, this would make the calling application
> > loop for an unbounded time (potentially forever), thus making the DoS
> > even more effective. Personally, I've never thought of syslog as a
> 
> I think that this is something very application-dependant, and it's probably
> best to implement as a timeout that it specifies.  But why would a DoSed
> syslogd lose messages on its end if we're getting ENOBUFS locally?  The
> protocol doesn't provide a way to acknowledge reception, does it?

On Sun, Oct 10, 2004 at 01:29:52AM +0400, Gleb Smirnoff wrote:
> Not forever. Actually this change can make applications work slower,
> when syslogd is DoSed. Anyway when an attacker is flooding syslog with
> messages your machine is _already_ slower, since syslogd consumes
> all CPU and IO resources.

I'll answer both at the same times as you bring up related points. I'm
sorry this became a very long email.


I agree the desired degree of reliability of logging is
application-dependent, and that is sort of my point; I don't think
however that using a timeout is a good solution. Our syslog
implementation is thread-safe; forcing all applications to use timeouts
(inherently a nightmare in multithreaded apps) is IMHO silly.

In addition, this change accomplishes nothing. We try very hard to send
out a UDP packet that might well be lost in transit, since syslog is as
unreliable as UDP. As to why the UDP packet might be lost in transit or
why the remove syslog may lose packet I have no answer, but IMHO it's
not relevant: what IS relevant is whether syslog() can make any
guarantee of delivery.
Note Brian that you ask "why would syslogd lose messages if we're
getting ENOBUFS". What I ask is: why loop try so hard locally when we
don't even know if anybody is listening? That is, the two failure cases
need not be simultaneous or related; but unless we close both, we can't
guarantee syslog() is reliable.

The scenario I'm thinking of that would be mostly affected by this is
admittedly complicated:

 - attacker needs a way to cause ENOBUFS condition on the target
   machine.
 - target machine runs an unrelated service; let's say for discussion
   sake that it's not network-bound (i.e. an application server or a
   service gateway). To make things worse, let's say the app uses a
   multiprocess and multithread model.

In this scenarion, as soon as the application needs to log anything,
threads start to get stuck in syslog() calls--maybe for milliseconds,
maybe for seconds, maybe forever. Those threads could be holding
application-level resources (say mutex), thus slowing down or
potentially blocking forever other threads.

Gleb, you mentioned that a DoS syslogd would mean an overloaded machine,
but the ENOBUFS machine is the client. Are you sure that an ENOBUFS case
must imply the client is so swamped with CPU and IO time that the above
scenario doesn't actually make things worse?

Above all however, how can you say "not forever"? What kind of guarantee
do you see that the application will never succeed its send() call?
Sure, statistically it will succeed, but that is not good enough.

Note that I'm not advocating that "since it can fail, don't bother
retrying". The concept of trying again is fine--my only gripe is with
retrying an unbounded number of times.


What this all boils down to is:

 - syslog is an unreliable protocol by definition (see RFC3164 6.4);
 - syslog() and family are unreliable functions (to the extent that they
   even return void;
 - if the change stays, I think it should be documented in the syslog(3)
   man page;
 - I strongly object to MFC'ing it;
 - look into a better way to accomplish the goal.

Bye,
	Andrea

-- 
               Speak softly and carry a cellular phone.