expand_number() for fetch'es -B and -S switches

Alexander Best arundel at freebsd.org
Sun Sep 5 20:37:57 UTC 2010


On Thu Sep  2 10, Dag-Erling Smørgrav wrote:
> Alexander Best <arundel at freebsd.org> writes:
> > the current maximum buffer limit of fetch(1) actually is around 1G. i
> > think 1M is not enough, because if people are pulling data over fast
> > lines they'll have almost constant disk writes. how about 100M then?
> > ;)
> 
> Large buffer sizes are *not* better, since fetch(1) will alternate
> between filling the buffer and writing it to disk.  The buffer should
> not be too small, but it should not be too large either; the sweet spot
> is somewhere around 128 kB.

hmm. so if there should be a maximum buffer size of let's say 1 megabyte,
there's no point really in having expand_number() then.

so how about forgetting about expand_number() and simply introducing a
maximum buffer size of 1 megabyte? everything higher than that will get
reduced to 1 megabyte. just like if you want to burn a cd and specify a write
speed of 1000 it simply gets reduced to someting like MAX_SPEED.

also that would correspond with the semantics of the minimum buffer size.

> 
> > on the other hand why have a maximum limit? if people want to have a
> > buffer of 100 gigabyte why shouldn't they? it's their decision
> > actually.
> 
> Good point...  although if they set it too high, either malloc(3) will
> fail - if they're lucky - or fetch(1) will crash when the system runs
> out of physical RAM and swap, and they'll have to start over.

yeah. users might start using buffer sizes of a few gigabyte which will cause
all kinds of problems. don't really wanna go there.

> 
> DES
> -- 
> Dag-Erling Smørgrav - des at des.no

cheers.
alex

-- 
a13x


More information about the freebsd-hackers mailing list