read(2) and thus bsdiff is limited to 2^31 bytes

[Cedric Blancher]

Nothing in POSIX mandates that read()/write() are atomic. Old UNIX,
SystemV, AIX, Solaris and HP-UX don't do that nor do they guarantee
that.

Ced

On 23 May 2016 at 14:21, Joerg Sonnenberger <joerg at bec.de> wrote:
> On Sun, May 22, 2016 at 06:22:58PM -0700, Matthew Macy wrote:
>>
>>
>>
>>  ---- On Sun, 22 May 2016 16:12:03 -0700 Joerg Sonnenberger <joerg at bec.de> wrote ----
>>  > On Sun, May 22, 2016 at 04:02:02PM -0700, Matthew Macy wrote:
>>  > >
>>  > >
>>  > >
>>  > >  ---- On Sun, 22 May 2016 15:54:14 -0700 Joerg Sonnenberger <joerg at bec.de> wrote ----
>>  > >  > On Sun, May 22, 2016 at 10:54:30PM +0200, Dirk Engling wrote:
>>  > >  > > When trying to bsdiff two DVD images, I noticed it failing due to
>>  > >  > > read(2) returning EINVAL to the tool. man 2 read says, this would only
>>  > >  > > happen for a negative value for fildes, which clearly was not true.
>>  > >  >
>>  > >  > I would classify that as implementation bug. It seems perfectly sensible
>>  > >  > to turn overly large requests into a short read/write, even for blocking
>>  > >  > files. But erroring out seems to be quite wrong to me.
>>  > >  >
>>  > >
>>  > > read(2) takes a size_t so this is clearly an internal bug where it's an int and treating it as a negative value.
>>  >
>>  > Not exactly. The reason for cutting it off are many fold. Using int in
>>  > the kernel is one argument. The requirement for locking the IO range for
>>  > concurrent read/write operations from other threads is a bigger
>>  > argument.
>>  >
>>  That still doesn't justify EINVAL as a return. Does read(2) need to
>> make atomicity guarantees?
>
> See my first sentence. I consider returning EINVAL for too large buffer
> size a bug. Yes, read/write operations should be atomic with regard to
> other processes on the system. Atomic meaning in this context that the
> read can be observed either completely or not at all. This still doesn't
> mean that read must execute the full size. Other cases for short
> read/writes are socket, pipes etc.
>
> Joerg
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"



-- 
Cedric Blancher <cedric.blancher at gmail.com>
Institute Pasteur