read(2) and thus bsdiff is limited to 2^31 bytes

[Joerg Sonnenberger]

On Sun, May 22, 2016 at 04:02:02PM -0700, Matthew Macy wrote:
> 
> 
> 
>  ---- On Sun, 22 May 2016 15:54:14 -0700 Joerg Sonnenberger <joerg at bec.de> wrote ---- 
>  > On Sun, May 22, 2016 at 10:54:30PM +0200, Dirk Engling wrote: 
>  > > When trying to bsdiff two DVD images, I noticed it failing due to 
>  > > read(2) returning EINVAL to the tool. man 2 read says, this would only 
>  > > happen for a negative value for fildes, which clearly was not true. 
>  >  
>  > I would classify that as implementation bug. It seems perfectly sensible 
>  > to turn overly large requests into a short read/write, even for blocking 
>  > files. But erroring out seems to be quite wrong to me. 
>  >  
> 
> read(2) takes a size_t so this is clearly an internal bug where it's an int and treating it as a negative value.

Not exactly. The reason for cutting it off are many fold. Using int in
the kernel is one argument. The requirement for locking the IO range for
concurrent read/write operations from other threads is a bigger
argument.

Joerg