rsync and smb.

[Antony Mawer]

On 2/08/2007 4:42 AM, Julian Elischer wrote:
> Remko Lodder wrote:
>> On Wed, Aug 01, 2007 at 12:37:14PM -0400, Craig Rodrigues wrote:
>>> On Wed, Aug 01, 2007 at 06:30:12PM +0200, Remko Lodder wrote:
>>>> I cannot seem to recreate the problem on 6-STABLE at all; I already 
>>>> copied more then two gb in
>>>> just a few minutes and I dont see any problems passing by..
>>> According to http://www.freebsd.org/cgi/query-pr.cgi?pr=78953 ,
>>> this bug is triggered if you have a directory with exactly 50
>>> files in it.
>>> -- 
>>> Craig Rodrigues        rodrigc at crodrigues.org
>>
>> I am not seeing that as well, I created 50 simple text files:
>>
>> [remko at guardian /mnt]$ rsync --ignore-errors -r /mnt/test/ /opt/share3
>> [remko at guardian /mnt]$ ls -l /opt/share3 | wc -l
>>       50
>> [remko at guardian /mnt]$ ls -l /mnt/test | wc -l
>>       50
> 
> probably 50 files in a new directory

Peter, the patch you Craig Rodrigues provided is the one from the bug 
report, and is NOT included in -STABLE ... you will need to apply it 
manually and then see if this solves the problem. Please let us know if 
it solves the problem, as another report of it solving the issue without 
causing any regressions will assist in getting this committed to 
-STABLE. It is included in -CURRENT, so will be there when 7.0-RELEASE 
comes out.


I have found the patch works quite well. The error it reports is in 
itself, harmless, but is problematic with rsync because rsync changes 
its behaviour when errors are encountered.

The problem does not appears using ls, which is probably why you are not 
seeing it when using ls on a directory.

The issue is that smbfs requests directory listings in groups of 52 
items, and if the number of files/directories in a listing falls on a 
multiple of 52 (including . and ..), then SMBFS is attempting to request 
the "next 52 items" on the listing when it is actually at the end of the 
listing. The remote SMB server then sends back an error because there 
are no more items.

As such, no files are missing from the listing -- it is simply because 
the smbfs client is attempting to read past the end of the directory 
listing, so the server sends an error.

Jim Carroll did all the hard work tracking down and fixing this issue 18 
months ago -- he deserves full credit for his persistence in tracking 
this down! It was committed to -CURRENT by bp@ (original smbfs author), 
but never got MFC'd. I bugged re@ to see if this could get MFC'd when I 
realised it wasn't included in 6.2-PRERELEASE, and bmah@ had a quick 
look and updated the bug status... but I haven't seen any action beyond 
that in terms of an MFC.


Unfortunately smbfs suffers from lack of an active maintainer -- there 
are lots of rough edges that have had fixes in Darwin that would be nice 
for someone with the time to pick up and port across where they make 
sense. Maybe this should be added to the Ideas page?

There are also these patches I am led to believe may potentially help 
with communicating with OSX versions of Samba (which likes to talk 
unicode). They may have bit-rotted since and so may need updating for 
-CURRENT:

     http://people.freebsd.org/~imura/kiconv/


... in short, the patch by Jim Carroll appears to fix this problem in my 
experience, and I am running with it in production without causing any 
regressions (at least that I have observed -- more eyes are always 
welcome). Thanks!

--Antony