Occassional "permission denied" in the middle of a large transfer over NFS

[Vincent Hoffman]

On 07/07/2012 13:26, Vincent Hoffman wrote:
> On 01/07/2012 12:18, Rick Macklem wrote:
>> Vincent Hoffman wrote:
>>> On 01/07/2012 01:53, Rick Macklem wrote:
>>>> To modify mountd to use the kernel changes is more work than I have
>>>> time for, in part because mountd.c is a very ugly old piece of C
>>>> code, imho.
>>>>
>>>> I do have a patch that suspends/resumes execution of the nfsd
>>>> threads
>>>> (new, experimental for FreeBSD8.n only) when mountd reloads
>>>> /etc/exports.
>>>> This approach has certain disadvantages vs Andrey's transactional
>>>> load/commit
>>>> model, but it is simple and might be sufficient for your problem.
>>>> If you want to try this patch, it is at:
>>>>    http://people.freebsd.org/~rmacklem/atomic-export.patch
>>>> (The patch affects both the kernel and mountd.c.)
>>> Happy to try it, to be certain I have understood, this is for the
>>> newnfs
>>> server (experimental in 8.x default for 8
>>> 9+)
>> Yes, that is correct. For the old (default on 8.x) it will have no effect.
>>
>>> I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when
>>> i
>>> get a minute.
>> Also, to enable it, you must add a "-S" flag to mountd_flags, after you've
>> replaced the mountd executable.
>>
>> The patch is pretty straightforward, but has not seen much testing.
>> Hopefully, it might be useful for you, rick
> Hi Rick,
>
> I'm afraid this didnt make any real difference for me.
> Since I couldnt test it on the live system I tried it on a test vm.
> on the vm (nfs server) I set a looping mount/umount
> while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ;  done
>
> and on the client I set a loop of tars of large directorys to the nfs
> mount running under time to see how well it survived. Then replicated
> the test with the patch and without.
>
>
> [root at seaurchin ~]# ministat nopatch.txt atomicpatch.txt
> x nopatch.txt
> + atomicpatch.txt
> +--------------------------------------------------------------------------------------------------+
> |    
> *                                                                                           
> |
> |    
> *                                                                                           
> |
> |   
> x*                                                                                           
> |
> |   xx* 
> x                                                                                        
> |
> |  +x** 
> xx                                                                                       
> |
> |  **** xxx 
> x                                                                                    
> |
> |  **** xxx +x+      
> +                                                                           
> |
> |  ****+*xx +x+   x  
> +                                                                           
> |
> |  ****+*x****++++x + +                        
> x                                                  |
> |  *************+*xx+ +++x *  x                
> x                                                  |
> |  ****************x**++*x+***x+ x*+ x  ++*+  + x+      +x +      
> +                              +|
> |||_______M_M_A__A_______|______|                                                                 
> |
> +--------------------------------------------------------------------------------------------------+
>     N           Min           Max        Median           Avg        Stddev
> x 101          1.25         106.8         14.08     21.892178     22.196005
> + 101          1.21        186.93         18.46     27.995842     30.523218
> No difference proven at 95.0% confidence
>
>
> (excuse wrapped ascii art)
>
> I think I'll have a look at the nfse patch set and see how that performs.
Replying to myself just as a record, I have tried nfse and I didnt get
the permission denied at all.
The only issue I had with it is that it strictly adheres to the syntax
in exports(5) while mountd is a little more flexible.

I had
/usr/local/export -alldirs -maproot=root 85.xx.xx.xx

/usr is the root of that filesystem

mountd - allowed this but actually silently exports /usr (and all dirs
below)

nfse - didnt allow this, it needed to be the correct
/usr  -alldirs -maproot=root 85.xx.xx.xx

which is I guess a POLA violation but follows the documentation.
this was using nfse in mountd compatibility mode. I havent played with
its more advanced features.

Vince
> Thanks for all your work on NFS on FreeBSD.
>
> Vince
>