[patch] giant-less quotas for UFS

[Nicolas KOWALSKI]

Eric Anderson <anderson at centtech.com> writes:

> Nicolas KOWALSKI wrote:
>> Eric Anderson <anderson at centtech.com> writes:
>>
>>> Nicolas KOWALSKI wrote:
>>>> Eric Anderson <anderson at centtech.com> writes:
>>>>
>>>>> Nicolas KOWALSKI wrote:
>>>>>> Yes, this is exactly what is happening. To add some precision, some
>>>>>> students here use calculation applications
>>>>>> that allocate a lot of disk space, ususally more than their allowed
>>>>>> home quotas; when by error they launch these apps in their home
>>>>>> directories, instead of their workstation dedicated space, it makes
>>>>>> the server go to its knees on the NFS client side.
>>>>> When you say 'to it's knees' - what do you mean exactly?  How many
>>>>> clients do you have, how much memory is on the server, and how many
>>>>> nfsd threads are you using?  What kind of load average do you see
>>>>> during this (on the server)?
>>>> Sorry for the imprecision.
>>>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320
>>>> 76GB disks and controller. It is accessed by NFS from ~100 Unix
>>>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The
>>>> network connection is GB ethernet.
>>>> During slowdowns, it's only from a NFS client view that the server
>>>> does not respond. For example, a simple 'ls' in my home directory is
>>>> almost immediate, but when it slows down, it can take up to 2 minutes.
>>>> On the server, the load average goes to 0.5, compared to a default
>>>> maximum of 0.15-0.20. The nfsd processus shows them in the state
>>>> "biowr" in top, but nothing is really written, because the quotas
>>>> system block any further writes to the user exceeding her/his quotas.
>>>>
>>> In this case (which is what I suspected), try bumping up your nfsd
>>> threads to 128.  I set mine very high (I have around 1000 clients),
>>> and I can say there aren't really ill-effects besides a bit of memory
>>> usage (which you have plenty of).  I suspect increasing the threads
>>> will neutralize this problem for you.
>> Using 128 nfsd threads, I stressed the server, by running on a NFS
>> client a small C program, writting continuously in a file, so that the
>> user "biguser" (account stored on /export/home2) exceeds his quota.
>> It half-works: during the test, users working on another disk
>> (/export/home) did not see any difference, but users working on the
>> same disk that "biguser" (/export/home2) where almost halted.
>> So, this is better, because before everybody was halted, but there is
>> still a problem.
>> Any other tips ?
>
> Watch gstat during the testing, and see if the disk that holds the
> full partition is really busy.  I'm betting it's thrashing the disk
> continually checking for free space.  I don't think there's any way to
> avoid that.

Mh, I did not find this "gstat" tool on the system or in the ports;
perhaps is it in >= 5.x ? (the server is running 4.11-p15).

It is sad I can not do anything about it: such a server pulled down by
a single NFS-client. :-(


However, Many Thanks for your help and tips.

Best regards,
-- 
Nicolas