[ZFS] refquota is very slow !

[Ben RUBSON]

Yes, I want to dedicate my ARC & L2ARC to my data pool (which anyway suffers from the same bug...).

I followed your advices and created a whole new SSD test pool, let all options by default, and added lz4 compression.
I then create 20.000.000 empty files in this test pool.

Then :

# zfs set quota=7.5G refquota=none test/test
# dd if=/dev/random of=/test/test/bf
dd: bf: Disc quota exceeded
1047553+0 records in
1047552+0 records out
536346624 bytes transferred in 16.769513 secs (31983434 bytes/sec)
# rm /test/test/bf

# zfs set quota=none refquota=7.5G test/test
# dd if=/dev/random of=/test/test/bf
dd: bf: Disc quota exceeded
1047520+0 records in
1047519+0 records out
536329728 bytes transferred in 215.986582 secs (2483162 bytes/sec)
# rm /test/test/bf

Additional tests give the same results.
6MB before the limit, write IOs are done very slowly and it takes minutes to fulfil these remaining MB.

How many files in the pool you performed this test in Juergen ?

Ben

> On 05 Sep 2016, at 10:04, InterNetX - Juergen Gotteswinter <juergen.gotteswinter at internetx.com> wrote:
> 
> any special reason for disabling secondarycache and limiting the
> primarycache to metadata? does it change something when you revert it to
> default?
> 
> compression -> lz4, even if its not compressable, it wont hurt
> 
> you probably got several smaller performance issues which end up in this
> mess all together.
> 
> Am 04.09.2016 um 17:16 schrieb Ben RUBSON:
>> Same kind of results with a single local (SSD) disk based pool, refquota takes much more time than quota around the limit.
>> 
>> Here is the output for this single disk based pool :
>> zfs get all   : http://pastebin.com/raw/TScgy0ps
>> zdb           : http://pastebin.com/raw/BxmQ4xNx
>> zpool get all : http://pastebin.com/raw/XugMbydy
>> 
>> Thank you !
>> 
>> Ben
>> 
>> 
>>> On 04 Sep 2016, at 13:42, InterNetX - Juergen Gotteswinter <juergen.gotteswinter at internetx.com> wrote:
>>> 
>>> Did you try the same in a single local disk based pool? And pls post output of
>>> zfs get all, zdb & zpool get all
>>> 
>>> 
>>>> Ben RUBSON <ben.rubson at gmail.com> hat am 4. September 2016 um 11:28
>>>> geschrieben:
>>>> 
>>>> 
>>>> Juergen & Bram,
>>>> 
>>>> Thank you for your feedback.
>>>> 
>>>> I then investigated further and think I found the root cause.
>>>> 
>>>> No issue with refquota in my zroot pool containing (in this example) 300.000
>>>> inodes used.
>>>> 
>>>> However, refquota is terribly slow in my data pool containing around
>>>> 12.000.000 inodes used.
>>>> 
>>>> I then created 12.000.000 empty file in my zroot pool, in a test dataset.
>>>> I put a refquota on this dataset and created a dd file to fulfil empty space.
>>>> And around the limit, it began to stall...
>>>> I then created an empty dataset in the same pool, refquota is even slow in
>>>> this dataset having no inode used.
>>>> The root cause seems then to be the total number of inodes used in the pool...
>>>> 
>>>> Some numbers :
>>>> Time to fulfil 512MB with quota : 17s
>>>> Time to fulfil 512MB with refquota : 3m35s
>>>> 
>>>> Very strange.
>>>> 
>>>> Do you experience the same thing ?
>>>> 
>>>> Thank you again,
>>>> 
>>>> Ben
>>>> 
>>>>> On 03 Sep 2016, at 16:59, Bram Vandoren <bram.vandoren at kuleuven.be> wrote:
>>>>> 
>>>>> I encountered the same problem over NFS. I didn't manage to reproduce it not
>>>>> using NFS. I think the userquota property works without any problem though.
>>>>> 
>>>>> Cheers,
>>>>> Bram.
>>>> 
>>>>> On 03 Sep 2016, at 12:26, InterNetX - Juergen Gotteswinter
>>>>> <juergen.gotteswinter at internetx.com> wrote:
>>>>> 
>>>>> cant confirm this, works like a charm without difference to normal quota
>>>>> setting