zfs, a directory that used to hold lot of files and listing pause

[Jonathan Chen]

On 21 October 2016 at 11:27, Steven Hartland <killing at multiplay.co.uk> wrote:
> On 20/10/2016 22:18, Jonathan Chen wrote:
>>
>> On 21 October 2016 at 09:09, Peter <pmc at citylink.dinoex.sub.org> wrote:
>> [...]
>>>
>>> I see this on my pgsql_tmp dirs (where Postgres stores intermediate
>>> query data that gets too big for mem - usually lots of files) - in
>>> normal operation these dirs are completely empty, but make heavy disk
>>> activity (even writing!) when doing ls.
>>> Seems normal, I dont care as long as the thing is stable. One would need
>>> to check how ZFS stores directories and what kind of fragmentation can
>>> happen there. Or wait for some future feature that would do
>>> housekeeping. ;)
>>
>> I'm seeing this as well with an Odoo ERP running on Postgresql. This
>> lag does matter to me as this is huge performance hit when running
>> Postgresql on ZFS, and it would be good to see this resolved.
>> pg_restores can make the system crawl as well.
>
> As mentioned before could you confirm you have disable atime?

Yup, also set the blocksize to 4K.

11:46am# zfs get all irontree/postgresql
NAME                 PROPERTY              VALUE                  SOURCE
irontree/postgresql  type                  filesystem             -
irontree/postgresql  creation              Wed Sep 23 15:07 2015  -
irontree/postgresql  used                  43.8G                  -
irontree/postgresql  available             592G                   -
irontree/postgresql  referenced            43.8G                  -
irontree/postgresql  compressratio         1.00x                  -
irontree/postgresql  mounted               yes                    -
irontree/postgresql  quota                 none                   default
irontree/postgresql  reservation           none                   default
irontree/postgresql  recordsize            8K                     local
irontree/postgresql  mountpoint            /postgresql
inherited from irontree
irontree/postgresql  sharenfs              off                    default
irontree/postgresql  checksum              on                     default
irontree/postgresql  compression           off                    default
irontree/postgresql  atime                 off                    local
irontree/postgresql  devices               on                     default
irontree/postgresql  exec                  on                     default
irontree/postgresql  setuid                on                     default
irontree/postgresql  readonly              off                    default
irontree/postgresql  jailed                off                    default
irontree/postgresql  snapdir               hidden                 default
irontree/postgresql  aclmode               discard                default
irontree/postgresql  aclinherit            restricted             default
irontree/postgresql  canmount              on                     default
irontree/postgresql  xattr                 off                    temporary
irontree/postgresql  copies                1                      default
irontree/postgresql  version               5                      -
irontree/postgresql  utf8only              off                    -
irontree/postgresql  normalization         none                   -
irontree/postgresql  casesensitivity       sensitive              -
irontree/postgresql  vscan                 off                    default
irontree/postgresql  nbmand                off                    default
irontree/postgresql  sharesmb              off                    default
irontree/postgresql  refquota              none                   default
irontree/postgresql  refreservation        none                   default
irontree/postgresql  primarycache          all                    default
irontree/postgresql  secondarycache        all                    default
irontree/postgresql  usedbysnapshots       0                      -
irontree/postgresql  usedbydataset         43.8G                  -
irontree/postgresql  usedbychildren        0                      -
irontree/postgresql  usedbyrefreservation  0                      -
irontree/postgresql  logbias               latency                default
irontree/postgresql  dedup                 off                    default
irontree/postgresql  mlslabel                                     -
irontree/postgresql  sync                  standard               default
irontree/postgresql  refcompressratio      1.00x                  -
irontree/postgresql  written               43.8G                  -
irontree/postgresql  logicalused           43.4G                  -
irontree/postgresql  logicalreferenced     43.4G                  -
irontree/postgresql  volmode               default                default
irontree/postgresql  filesystem_limit      none                   default
irontree/postgresql  snapshot_limit        none                   default
irontree/postgresql  filesystem_count      none                   default
irontree/postgresql  snapshot_count        none                   default
irontree/postgresql  redundant_metadata    all                    default


-- 
Jonathan Chen <jonc at chen.org.nz>