Performance issues with 8.0 ZFS and sendfile/lighttpd

[Miroslav Lachman]

Ivan Voras wrote:
> 2009/11/6 Miroslav Lachman<000.fbsd at quip.cz>:
>
>> I do not understand why there are 10MB/s read from disks when network
>> traffic dropped to around 1MB/s (8Mbps)
>>
>> root at cage ~/# iostat -w 20
>>       tty             ad4              ad6             cpu
>>   tin tout  KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
>>    0   14 41.66  53  2.17  41.82  53  2.18   0  0  2  0 97
>>    0   18 50.92  96  4.77  54.82 114  6.12   0  0  3  1 96
>>    0    6 53.52 101  5.29  54.98 108  5.81   1  0  4  1 94
>>    0    6 54.82  98  5.26  55.89 108  5.89   0  0  3  1 96
>
> Yes, this could limit your IO if the requests are random enough.
> Unfortunately I don't know how would you track down what is really
> going on. Maybe some tracing with DTrace?
>
> I'd tell you to use "top -m io" to see if there is a process
> responsible, but apparently these statistics are not updated for ZFS,
> which in itself may be a bug (which is why I'm crossposting to
> freebsd-fs).

DTrace is totally out of my skills ;(

There is otput of top -m io sorted by VCSW displaying JID.

last pid: 17724;  load averages:  0.01,  0.07,  0.08      up 74+20:49:49 
  21:03:40
195 processes: 1 running, 193 sleeping, 1 zombie
CPU:  0.0% user,  0.0% nice,  3.6% system,  0.4% interrupt, 96.1% idle
Mem: 462M Active, 2385M Inact, 977M Wired, 21M Cache, 399M Buf, 100M Free
Swap: 6144M Total, 2024K Used, 6142M Free

   PID JID USERNAME   VCSW  IVCSW   READ  WRITE  FAULT  TOTAL PERCENT 
COMMAND
17681   8 www         657     64      0      0      0      0   0.00% 
lighttpd
17683   8 www         379     41      0      0      0      0   0.00% 
lighttpd
17680   8 www         136      5      0      0      0      0   0.00% 
lighttpd
17682   8 www          85      0      0      0      0      0   0.00% 
lighttpd
  4689   1     90       10      0      0      0      0      0   0.00% 
fb_inet_server
  3403   1     90       10      0      0      0      0      0   0.00% 
fb_inet_server
  2632   1     90       10      0      0      0      0      0   0.00% 
fb_inet_server

All four top consumers is Lighttpd workers.

And as you noted, read, write, fault, total and percent are not updated 
on machine with ZFS, so I can't compare it with UFS2 based machine.
Is this bug in top fixed in 8.x? Will you file a PR? (you know more 
about FS related things than me :])

Miroslav Lachman