Performance issues with 8.0 ZFS and sendfile/lighttpd

[Michael Loftis]

--On Monday, November 02, 2009 10:52 AM +0100 Ivan Voras 
<ivoras at freebsd.org> wrote:

> AFAIK, ZFS is incompatible (currently) with some advanced VM operations
> (like mmap, and I think sendfile relies on the same mechanism as mmap),
> so that could be a cause of the slowdown. Though I'm surprised you can
> only get 200 MBit/s - that's 25 MB/s and I think that even with multiple
> memcpy-ing data around the kernel you should be able to get hundreds of
> MB/s on newer hardware (which normally really can achieve tens of
> gigabytes/s of sustained memory access).
>
> What else can you observe from your system? Do you have exceedingly high
> sys times and load numbers? I'm also interested in what does 10 seconds
> of running 'vmstat 1' looks like on your system. Is it a bare machine or
> a virtual machine?

Real hardware, dual quad core opteron with 64GB memory, a 3Ware 9650SE for 
disks.  The rest of the machine is not doing much if anything when the 
issue happens.  I've had to remove ZFS from all of the media streaming 
servers.  I ca probably get one up for testing again over the next few 
weeks.  I've some more hardware coming in.