listening sockets as non sockets

[Sepherosa Ziehau]

On Thu, Feb 23, 2017 at 7:27 AM, Gleb Smirnoff <glebius at freebsd.org> wrote:
> On Tue, Feb 21, 2017 at 09:58:45AM +0100, Julien Charbon wrote:
> J> > T> 1) The patch worked pretty okay, but the idea of separate file type is
> J> > T>    abandoned. With current filedescriptor code it is almost impossible
> J> > T>    to racelessly switch fileops and f_data.
> J> > T>    For more details read the commit message, URL below.
> J> > T>
> J> > T> 2) I moved the work to git. It allows for easier code sharing.
> J> > T>
> J> > T> https://github.com/glebius/FreeBSD/commit/3a52df429889ea9c6e61013f6913aad95939f159
> J> >
> J> > The current 'solisten' branch at https://github.com/glebius/FreeBSD has
> J> > been running successfully for 48 hours at Netflix. But as mentioned before,
> J> > our connection rate is pretty low.
> J> >
> J> > So, testing from Julien and Hiren will be much appreciated.
> J>
> J>  Sound good, I will have spare time to get performance results on our
> J> side w/ and w/o your change before the end of this week.
>
> Thanks, Julien! Looking forward for your results. Meanwhile, the code has
> been running for a week on a single Netflix box. And for 24 hours on a
> set of boxes as A/B test against non-patched code. Since our connection
> rate is low, and we never seen accept_mtx to be a major problem in our
> profiling, the A/B test didn't show any statistically significant
> improvements. But my biased eye of course sees some improvements in the
> graphs :)

Just got enough time to do some benchmark.  The configuration is
pretty simple, one server box w/ 2x82599, two client boxes w/ 1x82599,
connected through DAC.  nginx-devel is installed through pkg; access
log is disabled, 24 workers, each worker w/ 16384 max connections.  I
use the wrk I mentioned to you.  HTTP/1.1 and 1 request/connection,
15K concurrent connections from each client boxes, so 30K concurrent
connections total.  Each client has been verified to be able to do
160Kreqs/s w/ 1KB web object w/ 1 request/connection.  4 different
sized staic web objects are benched, see below.  The solisten was
checked out on Feb 25th.  Several GENERIC options (NFS and SCTP
related) have to be commented out and kernel is built w/ -DNO_MODULES
(CAM ctl module does not compile) and GENERIC-NODEBUG on the solisten
branch.  Each test lasts 2 minutes.  MSL on the server side has been
changed to 10ms, though I don't think it matters here.  Server hw:
2x2620v2 (HT enabled), 32GB ddr3-1600.  Client hw: i7-3770 (HT
enabled), 16GB ddr3-1600.

r314268 -> solisten

1KB:
Performance (reqs/s)
77916.71 -> 26240.37
Latency average
121ms -> 294ms

8KB:
Performance (reqs/s)
77803.72 -> 25968.87
Latency average
121ms -> 251ms

16KB:
Performance (reqs/s)
75698.77 -> 25047.23
Latency average
173ms -> 377ms

24KB:
Performance (reqs/s)
73736.80 -> 24946.36
Latency average
180ms -> 379ms

So what I have seen is solisten's performance is 1/3 of r314268, and
average latency doubles.

Thanks,
sephe

-- 
Tomorrow Will Never Die