ports/94997: [patch] port databases/memcached performance (TCP_NODELAY)
Anton Yuzhaninov
citrin at rambler-co.ru
Mon Mar 27 09:20:15 UTC 2006
>Number: 94997
>Category: ports
>Synopsis: [patch] port databases/memcached performance (TCP_NODELAY)
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-ports-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Mon Mar 27 09:20:13 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator: Anton Yuzhaninov
>Release: FreeBSD 6.1-BETA4
>Organization:
"Rambler Internet Holding" OJSC
>Environment:
Server: FreeBSD clam4.rambler.ru 6.1-BETA4 FreeBSD 6.1-BETA4 #0: Tue Mar 14 13:59:38 UTC 2006 root at wv1u.samsco.home:/usr/obj/usr/src/sys/GENERIC i386
memcached-1.1.12_2
Client: FreeBSD mcgi18.rambler.ru 6.1-BETA4 FreeBSD 6.1-BETA4 #0: Tue Mar 14 13:59:38 UTC 2006 root at wv1u.samsco.home:/usr/obj/usr/src/sys/GENERIC i386
p5-Cache-Memcached-1.15
>Description:
memcached has very big response time when reading stored objects size approximately
from 1400 to 2800 - when reply from server sent in two ip packets.
This caused by badly interaction between TCP delayed acknowledgements
(enabled in FreeBSD by default) and Nagle's algorithm.
Nagle's explanation of why interaction between delayed ACK and Nagle's
algorithm is bad:
http://developers.slashdot.org/comments.pl?sid=174457&threshold=1&commentsort=0&mode=thread&cid=14515105
So solution to improve latency is disable Nagle's algorithm for
memcached via socket option TCP_NODELAY.
This don't done by developers of memcached because on Linux TCP_CORK
(analog of TCP_NOPUSH) not compatible with TCP_NODELAY. And on
TCP_CORK show on Linux better performance than TCP_NODLEAY
>From memcached Changelog:
2003-08-12 (Brad Fitzpatrick)
* use TCP_CORK on Linux or TCP_PUSH on BSD
* only use TCP_NODELAY when we don't have alternatives
2003-08-10
* disable Nagel's Algorithm (TCP_NODELAY) for better performance (avva)
I run tests and it show, that TCP_NODELAY significantly decrease reading response time:
http://citrin.ru/stuff/memcached/read.png
and practically don't affect writing response time:
http://citrin.ru/stuff/memcached/write.png
Script used for this test:
http://citrin.ru/stuff/memcached/bench.pl
In result of using TCP_NODLEAY will be some network overhead, but in
environment where usually used memcached (100Mb or 1Gb LAN) it
negligible.
>How-To-Repeat:
Measure response time when reading from memcached objets size 1000, 2000 and 3000 bytes
(or in more wide range of sizes), e. g. using this script:
http://citrin.ru/stuff/memcached/test.pl
>Fix:
Apply this patch:
--- memcached.c.orig Thu Mar 23 18:51:47 2006
+++ memcached.c Thu Mar 23 18:57:56 2006
@@ -1146,9 +1146,7 @@
setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR, &flags, sizeof(flags));
setsockopt(sfd, SOL_SOCKET, SO_KEEPALIVE, &flags, sizeof(flags));
setsockopt(sfd, SOL_SOCKET, SO_LINGER, &ling, sizeof(ling));
-#if !defined(TCP_NOPUSH)
setsockopt(sfd, IPPROTO_TCP, TCP_NODELAY, &flags, sizeof(flags));
-#endif
/*
* the memset call clears nonstandard fields in some impementations
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-ports-bugs
mailing list