packet delay because of blackhole
ap00 at mail.ru
Tue Apr 1 07:03:49 PDT 2008
Tuesday, April 01, 2008, 2:41:29 PM, you wrote:
RP> On Fri, Mar 28, 2008 at 08:14:58PM +0300, Anthony Pankov wrote:
>> Just for somebody convince.
>> While analyzing client<->server HTTPS conversation one second delay in
>> packet exchange was discovered (strongly reproducible):
>> N time
>> 6 0.002303 10.28.4.14 10.28.4.50 SSL Client Hello
>> 7 0.106710 10.28.4.50 10.28.4.14 TCP 443 > 1447 [ACK] Seq=1 Ack=103 Win=65535 Len=0
>> 8 1.045712 10.28.4.50 10.28.4.14 TLSv1 Server Hello, Certificate, Server Hello Done
>> Another sample:
>> 10 0.011722 10.28.4.14 10.28.4.50 TLSv1 Application Data
>> 11 0.115933 10.28.4.50 10.28.4.14 TCP 443 > 1442 [ACK] Seq=839 Ack=519 Win=65466 Len=0
>> 12 1.054037 10.28.4.50 10.28.4.14 TLSv1 Application Data
>> The reason for delay is sysctl tcp.blackhole value grater than 0, much to surprise.
>> So, turning tcp.blackhole to 0 eliminate any delay (strongly reproducible).
>> System: FreeBSD 6_2_stable
RP> I'm not sure how performance penalty can induce a cache miss and I
RP> it's very processor specific. So, you're best guess is to profile the
I'm not fully understand what cache miss do you mean.
I'll try to be more clear.
During client<->server HTTPS conversation there is a packet delay (see "sample" and
"Another sample") about 900 ms. Delay appear one per conversation in
random place (between 7-8 packet in "sample", 11-12 in "another
sample"). Of course, it's not depending from SSL session cache, SSL
negotiation or any other apache/mod_ssl/OpenSSL setting/performance,
otherwise i should write to another maillist.
I have disabled all my sysctl tuning, one by one. No effect has
But when i turn tcp.blackhole to zero, all things became fine. Maximum
delay between packet is 6 ms.
It is strange, so i've reported to all.
I suggest to keep tcp.blackhole=0 and use firewall for protection. If
one would raise tcp.blackhole value, than he should dump packets and
make sure that there is no strange delay between packets.
It most likely FreeBSD net issue.
P.S. "Another sample" is not a sequel of "Sample", it is a dump of
Anthony mailto:ap00 at mail.ru
More information about the freebsd-performance