amd64/134486: Wrong MSS in outgoing packets for non-default (1460) MSS

[Andrey Voitenkov]

>Number:         134486
>Category:       amd64
>Synopsis:       Wrong MSS in outgoing packets for non-default (1460) MSS
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-amd64
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 12 10:30:00 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Andrey Voitenkov
>Release:        7.2-RELEASE
>Organization:
>Environment:
FreeBSD thin.DOMAIN.ua 7.2-RELEASE FreeBSD 7.2-RELEASE #2: Mon May 11 01:03:51 EEST 2009     root at thin.DOMAIN.ua:/usr/obj/usr/src/sys/THIN  amd64
>Description:
Scheema:
HostA -> ADSL modem -> Internet -> HostB

HostA is a client, HostB - server, FreeBSD 7.2-RELEASE, amd64.
HostA has MTU 1500 on external interface, HostB too.
Modem reduces MSS to 1412. After this HostB sends tcp packets back to HostA with wrong MSS.

tcpdump on HostB:
18:50:14.888507 IP hostA.2709 > hostB.80: S 3410180266:3410180266(0) win 65535 <mss 1412,nop,wscale 0,nop,nop,sackOK>
18:50:14.888522 IP hostB.80 > hostA.2709: S 3741845445:3741845445(0) ack 3410180267 win 65535 <mss 1412,nop,wscale 3,sackOK,eol>
18:50:14.914346 IP hostA.2709 > hostB.80: . ack 1 win 65535
18:50:14.922517 IP hostA.2709 > hostB.80: P 1:434(433) ack 1 win 65535
18:50:15.022010 IP hostB.80 > hostA.2709: . ack 434 win 8295
18:50:15.089538 IP hostB.80 > hostA.2709: . 1:1461(1460) ack 434 win 8295  <-- here is the problem
18:50:18.089355 IP hostB.80 > hostA.2709: . 1:1413(1412) ack 434 win 8295  <-- and normal packet after 3 seconds timeout

tcpdump on HostA:
18:50:14.925084 IP hostA.2709 > hostB.80: S 3410180266:3410180266(0) win 65535 <mss 1426,nop,wscale 0,nop,nop,sackOK>
18:50:14.937834 IP hostB.80 > hostA.2709: S 3741845445:3741845445(0) ack 3410180267 win 65535 <mss 1412,nop,wscale 3,sackOK,eol>
18:50:14.952628 IP hostA.2709 > hostB.80: . ack 1 win 65535
18:50:14.954715 IP hostA.2709 > hostB.80: P 1:434(433) ack 1 win 65535
18:50:15.071258 IP hostB.80 > hostA.2709: . ack 434 win 8295
18:50:18.142391 IP hostB.80 > hostA.2709: . 1:1413(1412) ack 434 win 8295

HostB, more verbose:
# tcpdump -n -vvv -i fxp0 src hostA or dst hostA
tcpdump: listening on fxp0, link-type EN10MB (Ethernet), capture size 96 bytes
10:13:28.167171 IP (tos 0x0, ttl 121, id 21599, offset 0, flags [DF], proto TCP (6), length 48) hostA.1262 > hostB.80: S, cksum 0x2a48 (correct), 3449703460:3449703460(0) win 64240 <mss 1412,nop,nop,sackOK>
10:13:28.167183 IP (tos 0x0, ttl 64, id 2277, offset 0, flags [DF], proto TCP (6), length 48) hostB.80 > hostA.1262: S, cksum 0x3f38 (incorrect (-> 0x012b), 3492959435:3492959435(0) ack 3449703461 win 65535 <mss 1412,sackOK,eol>
10:13:28.180318 IP (tos 0x0, ttl 121, id 21600, offset 0, flags [DF], proto TCP (6), length 40) hostA.1262 > hostB.80: ., cksum 0x31cd (correct), 1:1(0) ack 1 win 64240
10:13:28.188108 IP (tos 0x0, ttl 121, id 21601, offset 0, flags [DF], proto TCP (6), length 430) hostA.1262 > hostB.80: P 1:391(390) ack 1 win 64240
10:13:28.287657 IP (tos 0x0, ttl 64, id 2283, offset 0, flags [DF], proto TCP (6), length 40) hostB.80 > hostA.1262: ., cksum 0x3f30 (incorrect (-> 0x2b38), 1:1(0) ack 391 win 65535
10:13:28.365461 IP (tos 0x0, ttl 64, id 2286, offset 0, flags [DF], proto TCP (6), length 1500, bad cksum 0 (->ed18)!) hostB.80 > hostA.1262: . 1:1461(1460) ack 391 win 65535
10:13:31.365039 IP (tos 0x0, ttl 64, id 2311, offset 0, flags [DF], proto TCP (6), length 1452) hostB.80 > hostA.1262: . 1:1413(1412) ack 391 win 65535
10:13:31.602996 IP (tos 0x0, ttl 121, id 21625, offset 0, flags [DF], proto TCP (6), length 40) hostA.1262 > hostB.80: ., cksum 0x2ac3 (correct), 391:391(0) ack 1413 win 64240
10:13:31.603009 IP (tos 0x0, ttl 64, id 2312, offset 0, flags [DF], proto TCP (6), length 1500, bad cksum 0 (->ecfe)!) hostB.80 > hostA.1262: . 1413:2873(1460) ack 391 win 65535
10:13:37.802884 IP (tos 0x0, ttl 64, id 2332, offset 0, flags [DF], proto TCP (6), length 1452) hostB.80 > hostA.1262: . 1413:2825(1412) ack 391 win 65535
10:13:37.931321 IP (tos 0x0, ttl 121, id 21626, offset 0, flags [DF], proto TCP (6), length 40) hostA.1262 > hostB.80: ., cksum 0x253f (correct), 391:391(0) ack 2825 win 64240
10:13:37.931330 IP (tos 0x0, ttl 64, id 2334, offset 0, flags [DF], proto TCP (6), length 1500, bad cksum 0 (->ece8)!) hostB.80 > hostA.1262: . 2825:4285(1460) ack 391 win 65535
10:13:50.132504 IP (tos 0x0, ttl 64, id 2364, offset 0, flags [DF], proto TCP (6), length 1452) hostB.80 > hostA.1262: . 2825:4237(1412) ack 391 win 65535
10:13:50.299392 IP (tos 0x0, ttl 121, id 21631, offset 0, flags [DF], proto TCP (6), length 40) hostA.1262 > hostB.80: ., cksum 0x1fbb (correct), 391:391(0) ack 4237 win 64240
10:13:50.299406 IP (tos 0x0, ttl 64, id 2365, offset 0, flags [DF], proto TCP (6), length 1500, bad cksum 0 (->ecc9)!) hostB.80 > hostA.1262: . 4237:5697(1460) ack 391 win 65535

Problem does not depend on www-server used in tests above. Situation is the same with ftp and scp.
>How-To-Repeat:
Just try to download anything from a web server running 7.2-RELEASE amd64 via ADSL connection.
Alternatively the problem can be repeated in LAN after reducing MSS on client manually, using PF's max-mss for example.

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: