IPv6 fragmentation weirdness

[Bjoern A. Zeeb]

On Thu, 14 May 2009, Kevin Oberman wrote:

Hi,

>> Date: Fri, 15 May 2009 00:09:02 +0200 (CEST)
>> From: sthaug at nethelp.no
>>
>>> First, why is the kernel fragmenting this at all as it fits in the
>>> interface MTU?
>>
>> Good question, I definitely disagree with this behavior and would say
>> that it breaks POLA. But it's documented (see the ping6 -m option).
>>
>>> Can anyone fetch anything from ftp.funet.fi via IPv6? I suspect it is
>>> something in the path that is blocking my traffic, so others may not see
>>> this, but I think the root issues is the kernel fragmenting packets way
>>> below MTU size.
>>
>> I just picked up a copy of the 7.2 bootonly ISO image using IPv6. Slow
>> but usable. My path (from Oslo, Norway) is:
>>
>> sthaug at lab1% traceroute6 ftp.funet.fi
>> traceroute6 to ftp.funet.fi (2001:708:10:9::20:1) from 2001:8c0:8b00:1::2, 64 hops max, 12 byte packets
>>  1  ge-0-0-9-515.br1.fn3.no.catchbone.net  0.254 ms  4.917 ms  0.203 ms
>>  2  c10G-ge-5-1-0.cr2.osls.no.catchbone.net  0.485 ms  0.408 ms  0.399 ms
>>  3  c10G-xe-4-1-0.br1.osls.no.catchbone.net  0.364 ms  0.351 ms  0.361 ms
>>  4  2001:2000:3083:6::1  9.006 ms  8.848 ms  8.966 ms
>>  5  s-ipv6-b1-link.ipv6.telia.net  19.481 ms  19.590 ms  19.412 ms
>>  6  2001:2000:3080:d::2  110.907 ms  109.056 ms  119.495 ms
>>  7  helsinki0-rtr.funet.fi  116.305 ms  123.534 ms  119.472 ms
>>  8  csc0-x0000-helsinki0.ipv6.funet.fi  118.873 ms  117.439 ms  116.054 ms
>>  9  ftp.funet.fi  115.777 ms  116.087 ms  117.735 ms
>>
>> Note that the IPv6 transit from Telia is tunnelled, and the RTT is awful
>> compared to IPv4 (IPv4 RTT to ftp.funet.fi from the same box is around
>> 17 ms).
>>
>> Steinar Haug, Nethelp consulting, sthaug at nethelp.no
>>
>
> Thanks, Steinar.
>
> I just re-read the man page and I had misunderstood what it was
> saying. That still leave me baffled as to what is happening.
>
> My path is, as would be expected, very different.
> traceroute6 to ftp.funet.fi (2001:708:10:9::20:1) from 2001:400:0:40::200:101, 64 hops max, 12 byte packets
> 1  esnet.rt1.ams.nl.geant2.net (2001:798:29:10aa::9)  83.998 ms  115.099 ms  85.969 ms
> 2  so-7-0-0.rt2.cop.dk.geant2.net (2001:798:cc:1501:2201::1)  101.692 ms  96.955 ms  96.868 ms
> 3  nordunet-gw.rt2.cop.dk.geant2.net (2001:798:15:10aa::2)  179.931 ms  205.407 ms  195.268 ms
> 4  dk-uni.nordu.net (2001:948:0:f055::2)  210.468 ms se-fre.nordu.net (2001:948:0:f03f::1)  187.479 ms dk-uni.nordu.net (2001:948:0:f055::2)  190.578 ms
> 5  se-tug.nordu.net (2001:948:0:f049::2)  188.170 ms  213.538 ms se-tug.nordu.net (2001:948:0:f056::1)  183.273 ms
> 6  helsinki0-rtr.funet.fi (2001:948:0:f035::2)  188.114 ms  189.214 ms  192.192 ms
> 7  csc0-x0000-helsinki0.ipv6.funet.fi (2001:708:0:f000::1:2)  186.166 ms  190.181 ms  186.669 ms
> 8  ftp.funet.fi (2001:708:10:9::20:1)  186.251 ms  198.591 ms  205.987 ms
>
> This is exactly the same as my IPv4 path. Something along this path is
> silently refusing to pass a packet at the start of the transfer and that
> screams MTU.
>
> If I ping from a Juniper router, I can get 1482 byte packets through, so
> I suspect that there is a tunnel somewhere. But FreeBSD boxes die at the
> lower limit.
>
> Does the kernel fragmentation only affect ICMP or are TCP packet also
> fragmented at 1280 bytes?

WRT to TCP you may also want to check the hostcache:
sysctl net.inet.tcp.hostcache.list

-- 
Bjoern A. Zeeb                      The greatest risk is not taking one.