IPv6 aliases: one doesn't work, the other do

[Frédéric Perrin]

Hello,

I have a box running 8.0-RELEASE on i386. It has several jails, each
one being given an IPv6 alias. I notice that some jails can be reached
from the outside, others can't. Conversely, if I set as the source
address alias1, nothing comes back; it I set as the source address
alias2, it works as expected. The following transcript may be clearer:

This is happening on papillon, the host (meaning not a jail), after a
fresh reboot.

,----
| root at papillon:~# grep 'ipv6\|vr0' < /etc/rc.conf
| ifconfig_vr0="inet 91.121.77.72 netmask 255.255.255.0 broadcast 91.121.77.255"
| ifconfig_vr0_alias0="87.98.132.43 netmask 255.255.255.255"
| ifconfig_vr0_alias1="188.165.50.152 netmask 255.255.255.255"
| ipv6_enable="YES"
| ipv6_network_interfaces="vr0"
| ipv6_defaultrouter="2001:41d0:1:82ff:ff:ff:ff:ff"
| ipv6_ifconfig_vr0="2001:41d0:1:8248::1 prefixlen 56"
| ipv6_ifconfig_vr0_alias0="2001:41d0:1:8248::2"
| ipv6_ifconfig_vr0_alias1="2001:41d0:1:8248::3"
| ipv6_ifconfig_vr0_alias2="2001:41d0:1:8248::4"
| ipv6_ifconfig_vr0_alias3="2001:41d0:1:8248::5"
`----

benoute.fr is a friend's box, located in the same datacenter as mine.
If I use as source address ::3 or ::5, I get no answer, with the other
aliases it works as expected.

,----
| root at papillon:~# traceroute6 -n -s 2001:41d0:1:8248::3 mail.benoute.fr
| traceroute6 to mail.benoute.fr (2001:41d0:1:c1d7::1) from 2001:41d0:1:8248::3, 64 hops max, 12 byte packets
|  1  * * *
|  2  * * *
|  3  * * *
|  4  * * *
|  5  * * *
| ^C
| root at papillon:~# traceroute6 -n -s 2001:41d0:1:8248::4 mail.benoute.fr
| traceroute6 to mail.benoute.fr (2001:41d0:1:c1d7::1) from 2001:41d0:1:8248::4, 64 hops max, 12 byte packets
|  1  * * *
|  2  2001:41d0:1:c1d7::1  0.396 ms  0.301 ms  0.296 ms
`----

Same thing, but using renater.fr (a French ISP peering directly with
the datacenter that hosts my box):

,----
| root at papillon:~# traceroute6 -n -s 2001:41d0:1:8248::4 www.renater.fr
| traceroute6 to www.renater.fr (2001:660:3001:4002::10) from 2001:41d0:1:8248::4, 64 hops max, 12 byte packets
|  1  *
|     2001:41d0:1:82ff:ff:ff:ff:ff  5.882 ms *
|  2  2001:41d0::592  150.251 ms  7.933 ms *
|  3  2001:41d0::522  220.006 ms  230.797 ms  229.390 ms
|  4  2001:7f8:4e:2::103  4.636 ms  5.934 ms  4.331 ms
|  5  2001:660:7903:e:1::2  4.877 ms  4.525 ms  4.401 ms
|  6  2001:660:7903:a:2::2  4.387 ms  4.375 ms  4.362 ms
|  7  2001:660:3000:1008:10:0:6:5051  4.867 ms !P  4.825 ms !P  5.083 ms !P
| root at papillon:~# traceroute6 -n -s 2001:41d0:1:8248::3 www.renater.fr
| traceroute6 to www.renater.fr (2001:660:3001:4002::10) from 2001:41d0:1:8248::3, 64 hops max, 12 byte packets
|  1  * * *
|  2  * * *
|  3  * * *
|  4  * * *
|  5  *^C
`----

If I go to gadget (a Linux host with IPv6 connectivity), I can ping
some aliases, but no others. The aliases that pong correctly are the
same as the ones I can successfully use as source addresses in
traceroute6 to remote hosts. A 'tcpdump -i vr0 icmp6' on papillon
while this is happening show that no packet is seen by papillon.

,----
| fperrin at gadget:~$ for i in $( seq 5); do echo -n "$i - "; ping6 -c1 2001:41d0:8248::$i|grep loss; done
| 1 - 1 packets transmitted, 1 received, 0% packet loss, time 0ms
| 2 - 1 packets transmitted, 1 received, 0% packet loss, time 0ms
| 3 - 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
| 4 - 1 packets transmitted, 1 received, 0% packet loss, time 0ms
| 5 - 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
`----

Even if my life depended on it, I couldn't explain why some aliases
work, and not others. The only difference that I can see is that they
have different jails (::2 runs httpd+postgres, ::3 has no listening
dæmons, ::4 runs named, ::5 runs postfix).

Any possible pointers?

-- 
Fred