kern/141134: "ioctl (SIOCAIFADDR): File exists" error when configuring ppp interfaces

[Alexander Zagrebin]

>Number:         141134
>Category:       kern
>Synopsis:       "ioctl (SIOCAIFADDR): File exists" error when configuring ppp interfaces
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 03 12:00:11 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Alexander Zagrebin
>Release:        FreeBSD 8.0-RELEASE
>Organization:
-
>Environment:
FreeBSD gw.vosz.local 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sun Nov 29 00:37:18 MSK 2009     root at gw.vosz.local:/usr/src/sys/amd64/compile/KERNEL  amd64
>Description:
I use mpd 5.3 (ports/net/mpd5) as vpn server.
After upgrading to 8.0-RELEASE from 7.2-RELEASE I began to receive complaints from users. They says, that the first attempt to connect to our vpn server always fails, and only the second attempt is successful.
The log files confirms this.
I have found how to reproduce this problem with command line.

I'll create the interface tun10:

# ifconfig tun10 create inet 192.168.100.1 192.168.100.10 netmask 0xffffffff
# ifconfig tun10
tun10: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        inet 192.168.100.1 --> 192.168.100.10 netmask 0xffffffff
# netstat -rn
..
192.168.100.1      link#13            UHS         0        0    lo0
192.168.100.10     link#13            UH          0        0  tun10
..
# ping -c1 192.168.100.1
PING 192.168.100.1 (192.168.100.1): 56 data bytes
64 bytes from 192.168.100.1: icmp_seq=0 ttl=64 time=0.050 ms

--- 192.168.100.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.050/0.050/0.050/0.000 ms

All right.
The interface was successfully created, loopback route was added, and i can ping self.
Now I'll create the interface tun11 for another peer:

# ifconfig tun11 create inet 192.168.100.1 192.168.100.11 netmask 0xffffffff
ifconfig: ioctl (SIOCAIFADDR): File exists
# ifconfig tun11
tun11: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        inet 192.168.100.1 --> 192.168.100.11 netmask 0xffffffff
# netstat -rn
..
192.168.100.1      link#13            UHS         0        1    lo0
192.168.100.10     link#13            UH          0        0  tun10
192.168.100.11     link#14            UH          0        0  tun11
..

The first problem:
ifconfig returned error, but interface was successfully configured.
The same error, returned from ioctl, forces ppp daemon to terminate ipcp and disconnect a peer.

Now I'll destroy tun11 and will create it again or will reconfigure it with the same parameters (the result is the same):

# ifconfig tun11 destroy
# ifconfig tun11 create inet 192.168.100.1 192.168.100.11 netmask 0xffffffff
or
# ifconfig tun11 inet 192.168.100.1 192.168.100.11 netmask 0xffffffff
# netstat -rn
..
192.168.100.1      link#14            UHS         0        0    lo0
192.168.100.10     link#13            UH          0        0  tun10
192.168.100.11     link#14            UH          0        0  tun11
..

There is no error from ifconfig now. It's strange.
loopback route was changed.
This explains why the second attempt of a peer to connect is successful.
If we'll create additional interfaces (tun12,tun13, etc), the behavior will the same.

The second problem:
Suppose the peer on tun11 terminates her session.
# ifconfig tun11 destroy
# netstat -rn
..
192.168.100.10     link#13            UH          0        0  tun10
..
# ping -c1 192.168.100.1
PING 192.168.100.1 (192.168.100.1): 56 data bytes

--- 192.168.100.1 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss

So loopback route to 192.168.100.1 was deleted and I'll can't ping self. It's correct for destroyed tun11 (link#14), but as still there is the tun10 (link#13), we need loopback route to 192.168.100.1

After looking to the sources and doing some debugging I have found, that the first problem (EEXIST from ioctl) occurs because of call ifa_add_loopback_route
from in_ifinit (in.c:928). So I temporary disabled adding loopback routes via 
net.link.ether.inet.useloopback, but this is bad idea.


>How-To-Repeat:
See the "full description"
>Fix:
To prevent adding of loopback routes I use
sysctl net.link.ether.inet.useloopback=0


>Release-Note:
>Audit-Trail:
>Unformatted: