HELP! "key_acquire2: invalid sequence number is passed" -- IPSEC VPN down...

[Steve Camp]

Hi,

[ I have already posted this question to the 'freebsd-questions' mailing list and several
newsgroups.  I found a question posted to this 'freebsd-net' mailing list back in 2001, but
apparently no summary or solution was posted.]

I need some help.  I am running a VPN between a FreeBSD 4.3 box and another FreeBSD 4.7 box.
I am using the IPSEC / Racoon setup that comes with FreeBSD.  I have not compiled anything:
I inherited sysadmin duties for these boxen from another fellow.  They had been working
just fine when I first "acquired" them.  Since that time, my customer has had two moves when
they physically consolidated their two offices into one new office.  At that time, the VPN
was torn down, as there was only one box.

Now they have opened a new "branch" office (actually a "home" office) and have tasked me
with re-establishing the VPN to this separate location.  I took the second box and re-located
it.  The only changes made were to the /etc/hosts (new host name(s) and IP addresses), 
/etc/resolv.conf (new dns servers), and some tweaks to /etc/rc.conf, and IP re-configurations
in the /usr/local/etc/rc.d/ipsec.sh startup script.

The IPSEC VPN has been up and down, but frustratingly mostly down since this latest "move".
However, the VPN *was* working, and working well just two days ago.  Today I checked, and it
is again down, and the "primary" company server is logging lots and lots of these messages:

    Aug 28 18:07:00 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 28 18:10:00 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 28 18:13:00 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 28 18:16:30 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 28 18:19:00 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 28 18:22:00 servername /kernel: key_acquire2: invalid sequence number is passed.

       .
       .
       .

    Aug 29 11:46:36 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 29 11:49:18 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 29 11:50:00 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 29 11:50:47 servername /kernel: key_acquire2: invalid sequence number is passed.
    Aug 29 11:54:52 servername /kernel: key_acquire2: invalid sequence number is passed.

    etc etc

Any pointers / links / help etc welcome in trying to figure this problem out.

Has anyone experienced this problem before?  How to resolve / fix it?

Could this behaviour be caused by an ISP restricting certain kinds of traffic?  More
specifically, the last time I checked a few days ago, I was able to ping the public
IP address of the remote (e.g. home office) box, but now I get ICMP error messages
about

        ICMP Communication Administratively Prohibited from gateway machine.isp.net (xxx.xx.xxx.xxx)

While I have posted this query to the comp.dcom.vpn, and comp.unix.*bsd*.misc newsgroups,
any pointers to any other apropos Usenet newsgroups, mailing lists, support websites 
appreciated.

--
Steve Camp
steve at camp.com