Weird behavior with -stable

[kniveton]

There is a box that's been running 4-stable for a long time. A few days ago, I changed the 
IP address of the box to a different address on the same subnet, and also cvsup'ed -stable 
and rebuilt world and the kernel. Recently, I also added the options IPSEC and IPSEC_ESP

Now for the past few days, in the evening this box stops responding to imap, http, and 
other ports. Weirdly, it is still responding to ssh, but gives the wrong host key and/or 
doesn't accept my password.

When I go into work in the morning, I can log in fine on the console, and things appear to 
still be running.. a reboot brings everything back fine.

Any idea what this could be?

Thanks...
TJ