jail issue

[Robert Watson]

On Fri, 13 Feb 2004, Jim Prewett wrote:

> I've been using jails (very happily) for quite some time and have
> *never* had a problem like this.  I really don't have a clue what to
> look for :) 

Could you check and see if sks is installed in the host environment, and
if so, make sure there are no cron jobs, etc, associated with it?

> I'm getting complaints from fellow keyserver ops as my IP seems to
> sometimes be the jail and sometimes the host, so some of my packets get
> rejected as that IP has not been configured (by the remote host) to be a
> peer.  (how strange is that?!) 

I think the question we need to answer for sure is "Which process is
generating the TCP connection"; the follow-up, if it's the process we
think it is, figure out what system call is being used.

> nox# sockstat | grep sks
> root     sks        276    5 tcp4   129.24.244.40:11371   *:*                  
> root     sks        271    4 tcp4   129.24.244.40:11370   *:*                  
> root     sks        276    6 stream ./db_com_sock                              
> root     sks        271    5 stream ./recon_com_sock                

This all looks fine.

>  5804  p2  S+     0:00.00 grep sks
>   271 con- S+J    0:03.29 sks recon
>   276 con- S+J    0:11.50 sks db

This too.

> nmap of host (nox) and jail (pgp):
> 
> nox# nmap nox pgp  -p 11370-11371

Ditto.

> If there is anything else that I can provide, please let me know.  I'm
> *very* interested in resolving this. 

I think what we should do is add some calls to log() or printf() in the
TCP connect code to figure out what processes are generating TCP
connections, and using what addresses.  I don't know how comfortable you
are with C programming, or for that matter kernel programming, but a good
start would be to add some calls to log() in
src/sys/netinet/tcp_usrreq.c:tcp_connect(), specifically just above the
call to soisconnecting() to print out the source and destination IP
addresses and port numbers, as well as p->p_pid (process ID) and p->p_comm
(cached filename string for the binary).  If you find that it is the
process you think it should be, and it's really binding the wrong address,
we should move the printfs higher in tcp_connect() and see what the
arguments to connect() are.  Depending on how you feel about a lot of
kernel output, you might want to strcmp("sks", p->p_comm) so you only get
output from processes you're interested in.

If you'd rather, I can generate a patch with these changes.  My
recollection from earlier e-mail is that you're running 4.9-STABLE?

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Senior Research Scientist, McAfee Research