TCP Port Capture

[Doug Hardie]

I have a process that is occasionally giving me a situation that I don't understand.  It listens to a well-known port and processes requests received on that report.  It does return data to the calling system.  When a request is received, a child process is created that actually does most of the work.  The child process uses another process via a pipe during the processing to obtain some data it uses.

Occasionally, the main process exits with nothing logged.  I suspect its doing an exit, but thats just a guess at this time.  Some of the time (fairly infrequently), the main process cannot be restarted.  The process at the end of the pipe is now shown as having the port open.  lsof shows that.  If I restart that process, then the main process can be restarted properly.

Why would the process at the end of the pipe end up being attached to the port?  Unfortunately, the system always seems to be quiet (no requests in process) when this happens so nothing is logged by any process.  Normally I have a monitoring process that detects the main process being gone and restarts it.  But that doesn't work when the port is held by the final process.