negative dns caching problem *resolved* (was Re: JDK 1.5.0 patchset 7 "South China")

[Nick Johnson]

After months of fighting with this, today I stumbled upon the answer.  It 
was a java command line test case that led me to it.

>From the command line, FreeBSD's JDK does -exactly- what it is supposed to 
with negative caching.  My test case involved restarting BIND, adding a 
firewall rule for a host's name servers, having Java try to resolve the 
name (and of course fail), having Java wait while removing the firewall 
rule and having Java try again to resolve the name.  All the while I was 
watching network traffic with tcpdump.

Exactly as it is supposed to, the JDK tried again and succeeded after I 
removed the firewall rule.

So this meant that it had to be something to do with Tomcat or 
commons-daemon/jsvc.

On a whim, I removed the entry for Tomcat's native libraries, and it 
started to work correctly.

Then I did an ldd against libtcnative, and found that I had compiled it 
with gcc 4.2.1.  Knowing that there are problems with gcc 4.2, I 
recompiled with gcc 3.4.6.  It continues to work correctly.

So this turned out to be a multi-pronged problem.  Earlier I was building 
a version of the JDK that somehow never read java.security, and that 
required wiping out the work directory entirely and bootstrapping off the 
Diablo JDK.  Then, something about compiling the Tomcat native libraries 
with gcc 4.2.1 was apparently creating this bad behaviour situation.

So if you're using Tomcat, and you want to use the Tomcat native 
libraries, do not compile them with gcc 4.2.  In fact, you probably will 
want to make sure you compile them with the same major.minor version of 
gcc you used to compile libapr just to be safe.  The Tomcat native library 
IS capable of screwing your DNS resolution.

   Nick

On Sun, 28 Oct 2007, Nick Johnson wrote:

> Sadly, this does NOT correct the eternal negative caching problem.  That 
> problem still remains, even though I've configured the JDK not to cache 
> negative results at all.
> 
> I can do a tcpdump and see that the JDK makes no attempt whatsoever to 
> query a host again after it has previously timed out.  Nothing I've tried 
> to fix the problem has had any effect.
> 
>    Nick
> 
> On Fri, 26 Oct 2007, Nick Johnson wrote:
> 
> > It remains to be seen whether this will resolve the problems I've been 
> > having with InetAddress caching, but I am hopeful that it will.
> 
> 

-- 
"Courage isn't just a matter of not being frightened, you know. It's being
 afraid and doing what you have to do anyway."
   Doctor Who - Planet of the Daleks
This message has been brought to you by Nick Johnson 2.3b1 and the number 6.
http://healerNick.com/       http://morons.org/        http://spatula.net/