performance impact of large /etc/hosts files

[Alex Zbyslaw]

Erich Dollansky wrote:

> But new sites have new stuff I would like to be filtered out. To make 
> these experiences as rare as possible, I collect from friends and the 
> Internet hosts files to filter as much as possible.
>
> This resulted in a pretty large file meanwhile.
>
> But the Internet looks much more usable for me now.

Assuming I've understood your initial post correctly, then I do the 
same, redirecting some dozen ad sites to a  local web server.  With a 
dozen or so aliases I've never noticed any difference in performance, 
but I suspect you have rather more than that :-)  I could never quite be 
bothered to maintain the list once I'd filtered ads from the sites I use 
most often.

I think the answer to your original question is going to be "look at the 
source code".  If your hosts file is really that large then I suspect it 
will be having a performance effect and only you can judge if it's 
significant or not.  Large hosts files are not the future, so 
performance improvements in the future are unlikely, I would say.

I'm pretty sure you could also do the same with a local DNS server, if 
you wanted to "abuse" it in this way, and that would *probably* be 
faster since the code would expect to deal with large lists of hosts.  
Been a while since I did anything like that, though, and never on the 
scale you seem to be describing.

There's no clean solutions to getting different lookups per-user that I 
am aware of, but unless your host is also performing some service that 
involves a lot of name resolution then why care?  (And if it is, you 
shouldn't be using it as a general web browser :-))

Unclean solutions might include something like making the hosts file 
point to some automounted directory which changed per user, but you'd 
have to be sure that you saw a valid hosts file at boot time.  Fiddling 
with symlinks in rc scripts could do that, I'm sure.

--Alex