transparent squid bridge

[Tony Frank]

Hi,

On Tue, Mar 16, 2004 at 11:17:25AM -0700, Dan Vande More wrote:
> On Tue, 2004-03-16 at 05:38, Tony Frank wrote:
> > On Mon, Mar 15, 2004 at 09:08:28AM -0700, Dan Vande More wrote:
> > > Ok, to show I did try here's my diff on the manual patching. I've triple
> > > checked my work, and everything appears to be the way it is supposed to
> > > be.
> > 
> > Your diff looks ok, though I only took a cursury look at it.
> > Are you running with 'options BRIDGE' in your kernel or loading bridge module?
> > Are you using ipfw2 or ipfw1 ?
> 
> Options bridge is on, I am using ipfw2 / 5.2.1
> 
> Well ok, here comes the dumb part.
> 
> If I'm using 5.2.1, it's using ipfw2 by default right? I don't need to
> do anything special to use it, is what I read. (And ipfw2 is not a
> kernel option like it is in 4.x)
> I've googled six ways from sunday and don't see how to check with
> version I'm using.
> ipfw -v/ipfw --version/strings /sbin/ipfw doesn't work/reveal anything.
> 
For me, I get the following at bootup (refer dmesg.boot):

DUMMYNET initialized (011031)
bpf: lo0 attached
ipfw2 initialized, divert enabled, rule-based forwarding enabled, default to deny, logging unlimited
IPsec: Initialized Security Association Processing.

> > > Although the counter increases on rule 400 when a client requests a
> > > webpage on the other side of the bridge, a 'tcpdump port 80' on
> > > {proxy_server_ip_address} sees no packets whatsoever. 
> > 
> > > A tcpdump on the bridge server (tcpdump -n port 80) shows the packets
> > > from the client going straight to the requested host, instead of being
> > > hijacked and sent to the proxy server.
> > 
> > Is your proxy and your bridge on the same system?
> > If it is, perhaps try building the setup with two separate servers to 
> > make things a bit clearer.
> > Either setup seemed to work for me.
> 
> Yeah, they are different servers, for the sake of simplicity. Once I get
> this working, I intend to experiment with keeping them on the same
> server.
> By saying either worked for your, are you saying that you have this
> working?

Well, the divert parts seemed to work - as already mentioned the squid part
did not do so well, but that was not my immediate goal.

> > Note:
> > The traffic that is 'captured' will not be rewritten to have a new destination
> > IP address.
> > Rather that traffic is instead sent to the new destination directly.
> > It is up to the destination to 'recognise' this traffic and treat it 
> > appropriately.
> > If the hosts are on separate servers you should see 'uncaptured' traffic
> > being sent on to the default gateway (check ethernet link headers with -E option
> > to tcpdump) and 'captured' traffic being sent to the proxy server.
> > This is where it helps to have separate box for each function to clearly 
> > see the ethernet headers being different.
> > 
> 
> It doesn't look like you have it enabled but should I have 
> 
> net.inet.ip.check_interface=0
> 
> on? I know I need it when I run dsr on my load balanced mail servers,
> because they need to accept packets destined for another ip.

Based on my understanding, this has no effect in this scenario, comment from 
ip_input.c regarding this check:

         * Enable a consistency check between the destination address
         * and the arrival interface for a unicast packet (the RFC 1122
         * strong ES model) if IP forwarding is disabled and the packet
         * is not locally generated and the packet is not subject to
         * 'ipfw fwd'.

In my case ip forwarding is enabled, hence I believe this check is bypassed.

> > >From luigi's original post:
> > 
> > >on bridge:
> > >sysctl net.link.ether.bridge_cfg="rl0 rl1"
> > >sysctl net.link.ether.bridge_ipfw=1
> > >sysctl net.link.ether.bridge=1
> > >sysctl net.inet.ip.forwarding=1
> > >ipfw add forward proxy proto tcp from any to any 80
> > >
> > >on proxy:
> > >ipfw add forward localhost,8080 tcp from not me to any 80
> > 
> > Note this second ipfw line on the proxy itself.
> > It needs to be configured to redirect the traffic to the proxy software
> > port otherwise the server might well forward the traffic someplace
> > else (like to a default next-hop)
> > 
> > This setup works for me in that the tcp connection to port 80 is 
> > intercepted and sent to the local bridge/squid system, however my
> > squid currently returns an error due to 'invalid URL' or similar.
> > 
> > Basically it seems the URL hostname is not seen by squid.
> > I'm running 2.4.STABLE7 and did not configure anything new in squid.
> > That aside, the point is the redirecting of traffic from a bridge is working 
> > with these patches and FreeBSD 4.9-STABLE cvsupped today.
> > 
> > If I have the fwd line in ipfw I get a squid error page on by client
> > browser.
> > If I stop squid I get a browser error due to TCP reset being received
> > as nothing is listening on the proxy for the traffic.
> 
> So you got past the ipfw stuff, and now you can't get squid working?
> 
> Did you try --enable-ipf-transparent in the configure?

No

> How about httpd_accel_uses_host_header on in the conf?

I did try this option but it did not appear to have an impact.

> I have that setup working on openbsd, works perfect. But openbsd
> doesn't:( (Can't handle the load)

A quick read of http://www.squid-cache.org/Doc/FAQ/FAQ-17.html sorted me out.
I didn't need the --enable-ipf-transparent just the httpd_accel lines.
ie:
  
http_port localhost:3128
httpd_accel_port 80
httpd_accel_host virtual
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

My version:
Starting Squid Cache version 2.4.STABLE7 for i386-portbld-freebsd4.6.

Not sure if it was built with any transparent options. (whatever was 
in the ports makefile at the time)

I've also updated my squid to:

Squid Cache: Version 2.5.STABLE4
configure options:  --bindir=/usr/local/sbin --sysconfdir=/usr/local/etc/squid --datadir=/usr/local/etc/squid --libexecdir=/usr/local/libexec/squid --localstatedir=/usr/local/squid '--enable-storeio=ufs diskd null' '--enable-removal-policies=lru heap' '--enable-auth=basic ntlm digest' '--enable-basic-auth-helpers=NCSA PAM YP MSNT winbind' --enable-digest-auth-helpers=password '--enable-external-acl-helpers=ip_user unix_group wbinfo_group winbind_group' '--enable-ntlm-auth-helpers=SMB winbind' --enable-delay-pools --enable-snmp --disable-wccp --enable-underscores --enable-useragent-log --enable-arp-acl '--enable-err-languages=Bulgarian Catalan Czech Danish Dutch English Estonian Finnish       French German Hebrew Hungarian Italian Japanese Korean Lithuanian   Polish Portuguese Romanian Russian-1251 Russian-koi8-r Serbian      Simplify_Chinese Slovak Spanish Swedish Traditional_Chinese Turkish' --enable-default-err-language=English --prefix=/usr/local i386-portbld-freebsd4.9


> > > *************************************
> > > bash-2.05b# egrep -v "^#" /etc/sysctl.conf
> > > sysctl net.link.ether.bridge_cfg='xl0 dc0'
> > > sysctl net.link.ether.bridge_ipfw=1
> > > sysctl net.link.ether.bridge=1
> > > sysctl net.inet.ip.forwarding=1
> > > *************************************
> > 
> > Umm, is this really what you're using?
> > 
> Yeah but the 3rd one is wrong, with 5.2 it appears it's:
> 
> sysctl net.link.ether.bridge.enable=1
> 
> (Which is what I used, I just forgot to include it above.)

Some things have obviously changed 4.9 <-> 5.2

> > I have squid listening on 127.0.0.1 port 3128.
> > I also got it working using a second server, albeit with the error
> > listed above (which is squid configuration, not ipfw)
> Thank you Tony, you've been a big help, considering I still didn't even
> know if this had been done by anybody but Mr. Rizzo.

I dont have a present need for this in bridging environment, just
thought I'd have a crack at it since I have the bits all together
at the moment.

> In your opinion, is it better for me to try this on 4.9?
> 

:)  Works for me on 4.9-STABLE (plus patches)

I might try 5.2.1 as well, will advise what happens.

Regards,

Tony