Problem with ipfw, in-kernel NAT and port redirection to jails

[Kiryanov Vassily]

Hello Alexey,

Thank you for this information, I have thoughts about using pf nat as
an alternative way and your example will be useful for me.

But Eugene Grosbein adviced me to turn off tso4 on network card
underlaying my VLANs and it was enough to solve problem with port
redirection. Without turning tso4 off ipfw + in-kernel NAT works
fine but port redirection fails. 

Saturday, February 6, 2016, 1:57:47 PM, you wrote:

ARvfn> Hello.
ARvfn> I have same problem when I'm trying redirect incoming traffic into the 
ARvfn> jailed web server.
ARvfn> I repeated my installation few times on different releases - problem 
ARvfn> with redirected ports was here all time (except 9.3 - there was random 
ARvfn> result).
ARvfn> As a temporary solution am using pf nat for redirect ports.

ARvfn> My test configuration:
ARvfn> /etc/rc.conf:
ARvfn> ifconfig_vtnet0="inet 192.168.1.18/24"
ARvfn> defaultrouter="192.168.1.1"
ARvfn> cloned_interfaces="lo1"

ARvfn> /etc/jail.conf:
ARvfn> exec.start = "/bin/sh /etc/rc";
ARvfn> exec.stop = "/bin/sh /etc/rc.shutdown";
ARvfn> exec.clean;
ARvfn> j1 {
ARvfn>          path = /home/jail1;
ARvfn>          mount.devfs;
ARvfn>          host.hostname = j1;
ARvfn>          interface = "lo1";
ARvfn>          ip4.addr = 10.8.0.1;
ARvfn>          persist;
ARvfn> }

ARvfn> rc.firewall:
ARvfn> ipfw nat 1 config if vtnet0 redirect_port tcp 10.8.0.1:80 80
ARvfn> ipfw add 500 nat 1 ip from any to 192.168.1.18 in via vtnet0
ARvfn> ipfw add 600 nat 1 ip from 10.8.0.1 to any out via vtnet0
ARvfn> ipfw add allow ip from any to any

ARvfn> pf.conf:
ARvfn> ext_if = "vtnet0"
ARvfn> int_if = "lo1"
ARvfn> jail_net = $int_if:network
ARvfn> nat on $ext_if from $jail_net to any -> ($ext_if)
ARvfn> rdr pass on $ext_if inet proto tcp from any to ($ext_if:0) port 80 -> 
ARvfn> 10.8.0.1 port 80

ARvfn> In jail I'm try nginx, apache24 and nc as source for redirection. Test 
ARvfn> file was generated: dd if/dev/random of=tmp.raw bs=1M count=2
ARvfn> On 10.1 and 10.2 there is no big differences, when using ipfw nat we can 
ARvfn> get only part of file (I'm using curl on different machine: curl 
ARvfn> http://192.168.1.18/tmp.raw > /dev/null):
ARvfn> with nginx: Received = 33045
ARvfn> with apache: Received = 33092
ARvfn> with nc: Received = 16384
ARvfn> and result seems to be very stable in numbers.
ARvfn> On 9.3:
ARvfn> nginx: random bytes received, has no successful downloads
ARvfn> apache: random bytes received, sometimes download entire file
ARvfn> nc: entire file received

ARvfn> My virtual environment is proxmox 3.
ARvfn> Maybe it's related to 
ARvfn> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137346 or just not 
ARvfn> properly configured ipfw nat?
ARvfn> _______________________________________________
ARvfn> freebsd-net at freebsd.org mailing list
ARvfn> https://lists.freebsd.org/mailman/listinfo/freebsd-net
ARvfn> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"



-- 
Best regards,
 Kiryanov                            mailto:kvas at bf.pstu.ru