Fwd: Re: IPFW: Blocking me out. How to debug?
W. D.
WD at US-Webmasters.com
Sun Dec 23 12:14:12 PST 2007
>Date: Sun, 23 Dec 2007 06:04:02 -0800 (PST)
>From: Nash Nipples <trashy_bumper at yahoo.com>
>To: freebsd-security at freebsd.org
>Subject: Re: IPFW: Blocking me out. How to debug?
>
>Dear W.D.
>
>oh come on. i have the same problem.
Which problem are we talking about?
>cut and paste logic:
>
>#!/bin/sh
>#1. count packets
>#2. allow everything on lo0 (loopback)
>#3. slow down and deny packets to buffer overflow enabled daemons
>#3.5 to list all the buffer overflow enabled daemons use this sockstat -46ul
>#4. allow everything in and out on the Ethernet interface fxp0.
>Remember - wires are long things!
What does this mean?
>#5. switch sshd to a different port like 55 and use keys to authenticate
>#6. do ipfw show every morning
Why?
>#7. do ipfw zero every evening or as often as your boss wants that
How about a cron job that sends out a report then zero's ipfw?
>#8. learn how to modify this script quickly just to plumb all the
>other things that leak
What other things?
>#9. you cant block yourself out if you run this script with a trailing
>'&' e.g. sh /etc/ipfw.rules &
Hmmm. I'll have to try that. Would something like this
work to reload the rules?
ipfw disable firewall; sh /etc/ipfw.rules &
>#TODO: write a program that sends bills to customers
>#BUGS: it cant smile
>
>cmd="/sbin/ipfw"
>ext1="fxp0"
>gentleports="21,25,514"
Why are these ports "gentle"?
>
>#accounting
>#i need these figures to see how bad things are going
>$cmd 100 count ip from any to any in via lo0
>$cmd 110 count ip from any to any out via lo0
>$cmd 120 count ip from any to any in via $ext1
>
>
>
>$cmd 130 count ip from any to any out via $ext1
>
>#if counters below grow too high u are screwed
How so?
> #this counter should not vary much comparing to the next one
>$cmd 210 count icmp from any to any out via $ext1
> #if there is an obvious difference someone's digging holes in the yard
What does this mean?
> $cmd 220 count icmp from any to any in via $ext1
> #too much of dns.
>$cmd 230 count ip from any to any 53 out via $ext1
>$cmd 240 count ip from any to any 53 in via $ext1
Why too much?
>
>#if counters below grow too high you have screwed someone else
Who? Why?
>#oh yes. someone's got mail.
>$cmd 300 count ip from any to any 25 out via $ext1
>#which way did it go
>$cmd 310 count ip from any to any 25 via lo0
>
>#policy
>$cmd 1000 allow all from any to any via lo0
>/sbin/ipfw add 1110 pipe 1 ip from any to me $gentleports in via $ext1
>/sbin/ipfw pipe 1 config bw 1Kbit/s queue 1Kbytes
>
>$cmd 1120 deny ip from any to me $gentleports in via $ext1
>$cmd 1130 allow all from any to any via ext1
>#you will wonder but the next rule still has a match
>$cmd 1140 deny log all from any to any
What is it?
>
>I'm sorry but i cant draw pretty pictures to make it any more obvious
To make what more obvious?
>
>Nash
>
>
>----- Original Message ----
>From: W. D. <WD at US-Webmasters.com>
>To: FreeBSD-Security at FreeBSD.org
>Cc: Ian Smith <info at plot.uz>
>Sent: Friday, December 21, 2007 9:30:11 AM
>Subject: Re: IPFW: Blocking me out. How to debug?
>
>
>At 05:45 12/20/2007, Ian Smith, wrote:
>
>Thanks for your reply Ian. This is the kind of
>information I am looking for.
>
>
>>Firstly, this really belongs over on freebsd-net@ if not
>>freebsd-questions@, but anyway ..
>
>I'll be glad to move it there if you would like. I
>figured that since IPFW/Firewalls are security
>related, that FreeBSD-Security would be the most
>appropriate place.
>
>
>>On Thu, 20 Dec 2007, W. D. wrote:
>>
>> > At 03:49 12/17/2007, Tuomo Latto wrote:
>> > >W. D. wrote:
>> > >> How do I tell which rule is blocking me out? SSH *is* working,
>> > >> but others are not.
>> > >
>> > >It all depends on what you mean by "blocking you out" and
> "others".
>>
>>True; it's not really clear what you're trying to do, whether this is
> a
>>single server with a single net interface with no NAT or what, but
> based
>>on your present rules I'll have to make that assumption.
>
>OK, sorry. I guess I just assumed that it would be obvious
>that this is a Web server. ("Never assume anything, my good
>fellow" - Sherlock Holmes).
>
>By the way, it is/will be running Plesk server management
>software, if it matters:
>http://www.swsoft.com/en/products/plesk/reqs/
>
>Also, this server is on an internal LAN before I subject
>it to the wild, untamed, InterWeb, with its dangerous
>internets darting back and forth inside all of the tubes.
>
>
>> > >> # Loopback:
>> > >> # Allow anything on the local loopback:
>> > >> add allow all from any to any via lo0
>> > >> add deny ip from any to 127.0.0.0/8
>> > >> add deny ip from 127.0.0.0/8 to any
>>
>>Meaning, these rules are ok and not the problem. Ignore Mr. Nipples.
>
>With a name like that, it's hard to take him seriously. ;^)
>
>
>>
>> > >> # Allow established connections:
>> > >> add allow tcp from any to any established
>>
>>That's ok. It may help you in debugging what's happening to use:
>>
>> allow [log] tcp from any to any in established
>> allow [log] tcp from any to any out established
>
>I assume here that "[log]" means to insert "log" for
>debugging like this:
>
> allow log tcp from any to any in established
> allow log tcp from any to any out established
>
>rather than including the square brackets, "[" & "]",
>correct?
>
>I have done that and have included my latest ruleset
>below.
>
>
>
>>and really, using 'any to any' without specifying on which interfaces
> or
>>whether 'any' is your box or the outside world is a bit too general,
> but
>>moving on ..
>
>OK. What should I do? I only plan on having one
>Ethernet interface. What would be more secure?
>
>
>
>> > >> # Deny fragmented packets:
>> > >> add deny ip from any to any frag
>
>> > >> # Show pings:
>> > >> add count icmp from any to any icmptypes 8 in
>> > >
>>
>>That's inbound ping requests. Don't forget that 'inbound' means
> coming
>>into the firewall, not necessarily from the outside world. Your own
>>ping requests _from_ this box also have to both come in, and go out.
>
>Hmmm. OK. Outbound Ping will be rarely used, but should
>be allowed. Isn't that included in the next rule?
>
>
>> > >> # Allow pings, ping replies, and host unreach:
>> > >> add allow icmp from any to any icmptypes 0,8,3
>> > >
>>
>>Add icmptype 11 as well if you want traceroutes to work ..
>>
>> > >> # Allow UDP traceroutes:
>> > >> add allow udp from any to any 33434-34458 in
>> > >> add allow udp from any 33434-34458 to any out
>> > >
>>
>>Ok, though udp rules are often better done statefully. See below.
>>
>> > >> # Allow DNS with name server
>> > >> add allow udp from any to any domain out
>> > >> add allow udp from any domain to any in
>> > >Nope.
>>
>>You want to watch out here. This allows udp packets from any address
>>with source port 53 to connect with any open udp port on your system,
>>and allows the responses as well. It's a simple matter using such as
>>netcat to source packets from port 53.
>
>Should I restrict it by specifically stating the service?
>How can I be safe? What would the rule look like?
>
>
>>I gather from this that you're not running a DNS server yourself, but
>>using upstream server/s? In that case a stateful rule is safer:
>
>Again, I apologize for not being clear. I will be running
>DNS on this box for the domains being hosted. So, it will
>be polled whenever a request for a hosted domain is needed.
>
>
>
>> allow udp from me to any 53 keep-state
>>
>>which allows after the return packets but denies connections not
>>initiated from your box.
>>
>> > >> # SSH
>> > >> # Note that /etc/hosts.allow has restrictions
>> > >> # on which IP addresses are allowed.
>> > >> #
>> > >> # Allow SSH:
>> > >> add allow tcp from any to any ssh in setup
>> > >
>>
>>By 'ssh working', I guess you mean ssh connections to this box from
>>elsewhere, rather than ssh connections from this box? Not clear.
>
>Sorry! I am using SSH into this box, since it is easier to
>cut and paste for editing and configuration.
>
>I can't really see a situation where I would normally need to SSH
>outbound, can you? I use the Windoze boxes for that.
>
>
>> > >> # HTTP & HTTPS:
>> > >> add allow tcp from any to any https in setup
>> > >> add allow tcp from any to any http in setup
>> > >
>>
>>So, you have a webserver running on this box, listening on ports 80
> and
>>443? You've verified with 'netstat -finet -a' that this is the case?
>
>Yes:
>
># netstat -finet -a
>Active Internet connections (including servers)
>Proto Recv-Q Send-Q Local Address Foreign Address
> (state)
>tcp4 0 0 192.168.1.109.ssh 192.168.1.107.3502
> ESTABLISHED
>tcp4 0 52 192.168.1.109.ssh 192.168.1.107.2266
> ESTABLISHED
>tcp4 0 0 *.poppassd *.*
> LISTEN
>tcp4 0 0 *.ftp *.*
> LISTEN
>tcp4 0 0 *.smtps *.*
> LISTEN
>tcp4 0 0 *.smtp *.*
> LISTEN
>tcp4 0 0 localhost.locald.3000 *.*
> LISTEN
>tcp4 0 0 *.pop3s *.*
> LISTEN
>tcp4 0 0 *.pop3 *.*
> LISTEN
>tcp4 0 0 *.imaps *.*
> LISTEN
>tcp4 0 0 *.imap *.*
> LISTEN
>tcp4 0 0 *.8443 *.*
> LISTEN
>tcp4 0 0 *.8880 *.*
> LISTEN
>tcp4 0 0 *.3306 *.*
> LISTEN
>tcp4 0 0 localhost.locald.8005 *.*
> LISTEN
>tcp4 0 0 *.9008 *.*
> LISTEN
>tcp4 0 0 *.8009 *.*
> LISTEN
>tcp46 0 0 *.https *.*
> LISTEN
>tcp46 0 0 *.http *.*
> LISTEN
>tcp4 0 0 *.9080 *.*
> LISTEN
>tcp4 0 0 *.8180 *.*
> LISTEN
>tcp4 0 0 localhost.locald.postg *.*
> LISTEN
>tcp4 0 0 localhost.locald.rndc *.*
> LISTEN
>tcp4 0 0 localhost.locald.domai *.*
> LISTEN
>tcp4 0 0 192.168.1.109.domain *.*
> LISTEN
>tcp4 0 0 *.ssh *.*
> LISTEN
>udp4 0 0 192.168.1.109.24889 ns1.ournameserver.net.53
>udp4 0 0 *.51750 *.*
>udp4 0 0 localhost.locald.domai *.*
>udp4 0 0 192.168.1.109.domain *.*
>udp4 0 0 *.syslog *.*
>
>
>
>
>>
>> > >> # Mail: SMTP & IMAP:
>> > >> add allow tcp from any to any smtp in setup
>> > >> add allow tcp from any to any imap in setup
>> > >
>>
>>You're running SMTP and IMAP servers, verified as above?
>>
>>You see, this also allows you (as 'any') to connect to any outside
> SMTP
>>server too. It really helps to differentiate connections into your
> box
>>from those you're making to outside boxes, which these don't do.
>>
>>Have a close look at the 'simple' section in rc.firewall.
>
>I have scanned various versions of "rc.firewall". I kinda understand
>what is going on, but there are so many places that seem anti-intuitive
>to me.
>
>Also, what are the differences between running a script and loading
>these rules on bootup?
>
>
>
>>There are
>>advantages to running a script such as that rather than rules in a
> file,
>>like variable substitution, at least while getting it all working
> right.
>
>I have mixed feelings about variables. I guess they make
>it easier if you change a network card or IP addresses--you
>only have to do it in one place.
>
>However, a search and replace command doesn't take much
>time at all. Also, they add a level of complexity to
>a situation that (to me) is complex enough already.
>And, I wonder if by not using variables, I can save a
>few microseconds when processing traffic. ;^)
>
>
>
>> > >> # FTP:
>> > >> add allow tcp from any to any ftp in setup
>> > >> add allow tcp from any to any ftp\-data in setup
>> > >> add allow tcp from any ftp\-data to any setup out
>> > >
>>
>>Mmm, I prefer using and enforcing FTP passive mode, but YMMV.
>
>How would I do that? This guy doesn't think it's even
>possible:
>http://tinyurl.com/2z6ynr
>
>
>> > >> # Allow NTP in and out
>> > >> add allow udp from any ntp to 128.252.19.1 ntp out
>> > >> add allow udp from 128.252.19.1 ntp to any ntp in
>> > >
>>
>>Unless running a time service for other boxes, something like:
>>
>> allow udp from me to any ntp keep-state # or to a specific server
>
>Well, I think that since NTP is such a minimal user
>of resources, that I would like to rely on this
>box for the correct time. That way, I don't have
>to bug the stratum 1 boxes. Shall I use my original?
>
>
>>
>> > >> # Deny and log everything else:
>> > >> add deny log all from any to any
>> > >Bingo!
>>
>>Ok, so you got rid of interface 'all', great.
>
>Is this better?
>
> add deny log ip from any to any
>
>If so, I just don't understand this. Here is what the
>"Fine Manual" says:
>
> ==============================================================
> protocol: [not] protocol-name | protocol-number
> An IP protocol specified by number or name (for a complete list
> see /etc/protocols), or one of the following keywords:
>
> ip4 | ipv4
> Matches IPv4 packets.
>
> ip6 | ipv6
> Matches IPv6 packets.
>
> ip | all
> Matches any packet.
> ==============================================================
>
>According to this, "ip" and "all" are synonymous. Criminy!
>What am I missing here?
>
>
>
>> > >"ipfw -a list" may also help (packet counts).
>>
>>It's only a short ruleset, it may help us if you show the output of
> say
>>'ipfw -t show' (or ipfw -at list, same thing), if you're still having
>>problems, but see below re 'inness' and 'outness'.
>>
>> > I've been banging my head against this for the past few
>> > days. I don't get it.
>> >
>> > My understanding of the way this is supposed to work is
>> > that:
>> >
>> > # HTTP & HTTPS:
>> > add allow tcp from any to any https in setup
>> > add allow tcp from any to any http in setup
>> >
>> > should let initial HTTP & HTTPS requests through,
>> > and that:
>> >
>> > # Allow established connections:
>> > add allow tcp from any to any established
>>
>>Not quite. Looks like you're allowing http/https setup packets in
> (ie,
>>into the firewall) but not letting them out (of the firewall, to the
>>webserver). For example in the 'simple' ruleset mentioned, we have:
>>
>> # Allow access to our WWW
>> ${fwcmd} add pass tcp from any to ${oip} 80 setup
>>
>> # Reject&Log all setup of incoming connections from the
> outside
>> ${fwcmd} add deny log tcp from any to any in via ${oif} setup
>>
>> # Allow setup of any other TCP connection
>> ${fwcmd} add pass tcp from any to any setup
>
>I really don't get the above rule. Isn't it saying that
>*any* kind of TCP connection can come in or go out initially?
>
>
>
>>Note there's no 'in' or 'out' on the port 80 rule, so this allows the
>>packets on both the in and out pass of the firewall. Also, the IP is
>>specified as our IP - 'me' will do fine if it's just this box.
>
>In my set, should I include some "out" rules like this:
>
> add allow tcp from any to any https out setup
> add allow tcp from any to any http out setup
>
>
>>
>> > should allow connections that are "setup" to
>> > continue. Do I need a "check-state" or "keep-state"
>> > statement somewhere?
>>
>>No, though you can use stateful TCP rules if you want to, in which
> case
>>you'll want to DENY established connections. Personally I find
> relying
>>on the TCP state established by using 'setup' and 'established' fine
> for
>>TCP, but tend to use keep-state for UDP and some ICMP rules.
>
>That sounds reasonable.
>
>
>> > I don't understand what is wrong with the last rule:
>> >
>> > # Deny and log everything else:
>> > add deny log all from any to any
>> >
>> > My understanding is that anything that doesn't match
>> > the previous rules will match this one and hence
>> > be logged and denied. Is this not correct?
>>
>>That's correct. Aren't you seeing any? Try show rather than tell.
>
>Showing:
>
># ipfw -a -S -N -t list
>00100 688 173384 Thu Dec 20 15:32:17 2007 set 0 allow log logamount
> 10 ip from any to any via lo0
>00200 0 0 set 0 deny log logamount 10
> ip from any to 127.0.0.0/8
>00300 0 0 set 0 deny log logamount 10
> ip from 127.0.0.0/8 to any
>00400 4344 1712050 Fri Dec 21 00:23:37 2007 set 0 allow log logamount
> 10 tcp from any to any established
>00500 0 0 set 0 deny log logamount 10
> ip from any to any frag
>00600 4 240 Wed Dec 19 23:05:31 2007 set 0 count icmp from any
> to any icmptypes 8 in
>00700 8 480 Wed Dec 19 23:05:31 2007 set 0 allow log logamount
> 10 icmp from any to any icmptypes 0,3,8
>00800 0 0 set 0 allow log logamount 10
> udp from any to any dst-port 33434-34458 in
>00900 0 0 set 0 allow log logamount 10
> udp from any 33434-34458 to any out
>01000 366 24038 Fri Dec 21 00:02:00 2007 set 0 allow log logamount
> 10 udp from any to any dst-port domain out
>01100 364 59582 Fri Dec 21 00:02:00 2007 set 0 allow log logamount
> 10 udp from any domain to any in
>01200 1 48 Thu Dec 20 16:49:47 2007 set 0 allow log logamount
> 10 tcp from any to any dst-port ssh in setup
>01300 0 0 set 0 allow log logamount 10
> tcp from any to any dst-port https in setup
>01400 6 288 Thu Dec 20 14:43:38 2007 set 0 allow log logamount
> 10 tcp from any to any dst-port http in setup
>01500 98 6272 Fri Dec 21 00:02:00 2007 set 0 allow log logamount
> 10 tcp from any to any dst-port http
>01600 1 64 Thu Dec 20 15:25:01 2007 set 0 allow log logamount
> 10 tcp from any to any dst-port https
>01700 0 0 set 0 allow log logamount 10
> tcp from any to any dst-port smtp in setup
>01800 0 0 set 0 allow log logamount 10
> tcp from any to any dst-port imap in setup
>01900 43 2064 Wed Dec 19 23:16:18 2007 set 0 allow log logamount
> 10 tcp from any to any dst-port ftp in setup
>02000 0 0 set 0 allow log logamount 10
> tcp from any to any dst-port ftp-data in setup
>02100 0 0 set 0 allow log logamount 10
> tcp from any ftp-data to any setup out
>02200 100 7600 Thu Dec 20 23:47:00 2007 set 0 allow log logamount
> 10 udp from any ntp to navobs1.wustl.edu dst-port ntp out
>02300 100 7600 Thu Dec 20 23:47:00 2007 set 0 allow log logamount
> 10 udp from navobs1.wustl.edu ntp to any dst-port ntp in
>02400 2058 226123 Fri Dec 21 00:17:20 2007 set 0 deny log logamount 10
> ip from any to any
>65535 7 909 Wed Dec 19 22:58:29 2007 set 31 deny ip from any to
> any
>
>Lot of stuff being denied. I think some of that
>is my HTTP and HTTPS initial requests. What to do?
>
>>
>> > Again, I am having a great deal of difficulty
>> > understanding why these rules don't work as expected.
>> > I've scoured the 'Net and printed out just about
>> > every coherent ruleset out there.
>> >
>> > Besides adding the "log" keyword on all of the rules,
>> > these are the debugging tools I have been using:
>> >
>> > ipfw disable firewall
>> > ipfw -f flush
>> > ipfw enable firewall
>> > /etc/rc.d/ipfw start
>> > ipfw -a -S -N -t list
>> > ipfw list
>> > tail -f /var/log/ipfw/ipfw.log
>> > tcpdump -i nve0 'proto \tcp && port http'
> netstat -finet -a
>> >
>> > Could anyone please throw this tired dog a bone?
>>
>>Getting the two-pass nature of ipfw understood seems to be your main
>>difficulty.
>
>I would definitely agree with you. I am completely lost.
>
>By "two-pass" do you mean "in" to the firewall, and then
>"in" to the webserver, and another "two-pass": "out" from the
>webserver, and "out" from the firewall?
>
>
>>I know it was for me back then. Have a look at the section
>>in ipfw(8) regarding packet flows, and although it sounds trite, RTFM
>>about 10 times :)
>>
>>Cheers, Ian
>
>Love the "Fine Manual" --not! Just not enough examples for
>me to understand everything. Too much abstraction--AAArrrrgh!
>
>Am using this link, since "man ipfw" doesn't work on 6.2. (I dare
>someone to explain to me how to get it to work):
>http://www.freebsd.org/cgi/man.cgi?query=ipfw&sektion=8
>
>Thanks for your help, Ian. Would appreciate it if you would
>kick my butt in the proper direction again.
>
>Any other takers/kickers?
>
>
>Latest grope in the dark:
>===============================================================
># ipfw.rules
># ipfw firewall ruleset
># 2007 Dec 20
>
># By default, everything is denied access. You
># need to specifically allow something for it
># to work.
>
># Loopback:
># Allow anything on the local loopback:
>add allow log all from any to any via lo0
>
># Disallow spoofed access to local:
>add deny log ip from any to 127.0.0.0/8
>add deny log ip from 127.0.0.0/8 to any
>
># Allow established connections:
>add allow log tcp from any to any established
>
># Deny fragmented packets:
>add deny log ip from any to any frag
>
># Show pings:
>add count icmp from any to any icmptypes 8 in
>
># Allow pings, ping replies, and host unreach:
>add allow log icmp from any to any icmptypes 0,8,3
>
># Allow UDP traceroutes:
>add allow log udp from any to any 33434-34458 in
>add allow log udp from any 33434-34458 to any out
>
># Allow DNS with name server
>add allow log udp from any to any domain out
>add allow log udp from any domain to any in
>
># SSH
># Note that /etc/hosts.allow has restrictions
># on which IP addresses are allowed.
>#
># Allow SSH:
>add allow log tcp from any to any ssh in setup
>
># HTTP & HTTPS:
>add allow log tcp from any to any https in setup
>add allow log tcp from any to any http in setup
>
>add allow log tcp from any to any dst-port 80
>add allow log tcp from any to any dst-port 443
>
># Mail: SMTP & IMAP:
>add allow log tcp from any to any smtp in setup
>add allow log tcp from any to any imap in setup
>
># FTP:
>add allow log tcp from any to any ftp in setup
>add allow log tcp from any to any ftp\-data in setup
>add allow log tcp from any ftp\-data to any setup out
>
># Allow NTP in and out
>add allow log udp from any ntp to 128.252.19.1 ntp out
>add allow log udp from 128.252.19.1 ntp to any ntp in
>
>
># Deny and log everything else:
># add deny log all from any to any
>add deny log ip from any to any
>===============================================================
>
>
>
>
>
>Start Here to Find It Fast! ->
> http://www.US-Webmasters.com/best-start-page/
>$8.77 Domain Names -> http://domains.us-webmasters.com/
>
>_______________________________________________
>freebsd-security at freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-security
>To unsubscribe, send any mail to
> "freebsd-security-unsubscribe at freebsd.org"
>
>
>
>
>
>
>
>______________________________________________________________________
>______________
>Never miss a thing. Make Yahoo your home page.
>http://www.yahoo.com/r/hs
>_______________________________________________
>freebsd-security at freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-security
>To unsubscribe, send any mail to "freebsd-security-unsubscribe at freebsd.org"
Start Here to Find It Fast! -> http://www.US-Webmasters.com/best-start-page/
$8.77 Domain Names -> http://domains.us-webmasters.com/
More information about the freebsd-questions
mailing list