network problems 7.0-p3: sendto: Operation not permitted
Robert Jameson
rj at dawnshosting.com
Thu Jul 24 11:01:02 UTC 2008
Hello everyone, I'm not sure about how this works, I was told to share this
information with this group because of a possible issue with PF
My rules are in place @ http://rj.dawnshosting.com/fbsd_ml/pf.conf
If anyone has a chance and can tell me what is wrong about them, it would
mean alot to me.
My configuration worked fine before the update to 7.0-P3 like i said, but if
we fix the rules then we can begin the process of elimanation.
Thank's so much guys
From: *Robert Jameson* <rj at dawnshosting.com>
Date: Thu, Jul 24, 2008 at 1:59 AM
To: freebsd-stable <freebsd-stable at freebsd.org>
Hello Everyone,
Recently I upgraded to freebsd 7.0-p3 from 7.0-p2, once i upgraded i began
to have problems with my network, nothing has changed configuration wise,
let me show you guy's an example.
(12:46 AM):(root at cube)/$ ping google.com
PING google.com (72.14.207.99): 56 data bytes
64 bytes from 72.14.207.99: icmp_seq=0 ttl=240 time=64.713 ms
^C
--- google.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 64.713/64.713/64.713/0.000 ms
(12:46 AM):(root at cube)/$ ping google.com
PING google.com (72.14.207.99): 56 data bytes
64 bytes from 72.14.207.99: icmp_seq=0 ttl=240 time=73.814 ms
64 bytes from 72.14.207.99: icmp_seq=1 ttl=240 time=64.943 ms
^C
--- google.com ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 64.943/69.379/73.814/4.435 ms
(12:46 AM):(root at cube)/$ ping google.com
PING google.com (72.14.207.99): 56 data bytes
ping: sendto: Operation not permitted
^C
--- google.com ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
(12:46 AM):(root at cube)/$
As you can see above, I issued the ping command (4) times waiting for output
and then doing CTRL+C to interrupt the commands quickly and send them again
on the 4th try i did not intterupt it and received the operation not
permitted. hitting ctrl+c on this error I can type ping again and it will
work correctly.
I have the same problem with almost every network command, wget, curl,
fetch, lynx, ssh, nslookup, host etc.
This appears to be an issue with the network.
I have attached my rc.conf and sysctl.conf and pf.conf please let me know if
any other information is required.
Errors from /var/log/console.log:
Jul 18 21:10:02 cube kernel: Jul 18 21:10:02 cube named[908]: socket: too
many open file descriptors
Jul 19 00:30:13 cube kernel: Jul 19 00:30:13 cube named[9748]: socket: too
many open file descriptors
Jul 19 00:30:54 cube kernel: Jul 19 00:30:14 cube last message repeated 28
times
Initially I figured this problem was bind related and since it has been a
planned project for the past few months to switch to djbdns, I took the time
to switch to djbdns, so bind is no longer running.
I was also receiving this in /var/log/messages:
Jul 20 22:15:39 cube kernel: Limiting open port RST response from 318 to 200
packets/sec
Jul 20 22:15:40 cube kernel: Limiting open port RST response from 624 to 200
packets/sec
Jul 20 22:15:42 cube kernel: Limiting open port RST response from 213 to 200
packets/sec
Jul 20 22:15:50 cube kernel: Limiting open port RST response from 439 to 200
packets/sec
Jul 20 22:15:51 cube kernel: Limiting open port RST response from 673 to 200
packets/sec
Jul 20 22:15:52 cube kernel: Limiting open port RST response from 730 to 200
packets/sec
Jul 20 22:15:53 cube kernel: Limiting open port RST response from 307 to 200
packets/sec
Jul 20 22:16:02 cube kernel: Limiting open port RST response from 435 to 200
packets/sec
Jul 20 22:16:03 cube kernel: Limiting open port RST response from 730 to 200
packets/sec
Jul 20 22:16:04 cube kernel: Limiting open port RST response from 287 to 200
packets/sec
Jul 20 22:16:13 cube kernel: Limiting open port RST response from 519 to 200
packets/sec
Jul 20 22:16:14 cube kernel: Limiting open port RST response from 740 to 200
packets/sec
Jul 20 22:16:15 cube kernel: Limiting open port RST response from 258 to 200
packets/sec
Jul 20 22:16:24 cube kernel: Limiting open port RST response from 407 to 200
packets/sec
Jul 20 22:16:25 cube kernel: Limiting open port RST response from 660 to 200
packets/sec
After spending some time on Google i came up with:
/etc/sysctl.conf
net.inet.icmp.icmplim=2000
I know it seems abit high, but i kept adjusting until the error went away.
(not really fixing the problem?)
If your mail client or the mailing list prevents you from seeing the
attached
You can view them here:
http://rj.dawnshosting.com/fbsd_ml/
PS: While running tcpdump I see this
tcpdump -i fxp0
Neither one of these ip's exist on my system is my cable company doing
something wrong?
01:47:12.135929 arp who-has 64.253.3.161.dyn-cm-pool73.pool.hargray.net tell
64.253.3.1.dyn-cm-pool73.pool.hargray.net
01:47:12.155931 arp who-has 216.16.218.141.dyn-cm-pool46.pool.hargray.nettell
216.16.218.1.dyn-cm-pool46.pool.hargray.net
01:47:12.196000 arp who-has 181.131.216.67.181.static.hargray.net tell
1.131.216.67.1.static.hargray.net
tcpdump -i fxp0 | grep ICMP:
Is this an attack?
01:55:41.231722 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37084, length 64
01:55:42.232794 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37085, length 64
01:55:43.285913 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37086, length 64
01:55:44.286340 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37087, length 64
01:55:45.287380 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37088, length 64
01:55:46.345843 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37089, length 64
01:55:47.346685 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37090, length 64
01:55:48.347366 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37091, length 64
01:55:49.348370 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37092, length 64
01:55:50.360130 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37093, length 64
01:55:51.596916 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37094, length 64
01:55:52.597659 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37095, length 64
01:55:53.640120 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37096, length 64
01:55:54.735275 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37097, length 64
01:55:55.735568 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37098, length 64
01:55:56.745012 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37099, length 64
01:55:57.835442 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37100, length 64
01:55:58.920583 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37101, length 64
01:56:00.022747 IP cube.dawnshosting.com > purple.haze.bluntroll.in: ICMP
echo request, id 22055, seq 37102, length 64
----------
From: *Alex Trull* <alex at trull.org>
Date: Thu, Jul 24, 2008 at 3:29 AM
To: Robert Jameson <rj at dawnshosting.com>
Robert,
The config files you attached were a series of 403 forbidden htmls.
The icmp pings (1 per second) do not constitute an attack.
It looks like you are genuinely running out of free states or file
descriptors.
Had you applied any tuning that may have been lost in the upgrade ?
How many packets and sessions is this host meant to be handling - and what
sort of traffic ?
--
Alex
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
----------
From: *Alex Trull* <alex at trull.org>
Date: Thu, Jul 24, 2008 at 3:31 AM
To: Robert Jameson <rj at dawnshosting.com>
Cc: freebsd-stable <freebsd-stable at freebsd.org>
----------
From: *Jeremy Chadwick* <koitsu at freebsd.org>
Date: Thu, Jul 24, 2008 at 3:49 AM
To: Robert Jameson <rj at dawnshosting.com>
Cc: freebsd-stable <freebsd-stable at freebsd.org>
Let's see if I can figure out the multitude of things you've posted
about, since a bunch are unrelated and you appear to be flailing around
with your arms in the air. :-)
This usually indicates firewall rules on the local machine, although I
believe there are some other operations where EPERM can be returned.
Can you provide uname -a output? There was a "cable modem compatibility
fix" applied to FreeBSD a while ago (a user informed me of such),
although I do not know if it applies to you, as I do not know the
original symptoms. I believe that fix was also just for TCP.
This indicates a completely different/unrelated problem.
This indicates a high number of ICMP packets being received. Keep in
mind this can also be seen due to TCP connections which are being reset
and other such things -- ICMP is at a higher layer than TCP.
I don't think there's necessarily anything "wrong" with that number (you
show up to 740), but it would be worthwhile investigating what's
soliciting that amount of ICMP traffic. Are you seeing this 24x7x365?
It's not a big high; FreeBSD's 200 default is too low for any production
server, if you ask me. Setting it to 2000 is probably fine.
You should discuss your firewalling rules on freebsd-pf, and not here.
I believe you may have some mistakes which are inducing said problem.
Nope. This is normal behaviour for a cable modem network; they
constantly spam layer 2 ARP for *everyone* on the entire cable network
segment. Yes, you read that right.
At this rate (1 ICMP packet a second), absolutely not. You also don't
mention which FQDN/IP is yours; I assume "cube.dawnshosting.com", based
on your local hostname in the above. Your machine is sending out an
ICMP ping packet to purple.haze.bluntroll.in every 1 second. If you
don't know why, you need to investigate why.
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
----------
From: *Robert Jameson* <rj at dawnshosting.com>
Date: Thu, Jul 24, 2008 at 5:55 AM
To: Jeremy Chadwick <koitsu at freebsd.org>
Sorry about that, bit of a information overload, i really am flailing my
arms around!
Tried running with my firewall disabled/wide problem still occurs
FreeBSD cube.dawnshosting.com 7.0-RELEASE-p3 FreeBSD 7.0-RELEASE-p3 #5: Wed
Jul 16 21:55:02 EDT 2008
root at cube.dawnshosting.com:/usr/obj/usr/src/sys/CUBE
i386
Was the patch applied upstream? if not and its not too much trouble can you
point me in the direction of it.
Ah, thought they were related, what's causing this :)!
Yes its constant. let it me known i also have a 2 network cards in the
machne, 1 into my cable modem and nother into a linksys 16port vpn router.
the defaultrouter is set to a WAN IP (not 10.192.240.1), not that any of
that matters, i dont think?
I read a bit about it from the handbook, i think it's a non issue.
Might be worth mentioning the only real service change to this machine was
an ircd daemon w/ about 500 users.
I will send them an e-mail shortly, thanks.
ah, ok, nothing to see here, keep moving.
Correct, cube.dawnshosting.com is the actual FreeBSD machinr.
sorry for the newbish question, off the top of your head how can i see
who/what is using this process?
>
> --
----------
From: *Robert Jameson* <rj at dawnshosting.com>
Date: Thu, Jul 24, 2008 at 6:21 AM
To: freebsd-stable <freebsd-stable at freebsd.org>
Still don't know whats going on, im currently sitting here with no firewall
between me and the internet (very nervous) seeing if it fixes the problems,
as of right this moment, still seeing permission denied errors.
I have fixed the 403 errors now.
http://rj.dawnshosting.com/fbsd_ml/ now contains sysctl.conf rc.conf pf.conf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rc.conf
Type: application/octet-stream
Size: 344 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20080724/dde533fb/rc.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sysctl.conf
Type: application/octet-stream
Size: 344 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20080724/dde533fb/sysctl.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pf.conf
Type: application/octet-stream
Size: 344 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20080724/dde533fb/pf.obj
-------------- next part --------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQBIiC/mey4m6/eWxTQRAvnQAJ9H2EeeOYcpNqxt1DLwG69stTDLBACgibvr
FIMP7ahxUHjP19f2LjTNc7k=
=UPoB
-----END PGP SIGNATURE-----
More information about the freebsd-pf
mailing list