FreeBSD 10.1-REL - network unaccessible after high traffic
Cs
bimmer at field.hu
Fri Jun 12 12:39:32 UTC 2015
but why is that machine runs fine except the network if it's memory
related? swap didn't increased before the network outage.
2015.06.12. 14:37 keltezéssel, Christopher Forgeron írta:
> rsycn burns memory - I'd say you have a good chance you're running out
> of mem before it's replenished.
>
> For vmstat 5 - Don't run it on console. Connect via a second box with
> ssh, and run it there - That way it's the last thing on the ssh
> terminal screen when the box dies, and you'll have your proof.
>
> On Fri, Jun 12, 2015 at 9:31 AM, Cs <bimmer at field.hu
> <mailto:bimmer at field.hu>> wrote:
>
> machine has been restarted before I could check the "vmstat 5"
> output. Yep, it's rsync. Anyway I disabled the backup transfer
> it'll solve, but I can't really accept this for solution.
>
>
> 2015.06.12. 14 <tel:2015.06.12.%2014>:29 keltezéssel, Christopher
> Forgeron írta:
>
> Well, even at low speed it could drop due to memory from what
> I've seen.
>
> What was the last line from vmstat 5 before it locked up?
>
> I find that the em driver isn't crap, but there is a deeper
> problem inside
> of FreeBSD that is being exposed now - For me it's due to
> faster network
> connections.
>
> Are you using rsync to move the files?
>
> On Fri, Jun 12, 2015 at 7:17 AM, Cs <bimmer at field.hu
> <mailto:bimmer at field.hu>> wrote:
>
> it seems it's not memory related. Server just died a few
> minutes ago
> during transporting the backup (400GB) around 800Mbps speed..
> will disable remote backup, it's a shame that em driver is
> such a crap.
>
>
> 2015.06.08. 5:01 keltezéssel, Christopher Forgeron írta:
>
> You know what helped me:
>
> 'vmstat 5'
>
> Leave that running. If the last thing on the console
> after a crash/hang is
> vmstat showing 8k of memory left, then you're in the
> same problem-park as
> me.
>
> My 10.1 96GiB RAM box is chewing ~8 GiB of RAM in less
> than 5 seconds, and
> then crashing/panicking/hanging.
>
> There's others with this issues if you search for it;
> a sysctl
> to vm.v_free_min to double or triple that value may
> help, but first let us
> know if that's what is bonking your sever.
>
>
>
> On Sun, Jun 7, 2015 at 11:03 AM, Cs <bimmer at field.hu
> <mailto:bimmer at field.hu>> wrote:
>
> ok, just lowered it to 1500 but please also note
> that it was on 1500 for
>
> 2
> years
>
> 2015.06.07. 14 <tel:2015.06.07.%2014>:57
> keltezéssel, Rick Macklem írta:
>
> Since disabling TSO didn't help, you could try
> dropping to 1500mtu
>
> on both interfaces. Some people run into
> problems when 9K jumbo clusters
> fragment the kernel address space used to
> allocate mbufs.
>
> Good luck with it, rick
>
> ----- Original Message -----
>
> Hi All,
>
> It worked fine for two weeks but I had a
> network outage 2 days ago
> then
> today. Tried to disable rxcsum and txcsum
> after the first one, didn't
> help. Don't know what else to do it's a
> shame that I can't use this
> card
> with fbsd i REALLY don't want to install
> linux instead but my
> production
> servers outages are not welcomed by the
> customers..
>
> 2015.05.26. 10 <tel:2015.05.26.%2010>:36
> keltezéssel, Cs írta:
>
> Thanks Mark, good idea. I found this
> thread which is exactly the
>
> same
> problem as mine:
>
>
> https://forums.freebsd.org/threads/workaround-freebsd-10-1-sudden-network-down.49264/
>
> Will see if it helps in a couple weeks.
>
> Regards,
> Csaba
>
> 2015.05.26. 10
> <tel:2015.05.26.%2010>:30 keltezéssel,
> Mark Schouten írta:
>
> Oh, didn't see your lowest remark.
> Then, the next thing that comes
>
> past here a few times per week is
> 'Try disabling TSO'.
>
>
> Met vriendelijke groeten,
>
> --
> Kerio Operator in de Cloud?
> https://www.kerioindecloud.nl/
> Mark Schouten | Tuxis Internet
> Engineering
> KvK: 61527076 | http://www.tuxis.nl/
> T: 0318 200208 | info at tuxis.nl
> <mailto:info at tuxis.nl>
>
>
>
> Van: Cs <bimmer at field.hu
> <mailto:bimmer at field.hu>>
> Aan: Mark Schouten
> <mark at tuxis.nl <mailto:mark at tuxis.nl>>
> Cc:
> <freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org>>
> Verzonden: 25-5-2015 11:12
> Onderwerp: Re: FreeBSD
> 10.1-REL - network unaccessible after
> high
> traffic
>
> It was on 1500 for ~3 years :)
> Regards,
> Csaba
> On May 25, 2015, 10:30,
> at 10:30, Mark Schouten
> <mark at tuxis.nl
> <mailto:mark at tuxis.nl>>
> wrote:
>
> Try lowering your mtu to 1500,
> that worked miracles for me..
>
> --
> Mark Schouten
> Tuxis Internet Engineering
> mark at tuxis.nl
> <mailto:mark at tuxis.nl> / 0318
> 200208
>
> On 25 May 2015, at 09:36,
> "Cs" <bimmer at field.hu
> <mailto:bimmer at field.hu>> wrote:
>
> Hi all,
> I have two FreeBSd
> 10.1-RELEASE servers
> connected to each
> other.
> They
>
> were connected via cross
> link, but they are
> connected to a cisco
>
> switch
> now (the problem was the same
> with cross link too). When
> transferring
> huge files (50-500GB backup
> files) via Gigabit (it is
> important!)
> the
> network randomly dies. The
> backup runs every day/week and
> sometimes the
> connection is ok for months
> sometimes it happens twice a week.
> When the
> network dies I can log in to
> the server via IPMI and use the
> console
> everything is OK, but can't
> send anything out on the network.
> ifconfig
> em0 down/up doesn't help nor
> netif restart. The problem never
> occured
> when I used 100Mbit connection
> between them, but it was 3com NIC
> (xl),
> gigabit adapter is Intel
> (em0). When I limit the
> transfer rate
> (rsync
> bandwith limit or ipfw pipe)
> the problem is much more rare.
>
> I tried to set these
> tuning parameters on both
> servers with
>
> different
>
> buffer size but nothing
> helped:
>
> # cat /etc/sysctl.conf
>
> security.bsd.see_other_uids=0
> net.inet.tcp.recvspace=512000
> net.route.netisr_maxqlen=2048
> kern.ipc.nmbclusters=1310720
> net.inet.tcp.sendbuf_max=16777216
> net.inet.tcp.recvbuf_max=16777216
> kern.ipc.soacceptqueue=32768
> # cat /boot/loader.conf
> geom_mirror_load="YES" #
> RAID1 disk driver (see
> gmirror(8))
> ipfw_load="YES"
> net.inet.ip.fw.default_to_accept=1
> kern.maxusers=4096
> accf_data_load="YES"
> The duplex settings
> are identical on both servers.
> Server A:
> em1:
> flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
> metric 0
> mtu
>
> 9000
>
>
> options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
>
>
> ether
> 00:25:90:24:52:66
>
> inet x.x.x.x
> netmask 0xfffffe00
> broadcast x.x.x.x
> nd6
> options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet
> autoselect (1000baseT
> <full-duplex>)
> status: active
> Server B:
> em0:
> flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
> metric 0
> mtu
>
> 9000
>
>
> options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
>
>
> ether
> 00:30:48:dd:fe:3e
>
> inet x.x.x.x
> netmask 0xfffffe00
> broadcast x.x.x.x
> nd6
> options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet
> autoselect (1000baseT
> <full-duplex>)
> status: active
> Today I tried to set
> mtu to 9000 but in tcpdump
> I see that
> during
> scp
>
> it is still 1500:
>
> x.x.x.x.222 >
> x.x.x.x.37612: Flags [.],
> cksum 0xb6ee
>
> (incorrect ->
>
> 0xda6f), seq 35749, ack
> 113701596, win 7986,
> options [nop,nop,TS
>
> val
> 3103966325
> <tel:3103966325> ecr
> 853712893], length 0
>
> 09:27:33.912354 IP (tos 0x8,
> ttl 64, id 1028, offset 0, flags
>
> [DF],
>
> proto TCP (6), length 1500)
>
> 09:27:33.912358 IP (tos 0x8,
> ttl 64, id 1029, offset 0, flags
>
> [DF],
>
> proto TCP (6), length 1500)
>
> Any ideas? Thanks guys!
>
> _______________________________________________
> freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org>
> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any
> mail to
>
>
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
> _______________________________________________
>
> freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org>
> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
>
> _______________________________________________
>
> freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org>
> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
> _______________________________________________
>
> freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
> _______________________________________________
>
> freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
> _______________________________________________
>
> freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
> _______________________________________________
> freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org>
> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
> _______________________________________________
> freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org>
> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
>
> _______________________________________________
> freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
>
More information about the freebsd-net
mailing list