From alexdehaini at gmail.com Fri Feb 6 07:14:35 2009 From: alexdehaini at gmail.com (Alex Dehaini) Date: Fri Feb 6 07:14:43 2009 Subject: Limiting open port RST response from 247 to 200 packets per second Message-ID: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> Hi Guys, I have some issues with Squid on Freebsd. I am running FreeBSD release 4.9 and Squid version 2.5. I have setup FreeBSD as a bridge so that all traffic from my network can transparently pass through the FreeBSD server. I am running Squid on the same server and I created an ipfw rule to redirect port 80 to port 3128. Normally, when Squid is not started - we see traffic close to 30MB flowing through the server. Immediately I start squid, the traffic drops to half and sometimes lower and stays there. When this happens, I have a lot of clients that will call and complain they can't access the Internet. At the same time, I get these log messages *Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 to 200 packets per second Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 to 200 packets per second Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 to 200 packets per second Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 to 200 packets per second Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from 273 to 200 packets per second Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from 273 to 200 packets per second Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from 228 to 200 packets per second Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from 228 to 200 packets per second Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from 225 to 200 packets per second Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from 225 to 200 packets per second Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from 244 to 200 packets per second Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from 244 to 200 packets per second Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from 259 to 200 packets per second Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from 259 to 200 packets per second Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from 234 to 200 packets per second Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from 234 to 200 packets per second Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from 243 to 200 packets per second Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from 243 to 200 packets per second Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from 218 to 200 packets per second Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from 218 to 200 packets per second Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from 233 to 200 packets per second Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from 233 to 200 packets per second Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from 241 to 200 packets per second Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from 241 to 200 packets per second Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from 220 to 200 packets per second Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from 220 to 200 packets per second Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from 206 to 200 packets per second Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from 206 to 200 packets per second Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from 223 to 200 packets per second Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from 223 to 200 packets per second* When I stop Squid, everything returns to normal. Any idea what is causing this. I will appreciate any help. Thanks -- Alex Dehaini Developer Site - www.alexdehaini.com Email - alexdehaini@gmail.com From alexdehaini at gmail.com Fri Feb 6 07:24:55 2009 From: alexdehaini at gmail.com (Alex Dehaini) Date: Fri Feb 6 07:25:02 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> Message-ID: <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> I increased net.inet.icmp.icmplim to 2000 but this does not make any change. Here is my output myserver# sysctl -a | grep net.inet.icmp.icmplim net.inet.icmp.icmplim: 2000 net.inet.icmp.icmplim_output: 1 After increasing inet.icmp.icmplim to 2000 and startign Squid, I don't get the errors below Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 to 200 packets per second Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 to 200 packets per second Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 to 200 packets per second But traffic still drops. Alex On Fri, Feb 6, 2009 at 3:19 PM, Istv?n Szuk?cs wrote: > Hi! > > > lix@test:~$sysctl -a | grep net.inet.icmp.icmplim > net.inet.icmp.icmplim: 200 > net.inet.icmp.icmplim_output: 1 > > > Regards, > Istvan > > On Fri, Feb 6, 2009 at 2:44 PM, Alex Dehaini wrote: > >> Hi Guys, >> >> I have some issues with Squid on Freebsd. I am running FreeBSD release 4.9 >> and Squid version 2.5. >> >> I have setup FreeBSD as a bridge so that all traffic from my network can >> transparently pass through the FreeBSD server. I am running Squid on the >> same server and I created an ipfw rule to redirect port 80 to port 3128. >> >> Normally, when Squid is not started - we see traffic close to 30MB flowing >> through the server. Immediately I start squid, the traffic drops to half >> and >> sometimes lower and stays there. When this happens, I have a lot of >> clients >> that will call and complain they can't access the Internet. At the same >> time, I get these log messages >> >> *Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from >> 247 >> to 200 packets per second >> Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 >> to 200 packets per second >> Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 >> to 200 packets per second >> Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 >> to 200 packets per second >> Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from 273 >> to 200 packets per second >> Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from 273 >> to 200 packets per second >> Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from 228 >> to 200 packets per second >> Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from 228 >> to 200 packets per second >> Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from 225 >> to 200 packets per second >> Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from 225 >> to 200 packets per second >> Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from 244 >> to 200 packets per second >> Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from 244 >> to 200 packets per second >> Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from 259 >> to 200 packets per second >> Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from 259 >> to 200 packets per second >> Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from 234 >> to 200 packets per second >> Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from 234 >> to 200 packets per second >> Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from 243 >> to 200 packets per second >> Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from 243 >> to 200 packets per second >> Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from 218 >> to 200 packets per second >> Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from 218 >> to 200 packets per second >> Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from 233 >> to 200 packets per second >> Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from 233 >> to 200 packets per second >> Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from 241 >> to 200 packets per second >> Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from 241 >> to 200 packets per second >> Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from 220 >> to 200 packets per second >> Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from 220 >> to 200 packets per second >> Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from 206 >> to 200 packets per second >> Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from 206 >> to 200 packets per second >> Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from 223 >> to 200 packets per second >> Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from 223 >> to 200 packets per second* >> >> When I stop Squid, everything returns to normal. Any idea what is causing >> this. I will appreciate any help. >> >> Thanks >> >> -- >> Alex Dehaini >> Developer >> Site - www.alexdehaini.com >> Email - alexdehaini@gmail.com >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to " >> freebsd-performance-unsubscribe@freebsd.org" >> > > > > -- > the sun shines for all > -- Alex Dehaini Developer Site - www.alexdehaini.com Email - alexdehaini@gmail.com From leccine at gmail.com Fri Feb 6 07:41:29 2009 From: leccine at gmail.com (=?ISO-8859-1?Q?Istv=E1n_Szuk=E1cs?=) Date: Fri Feb 6 07:41:36 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> Message-ID: Hi! lix@test:~$sysctl -a | grep net.inet.icmp.icmplim net.inet.icmp.icmplim: 200 net.inet.icmp.icmplim_output: 1 Regards, Istvan On Fri, Feb 6, 2009 at 2:44 PM, Alex Dehaini wrote: > Hi Guys, > > I have some issues with Squid on Freebsd. I am running FreeBSD release 4.9 > and Squid version 2.5. > > I have setup FreeBSD as a bridge so that all traffic from my network can > transparently pass through the FreeBSD server. I am running Squid on the > same server and I created an ipfw rule to redirect port 80 to port 3128. > > Normally, when Squid is not started - we see traffic close to 30MB flowing > through the server. Immediately I start squid, the traffic drops to half > and > sometimes lower and stays there. When this happens, I have a lot of clients > that will call and complain they can't access the Internet. At the same > time, I get these log messages > > *Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > to 200 packets per second > Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > to 200 packets per second > Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 > to 200 packets per second > Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 > to 200 packets per second > Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from 273 > to 200 packets per second > Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from 273 > to 200 packets per second > Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from 228 > to 200 packets per second > Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from 228 > to 200 packets per second > Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from 225 > to 200 packets per second > Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from 225 > to 200 packets per second > Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from 244 > to 200 packets per second > Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from 244 > to 200 packets per second > Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from 259 > to 200 packets per second > Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from 259 > to 200 packets per second > Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from 234 > to 200 packets per second > Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from 234 > to 200 packets per second > Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from 243 > to 200 packets per second > Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from 243 > to 200 packets per second > Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from 218 > to 200 packets per second > Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from 218 > to 200 packets per second > Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from 233 > to 200 packets per second > Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from 233 > to 200 packets per second > Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from 241 > to 200 packets per second > Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from 241 > to 200 packets per second > Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from 220 > to 200 packets per second > Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from 220 > to 200 packets per second > Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from 206 > to 200 packets per second > Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from 206 > to 200 packets per second > Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from 223 > to 200 packets per second > Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from 223 > to 200 packets per second* > > When I stop Squid, everything returns to normal. Any idea what is causing > this. I will appreciate any help. > > Thanks > > -- > Alex Dehaini > Developer > Site - www.alexdehaini.com > Email - alexdehaini@gmail.com > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to " > freebsd-performance-unsubscribe@freebsd.org" > -- the sun shines for all From alexdehaini at gmail.com Fri Feb 6 10:19:24 2009 From: alexdehaini at gmail.com (Alex Dehaini) Date: Fri Feb 6 10:19:30 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> Message-ID: <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> Any ideas On Fri, Feb 6, 2009 at 3:24 PM, Alex Dehaini wrote: > I increased net.inet.icmp.icmplim to 2000 but this does not make any > change. Here is my output > > myserver# sysctl -a | grep net.inet.icmp.icmplim > net.inet.icmp.icmplim: 2000 > net.inet.icmp.icmplim_output: 1 > > After increasing inet.icmp.icmplim to 2000 and startign Squid, I don't get > the errors below > > Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > to 200 packets per second > Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > to 200 packets per second > Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 > to 200 packets per second > > But traffic still drops. > > Alex > > > > On Fri, Feb 6, 2009 at 3:19 PM, Istv?n Szuk?cs wrote: > >> Hi! >> >> >> lix@test:~$sysctl -a | grep net.inet.icmp.icmplim >> net.inet.icmp.icmplim: 200 >> net.inet.icmp.icmplim_output: 1 >> >> >> Regards, >> Istvan >> >> On Fri, Feb 6, 2009 at 2:44 PM, Alex Dehaini wrote: >> >>> Hi Guys, >>> >>> I have some issues with Squid on Freebsd. I am running FreeBSD release >>> 4.9 >>> and Squid version 2.5. >>> >>> I have setup FreeBSD as a bridge so that all traffic from my network can >>> transparently pass through the FreeBSD server. I am running Squid on the >>> same server and I created an ipfw rule to redirect port 80 to port 3128. >>> >>> Normally, when Squid is not started - we see traffic close to 30MB >>> flowing >>> through the server. Immediately I start squid, the traffic drops to half >>> and >>> sometimes lower and stays there. When this happens, I have a lot of >>> clients >>> that will call and complain they can't access the Internet. At the same >>> time, I get these log messages >>> >>> *Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from >>> 247 >>> to 200 packets per second >>> Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from >>> 247 >>> to 200 packets per second >>> Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from >>> 239 >>> to 200 packets per second >>> Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from >>> 239 >>> to 200 packets per second >>> Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from >>> 273 >>> to 200 packets per second >>> Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from >>> 273 >>> to 200 packets per second >>> Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from >>> 228 >>> to 200 packets per second >>> Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from >>> 228 >>> to 200 packets per second >>> Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from >>> 225 >>> to 200 packets per second >>> Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from >>> 225 >>> to 200 packets per second >>> Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from >>> 244 >>> to 200 packets per second >>> Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from >>> 244 >>> to 200 packets per second >>> Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from >>> 259 >>> to 200 packets per second >>> Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from >>> 259 >>> to 200 packets per second >>> Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from >>> 234 >>> to 200 packets per second >>> Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from >>> 234 >>> to 200 packets per second >>> Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from >>> 243 >>> to 200 packets per second >>> Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from >>> 243 >>> to 200 packets per second >>> Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from >>> 218 >>> to 200 packets per second >>> Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from >>> 218 >>> to 200 packets per second >>> Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from >>> 233 >>> to 200 packets per second >>> Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from >>> 233 >>> to 200 packets per second >>> Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from >>> 241 >>> to 200 packets per second >>> Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from >>> 241 >>> to 200 packets per second >>> Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from >>> 220 >>> to 200 packets per second >>> Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from >>> 220 >>> to 200 packets per second >>> Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from >>> 206 >>> to 200 packets per second >>> Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from >>> 206 >>> to 200 packets per second >>> Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from >>> 223 >>> to 200 packets per second >>> Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from >>> 223 >>> to 200 packets per second* >>> >>> When I stop Squid, everything returns to normal. Any idea what is causing >>> this. I will appreciate any help. >>> >>> Thanks >>> >>> -- >>> Alex Dehaini >>> Developer >>> Site - www.alexdehaini.com >>> Email - alexdehaini@gmail.com >>> _______________________________________________ >>> freebsd-performance@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>> To unsubscribe, send any mail to " >>> freebsd-performance-unsubscribe@freebsd.org" >>> >> >> >> >> -- >> the sun shines for all >> > > > > -- > Alex Dehaini > Developer > Site - www.alexdehaini.com > Email - alexdehaini@gmail.com > -- Alex Dehaini Developer Site - www.alexdehaini.com Email - alexdehaini@gmail.com From maciej at suszko.eu Fri Feb 6 11:03:39 2009 From: maciej at suszko.eu (Maciej Suszko) Date: Fri Feb 6 11:03:48 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> Message-ID: <20090206193651.518c3347@suszko.eu> Alex Dehaini wrote: > Any ideas Show us the output of `vmstat -z` - may be helpful. -- regards, Maciej Suszko. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090206/209cb5db/signature.pgp From ntarmos at ceid.upatras.gr Fri Feb 6 11:13:30 2009 From: ntarmos at ceid.upatras.gr (Nikos Ntarmos) Date: Fri Feb 6 11:13:39 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> Message-ID: <20090206184835.GA59842@ace.netcins.ceid.upatras.gr> Alex, Traffic from and to the bridge interface that speaks with the internet dropping when you use a proxy server is quite normal. You did have an issue with icmplim but you have that fixed now. Do your clients still complain of problems with connectivity? Cheers. \n\n From alexdehaini at gmail.com Fri Feb 6 12:45:20 2009 From: alexdehaini at gmail.com (Alex Dehaini) Date: Fri Feb 6 12:45:26 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <20090206193651.518c3347@suszko.eu> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> <20090206193651.518c3347@suszko.eu> Message-ID: <4b008f7d0902061245g458ae7eh1bd7375b0c085d4@mail.gmail.com> Here is the output of 'vmstat -z' on my server myserver# vmstat -z ITEM SIZE LIMIT USED FREE REQUESTS PIPE: 160, 0, 6, 96, 5976 SWAPMETA: 160, 233016, 0, 0, 0 unpcb: 160, 0, 4, 46, 6460 ripcb: 192, 40000, 0, 42, 28 divcb: 192, 40000, 0, 0, 0 syncache: 160, 15359, 0, 1766, 10989 tcpcb: 576, 40000, 7, 92, 2115 udpcb: 192, 40000, 6, 36, 2158 socket: 192, 40000, 17, 111, 10762 DIRHASH: 1024, 0, 288, 4, 301 KNOTE: 64, 0, 0, 128, 24 NFSNODE: 352, 0, 0, 0, 0 NFSMOUNT: 544, 0, 0, 0, 0 VNODE: 192, 0, 2806, 56, 2806 NAMEI: 1024, 0, 0, 16, 261060 VMSPACE: 192, 0, 34, 94, 5671 PROC: 416, 0, 40, 58, 5678 DP fakepg: 64, 0, 0, 0, 0 PV ENTRY: 28, 3297134, 30022, 1002145, 4007245 MAP ENTRY: 48, 0, 753, 820, 278386 KMAP ENTRY: 48, 85223, 182, 202, 11578 MAP: 108, 0, 7, 3, 7 VM OBJECT: 92, 0, 1340, 412, 145002 I hope it helps Alex On Fri, Feb 6, 2009 at 6:36 PM, Maciej Suszko wrote: > Alex Dehaini wrote: > > Any ideas > > Show us the output of `vmstat -z` - may be helpful. > -- > regards, Maciej Suszko. > -- Alex Dehaini Developer Site - www.alexdehaini.com Email - alexdehaini@gmail.com From ivoras at freebsd.org Sat Feb 7 09:05:05 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Sat Feb 7 09:05:13 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> Message-ID: Alex Dehaini wrote: > Hi Guys, > > I have some issues with Squid on Freebsd. I am running FreeBSD release 4.9 > and Squid version 2.5. > > I have setup FreeBSD as a bridge so that all traffic from my network can > transparently pass through the FreeBSD server. I am running Squid on the > same server and I created an ipfw rule to redirect port 80 to port 3128. > > Normally, when Squid is not started - we see traffic close to 30MB flowing > through the server. Immediately I start squid, the traffic drops to half and > sometimes lower and stays there. When this happens, I have a lot of clients > that will call and complain they can't access the Internet. At the same > time, I get these log messages > > *Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > to 200 packets per second > Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > to 200 packets per second > When I stop Squid, everything returns to normal. Any idea what is causing > this. I will appreciate any help. RST response means a client has tried to connect to a TCP port and didn't succeed. In your case, is squid actually doing anything? Are there any traffic in your logs? By increasing icmplim you only lifted the supression of the outgoing RST packets, you didn't solve your problem. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090207/5ab92dd4/signature.pgp From alexdehaini at gmail.com Mon Feb 9 03:49:32 2009 From: alexdehaini at gmail.com (Alex Dehaini) Date: Mon Feb 9 03:49:45 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902061245g458ae7eh1bd7375b0c085d4@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> <20090206193651.518c3347@suszko.eu> <4b008f7d0902061245g458ae7eh1bd7375b0c085d4@mail.gmail.com> Message-ID: <4b008f7d0902090349m7405d819l4468168441dbca5b@mail.gmail.com> Any help guys? On Fri, Feb 6, 2009 at 8:45 PM, Alex Dehaini wrote: > Here is the output of 'vmstat -z' on my server > > myserver# vmstat -z > > ITEM SIZE LIMIT USED FREE REQUESTS > > PIPE: 160, 0, 6, 96, 5976 > SWAPMETA: 160, 233016, 0, 0, 0 > unpcb: 160, 0, 4, 46, 6460 > ripcb: 192, 40000, 0, 42, 28 > divcb: 192, 40000, 0, 0, 0 > syncache: 160, 15359, 0, 1766, 10989 > tcpcb: 576, 40000, 7, 92, 2115 > udpcb: 192, 40000, 6, 36, 2158 > socket: 192, 40000, 17, 111, 10762 > DIRHASH: 1024, 0, 288, 4, 301 > KNOTE: 64, 0, 0, 128, 24 > NFSNODE: 352, 0, 0, 0, 0 > NFSMOUNT: 544, 0, 0, 0, 0 > VNODE: 192, 0, 2806, 56, 2806 > NAMEI: 1024, 0, 0, 16, 261060 > VMSPACE: 192, 0, 34, 94, 5671 > PROC: 416, 0, 40, 58, 5678 > DP fakepg: 64, 0, 0, 0, 0 > PV ENTRY: 28, 3297134, 30022, 1002145, 4007245 > MAP ENTRY: 48, 0, 753, 820, 278386 > KMAP ENTRY: 48, 85223, 182, 202, 11578 > MAP: 108, 0, 7, 3, 7 > VM OBJECT: 92, 0, 1340, 412, 145002 > > I hope it helps > > Alex > > > > On Fri, Feb 6, 2009 at 6:36 PM, Maciej Suszko wrote: > >> Alex Dehaini wrote: >> > Any ideas >> >> Show us the output of `vmstat -z` - may be helpful. >> -- >> regards, Maciej Suszko. >> > > > > -- > Alex Dehaini > Developer > Site - www.alexdehaini.com > Email - alexdehaini@gmail.com > -- Alex Dehaini Developer Site - www.alexdehaini.com Email - alexdehaini@gmail.com From maciej at suszko.eu Mon Feb 9 09:02:27 2009 From: maciej at suszko.eu (Maciej Suszko) Date: Mon Feb 9 09:02:34 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902090349m7405d819l4468168441dbca5b@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> <20090206193651.518c3347@suszko.eu> <4b008f7d0902061245g458ae7eh1bd7375b0c085d4@mail.gmail.com> <4b008f7d0902090349m7405d819l4468168441dbca5b@mail.gmail.com> Message-ID: <20090209180220.3bc453db@suszko.eu> Alex Dehaini wrote: > Any help guys? It doesn't look suspicious to me. Watch your logs and while you get those messages about limiting open port rst response, try to check `netstat -na` output - maybe you'll find some ,,anomalies'', do it several times, post the observation results. We'll see :) > On Fri, Feb 6, 2009 at 8:45 PM, Alex Dehaini > wrote: > > > Here is the output of 'vmstat -z' on my server > > > > myserver# vmstat -z > > > > ITEM SIZE LIMIT USED FREE REQUESTS > > > > PIPE: 160, 0, 6, 96, 5976 > > SWAPMETA: 160, 233016, 0, 0, 0 > > unpcb: 160, 0, 4, 46, 6460 > > ripcb: 192, 40000, 0, 42, 28 > > divcb: 192, 40000, 0, 0, 0 > > syncache: 160, 15359, 0, 1766, 10989 > > tcpcb: 576, 40000, 7, 92, 2115 > > udpcb: 192, 40000, 6, 36, 2158 > > socket: 192, 40000, 17, 111, 10762 > > DIRHASH: 1024, 0, 288, 4, 301 > > KNOTE: 64, 0, 0, 128, 24 > > NFSNODE: 352, 0, 0, 0, 0 > > NFSMOUNT: 544, 0, 0, 0, 0 > > VNODE: 192, 0, 2806, 56, 2806 > > NAMEI: 1024, 0, 0, 16, 261060 > > VMSPACE: 192, 0, 34, 94, 5671 > > PROC: 416, 0, 40, 58, 5678 > > DP fakepg: 64, 0, 0, 0, 0 > > PV ENTRY: 28, 3297134, 30022, 1002145, 4007245 > > MAP ENTRY: 48, 0, 753, 820, 278386 > > KMAP ENTRY: 48, 85223, 182, 202, 11578 > > MAP: 108, 0, 7, 3, 7 > > VM OBJECT: 92, 0, 1340, 412, 145002 > > > > I hope it helps -- regards, Maciej Suszko. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090209/18bd02fc/signature.pgp From kama at pvp.se Tue Feb 10 01:07:16 2009 From: kama at pvp.se (kama) Date: Tue Feb 10 01:07:24 2009 Subject: Limiting open port RST response from 247 to 200 packets per second In-Reply-To: <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> References: <4b008f7d0902060644o62a3942lf63ff6689c3b4d94@mail.gmail.com> <4b008f7d0902060724o6817f822ufb3ce8a8f9060fa8@mail.gmail.com> <4b008f7d0902061019v414ef35do1b84a3f2e6f0b48d@mail.gmail.com> Message-ID: <20090210093611.V67848@ns1.as.pvp.se> Ehm, you have posted the exact same lines from the log... Until you find new lines you are probably fine... /Bjorn On Fri, 6 Feb 2009, Alex Dehaini wrote: > Any ideas > > On Fri, Feb 6, 2009 at 3:24 PM, Alex Dehaini wrote: > > > I increased net.inet.icmp.icmplim to 2000 but this does not make any > > change. Here is my output > > > > myserver# sysctl -a | grep net.inet.icmp.icmplim > > net.inet.icmp.icmplim: 2000 > > net.inet.icmp.icmplim_output: 1 > > > > After increasing inet.icmp.icmplim to 2000 and startign Squid, I don't get > > the errors below > > > > Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > > to 200 packets per second > > Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from 247 > > to 200 packets per second > > Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from 239 > > to 200 packets per second > > > > But traffic still drops. > > > > Alex > > > > > > > > On Fri, Feb 6, 2009 at 3:19 PM, Istv?n Szuk?cs wrote: > > > >> Hi! > >> > >> > >> lix@test:~$sysctl -a | grep net.inet.icmp.icmplim > >> net.inet.icmp.icmplim: 200 > >> net.inet.icmp.icmplim_output: 1 > >> > >> > >> Regards, > >> Istvan > >> > >> On Fri, Feb 6, 2009 at 2:44 PM, Alex Dehaini wrote: > >> > >>> Hi Guys, > >>> > >>> I have some issues with Squid on Freebsd. I am running FreeBSD release > >>> 4.9 > >>> and Squid version 2.5. > >>> > >>> I have setup FreeBSD as a bridge so that all traffic from my network can > >>> transparently pass through the FreeBSD server. I am running Squid on the > >>> same server and I created an ipfw rule to redirect port 80 to port 3128. > >>> > >>> Normally, when Squid is not started - we see traffic close to 30MB > >>> flowing > >>> through the server. Immediately I start squid, the traffic drops to half > >>> and > >>> sometimes lower and stays there. When this happens, I have a lot of > >>> clients > >>> that will call and complain they can't access the Internet. At the same > >>> time, I get these log messages > >>> > >>> *Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from > >>> 247 > >>> to 200 packets per second > >>> Feb 5 20:39:44 myserver /kernel: Limiting open port RST response from > >>> 247 > >>> to 200 packets per second > >>> Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from > >>> 239 > >>> to 200 packets per second > >>> Feb 5 20:39:45 myserver /kernel: Limiting open port RST response from > >>> 239 > >>> to 200 packets per second > >>> Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from > >>> 273 > >>> to 200 packets per second > >>> Feb 5 20:39:46 myserver /kernel: Limiting open port RST response from > >>> 273 > >>> to 200 packets per second > >>> Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from > >>> 228 > >>> to 200 packets per second > >>> Feb 5 20:39:47 myserver /kernel: Limiting open port RST response from > >>> 228 > >>> to 200 packets per second > >>> Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from > >>> 225 > >>> to 200 packets per second > >>> Feb 5 20:39:48 myserver /kernel: Limiting open port RST response from > >>> 225 > >>> to 200 packets per second > >>> Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from > >>> 244 > >>> to 200 packets per second > >>> Feb 5 20:39:49 myserver /kernel: Limiting open port RST response from > >>> 244 > >>> to 200 packets per second > >>> Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from > >>> 259 > >>> to 200 packets per second > >>> Feb 5 20:39:50 myserver /kernel: Limiting open port RST response from > >>> 259 > >>> to 200 packets per second > >>> Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from > >>> 234 > >>> to 200 packets per second > >>> Feb 5 20:39:51 myserver /kernel: Limiting open port RST response from > >>> 234 > >>> to 200 packets per second > >>> Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from > >>> 243 > >>> to 200 packets per second > >>> Feb 5 20:39:52 myserver /kernel: Limiting open port RST response from > >>> 243 > >>> to 200 packets per second > >>> Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from > >>> 218 > >>> to 200 packets per second > >>> Feb 5 20:39:53 myserver /kernel: Limiting open port RST response from > >>> 218 > >>> to 200 packets per second > >>> Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from > >>> 233 > >>> to 200 packets per second > >>> Feb 5 20:39:55 myserver /kernel: Limiting open port RST response from > >>> 233 > >>> to 200 packets per second > >>> Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from > >>> 241 > >>> to 200 packets per second > >>> Feb 5 20:39:56 myserver /kernel: Limiting open port RST response from > >>> 241 > >>> to 200 packets per second > >>> Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from > >>> 220 > >>> to 200 packets per second > >>> Feb 5 20:39:57 myserver /kernel: Limiting open port RST response from > >>> 220 > >>> to 200 packets per second > >>> Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from > >>> 206 > >>> to 200 packets per second > >>> Feb 5 20:39:58 myserver /kernel: Limiting open port RST response from > >>> 206 > >>> to 200 packets per second > >>> Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from > >>> 223 > >>> to 200 packets per second > >>> Feb 5 20:40:01 myserver /kernel: Limiting open port RST response from > >>> 223 > >>> to 200 packets per second* > >>> > >>> When I stop Squid, everything returns to normal. Any idea what is causing > >>> this. I will appreciate any help. > >>> > >>> Thanks > >>> > >>> -- > >>> Alex Dehaini > >>> Developer > >>> Site - www.alexdehaini.com > >>> Email - alexdehaini@gmail.com > >>> _______________________________________________ > >>> freebsd-performance@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance > >>> To unsubscribe, send any mail to " > >>> freebsd-performance-unsubscribe@freebsd.org" > >>> > >> > >> > >> > >> -- > >> the sun shines for all > >> > > > > > > > > -- > > Alex Dehaini > > Developer > > Site - www.alexdehaini.com > > Email - alexdehaini@gmail.com > > > > > > -- > Alex Dehaini > Developer > Site - www.alexdehaini.com > Email - alexdehaini@gmail.com > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > From sebster at sebster.com Tue Feb 10 03:59:50 2009 From: sebster at sebster.com (Sebastiaan van Erk) Date: Tue Feb 10 03:59:57 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 Message-ID: <499165F3.6050803@sebster.com> Hi, I want to deploy a production FreeBSD web site (database cluster, apache cluster, ip failover using carp, etc.), however I'm experiencing painful disk I/O throughput problems which currently does not make the above project viable. I've done some rudimentary benchmarking of two identically configured virtual machines (2 VCPUs, 512MB memory, 8GB disk) and installed one with FreeBSD 7.1-amd64 and one with Linux Ubuntu 8.10-amd64. These are the results I'm getting with dbench : 1 2 4 freebsd 12.0009 13.6348 12.9402 (MB/s) linux 376.145 651.314 634.649 (MB/s) Both virtual machines run dbench 3.04 and the results are extremely stable over repeated runs. The virtual hardware detected by the FreeBSD machine is as follows: mpt0: port 0x1080-0x10ff mem 0xf4810000-0xf4810fff irq 17 at device 16.0 on pci0 mpt0: [ITHREAD] mpt0: MPI Version=1.2.0.0 And: da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 3.300MB/s transfers da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) I've also run unixbench (4.1 and 5.1.2) and the performance of the FreeBSD machine is horrible compared to Linux on many of the tests, though my first guess is that it all comes back down the disk performance (on the CPU-only tests the results are about the same). Online when I see logs of da0 specs via google, they more look more like this (much higher transfer rate, and SCSI-n, n>2): da0: Fixed Direct Access SCSI-5 device da0: 300.000MB/s transfers Does anybody know how I can get proper performance for the drive under ESXi? Regards, Sebastiaan -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3328 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090210/29fea209/smime.bin From ivoras at freebsd.org Tue Feb 10 06:05:10 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Feb 10 06:05:19 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <499165F3.6050803@sebster.com> References: <499165F3.6050803@sebster.com> Message-ID: Sebastiaan van Erk wrote: > Hi, > > I want to deploy a production FreeBSD web site (database cluster, apache > cluster, ip failover using carp, etc.), however I'm experiencing painful > disk I/O throughput problems which currently does not make the above > project viable. I've done some rudimentary benchmarking of two > identically configured virtual machines (2 VCPUs, 512MB memory, 8GB > disk) and installed one with FreeBSD 7.1-amd64 and one with Linux Ubuntu > 8.10-amd64. These are the results I'm getting with dbench : > > 1 2 4 > freebsd 12.0009 13.6348 12.9402 (MB/s) > linux 376.145 651.314 634.649 (MB/s) > > Both virtual machines run dbench 3.04 and the results are extremely > stable over repeated runs. > > The virtual hardware detected by the FreeBSD machine is as follows: > Does anybody know how I can get proper performance for the drive under > ESXi? VMWare has many optimizations for Linux that are not used with FreeBSD. VMI, for example, makes the Linux guest paravirtualized, and then there are special drivers for networking, its vmotion driver (this one probably doesn't contribute to performance much), etc. and Linux is in any case much better tested and supported. If VMWare allows, you may try changing the type of the controller (I don't know about ESXi but VMWare Server supports LSI or Buslogic SCSI emulation) or switch to ATA emulation and try again. A generic optimization is to reduce kern.hz to something like 50 but it probably won't help your disk performance. As for unixbench, you need to examine and compare each microbenchmark result individually before drawing a conclusion. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090210/0505b4f3/signature.pgp From sebster at sebster.com Tue Feb 10 06:22:33 2009 From: sebster at sebster.com (Sebastiaan van Erk) Date: Tue Feb 10 06:22:40 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: References: <499165F3.6050803@sebster.com> Message-ID: <49918DA6.4020608@sebster.com> Ivan Voras wrote: Hi, Thanks for the reply. > Sebastiaan van Erk wrote: >> Hi, [snip] >> 1 2 4 >> freebsd 12.0009 13.6348 12.9402 (MB/s) >> linux 376.145 651.314 634.649 (MB/s) >> >> Both virtual machines run dbench 3.04 and the results are extremely >> stable over repeated runs. > VMWare has many optimizations for Linux that are not used with FreeBSD. > VMI, for example, makes the Linux guest paravirtualized, and then there > are special drivers for networking, its vmotion driver (this one > probably doesn't contribute to performance much), etc. and Linux is in > any case much better tested and supported. VMI/paravirtualization is not enabled for this Linux host. Neither is VMotion. Networking is performing extremely well (see also below). > If VMWare allows, you may try changing the type of the controller (I > don't know about ESXi but VMWare Server supports LSI or Buslogic SCSI > emulation) or switch to ATA emulation and try again. I tried this, and it has no significant effect. Just for completeness, here's the relevant output of dmesg: bt0: port 0x1060-0x107f mem 0xf4810000-0xf481001f irq 17 at device 16.0 on pci0 bt0: BT-958 FW Rev. 5.07B Ultra Wide SCSI Host Adapter, SCSI ID 7, 192 CCBs bt0: [GIANT-LOCKED] bt0: [ITHREAD] da0 at bt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz DT, offset 15, 16bit) da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) The transfer rate for dbench 1 is 15.0118 MB/s. > A generic optimization is to reduce kern.hz to something like 50 but it > probably won't help your disk performance. I already had this (not 50, but 100), but this doesn't do anything for the disk performance. > As for unixbench, you need to examine and compare each microbenchmark > result individually before drawing a conclusion. Yes, I realize that. However the dbench result is my first priority, and when (if) that is fixed, I'll run the unixbench again and see what my next priority is. (However, just to give you an idea I attached the basic 5.1.2 unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the output under linux to the "procinfo" file.) Finally, I also ran some network benchmarks such as netio, and tested VM to VM communication on *different* ESXi machines connected via Gigabit ethernet, and it achieved more than 100MB/s throughput. Since CPU speed + Network IO are doing just fine, I'm guessing this is a pure disk (driver?) related issue. However, to go into production with FreeBSD I *must* be able to fix it. Note also the discrepency: 12 MB/s vs 350 MB/s on disk access! My lousy home machine (FreeBSD) is even 5 times faster at 60 MB/s. This machine has extremely fast disks in a RAID10 configuration. Any ideas are welcome! Regards, Sebastiaan -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3328 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090210/2acea3e0/smime.bin From sebster at sebster.com Tue Feb 10 06:24:14 2009 From: sebster at sebster.com (Sebastiaan van Erk) Date: Tue Feb 10 06:24:21 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <49918DA6.4020608@sebster.com> References: <499165F3.6050803@sebster.com> <49918DA6.4020608@sebster.com> Message-ID: <49918E0A.1060500@sebster.com> Skipped content of type multipart/mixed-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3328 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090210/50501a6d/smime.bin From ivoras at freebsd.org Tue Feb 10 06:44:45 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Feb 10 06:44:53 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <49918E0A.1060500@sebster.com> References: <499165F3.6050803@sebster.com> <49918DA6.4020608@sebster.com> <49918E0A.1060500@sebster.com> Message-ID: Sebastiaan van Erk wrote: > Sebastiaan van Erk wrote: >> (However, just to give you an idea I attached the basic 5.1.2 >> unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench >> does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the >> output under linux to the "procinfo" file.) System: test-fbsd.vpn1.sebster.com: FreeBSD ------------------------------------------------------------------------ Benchmark Run: Tue Feb 10 2009 06:25:49 - 06:54:08 2 CPUs in system; running 1 parallel copy of tests System Benchmarks Index Values BASELINE RESULT INDEX Dhrystone 2 using register variables 116700.0 14144383.9 1212.0 Double-Precision Whetstone 55.0 3238.7 588.9 Execl Throughput 43.0 630.0 146.5 File Copy 1024 bufsize 2000 maxblocks 3960.0 28793.2 72.7 File Copy 256 bufsize 500 maxblocks 1655.0 33410.0 201.9 File Copy 4096 bufsize 8000 maxblocks 5800.0 33536.8 57.8 Pipe Throughput 12440.0 1146784.7 921.9 Pipe-based Context Switching 4000.0 36203.6 90.5 Process Creation 126.0 783.3 62.2 Shell Scripts (1 concurrent) 42.4 645.1 152.2 Shell Scripts (8 concurrent) 6.0 115.4 192.3 System Call Overhead 15000.0 939647.5 626.4 ======== System Benchmarks Index Score 212.4 System: test-ubuntu: GNU/Linux ------------------------------------------------------------------------ Benchmark Run: Mon Feb 09 2009 15:15:06 - 15:43:20 2 CPUs in system; running 1 parallel copy of tests System Benchmarks Index Values BASELINE RESULT INDEX Dhrystone 2 using register variables 116700.0 18610575.3 1594.7 Double-Precision Whetstone 55.0 2990.1 543.7 Execl Throughput 43.0 1058.6 246.2 File Copy 1024 bufsize 2000 maxblocks 3960.0 468973.2 1184.3 File Copy 256 bufsize 500 maxblocks 1655.0 132022.2 797.7 File Copy 4096 bufsize 8000 maxblocks 5800.0 921448.5 1588.7 Pipe Throughput 12440.0 1132933.6 910.7 Pipe-based Context Switching 4000.0 93429.0 233.6 Process Creation 126.0 1744.3 138.4 Shell Scripts (1 concurrent) 42.4 2566.9 605.4 Shell Scripts (8 concurrent) 6.0 518.4 864.0 System Call Overhead 15000.0 1935577.0 1290.4 ======== System Benchmarks Index Score 656.1 The results are ... interesting. It seems that FreeBSD simply dies in any test having a high context switch rate. Hmmm, this looks familiar. Either I or a collegue of mine had a similar situation some time ago, with the same discrepancy in disk speeds and the same difference in context switches. Unfortunately, there was no solution. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090210/46dfb036/signature.pgp From fbsd-performance at mawer.org Tue Feb 10 23:07:02 2009 From: fbsd-performance at mawer.org (Antony Mawer) Date: Tue Feb 10 23:07:09 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: References: <499165F3.6050803@sebster.com> <49918DA6.4020608@sebster.com> <49918E0A.1060500@sebster.com> Message-ID: <49927151.2030100@mawer.org> Ivan Voras wrote: > Sebastiaan van Erk wrote: >> Sebastiaan van Erk wrote: >>> (However, just to give you an idea I attached the basic 5.1.2 >>> unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench >>> does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the >>> output under linux to the "procinfo" file.) > ... benchmark results snipped ... > > The results are ... interesting. It seems that FreeBSD simply dies in > any test having a high context switch rate. Hmmm, this looks familiar. > Either I or a collegue of mine had a similar situation some time ago, > with the same discrepancy in disk speeds and the same difference in > context switches. Unfortunately, there was no solution. How would one go about gathering data on such a scenario to help improve this? We were planning a project involving VMware deployments with FreeBSD 7.1 systems in the near future, but if performance is that bad it is likely to be a show stopper. Where do we start looking and who should we be talking to? -- Antony From ivoras at freebsd.org Wed Feb 11 02:12:41 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Wed Feb 11 02:12:49 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <49927151.2030100@mawer.org> References: <499165F3.6050803@sebster.com> <49918DA6.4020608@sebster.com> <49918E0A.1060500@sebster.com> <49927151.2030100@mawer.org> Message-ID: Antony Mawer wrote: > Ivan Voras wrote: >> Sebastiaan van Erk wrote: >>> Sebastiaan van Erk wrote: >>>> (However, just to give you an idea I attached the basic 5.1.2 >>>> unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench >>>> does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the >>>> output under linux to the "procinfo" file.) >> > ... benchmark results snipped ... >> >> The results are ... interesting. It seems that FreeBSD simply dies in >> any test having a high context switch rate. Hmmm, this looks familiar. >> Either I or a collegue of mine had a similar situation some time ago, >> with the same discrepancy in disk speeds and the same difference in >> context switches. Unfortunately, there was no solution. > > How would one go about gathering data on such a scenario to help improve > this? We were planning a project involving VMware deployments with > FreeBSD 7.1 systems in the near future, but if performance is that bad > it is likely to be a show stopper. > > Where do we start looking and who should we be talking to? Relax, we didn't yet actually establish that it isn't a local problem. I've talked a little with OP but nothing we did yet made it better. For what it's worth, my experience is that on VMWare Server, whose emulated SCSI hardware is detected exactly the same as on ESX, the performance is "normal" (i.e. as expected), and on an ESX 3.0 on a slow array (and sharing it with other active machines) I get results around 20 MB/s in dbench - which is better than the OP gets on a fast array. All this is with 32-bit guests, FreeBSD 7.0 or 7.1. If I get the chance, I'll try ESXi within the next few days and try to replicate the problem. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090211/26b87a10/signature.pgp From Bradley.Radjoo at is.co.za Wed Feb 11 07:38:29 2009 From: Bradley.Radjoo at is.co.za (Bradley Radjoo) Date: Wed Feb 11 07:38:36 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter Message-ID: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> Greetings, Please can you assist. I have attempted most of the popular BSD Variants such as NetBSD, Open BSD, FreeBSD... The only distrubution that seems to work on the is Dell SAS 6/iR Adapter is FreeBSD 7.1 Firmware version & date : 9/29/2008 00.25.47.00 But the issue is that writing to disk is extremely slow...i.e. 14Mb per second.... If I install any other linux distribution, like CentOS, write speek is the way it should be..it averages around 190Mb+ per second I was hoping if it were at all possible for you to assist me in with a later RAID Controller Driver for the SAS 6/iR Adapter This is on a Dell PowerEdge R300 Server. BIOS Version 1.3.0 Running Raid 1 (ext3) NOTE : - Disabling ACPI does not make a difference - I do have the latest Firmware across the hardware Kind Regards, Bradley Please note: This email and its content are subject to the disclaimer as displayed at the following link http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. Should you not have Web access, send a mail to disclaimers@is.co.za and a copy will be emailed to you. From mui at mui.fi Wed Feb 11 10:23:09 2009 From: mui at mui.fi (Markus Kovero) Date: Wed Feb 11 10:30:39 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> Message-ID: <153301c98c73$aa948e80$ffbdab80$@fi> Hi, have you tried difference between UP and SMP kernels, years ago there were serious performance hit with SMP-kernel and adaptec/dell perc Yours Markus Kovero -----Original Message----- From: owner-freebsd-performance@freebsd.org [mailto:owner-freebsd-performance@freebsd.org] On Behalf Of Bradley Radjoo Sent: 11. helmikuuta 2009 17:18 To: freebsd-performance@freebsd.org Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter Greetings, Please can you assist. I have attempted most of the popular BSD Variants such as NetBSD, Open BSD, FreeBSD... The only distrubution that seems to work on the is Dell SAS 6/iR Adapter is FreeBSD 7.1 Firmware version & date : 9/29/2008 00.25.47.00 But the issue is that writing to disk is extremely slow...i.e. 14Mb per second.... If I install any other linux distribution, like CentOS, write speek is the way it should be..it averages around 190Mb+ per second I was hoping if it were at all possible for you to assist me in with a later RAID Controller Driver for the SAS 6/iR Adapter This is on a Dell PowerEdge R300 Server. BIOS Version 1.3.0 Running Raid 1 (ext3) NOTE : - Disabling ACPI does not make a difference - I do have the latest Firmware across the hardware Kind Regards, Bradley Please note: This email and its content are subject to the disclaimer as displayed at the following link http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. Should you not have Web access, send a mail to disclaimers@is.co.za and a copy will be emailed to you. _______________________________________________ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" From ivoras at freebsd.org Wed Feb 11 11:33:30 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Wed Feb 11 11:34:03 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <49927151.2030100@mawer.org> References: <499165F3.6050803@sebster.com> <49918DA6.4020608@sebster.com> <49918E0A.1060500@sebster.com> <49927151.2030100@mawer.org> Message-ID: <9bbcef730902111102j5ef13491md4aa887de83e55f4@mail.gmail.com> 2009/2/11 Antony Mawer : > How would one go about gathering data on such a scenario to help improve > this? We were planning a project involving VMware deployments with FreeBSD > 7.1 systems in the near future, but if performance is that bad it is likely > to be a show stopper. I have now tested it under ESXi 3.5, and here's what I find: In FreeBSD 7.1 amd64, 4 vCPUs performance for dbench is : 1 proc : 155 MB/s, 2 proc: 175 MB/s, 4 proc: 188 MB/s The same performance *as reported by VMWare's Infrastructure Client* ("performance" tab): around 50 MB/s in all cases Visual inspection of drives' LED indicators (2 drive 10k RPM RAID0 hw array) confirms constant activity. In Ubuntu 8.10 amd64, 4 vCPUs, performance for dbench is : 1 proc: 375 MB/s, 2 proc: 660 MB/s, 4 proc: 1055 MB/s (sic!) The same performance *as reported by VMWare Infrastructure Client*: around 25 MB/s in all cases (sic!) Visual inspection of drives: very sporadic activity The maximum performance expected from this array is around 150 MB/s *at peaks* - there is physically no way it can go above this, so I judge the above measurements bogus. This is all very strange. Someone here is caching more than it should be, and it looks like it's VMWare. It doesn't look as clock skew in the guests since "iostat 1" et al work at about 1sec wallclock time. The "visual inspection" oddity inspired me to do another benchmark: Bonnie++ reports: For FreeBSD: write: 52 MB/s, rewrite: 21 MB/s, read: 45 MB/s For Linux: write: 141 MB/s, rewrite: 55 MB/s, read: 168 MB/s VMWare's Infrastructure Client agrees with these performance measurements in both cases, and drives are blinking as expected. As previously demonstrated by me and others, Linux usually has significantly better file system performance in the non-virtualized case, so the difference could be simply increased by the virtualization. From richardtector at thekeelecentre.com Thu Feb 12 03:57:16 2009 From: richardtector at thekeelecentre.com (Richard Tector) Date: Thu Feb 12 03:57:28 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> Message-ID: <49940E8E.1080107@thekeelecentre.com> Bradley Radjoo wrote: > Greetings, > > Please can you assist. > > I have attempted most of the popular BSD Variants such as NetBSD, Open > BSD, FreeBSD... > The only distrubution that seems to work on the is Dell SAS 6/iR Adapter > is FreeBSD 7.1 > Firmware version & date : 9/29/2008 00.25.47.00 > > But the issue is that writing to disk is extremely slow...i.e. 14Mb per > second.... > > If I install any other linux distribution, like CentOS, write speek is > the way it should be..it averages around 190Mb+ per second > > I was hoping if it were at all possible for you to assist me in with a > later RAID Controller Driver for the SAS 6/iR Adapter Try: hw.mpt.enable_sata_wc=1 in /boot/loader.conf and reboot. Assuming you are using SATA drives, the write cache is disabled by default. This reenables it. The mpt manpage holds more information. Regards, Richard From Bradley.Radjoo at is.co.za Thu Feb 12 06:43:23 2009 From: Bradley.Radjoo at is.co.za (Bradley Radjoo) Date: Thu Feb 12 06:43:30 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> <49940E8E.1080107@thekeelecentre.com> Message-ID: <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> Thanks very much Richard, (They are 2 X 73Gig Ultra 320 scsi SAS drives) BUT seems that even with the UP kernel, I still get issues.. This is what I see in dmesg : mpt0: port 0xec00-0xecff mem 0xdfcec000-0xdfceffff,0xdfcf0000-0xdfcfffff irq 15 at device 0.0 on pci5 mpt0: [ITHREAD] mpt0: MPI Version=1.5.18.0 mpt0: mpt_wait_req(6) timed out mpt0: port 0 enable timed out mpt0: failed to enable port 0 mpt0: unable to initialize IOC da0 at mpt0 bus 0 target 0 lun 0 ses0 at mpt0 bus 0 target 8 lun 0 Please can you assist and advise -----Original Message----- From: Richard Tector [mailto:richardtector@thekeelecentre.com] Sent: 12 February 2009 01:57 PM To: Bradley Radjoo Cc: freebsd-performance@freebsd.org Subject: Re: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter Bradley Radjoo wrote: > Greetings, > > Please can you assist. > > I have attempted most of the popular BSD Variants such as NetBSD, Open > BSD, FreeBSD... > The only distrubution that seems to work on the is Dell SAS 6/iR Adapter > is FreeBSD 7.1 > Firmware version & date : 9/29/2008 00.25.47.00 > > But the issue is that writing to disk is extremely slow...i.e. 14Mb per > second.... > > If I install any other linux distribution, like CentOS, write speek is > the way it should be..it averages around 190Mb+ per second > > I was hoping if it were at all possible for you to assist me in with a > later RAID Controller Driver for the SAS 6/iR Adapter Try: hw.mpt.enable_sata_wc=1 in /boot/loader.conf and reboot. Assuming you are using SATA drives, the write cache is disabled by default. This reenables it. The mpt manpage holds more information. Regards, Richard Please note: This email and its content are subject to the disclaimer as displayed at the following link http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. Should you not have Web access, send a mail to disclaimers@is.co.za and a copy will be emailed to you. From ohartman at zedat.fu-berlin.de Thu Feb 12 07:18:09 2009 From: ohartman at zedat.fu-berlin.de (O. Hartmann) Date: Thu Feb 12 07:18:16 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> <49940E8E.1080107@thekeelecentre.com> <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> Message-ID: <499439A8.1080208@zedat.fu-berlin.de> Bradley Radjoo wrote: > Thanks very much Richard, > (They are 2 X 73Gig Ultra 320 scsi SAS drives) > > BUT seems that even with the UP kernel, I still get issues.. > This is what I see in dmesg : > > mpt0: port 0xec00-0xecff mem > 0xdfcec000-0xdfceffff,0xdfcf0000-0xdfcfffff irq 15 at device 0.0 on pci5 > mpt0: [ITHREAD] > mpt0: MPI Version=1.5.18.0 > mpt0: mpt_wait_req(6) timed out > mpt0: port 0 enable timed out > mpt0: failed to enable port 0 > mpt0: unable to initialize IOC > da0 at mpt0 bus 0 target 0 lun 0 > ses0 at mpt0 bus 0 target 8 lun 0 > > Please can you assist and advise > > -----Original Message----- > From: Richard Tector [mailto:richardtector@thekeelecentre.com] > Sent: 12 February 2009 01:57 PM > To: Bradley Radjoo > Cc: freebsd-performance@freebsd.org > Subject: Re: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell > SAS 6/iR Adapter > > Bradley Radjoo wrote: >> Greetings, >> >> Please can you assist. >> >> I have attempted most of the popular BSD Variants such as NetBSD, Open >> BSD, FreeBSD... >> The only distrubution that seems to work on the is Dell SAS 6/iR > Adapter >> is FreeBSD 7.1 >> Firmware version & date : 9/29/2008 00.25.47.00 >> >> But the issue is that writing to disk is extremely slow...i.e. 14Mb > per >> second.... >> >> If I install any other linux distribution, like CentOS, write speek is >> the way it should be..it averages around 190Mb+ per second >> >> I was hoping if it were at all possible for you to assist me in with a >> later RAID Controller Driver for the SAS 6/iR Adapter > > Try: > hw.mpt.enable_sata_wc=1 > in /boot/loader.conf and reboot. > > Assuming you are using SATA drives, the write cache is disabled by > default. This reenables it. The mpt manpage holds more information. > > Regards, > > Richard > Please note: This email and its content are subject to the disclaimer as displayed at the following link http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. Should you not have Web access, send a mail to disclaimers@is.co.za and a copy will be emailed to you. > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" We're using a Dell 1950-III Poweredge server with two XEON 4-Core-CPUs and at the momen one 750GB SATA drive. This box suffers from incredible slow drive access speeds - approximately a half of what I see on other boxes around. When I read this posting I was quite happy having found a possible solution, but I can't find the above mentioned OID, FreeBSD 8.0-CURRENT/amd64 (running on this box) shows only "hw.ata.wc: 1" as a relevant OID. No hw.mpt.XXX OIDs anywere. This is the part from DMESG reflecting MPT: mpt0: port 0xec00-0xecff mem 0xfc4fc000-0xfc4fffff,0xfc4e0000-0xfc4effff irq 16 at device 0.0 on pci1 mpt0: [ITHREAD] mpt0: MPI Version=1.5.14.0 mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 ) mpt0: 0 Active Volumes (2 Max) mpt0: 0 Hidden Drive Members (14 Max) I can not tell you, at this very moment, what configuration has been done in the BIOS. The SATA HD is recognized as da0: da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 300.000MB/s transfers da0: Command Queueing Enabled da0: 715404MB (1465149168 512 byte sectors: 255H 63S/T 91201C) Maybe someon can followup on this if there is some kind of issue ... Thanks in advance, Oliver From richardtector at thekeelecentre.com Thu Feb 12 10:04:50 2009 From: richardtector at thekeelecentre.com (Richard Tector) Date: Thu Feb 12 10:04:56 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <499439A8.1080208@zedat.fu-berlin.de> References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> <49940E8E.1080107@thekeelecentre.com> <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> <499439A8.1080208@zedat.fu-berlin.de> Message-ID: <499464B5.3090001@thekeelecentre.com> O. Hartmann wrote: > When I read this posting I was quite happy having found a possible > solution, but I can't find the above mentioned OID, FreeBSD > 8.0-CURRENT/amd64 (running on this box) shows only "hw.ata.wc: 1" as > a relevant OID. No hw.mpt.XXX OIDs anywere. This is the part from DMESG > reflecting MPT: > It's a driver tunable, not a sysctl. Richard From richardtector at thekeelecentre.com Thu Feb 12 10:05:42 2009 From: richardtector at thekeelecentre.com (Richard Tector) Date: Thu Feb 12 10:05:48 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> <49940E8E.1080107@thekeelecentre.com> <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> Message-ID: <499464E9.6060309@thekeelecentre.com> Bradley Radjoo wrote: > Thanks very much Richard, > (They are 2 X 73Gig Ultra 320 scsi SAS drives) No idea then, sorry. But I do have 3 R200's which also use the SAS6 controller due to arrive tomorrow, so I'll see what results I get. Richard From Bradley.Radjoo at is.co.za Thu Feb 12 23:32:03 2009 From: Bradley.Radjoo at is.co.za (Bradley Radjoo) Date: Thu Feb 12 23:32:10 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> <49940E8E.1080107@thekeelecentre.com> <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> <499464E9.6060309@thekeelecentre.com> Message-ID: <89D2AE9E4EAAB34FABDBF2913867C62F1CC159F8@ZABRYSVISEX04.af.didata.local> Indeed, driver tunable ;-) All the best Richard, would like to know it goes Bradley -----Original Message----- From: Richard Tector [mailto:richardtector@thekeelecentre.com] Sent: 12 February 2009 08:05 PM To: Bradley Radjoo Cc: freebsd-performance@freebsd.org Subject: Re: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter Bradley Radjoo wrote: > Thanks very much Richard, > (They are 2 X 73Gig Ultra 320 scsi SAS drives) No idea then, sorry. But I do have 3 R200's which also use the SAS6 controller due to arrive tomorrow, so I'll see what results I get. Richard Please note: This email and its content are subject to the disclaimer as displayed at the following link http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. Should you not have Web access, send a mail to disclaimers@is.co.za and a copy will be emailed to you. From josh at tcbug.org Fri Feb 13 11:46:13 2009 From: josh at tcbug.org (Josh Paetzel) Date: Fri Feb 13 11:46:20 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <9bbcef730902111102j5ef13491md4aa887de83e55f4@mail.gmail.com> References: <499165F3.6050803@sebster.com> <49918DA6.4020608@sebster.com> <49918E0A.1060500@sebster.com> <49927151.2030100@mawer.org> <9bbcef730902111102j5ef13491md4aa887de83e55f4@mail.gmail.com> Message-ID: <666B517C-6E30-465D-8BAD-B6266C4D3743@tcbug.org> On Feb 11, 2009, at 1:02 PM, Ivan Voras wrote: > > > As previously demonstrated by me and others, Linux usually has > significantly better file system performance in the non-virtualized > case, so the difference could be simply increased by the > virtualization. In my limited experience with VMWare linux seems to have near bare metal disk performance. FreeBSD seems to incur a significant performance penalty. For instance on my laptop, running OSX and VMWare Fusion, FreeBSd virtual machines can't saturate 100TX off the disk, raw dd manages about 7 Megs/sec, which is in line with what I get shovelling big files around. Disk is a 7200 RPM SATA2. Thanks, Josh Paetzel From ivoras at gmail.com Fri Feb 13 12:01:23 2009 From: ivoras at gmail.com (ivoras@gmail.com) Date: Fri Feb 13 12:42:05 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <666B517C-6E30-465D-8BAD-B6266C4D3743@tcbug.org> Message-ID: <000e0cd28750065bb80462d24d55@google.com> On Feb 13, 2009 8:27pm, Josh Paetzel wrote: > In my limited experience with VMWare linux seems to have near bare metal disk performance. FreeBSD seems to incur a significant performance penalty. For instance on my laptop, running OSX and VMWare Fusion, FreeBSd virtual machines can't saturate 100TX off the disk, raw dd manages about 7 Megs/sec, which is in line with what I get shovelling big files around. Disk is a 7200 RPM SATA2. > You might want to try the patch Scott Long made recently (http://svn.freebsd.org/changeset/base/188570) - I found it doubles the performance in some cases for VMWare (writing mostly, though sequential reading can be similarly improved by the combination of this patch and increasing vfs.read_max), but it's still worse than with Linux (100 MB/s vs 150 MB/s). As for the original thread topic: I've communicated with the OP and it appears his method of benchmarking had an error so the problems that appear in his post are bogus. From sebster at sebster.com Fri Feb 13 13:59:57 2009 From: sebster at sebster.com (Sebastiaan van Erk) Date: Fri Feb 13 14:00:04 2009 Subject: FreeBSD 7.1 disk performance issue on ESXi 3.5 In-Reply-To: <000e0cd28750065bb80462d24d55@google.com> References: <000e0cd28750065bb80462d24d55@google.com> Message-ID: <4995ED59.2090708@sebster.com> Hi, ivoras@gmail.com wrote: > As for the original thread topic: I've communicated with the OP and it > appears his method of benchmarking had an error so the problems that > appear in his post are bogus. It is not quite true that the "method" is bogus, there just seems to be a huge difference between a soft updates vs non-soft-updates disk. These are the results I get now: dbench -D -t 60 1 on / (ufs, local): Throughput 13.4561 MB/sec 1 procs on /tmp (ufs, local, soft-updates): Throughput 92.299 MB/sec 1 procs However, whether it is caching or not, Linux gets 350 MB/s using 1 process and even 650 MB/s using 2. As I understand it, this shouldn't be possible on the physical disks, but still, the *virtual* disk seems to get this performance. When I benchmark the linux vs the freebsd using Unixbench 4.1/5.1 (I tried both) I also get ***HUGE*** differences: System: test-fbsd.vpn1.sebster.com: FreeBSD ------------------------------------------------------------------------ Benchmark Run: Tue Feb 10 2009 06:25:49 - 06:54:08 2 CPUs in system; running 1 parallel copy of tests System Benchmarks Index Values BASELINE RESULT INDEX Dhrystone 2 using register variables 116700.0 14144383.9 1212.0 Double-Precision Whetstone 55.0 3238.7 588.9 Execl Throughput 43.0 630.0 146.5 File Copy 1024 bufsize 2000 maxblocks 3960.0 28793.2 72.7 File Copy 256 bufsize 500 maxblocks 1655.0 33410.0 201.9 File Copy 4096 bufsize 8000 maxblocks 5800.0 33536.8 57.8 Pipe Throughput 12440.0 1146784.7 921.9 Pipe-based Context Switching 4000.0 36203.6 90.5 Process Creation 126.0 783.3 62.2 Shell Scripts (1 concurrent) 42.4 645.1 152.2 Shell Scripts (8 concurrent) 6.0 115.4 192.3 System Call Overhead 15000.0 939647.5 626.4 ======== System Benchmarks Index Score 212.4 System: test-ubuntu: GNU/Linux ------------------------------------------------------------------------ Benchmark Run: Mon Feb 09 2009 15:15:06 - 15:43:20 2 CPUs in system; running 1 parallel copy of tests System Benchmarks Index Values BASELINE RESULT INDEX Dhrystone 2 using register variables 116700.0 18610575.3 1594.7 Double-Precision Whetstone 55.0 2990.1 543.7 Execl Throughput 43.0 1058.6 246.2 File Copy 1024 bufsize 2000 maxblocks 3960.0 468973.2 1184.3 File Copy 256 bufsize 500 maxblocks 1655.0 132022.2 797.7 File Copy 4096 bufsize 8000 maxblocks 5800.0 921448.5 1588.7 Pipe Throughput 12440.0 1132933.6 910.7 Pipe-based Context Switching 4000.0 93429.0 233.6 Process Creation 126.0 1744.3 138.4 Shell Scripts (1 concurrent) 42.4 2566.9 605.4 Shell Scripts (8 concurrent) 6.0 518.4 864.0 System Call Overhead 15000.0 1935577.0 1290.4 ======== System Benchmarks Index Score 656.1 Here the disk intensive test (file copy) and context switch/process creation test do terrible. For all my personal servers this is not an issue for me at all. But for a big high traffic web site I'm building, I'm afraid I'm going to have to go for Linux. :-( Regards, Sebastiaan > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3328 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090213/07fb0fa9/smime.bin From ohartman at mail.zedat.fu-berlin.de Fri Feb 13 17:00:53 2009 From: ohartman at mail.zedat.fu-berlin.de (O. Hartmann) Date: Fri Feb 13 18:46:35 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <499464B5.3090001@thekeelecentre.com> References: <89D2AE9E4EAAB34FABDBF2913867C62F1CB4B133@ZABRYSVISEX04.af.didata.local> <49940E8E.1080107@thekeelecentre.com> <89D2AE9E4EAAB34FABDBF2913867C62F1CBB66F5@ZABRYSVISEX04.af.didata.local> <499439A8.1080208@zedat.fu-berlin.de> <499464B5.3090001@thekeelecentre.com> Message-ID: <4996133E.9040302@mail.zedat.fu-berlin.de> Richard Tector wrote: > O. Hartmann wrote: >> When I read this posting I was quite happy having found a possible >> solution, but I can't find the above mentioned OID, FreeBSD >> 8.0-CURRENT/amd64 (running on this box) shows only "hw.ata.wc: 1" as >> a relevant OID. No hw.mpt.XXX OIDs anywere. This is the part from >> DMESG reflecting MPT: >> > > It's a driver tunable, not a sysctl. > > Richard Oh, thanks, I forgot. From freebsd at sopwith.solgatos.com Fri Feb 13 23:08:57 2009 From: freebsd at sopwith.solgatos.com (Dieter) Date: Fri Feb 13 23:09:04 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: Your message of "Thu, 12 Feb 2009 11:57:02 GMT." <49940E8E.1080107@thekeelecentre.com> Message-ID: <200902210616.GAA14352@sopwith.solgatos.com> > > But the issue is that writing to disk is extremely slow...i.e. 14Mb per > > second.... > > > > If I install any other linux distribution, like CentOS, write speek is > > the way it should be..it averages around 190Mb+ per second > > > > I was hoping if it were at all possible for you to assist me in with a > > later RAID Controller Driver for the SAS 6/iR Adapter > > Try: > hw.mpt.enable_sata_wc=1 > in /boot/loader.conf and reboot. > > Assuming you are using SATA drives, the write cache is disabled by > default. This reenables it. The mpt manpage holds more information. Just keep in mind that turning the disk's write cache on puts your data at risk. :-( The correct solution is NCQ. From sebster at sebster.com Fri Feb 13 23:58:14 2009 From: sebster at sebster.com (Sebastiaan van Erk) Date: Fri Feb 13 23:58:21 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <200902210616.GAA14352@sopwith.solgatos.com> References: <200902210616.GAA14352@sopwith.solgatos.com> Message-ID: <499672F1.4060109@sebster.com> Dieter wrote: >>> But the issue is that writing to disk is extremely slow...i.e. 14Mb per >>> second.... >>> >>> If I install any other linux distribution, like CentOS, write speek is >>> the way it should be..it averages around 190Mb+ per second >>> >>> I was hoping if it were at all possible for you to assist me in with a >>> later RAID Controller Driver for the SAS 6/iR Adapter >>> >> Try: >> hw.mpt.enable_sata_wc=1 >> in /boot/loader.conf and reboot. >> >> Assuming you are using SATA drives, the write cache is disabled by >> default. This reenables it. The mpt manpage holds more information. >> > > Just keep in mind that turning the disk's write cache on puts your > data at risk. :-( > > The correct solution is NCQ. Just wondering, if you have a battery on your RAID array, is this a problem? Wouldn't the cache get written anyway in case of a crash? Regards, Sebastiaan -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3328 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20090214/626e16cf/smime.bin From tom at tomjudge.com Tue Feb 17 13:43:32 2009 From: tom at tomjudge.com (Tom Judge) Date: Tue Feb 17 13:43:39 2009 Subject: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter In-Reply-To: <499672F1.4060109@sebster.com> References: <200902210616.GAA14352@sopwith.solgatos.com> <499672F1.4060109@sebster.com> Message-ID: <499B2B47.2000300@tomjudge.com> Sebastiaan van Erk wrote: > Dieter wrote: >>>> But the issue is that writing to disk is extremely slow...i.e. 14Mb >>>> per >>>> second.... >>>> >>>> If I install any other linux distribution, like CentOS, write speek is >>>> the way it should be..it averages around 190Mb+ per second >>>> >>>> I was hoping if it were at all possible for you to assist me in with a >>>> later RAID Controller Driver for the SAS 6/iR Adapter >>>> >>> Try: >>> hw.mpt.enable_sata_wc=1 >>> in /boot/loader.conf and reboot. >>> >>> Assuming you are using SATA drives, the write cache is disabled by >>> default. This reenables it. The mpt manpage holds more information. >>> >> >> Just keep in mind that turning the disk's write cache on puts your >> data at risk. :-( >> >> The correct solution is NCQ. > Just wondering, if you have a battery on your RAID array, is this a > problem? Wouldn't the cache get written anyway in case of a crash? > > Regards, > Sebastiaan Hi, hw.mpt.enable_sata_wc tunable enables the on disk write cache not the controller write cache. Tom