Online gaming and file downloads - latency hell!

[Ian Smith]

In freebsd-questions Digest, Vol 315, Issue 11, Message: 9
On Fri, 18 Jun 2010 12:11:48 +0200
Morgan Wesstr?m <freebsd-questions at pp.dyndns.biz> wrote:
 > On 2010-06-16 02:51, Modulok wrote:
 > > Yo,
 > > 
 > > I have a FreeBSD box acting as a router between me and the Internet.
 > > Whenever someone on the local network downloads something, the other
 > > connections have a really high latency. A second or more. For people
 > > who like to download large files and play online games, it's not good.
 > > 
 > > I tried traffic shaping with PF, which works - almost: I tried the
 > > home example in the PF book, but it doesn't work out so well. I can
 > > throttle users with no trouble, but even so that doesn't seem to help
 > > the latency issue unless I choke the 'big file download' users almost
 > > completely off. It's like nothing helps. I tried a priority based
 > > queue where all traffic on the gaming ports was placed in front of all
 > > other traffic, and while I saw a very mild improvement, latency was
 > > still really pitiful.
 > > 
 > > Is there anything else I can do? Anyone have a similar setup and wish
 > > to share config files? Are there some sysctl's that would help this
 > > out or something. I'm almost ready to just buy a 'gaming' *gag* router
 > > which implements their own brand of QoS, but don't want to sink to
 > > that level if I don't have to.
 > > 
 > > Help!
 > > -Modulok-
 > 
 > Traffic shaping on your side when downloading unfortunately doesn't help
 > you. The data has already been transferred across your cable or DSL
 > connection by then and reordering any packets on your side will not
 > change the latency. Traffic shaping your download has to be performed at
 > your upstream router which you probably don't control. PF can help you
 > traffic shape your outgoing traffic. I have used it for this for the
 > past 6 years to help me maintain a low and stable ping while I play
 > online, even if I upload simultaneously. I've read about people trying
 > to throttle outgoing ACKs to slow down their download but that still
 > wouldn't rearrange any incoming data packets so I don't see how that
 > would help. I haven't tried it myself though but neither have I read
 > about anyone successfully accomplishing this.
 > 
 > Regards
 > Morgan

A short story:

About 15 months ago, before becoming aware that Luigi and colleagues had 
been busy porting ipfw and dummynet to Linux, I was asked to implement a 
shaping solution for a very limited (512/512kbps) ADSL connection for a 
community radio station using a Linux firewall-in-a-box called IPCop as 
router, whose shaping was based on Bert Hubert's WonderShaper script, 
using Linux' tc module to prioritise and shape only outbound traffic.

Having used ipfw+dummynet successfully for some years to shape traffic 
for a local voluntary organisation 'Community Technology Centre', I was 
staggered to find that all of the collective Linux wisdom on the subject 
chanted that same mantra .. that you can't prioritise download traffic, 
as the ISP will have 'gigantic queues' of TCP traffic that you can't 
control, and that prioritising ACKs, QoS and ICMP traffic and such is 
the best you can do.  By this philosophy, tc only implements limiting 
total bandwidth of inbound traffic, shaping outbound by QoS and classes.

To disprove this pervasive myth I had to implement inbound shaping by 
using tc to control the _outbound_ traffic to the _inside_ interface, 
where all sorts of random clients are doing big downloads, yootoobing 
and such plus some big uploads, while guaranteeing that the station's 
outbound audio stream had fully half the outbound-to-net bandwidth free 
without undue pressure and that remote ssh sessions etc remained snappy.  

This involves queuing inbound (mostly TCP) traffic on the local router, 
dropping any excess, which works most effectively to maintain a hard 
limit to downloads (at around 85% of 512kbps) while keeping the outbound 
(to-net) channel lightly loaded after streaming, ACKs, and uploads.

I don't know how pf works (or can be made to work) in this regard, nor 
can I speculate about gaming latency particularly, but hope to find out 
soon by either replacing the old IPCop box with pfSense, or trying ipfw 
and dummynet on Linux .. I know, but they're still reluctant to shop 
other than Linux, and the idea of implementing a FreeBSD-derived 
firewall and shaping solution on Linux has a good deal of appeal ..

HTH (or at least, doesn't hurt :)

cheers, Ian