Online gaming and file downloads - latency hell!

[Modulok]

Yo,

I have a FreeBSD box acting as a router between me and the Internet.
Whenever someone on the local network downloads something, the other
connections have a really high latency. A second or more. For people
who like to download large files and play online games, it's not good.

I tried traffic shaping with PF, which works - almost: I tried the
home example in the PF book, but it doesn't work out so well. I can
throttle users with no trouble, but even so that doesn't seem to help
the latency issue unless I choke the 'big file download' users almost
completely off. It's like nothing helps. I tried a priority based
queue where all traffic on the gaming ports was placed in front of all
other traffic, and while I saw a very mild improvement, latency was
still really pitiful.

Is there anything else I can do? Anyone have a similar setup and wish
to share config files? Are there some sysctl's that would help this
out or something. I'm almost ready to just buy a 'gaming' *gag* router
which implements their own brand of QoS, but don't want to sink to
that level if I don't have to.

Help!
-Modulok-