ALTQ bandwidth limiting only from internet IPs

[Josh Finlay]

I use the following rules for PF:

ExtIF="ng0"
IntIF="de0"
Delta="192.168.0.6"
Fear="192.168.0.1"

altq on $ExtIF cbq bandwidth 128Kb queue { q_network_out }
altq on $IntIF cbq bandwidth 512Kb queue { q_network_in }
queue q_network_out bandwidth 100% { q_delta_out, q_fear_out }
        queue q_delta_out bandwidth 50% cbq(default borrow)
        queue q_fear_out bandwidth 50% cbq(borrow)
queue q_network_in bandwidth 100% { q_delta_in, q_fear_in }
        queue q_delta_in bandwidth 50% cbq(default borrow)
        queue q_fear_in bandwidth 50% cbq(borrow)

pass out on $ExtIF from $Delta to any keep state queue q_delta_out
pass out on $ExtIF from $Fear to any keep state queue q_fear_out
pass out on $IntIF from $Delta to any keep state queue q_delta_in
pass out on $IntIF from $Fear to any keep state queue q_fear_in

This config seems to work quite well
but its also queueing local traffic aswell
so if I'm uploading from "Delta" to somewhere on the internet, my local ssh 
sessions (to the machine running pf) lag due to lack of free bandwidth

So how do I tell PF to only queue if its an internet ip? or perhaps a better 
way of saying it, is to *not* queue local traffic (to/from local ips).