Problems getting result from DNSMASQ service

[dweimer]

I am trying to setup DNSMASQ to use as a resolver for NGINX so that I 
can return a a result form the local hosts file for a reverse proxy 
resolution.  I installed DNSMASQ from ports and configured it, it 
appears to be running correctly, but the client just times out.

If I run it in foreground and watch it, DNSMASQ receives the query, 
forwards it to upstream server and sends reply.

# dnsmasq -d -q
dnsmasq: started, version 2.72 cachesize 150
dnsmasq: compile time options: IPv6 GNU-getopt no-DBus i18n IDN DHCP 
DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect
dnsmasq: reading /etc/resolv.conf
dnsmasq: ignoring nameserver 10.50.20.6 - local interface
dnsmasq: using nameserver 10.50.20.5#53
dnsmasq: using nameserver 10.50.20.4#53
dnsmasq: read /etc/hosts - 3 addresses
dnsmasq: query[A] www.yahoo.com from 10.50.20.4
dnsmasq: forwarded www.yahoo.com to 10.50.20.5
dnsmasq: forwarded www.yahoo.com to 10.50.20.4
dnsmasq: reply www.yahoo.com is <CNAME>
dnsmasq: reply fd-fp3.wg1.b.yahoo.com is 98.139.180.149
dnsmasq: reply fd-fp3.wg1.b.yahoo.com is 98.139.183.24
dnsmasq: query[A] www.yahoo.com from 10.50.20.4
dnsmasq: cached www.yahoo.com is <CNAME>
dnsmasq: cached fd-fp3.wg1.b.yahoo.com is 98.139.183.24
dnsmasq: cached fd-fp3.wg1.b.yahoo.com is 98.139.180.149
dnsmasq: query[A] www.yahoo.com from 10.50.20.4
dnsmasq: cached www.yahoo.com is <CNAME>
dnsmasq: cached fd-fp3.wg1.b.yahoo.com is 98.139.180.149
dnsmasq: cached fd-fp3.wg1.b.yahoo.com is 98.139.183.24

The client doesn't get it though.
# nslookup
> server 10.50.20.6
Default server: 10.50.20.6
Address: 10.50.20.6#53
> www.yahoo.com
;; connection timed out; no servers could be reached

Does anyone have any Ideas what I am missing, I tried searching online 
but I can't seem to find anyone else that has had this problem. I don't 
have any firewall between the devices, I get the same result running the 
query from the same machine as DNSMASQ service. I am running the service 
in a jail though, maybe that is having some sort of impact on the 
results?

-- 
Thanks,
    Dean E. Weimer
    http://www.dweimer.net/