From nobody Thu Jul 18 16:07:35 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WPyQQ3p4bz5RfXt for ; Thu, 18 Jul 2024 16:08:14 +0000 (UTC) (envelope-from odhiambo@gmail.com) Received: from mail-oa1-x2b.google.com (mail-oa1-x2b.google.com [IPv6:2001:4860:4864:20::2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WPyQP49mZz4Kf6 for ; Thu, 18 Jul 2024 16:08:13 +0000 (UTC) (envelope-from odhiambo@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=nhDWMhh3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of odhiambo@gmail.com designates 2001:4860:4864:20::2b as permitted sender) smtp.mailfrom=odhiambo@gmail.com Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-260e6dfc701so530107fac.3 for ; Thu, 18 Jul 2024 09:08:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721318892; x=1721923692; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=eOFP5/rWKUz76UsEWkEUZQvNd+TZlITNcnMX+cq67Pw=; b=nhDWMhh3szzwB6Rv5jxdbHgCbofF5nnE0mWt9sbKhJOR/UWbHzMJWGNSmYiJXZ0SYW ipPFERMO9lSvsil7hlDmt2W2XrQ0m8TaA5wXGNwHcjR/NSFqKSJQW1h+Dn+6MlaRmjd4 OQXK6uN3QnzKGfvu2uLE2emW2rDjguYH/+nOmVNav9n+XqPNZ6hk8XMJpVi05xZ1Mgew JO4kA8Gz0dk+lZ65NLwY58jQOyaJo7111/9iPpF9qbwOoPpdEnXWY8t3o6LxvpCFCkEA 05Nqs+NGJA9uTb3eJffKujtMEcLiLUZu+ZywyL8oVyJqARxuV5VhH/4SsjMauE1J7rB4 25wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721318892; x=1721923692; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eOFP5/rWKUz76UsEWkEUZQvNd+TZlITNcnMX+cq67Pw=; b=CHYfwcmHfpLxOex1yyZOch7JRQiLidxI/JnkSX96xwPwTg6aATPrydqF3hPN+19qmr pUSgNFf5mxx0BP7APRk08af9/u3JDZ0fAeR3kxfGFBMn+xmmhBoVlAY1b2sCYHG073QP 5WFipUWLhxdQHhrLOAXtG8M5XjsHv+d3KmWiAhQHJ8Jw7Suu0jjoA2tsIhttIeMt/dTr yJbqt21s8/xt1KsTkBVu/x6MbS2Tx6mL8GdwNdh0QVe1MCggujJIByCNvwKCZSaLmhAs PqwiaoA05xG4+KXipHBJceublc1r1UnPRlgXlHgPelmGYZFPrL4SaU7jxvp9kBFjqBy4 VOJQ== X-Gm-Message-State: AOJu0YxIWsoVPax/zbyJCfFS49WcI65O4Fr6K4OedFEz/FwrD2ndZTBA EbCWDGr3204ynKM5mf4eEklDwkFieAN+TGWdV8Li4n20TiZSjSmX5ZbrqRDSR4Q7nl5rkbMj/Y+ gZ8MKYS0bwUHE6OcpJXAAtKf8Z7oj6QHk4O0= X-Google-Smtp-Source: AGHT+IGnnK6dOsii89bi5+AQmzslOR8jKcMV+9K/e+S9/nqUE6mFvtYJdRRRk9MjW2O1bTStC/45e0kAIFgWZeYKRfA= X-Received: by 2002:a05:6870:f610:b0:261:87:fe1f with SMTP id 586e51a60fabf-26100880120mr874843fac.44.1721318891447; Thu, 18 Jul 2024 09:08:11 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <202407111449.46BEnLoP051380@gndrsh.dnsmgr.net> In-Reply-To: From: Odhiambo Washington Date: Thu, 18 Jul 2024 19:07:35 +0300 Message-ID: Subject: Re: Suddenly unable to access VMs To: FreeBSD virtualization Content-Type: multipart/alternative; boundary="00000000000011efcf061d87c97a" X-Spamd-Bar: - X-Spamd-Result: default: False [-1.28 / 15.00]; URI_COUNT_ODD(1.00)[15]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.97)[-0.966]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.32)[-0.318]; R_SPF_ALLOW(-0.20)[+ip6:2001:4860:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; MISSING_XM_UA(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[2001:4860:4864:20::2b:from] X-Rspamd-Queue-Id: 4WPyQP49mZz4Kf6 --00000000000011efcf061d87c97a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jul 11, 2024 at 6:23=E2=80=AFPM Odhiambo Washington wrote: > > > On Thu, Jul 11, 2024 at 5:49=E2=80=AFPM Rodney W. Grimes < > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > >> > My bhyve VMs have been all fine until now. >> > I can't ping them and can't SSH into them. However, I can connect to >> them >> > with VNCViewer from a remote host (my PC from my house) :-( >> > >> > I haven't done any changes on the host at all. >> > dnsmasq is running, but seems like the VMs aren't getting the IPs for >> some >> > reason. >> > >> > ``` >> > cloned_interfaces=3D"bridge0 tap0 tap1 tap2 tap3 tap4 tap5" >> > ifconfig_bridge0_name=3D"vmbridge" >> > ifconfig_vmbridge=3D"addm em1 addm tap0 addm tap1 addm tap2 addm tap3 = addm >> > tap4 addm tap5 up" >> > ifconfig_vmbridge_alias0=3D"inet 172.16.0.1 netmask 255.255.255.0" >> > ``` >> > What might have happened? >> > >> > >> > root@gw:/home/wash # ifconfig vmbridge >> > vmbridge: flags=3D1008843 >> > metric 0 mtu 1500 >> > options=3D0 >> > ether 58:9c:fc:10:df:1d >> > inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 >> > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >> > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 >> > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >> > member: tap5 flags=3D143 >> > ifmaxaddr 0 port 10 priority 128 path cost 2000000 >> > member: tap4 flags=3D143 >> > ifmaxaddr 0 port 9 priority 128 path cost 2000000 >> > member: tap3 flags=3D143 >> > ifmaxaddr 0 port 8 priority 128 path cost 2000000 >> > member: tap2 flags=3D143 >> > ifmaxaddr 0 port 7 priority 128 path cost 2000000 >> > member: tap1 flags=3D143 >> > ifmaxaddr 0 port 6 priority 128 path cost 2000000 >> > member: tap0 flags=3D143 >> > ifmaxaddr 0 port 5 priority 128 path cost 2000000 >> > member: em1 flags=3D143 >> > ifmaxaddr 0 port 2 priority 128 path cost 55 >> > groups: bridge >> > nd6 options=3D9 >> > root@gw:/home/wash # ssh 172.16.0.99 >> > ssh: connect to host 172.16.0.99 port 22: Permission denied >> > root@gw:/home/wash # ssh 172.16.0.100 >> > ssh: connect to host 172.16.0.100 port 22: Permission denied >> > root@gw:/home/wash # ping 172.16.0.100 >> > PING 172.16.0.100 (172.16.0.100): 56 data bytes >> > ping: sendto: Permission denied >> > ping: sendto: Permission denied >> > ping: sendto: Permission denied >> > ping: sendto: Permission denied >> > ^C >> > --- 172.16.0.100 ping statistics --- >> > 4 packets transmitted, 0 packets received, 100.0% packet loss >> > root@gw:/home/wash # ping 172.16.0.99 >> > PING 172.16.0.99 (172.16.0.99): 56 data bytes >> > ping: sendto: Permission denied >> > ping: sendto: Permission denied >> > ping: sendto: Permission denied >> > ^C >> > --- 172.16.0.99 ping statistics --- >> > 3 packets transmitted, 0 packets received, 100.0% packet loss >> > root@gw:/home/wash # service dnsmasq status >> > dnsmasq is running as pid 4190. >> > root@gw:/home/wash # >> >> Permission denied is almost certainly coming from firewall, >> either ipfw or pf. >> > > I haven't changed anything in my pf.conf either. > What also baffles me is that the VMs are not obtaining IP addresses from > dnsmasq. > Is anyone able to spot something obvious from the following pf.conf that could be causing the problem I am having? Thanking you in advance. ``` #--------------------------------------------------------------------------= ----- # PF: List and Macros #--------------------------------------------------------------------------= ----- # Interfaces ext_if =3D "em0" # macro for external interface - use tun0 for PPPoE int_if =3D "em1" # macro for internal interface jail_if =3D "lo1" # the interface we chose for communication between jails # # bhyve bhyve_net=3D"172.16.0.0/24" int_addr =3D "192.168.55.254" # Internal IPv4 address (i.e., gateway for private network) int_network =3D "192.168.54.0/23" # Internal IPv4 network # External services - port 25214 is used instead of 514 for syslog ext_tcp_services =3D "{ 21 22 25 26 53 80 110 123 143 443 465 587 2222 2525 2587 25214 993 995 3000 5000><5100 6276 6277 8000 8069 8080 8081 8082 8083 8999 10000 5900><6000 30000><50000 1024><6277 8765 }" ext_udp_services =3D "{ ntp, 1194 }" # Internal services int_tcp_services =3D "{ domain, bootps, dhcpv6-server, ntp, http, https, http-alt, \ smtp, smtps, pop3, pop3s, imap, imaps, ftp-data, ftp, ssh, svn, 2222, 2232, 4444, \ 3128, 3129, 13128, 13129, 23129, 9050, 8123, 8056, 2199, 8191, 82, 2087, 8333, 1157, \ 2083, 8030, 10443, 3389, 8080, 8081, 9091, 81, 8086, 8000, 8001, 8002, 8005, 8006, 8090, 8100, 8800, 443, 465, 587, 8444, 8443, 9443, \ 18082, 18087, 18092, 18093, 9447, 7005, 115, 8030, 18090, 18083, 18084, 15001, 15002, 15003, 2082 }" int_udp_services =3D "{ domain, bootps, dhcpv6-server, ntp, 1194, svn, sip, 8056, 500, 1000, 10000 }" # The martians table denotes the RFC 1918 addresses and a few other ranges which # are mandated by various RFCs not to be in circulation on the open Internet. martians =3D "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \ 10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \ 0.0.0.0/8, 240.0.0.0/4 }" #--------------------------------------------------------------------------= ----- # PF: Tables #--------------------------------------------------------------------------= ----- table persist file "/etc/firewall/whatsapp-cidr.txt" table file "/etc/firewall/bruteforce_attackers" persist table persist file "/etc/firewall/f2b" table persist table persist #--------------------------------------------------------------------------= ----- # PF: Options #--------------------------------------------------------------------------= ----- # No restrictions on jail network set skip on $jail_if set skip on lo #--------------------------------------------------------------------------= ----- # PF: Scrub (Packet Normalization) #--------------------------------------------------------------------------= ----- set limit { states 40000, frags 20000, src-nodes 20000 } set timeout { adaptive.start 18000, adaptive.end 39000 } #--------------------------------------------------------------------------= ----- # PF: Packet Queueing and Priorization #--------------------------------------------------------------------------= ----- #--------------------------------------------------------------------------= ----- # PF: Netkwork Address Translation (NAT) and Packet Redirection #--------------------------------------------------------------------------= ----- # Network address translation # Internet nat on $ext_if inet from any to any -> $ext_if # Bhyve nat on $ext_if from $bhyve_net to any -> ($ext_if) # Traccar rdr pass on $ext_if inet proto { tcp, udp } from any to port 5055 -> 127.0.0.1 port 5055 # Nominatim Test rdr pass on $ext_if inet proto { tcp, udp } from any to port 8089 -> 127.0.0.1 port 8089 #--------------------------------------------------------------------------= ----- # PF: Packet Filtering #--------------------------------------------------------------------------= ----- # Restrictive default rules block all # Block packets and reply with a TCP RST or ICMP Unreachable response block return # FTP-Proxy # We need to have an anchor for ftp-proxy anchor "ftp-proxy/*" # Anchor for fail2ban anchor "f2b/*" # Anchor for blacklistd # This makes sure that the rules within blacklistd are only used for incoming data on ext_if anchor "blacklistd/*" in on $ext_if # Blocking Spoofed Packets #--------------------------------------------------------------------------= ----- # Filter rules for $ext_if inbound # Temporarily let go everything, do not leave active! #pass in on $ext_if inet #pass in on $ext_if inet6 # Allow ping pass in on $ext_if inet proto icmp all pass in on $ext_if inet6 proto icmp6 all # External services pass in on $ext_if inet proto tcp to port $ext_tcp_services pass in on $ext_if inet proto udp to port $ext_udp_services block drop in quick on $ext_if inet from to any block drop in quick on $ext_if inet from to any block in quick from # Custom blocks block drop in quick on $ext_if inet from 31.130.184.0/24 to any block in proto tcp from to any # Now block the ssh bruteforce block drop in quick on $ext_if inet from block drop in quick on $ext_if inet6 from # PF "Self-Protecting" an FTP Server (passive) pass in on $ext_if inet proto tcp from any to any port { ftp, > 49151 } pass in on $ext_if inet6 proto tcp from any to any port { ftp, > 49151 } # Traccar pass in on $ext_if inet proto tcp from any to any port { 5000><5100 } pass in on $ext_if inet6 proto tcp from any to any port { 5000><5100 } # Nominatim Test pass in on $ext_if inet proto tcp from any to any port 8089 #--------------------------------------------------------------------------= ----- # Filter rules for $ext_if outbound pass out on $ext_if inet pass out on $ext_if inet6 #--------------------------------------------------------------------------= ----- # Filter rules for $int_if inbound # block drop in quick on $int_if inet from 192.168.55.x to any port { 443 993 } #block drop in quick on $int_if from 192.168.54.190 to any #block drop in quick on $int_if from any to 13.107.4.50 # Temporarily let go everything, do not leave active! #pass in on $int_if inet #pass in on $int_if inet6 # Internal services pass in on $int_if inet proto tcp to port $int_tcp_services pass in on $int_if inet proto udp to port $int_udp_services # Filter rules for $int_if outbound pass out on $int_if inet pass out on $int_if inet6 #--------------------------------------------------------------------------= ----- # DEBUG: RULES FOR VMM pass in quick on bridge0 all keep state pass in quick on tap0 all keep state pass in quick on tap1 all keep state pass in quick on tap2 all keep state pass in quick on tap3 all keep state pass out quick on bridge0 all keep state pass out quick on tap0 all keep state pass out quick on tap1 all keep state pass out quick on tap2 all keep state pass out quick on tap3 all keep state # Bhyve hosts pass in on tap0 pass in on tap1 pass in on tap2 pass in on tap3 ``` --=20 Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html] --00000000000011efcf061d87c97a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Jul 11, 2024 at 6:23=E2=80=AF= PM Odhiambo Washington <odhiambo@g= mail.com> wrote:


On Thu, Jul 11, 2024 at 5:49= =E2=80=AFPM Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> My bhyve VMs hav= e been all fine until now.
> I can't ping them and can't SSH into them. However, I can conn= ect to them
> with VNCViewer from a remote host (my PC from my house) :-(
>
> I haven't done any changes on the host at all.
> dnsmasq is running, but seems like the VMs aren't getting the IPs = for some
> reason.
>
> ```
> cloned_interfaces=3D"bridge0 tap0 tap1 tap2 tap3 tap4 tap5"<= br> > ifconfig_bridge0_name=3D"vmbridge"
> ifconfig_vmbridge=3D"addm em1 addm tap0 addm tap1 addm tap2 addm = tap3 addm
> tap4 addm tap5 up"
> ifconfig_vmbridge_alias0=3D"inet 172.16.0.1 netmask 255.255.255.0= "
> ```
> What might have happened?
>
>
> root@gw:/home/wash # ifconfig vmbridge
> vmbridge: flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LO= WER_UP>
> metric 0 mtu 1500
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0options=3D0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ether 58:9c:fc:10:df:1d
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet 172.16.0.1 netmask 0xffffff00 br= oadcast 172.16.0.255
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0id 00:00:00:00:00:00 priority 32768 h= ellotime 2 fwddelay 15
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0maxage 20 holdcnt 6 proto rstp maxadd= r 2000 timeout 1200
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0root id 00:00:00:00:00:00 priority 32= 768 ifcost 0 port 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap5 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 10 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap4 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 9 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap3 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 8 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap2 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 7 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap1 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 6 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap0 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 5 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: em1 flags=3D143<LEARNING,D= ISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 2 priority 128 path cost 55
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0groups: bridge
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nd6 options=3D9<PERFORMNUD,IFDISAB= LED>
> root@gw:/home/wash # ssh 172.16.0.99
> ssh: connect to host 172.16.0.99 port 22: Permission denied
> root@gw:/home/wash # ssh 172.16.0.100
> ssh: connect to host 172.16.0.100 port 22: Permission denied
> root@gw:/home/wash # ping 172.16.0.100
> PING 172.16.0.100 (172.16.0.100): 56 data bytes
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ^C
> --- 172.16.0.100 ping statistics ---
> 4 packets transmitted, 0 packets received, 100.0% packet loss
> root@gw:/home/wash # ping 172.16.0.99
> PING 172.16.0.99 (172.16.0.99): 56 data bytes
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ^C
> --- 172.16.0.99 ping statistics ---
> 3 packets transmitted, 0 packets received, 100.0% packet loss
> root@gw:/home/wash # service dnsmasq status
> dnsmasq is running as pid 4190.
> root@gw:/home/wash #

Permission denied is almost certainly coming from firewall,
either ipfw or pf.

I haven't change= d anything in my pf.conf either.
What also baffles me is that the= VMs are not obtaining IP addresses from dnsmasq.=C2=A0

Is anyone able to spot something obvious fro= m the following pf.conf that could be causing the problem I am having?
Thanking you in advance.

```

#= ---------------------------------------------------------------------------= ----
# PF: List and Macros
#-----------------------------------------= --------------------------------------

# Interfaces
ext_if =3D &q= uot;em0" # macro for external interface - use tun0 for PPPoE
int= _if =3D "em1" # macro for internal interface

jail_if = =3D "lo1" # the interface we chose for communication between j= ails
#
# bhyve
bhyve_net=3D"= 172.16.0.0/24"

int_addr =3D "192.168.55.254" =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Internal IPv4 address (i.e., gatew= ay for private network)
int_network =3D "192.168.54.0/23" =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Internal I= Pv4 network

# External services - port 25214 is used instead of 514 = for syslog
ext_tcp_services =3D "{ 21 22 25 26 53 80 110 123 143 44= 3 465 587 2222 2525 2587 25214 993 995 3000 5000><5100 6276 6277 8000= 8069 8080 8081 8082 8083 8999 10000 5900><6000 30000><50000 10= 24><6277 8765 }"
ext_udp_services =3D "{ ntp, 1194 }&quo= t;

# Internal services
int_tcp_services =3D "{ domain, bootp= s, dhcpv6-server, ntp, http, https, http-alt, \
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 smtp, smtps, pop3, pop3s, imap, imaps, ftp-data, ftp, ssh, svn, 2222, 2= 232, 4444, \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 3128, 3129, 13128, 13129, 23129= , 9050, 8123, 8056, 2199, 8191, 82, 2087, 8333, 1157, \
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 2083, 8030, 10443, 3389, 8080, 8081, 9091, 81, 8086, 8000, 80= 01, 8002, 8005, 8006, 8090, 8100, 8800, 443, 465, 587, 8444, 8443, 9443, \<= br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 18082, 18087, 18092, 18093, 9447, 7005, 115,= 8030, 18090, 18083, 18084, 15001, 15002, 15003, 2082 }"
int_udp_se= rvices =3D "{ domain, bootps, dhcpv6-server, ntp, 1194, svn, sip, 8056= , 500, 1000, 10000 }"

# The martians table denotes the RFC 1918= addresses and a few other ranges which
# are mandated by various RFCs n= ot to be in circulation on the open Internet.
martians =3D "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, \=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 0.0.0.0/8, 240.0.0.0/= 4 }"

#-----------------------------------------------------= --------------------------
# PF: Tables
#----------------------------= ---------------------------------------------------

table <whatsa= pp-cidr> persist file "/etc/firewall/whatsapp-cidr.txt"
tab= le <bruteforce> file "/etc/firewall/bruteforce_attackers" p= ersist
table <f2b> persist file "/etc/firewall/f2b"
t= able <fail2ban> persist
table <sshguard> persist

#---= ---------------------------------------------------------------------------= -
# PF: Options
#----------------------------------------------------= ---------------------------

# No restrictions on jail network
set= skip on $jail_if
set skip on lo

#-------------------------------= ------------------------------------------------
# PF: Scrub (Packet Nor= malization)
#-----------------------------------------------------------= --------------------

set limit { states 40000, frags 20000, src-node= s 20000 }
set timeout { adaptive.start 18000, adaptive.end 39000 }
#------------------------------------------------------------------------= -------
# PF: Packet Queueing and Priorization
#---------------------= ----------------------------------------------------------

#--------= -----------------------------------------------------------------------
= # PF: Netkwork Address Translation (NAT) and Packet Redirection
#-------= ------------------------------------------------------------------------
# Network address translation
# Internet
nat on $ext_if inet fro= m any to any -> $ext_if

# Bhyve
nat on $ext_if from $bhyve_net= to any -> ($ext_if)

# Traccar
rdr pass on $ext_if inet proto = { tcp, udp } from any to port 5055 -> 127.0.0.1 port 5055
# Nominatim= Test
rdr pass on $ext_if inet proto { tcp, udp } from any to port 8089 = -> 127.0.0.1 port 8089


#-------------------------------------= ------------------------------------------
# PF: Packet Filtering
#--= ---------------------------------------------------------------------------= --

# Restrictive default rules
block all

# Block packets a= nd reply with a TCP RST or ICMP Unreachable response
block return
# FTP-Proxy
# We need to have an anchor for ftp-proxy
anchor "f= tp-proxy/*"

# Anchor for fail2ban
anchor "f2b/*"
# Anchor for blacklistd
# This makes sure that the rules within bl= acklistd are only used for incoming data on ext_if
anchor "blacklis= td/*" in on $ext_if

# Blocking Spoofed Packets

#--------= -----------------------------------------------------------------------
= # Filter rules for $ext_if inbound

# Temporarily let go everything, = do not leave active!
#pass in on $ext_if inet
#pass in on $ext_if ine= t6

# Allow ping
pass in on $ext_if inet =C2=A0proto icmp =C2=A0al= l
pass in on $ext_if inet6 proto icmp6 all

# External servicespass in on $ext_if inet =C2=A0proto tcp to port $ext_tcp_services
pass = in on $ext_if inet =C2=A0proto udp to port $ext_udp_services

block d= rop in quick on $ext_if inet from <bruteforce> to any
block drop i= n quick on $ext_if inet from <f2b> to any
block in quick from <= fail2ban>

# Custom blocks
block drop in quick on $ext_if inet = from 31.130.184.0/24 to any

<= br>block in proto tcp from <sshguard> to any

# Now block the s= sh bruteforce=C2=A0
block drop in quick on $ext_if inet =C2=A0from <s= sh-bruteforce>
block drop in quick on $ext_if inet6 from <ssh-brut= eforce>

# PF "Self-Protecting" an FTP Server (passive)<= br>pass in on $ext_if inet =C2=A0proto tcp from any to any port { ftp, >= 49151 }
pass in on $ext_if inet6 proto tcp from any to any port { ftp, = > 49151 }

# Traccar
pass in on $ext_if inet proto tcp from any= to any port { 5000><5100 }
pass in on $ext_if inet6 proto tcp fro= m any to any port { 5000><5100 }
# Nominatim Test
pass in on $e= xt_if inet proto tcp from any to any port 8089

#--------------------= -----------------------------------------------------------
# Filter rul= es for $ext_if outbound

pass out on $ext_if inet
pass out on $ext= _if inet6

#---------------------------------------------------------= ----------------------
# Filter rules for $int_if inbound

# block= drop in quick on $int_if inet from 192.168.55.x to any port { 443 993 }#block drop in quick on $int_if from 192.168.54.190 to any
#block drop = in quick on $int_if from any to 13.107.4.50


# Temporarily let go= everything, do not leave active!
#pass in on $int_if inet
#pass in o= n $int_if inet6


# Internal services
pass in on $int_if inet = =C2=A0proto tcp to port $int_tcp_services
pass in on $int_if inet =C2=A0= proto udp to port $int_udp_services


# Filter rules for $int_if o= utbound

pass out on $int_if inet
pass out on $int_if inet6
#-------------------------------------------------------------------------= ------

# DEBUG: RULES FOR VMM

pass in quick on bridge0 all ke= ep state
pass in quick on tap0 all keep state
pass in quick on tap1 a= ll keep state
pass in quick on tap2 all keep state
pass in quick on t= ap3 all keep state
pass out quick on bridge0 all keep state
pass out = quick on tap0 all keep state
pass out quick on tap1 all keep state
pa= ss out quick on tap2 all keep state
pass out quick on tap3 all keep stat= e

# Bhyve hosts
pass in on tap0
pass in on tap1
pass in on = tap2
pass in on tap3
```

--
Best regards,
Odhia= mbo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
=C2=A0In=C2=A0an Internet failure case, the #1 suspect is a constant:= DNS.
"Oh, the cruft.",=C2=A0egrep -v '^$|^.*#'=C2=A0=C2= =AF\_(=E3=83=84)_/=C2=AF=C2=A0:-)
[How to ask smart question= s:=C2=A0http://www.catb.org/~= esr/faqs/smart-questions.html]
--00000000000011efcf061d87c97a--