Hyper-V networking: problem after upgrade to 10.2

Sun Jan 31 13:36:49 UTC 2016

Hey Jac.
The setting was enabled in the VM configuration in Hyper-V, for the network
card. No such configuration in the guest OS itself.
As you've not enabled these features, it doesn't seem those are related to
the problem.

Regards,
Marie Helene

On Sun, Jan 31, 2016 at 1:45 PM Jac Backus <j.backus at bugworks.com> wrote:

> Hello Marie Helene,
>
>
>
> Thanks for your reply!
>
>
>
> All hardware acceleration is disabled in the Hyper-V settings, so is DHCP
> guard. All addresses of the VM are fixed.
>
> How do I check this enabled or disabled in the FreeBSD VM? I did not
> configure it.
>
>
>
> With kind regards,
>
>
>
> Jac
>
>
>
> *Van:* Marie Helene Kvello-Aune [mailto:marieheleneka at gmail.com]
> *Verzonden:* zondag 31 januari 2016 13:25
> *Aan:* Dexuan Cui; Jac Backus; Sephe Qiao (Wicresoft); Kylie Liang;
> freebsd-virtualization at freebsd.org; BSD Integration Components for Hyper-V
> *Onderwerp:* Re: Hyper-V networking: problem after upgrade to 10.2
>
>
>
> I had this issue with a Windows Server 2012 R2 Hyper-V host and FreeBSD
> 10.1 sometime last year, but chalked it up to a random fault during
> upgrade. It wasn't a production VM so I didn't put much effort into
> debugging it at the time.
>
>
>
> I remember I was using SR-IOV on the virtual network switch, and had it
> enabled on the VM as well. I had also enabled DHCP guard. Maybe some of
> these settings are relevant to reproducing the bug?
>
>
>
> Regards,
>
> Marie Helene Kvello-Aune
>
>
>
> On Sun, Jan 31, 2016 at 1:15 PM Dexuan Cui <decui at microsoft.com> wrote:
>
> Hi Jac,
> Good to know the information.
>
> Since I can't repro the issue, it's difficult for me to debug it. :(
> I'm guessing if it would help if you use a permanent ARP entry in the VM
> ("arp -s hostname ether_addr") for the other end - surely this is only for
> debug purpose.
>
> During the VM boot-up, can you keep pinging the VM from the other host. I
> mean: it looks the NIC never works since it becomes UP in the VM?
>
> BTW, I'm not sure if it's easy for you to do the same test as mine, i.e.,
> do a fresh installation of 10.1 VM and upgrade it to 10.2.
>
> @Sephe, any idea?
>
> Thanks,
> -- Dexuan
>
> From: Jac Backus [mailto:j.backus at bugworks.com]
> Sent: Sunday, January 31, 2016 3:42
> To: Dexuan Cui <decui at microsoft.com<mailto:decui at microsoft.com>>; Sephe
> Qiao (Wicresoft) <v-yanqia at microsoft.com<mailto:v-yanqia at microsoft.com>>;
> Kylie Liang <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>; '
> freebsd-virtualization at freebsd.org' <freebsd-virtualization at freebsd.org
> <mailto:freebsd-virtualization at freebsd.org>>; BSD Integration Components
> for Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
> Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Good day Dexuan,
>
> There is something wrong with getting mac addresses for host on the lan,
> it seems.
>
> When I ping the 10.2 server from a host on the net, I see on that host
> arp requests  (Wireshark: who has ... Tell ...) for the 10.2 server.
>
> Arp -a on the 10.2 server itself says for the non-server entries
>  ? <address> at (imcomplete) on hn0 expired [ethernet]
>
> Tcpdump on the 10.2 server only shows arp requests:
> ARP, Request who-has ... tell ...
>
> Does this help?
>
> With kind regards,
>
> Jac
> Van: Dexuan Cui [mailto:decui at microsoft.com]
> Verzonden: vrijdag 29 januari 2016 9:59
> Aan: Jac Backus; Sephe Qiao (Wicresoft); Kylie Liang; '
> freebsd-virtualization at freebsd.org'; BSD Integration Components for
> Hyper-V
> Onderwerp: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Hmm, it's strange we can't repro.
>
> I suppose you can't ping the netgate VM (or machine) 's IP address either?
> When this happens, can you check the arp table in both sides?
> Can you please run tcpdump in the VM and in the gateway to diagnose the
> issue?
>
> Thanks,
> -- Dexuan
>
> From: Jac Backus [mailto:j.backus at bugworks.com]
> Sent: Friday, January 29, 2016 15:36
> To: Dexuan Cui <decui at microsoft.com<mailto:decui at microsoft.com>>; Sephe
> Qiao (Wicresoft) <v-yanqia at microsoft.com<mailto:v-yanqia at microsoft.com>>;
> Kylie Liang <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>; '
> freebsd-virtualization at freebsd.org' <freebsd-virtualization at freebsd.org
> <mailto:freebsd-virtualization at freebsd.org>>; BSD Integration Components
> for Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
> Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Hello Dexuan,
>
> That remarkable.
> My uname -a is indentical.
>
> I can ping local interfaces. Ping to other addresses in local subnet
> gives: Ping: sendto: Host is down. Ping to other addresses gives: Ping:
> sendto: No route to host.
> Routing tables (netstat -rn) for both versions look the same.
>
> Is there something I can test?
>
> With kind regards,
>
> Jac
>
> Van: Dexuan Cui [mailto:decui at microsoft.com]
> Verzonden: vrijdag 29 januari 2016 4:25
> Aan: Jac Backus; Sephe Qiao (Wicresoft); Kylie Liang; '
> freebsd-virtualization at freebsd.org'; BSD Integration Components for
> Hyper-V
> Onderwerp: RE: Hyper-V networking: problem after upgrade to 10.2
>
>
> Hi Jac,
>
> I installed a 10.1 VM with FreeBSD-10.1-RELEASE-amd64-dvd1.iso and
> upgraded it to 10.2 by running "freebsd-update upgrade -r 10.2-RELEASE".
> Everything worked just fine. With the new kernel (see the below), ssh and
> scp still works fine for me.
>
> # uname -a
> FreeBSD bsd101 10.2-RELEASE-p9 FreeBSD 10.2-RELEASE-p9 #0: Thu Jan 14
> 01:32:46 UTC 2016     root at amd64-builder.daemonology.net:
> /usr/obj/usr/src/sys/GENERIC<mailto:root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC>
> amd64
>
> What's the specific symptom for "networking does not work anymore" in your
> side(upgrading from 10.1 to 10.2)?
>
> Thanks,
> -- Dexuan
>
> From: Jac Backus [mailto:j.backus at bugworks.com]
> Sent: Wednesday, January 27, 2016 17:35
> To: Dexuan Cui <decui at microsoft.com<mailto:decui at microsoft.com>>; Sephe
> Qiao (Wicresoft) <v-yanqia at microsoft.com<mailto:v-yanqia at microsoft.com>>;
> Kylie Liang <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>; '
> freebsd-virtualization at freebsd.org' <freebsd-virtualization at freebsd.org
> <mailto:freebsd-virtualization at freebsd.org>>; BSD Integration Components
> for Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
> Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Hello Dexuan,
>
> Unfortunetely, no OACTIVE flag:
>
> hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>         options=31b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,TSO6>
>
> With kind regards,
>
> Jac
>
> Van: Dexuan Cui [mailto:decui at microsoft.com]
> Verzonden: woensdag 27 januari 2016 4:09
> Aan: Sephe Qiao (Wicresoft); Jac Backus; Kylie Liang; '
> freebsd-virtualization at freebsd.org'; BSD Integration Components for
> Hyper-V
> Onderwerp: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Hi Jac,
> Please show 'ifconfig -a' when the issue happens (when you upgrade 10.1
> from 10.2).
> We suspect it may be a known OACTIVE issue and "ifconfig -a' can confirm
> this, the output has the string "OACTIVE".
>
> It looks somehow the issue doesn't happen when we use a 10.2 fresh
> installation.
>
> Thanks,
> -- Dexuan
>
> From: Sephe Qiao (Wicresoft)
> Sent: Wednesday, January 27, 2016 9:13
> To: Jac Backus <j.backus at bugworks.com<mailto:j.backus at bugworks.com>>;
> Dexuan Cui <decui at microsoft.com<mailto:decui at microsoft.com>>; Kylie Liang
> <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>; '
> freebsd-virtualization at freebsd.org' <freebsd-virtualization at freebsd.org
> <mailto:freebsd-virtualization at freebsd.org>>; BSD Integration Components
> for Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
> Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Oh, please ignore this, I think its solved :)
>
> From: Sephe Qiao (Wicresoft)
> Sent: Wednesday, January 27, 2016 9:10 AM
> To: Jac Backus <j.backus at bugworks.com<mailto:j.backus at bugworks.com>>;
> Dexuan Cui <decui at microsoft.com<mailto:decui at microsoft.com>>; Kylie Liang
> <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>; '
> freebsd-virtualization at freebsd.org' <freebsd-virtualization at freebsd.org
> <mailto:freebsd-virtualization at freebsd.org>>; BSD Integration Components
> for Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
> Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> Hi Jac,
>
> What's the output of 'ifconfig -a' when this happened?
>
> Thanks,
> sephe
>
> From: Jac Backus [mailto:j.backus at bugworks.com]
> Sent: Tuesday, January 26, 2016 6:37 PM
> To: Dexuan Cui <decui at microsoft.com<mailto:decui at microsoft.com>>; Kylie
> Liang <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>; '
> freebsd-virtualization at freebsd.org' <freebsd-virtualization at freebsd.org
> <mailto:freebsd-virtualization at freebsd.org>>; BSD Integration Components
> for Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
> Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
>
> Hello Dexuan,
>
>
>
> It seems, it is not completely correct, although the effect is as if it is
> not working.
>
> Systat -ifstat 1 shows this:
>
> [cid:image001.png at 01D159DA.65A3A0E0]
>
>
>
> So something is happening. But I can not reach anything. And the server
> can not be reached from the lan (hn0) or internet (hn1 and hn2).
>
>
>
> I get a firewall message in /var/log/messages (first message from
> 11:18:55):
>
> [cid:image002.png at 01D159DA.65A3A0E0]
>
> But this is just caused by the problem?
>
>
>
> If I can help with further information, please let me know.
>
>
>
> Regarding Bug 187006, all interfaces have fixed addresses.
>
>
>
> With kind regards,
>
>
>
> Jac
>
>
>
>
>
>
>
> -----Oorspronkelijk bericht-----
> Van: Dexuan Cui [mailto:decui at microsoft.com]
> Verzonden: dinsdag 26 januari 2016 7:55
> Aan: Kylie Liang; Jac Backus; 'freebsd-virtualization at freebsd.org'; BSD
> Integration Components for Hyper-V
> Onderwerp: RE: Hyper-V networking: problem after upgrade to 10.2
>
>
>
> Hi Jac,
>
> BTW,  what do you mean by saying "networking does not work anymore" -- can
> you please check if your issue is the same as
>
>
>
> Bug 187006 - [hyper-v] dynamic address (dhcp) obtaining doesn't work on
> HYPER-V OS 2012 R2
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187006<
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs.freebsd.org%2fbugzilla%2fshow_bug.cgi%3fid%3d187006&data=01%7c01%7cv-yanqia%40064d.mgd.microsoft.com%7cef82474449e745da88c908d3263e7e48%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=1IyQb3x7ecm%2f6uESQmVAAyAyOQr4ZMJ3Fkawp93dZgQ%3d>
> ?
>
>
>
> Thanks,
>
> -- Dexuan
>
>
>
>
>
> > -----Original Message-----
>
> > From: Dexuan Cui
>
> > Sent: Tuesday, January 26, 2016 14:49
>
> > To: Kylie Liang <kyliel at microsoft.com<mailto:kyliel at microsoft.com>>;
> Jac Backus
>
> > <j.backus at bugworks.com<mailto:j.backus at bugworks.com>>; '
> freebsd-virtualization at freebsd.org'
>
> > <freebsd-virtualization at freebsd.org<mailto:
> freebsd-virtualization at freebsd.org>>; BSD Integration Components for
>
> > Hyper-V <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
>
> > Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> >
>
> > Hi Jac,
>
> > Kylie meant disabling TSO. Please try this ("ifconfig hn0 -tso").
>
> >
>
> > The message " hn0: unknown status 1073872902 received" should be an
>
> > unnecessary warning only.
>
> > My 10.2 VM can work fine even if I see the message too.
>
> >
>
> > Can you please install a 10.2 VM from the 10.2 .ISO file directly as I
>
> > did and see if it works for you?
>
> >
>
> > I guess we never tried upgrading 10.1 from 10.2.
>
> > Can you please list the steps how you did the upgrading? We'll try the
>
> > same steps.
>
> >
>
> > Thanks,
>
> > -- Dexuan
>
> >
>
> > > -----Original Message-----
>
> > > From: Kylie Liang
>
> > > Sent: Tuesday, January 26, 2016 8:01
>
> > > To: Jac Backus <j.backus at bugworks.com<mailto:j.backus at bugworks.com>>;
> 'freebsd-
>
> > virtualization at freebsd.org<mailto:virtualization at freebsd.org>'
>
> > > <freebsd-virtualization at freebsd.org<mailto:
> freebsd-virtualization at freebsd.org>>; BSD Integration Components for
>
> > > Hyper-
>
> > V
>
> > > <bsdic at microsoft.com<mailto:bsdic at microsoft.com>>
>
> > > Subject: RE: Hyper-V networking: problem after upgrade to 10.2
>
> > >
>
> > > Hi Jac,
>
> > >
>
> > > Thank you for asking. To isolate your issue, could you please try
>
> > > disabling SO
>
> > on
>
> > > your 10.2 system first? Thank you.
>
> > >
>
> > > And I would like to confirm with you
>
> > > 1) You met issue for 10.2 kernel + 10.2 system
>
> > > 2) No issue for 10.1 kernel + 10.1 system
>
> > > 3) No issue for 10.1 kernel + 10.2 system
>
> > >
>
> > > Right? And add our engineers in the list.
>
> > >
>
> > > Thanks,
>
> > > Kylie Liang
>
> > >
>
> > > -----Original Message-----
>
> > > From: owner-freebsd-virtualization at freebsd.org<mailto:
> owner-freebsd-virtualization at freebsd.org>
>
> > > [mailto:owner-freebsd- virtualization at freebsd.org<mailto:
> virtualization at freebsd.org>] On Behalf Of Jac
>
> > > Backus
>
> > > Sent: Tuesday, January 26, 2016 5:56 AM
>
> > > To: 'freebsd-virtualization at freebsd.org'
>
> > > <freebsd-virtualization at freebsd.org<mailto:
> freebsd-virtualization at freebsd.org>>
>
> > > Subject: Hyper-V networking: problem after upgrade to 10.2
>
> > >
>
> > > Dear reader,
>
> > >
>
> > > Today, I did  upgrade  FreeBSD 10.1 to 10.2 running on Hyper-V on a
>
> > > full
>
> > patched
>
> > > Windows Server 2012 R2 x64 version.
>
> > >
>
> > > After the update, networking does not work anymore.
>
> > >
>
> > > In /var/log/messages is this:
>
> > >
>
> > > Jan 25 21:02:01 mercurius kernel: hn0: <Synthetic Network Interface>
>
> > > on
>
> > > vmbus0 Jan 25 21:02:01 mercurius kernel: hn0: unknown status
>
> > > 1073872902 received Jan 25 21:02:01 mercurius kernel: hn0: unknown
>
> > > status 1073872902 received Jan 25 21:02:01 mercurius kernel: hn0: hv
>
> > > send offload request succeeded Jan 25 21:02:01 mercurius kernel: hn0:
> Using defaults for TSO:
>
> > > 65518/35/2048 Jan 25 21:02:01 mercurius kernel: hn0: Ethernet address:
>
> > > 00:15:5d:ac:11:08 Jan 25 21:02:01 mercurius kernel: hn1: <Synthetic
>
> > > Network
>
> > > Interface> on vmbus0 Jan 25 21:02:01 mercurius kernel: hn1: unknown
>
> > > Interface> status
>
> > > 1073872902 received Jan 25 21:02:01 mercurius kernel: hn1: unknown
>
> > > status
>
> > > 1073872902 received Jan 25 21:02:01 mercurius kernel: hn1: hv send
>
> > > offload request succeeded Jan 25 21:02:01 mercurius kernel: hn1:
>
> > > Using defaults for
>
> > TSO:
>
> > > 65518/35/2048 Jan 25 21:02:01 mercurius kernel: hn1: Ethernet address:
>
> > > 00:15:5d:ac:11:09 Jan 25 21:02:01 mercurius kernel: hn2: <Synthetic
>
> > > Network
>
> > > Interface> on vmbus0 Jan 25 21:02:01 mercurius kernel: hn2: unknown
>
> > > Interface> status
>
> > > 1073872902 received Jan 25 21:02:01 mercurius kernel: hn2: unknown
>
> > > status
>
> > > 1073872902 received Jan 25 21:02:01 mercurius kernel: hn2: hv send
>
> > > offload request succeeded Jan 25 21:02:01 mercurius kernel: hn2:
>
> > > Using defaults for
>
> > TSO:
>
> > > 65518/35/2048 Jan 25 21:02:01 mercurius kernel: hn2: Ethernet address:
>
> > > 00:15:5d:ac:11:07
>
> > >
>
> > > It worked fine with the 10.1 kernel, and when I boot this kernel, it
> works again:
>
> > >
>
> > > Jan 25 22:20:02 mercurius kernel: hn0: <Synthetic Network Interface>
>
> > > on
>
> > > vmbus0 Jan 25 22:20:02 mercurius kernel: hn0: Ethernet address:
>
> > > 00:15:5d:ac:11:07 Jan 25 22:20:02 mercurius kernel: hn1: <Synthetic
>
> > > Network
>
> > > Interface> on vmbus0 Jan 25 22:20:02 mercurius kernel: hn1: Ethernet
> address:
>
> > > 00:15:5d:ac:11:08 Jan 25 22:20:02 mercurius kernel: hn2: <Synthetic
>
> > > Network
>
> > > Interface> on vmbus0 Jan 25 22:20:02 mercurius kernel: hn2: Ethernet
> address:
>
> > > 00:15:5d:ac:11:09
>
> > >
>
> > > So I am running  a 10.2 system on a 10.1 kernel at the moment.
>
> > >
>
> > > I found nothing in /usr/src/UPDATING and not really anything on the
> net.
>
> > >
>
> > > So, could you tell why does this happen, and how can I solve this?
>
> > >
>
> > > Thanks for the help!
>
> > >
>
> > > With kind regards,
>
> > >
>
> > > Jac Backus
>
> > >
>
> > >
>
> > >
>
> > > _______________________________________________
>
> > > freebsd-virtualization at freebsd.org<mailto:
> freebsd-virtualization at freebsd.org> mailing list
>
> > >
>
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flists
>
> > .freebs
>
> > > d.org%2fmailman%2flistinfo%2ffreebsd-
>
> > >
>
> > virtualization&data=01%7c01%7ckyliel%40064d.mgd.microsoft.com%7cc9ca2e
>
> > >
>
> > 0d0fef482b553f08d325d3aefb%7c72f988bf86f141af91ab2d7cd011db47%7c1&s
>
> > > data=o%2bMZGuBW0frrQhjAPkhrWlLgNEH8LJ7BiLUyiO4tvR0%3d
>
> > > To unsubscribe, send any mail to "freebsd-virtualization-
>
> > > unsubscribe at freebsd.org<mailto:unsubscribe at freebsd.org>"
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe at freebsd.org"
>
>