bhyve hangs on going multi user

Thu Feb 7 23:19:19 UTC 2013

On 07/02/2013 22:16, Neel Natu wrote:
> Hi Sevan,
>
> On Thu, Feb 7, 2013 at 7:12 AM, Sevan / Venture37
> <venture37 at gmail.com> wrote:
>> On 7 Feb 2013, at 12:40 PM, Sevan / Venture37 <venture37 at gmail.com>
>> wrote:
>>
>>> Will try building a new world with without a make.conf & use that
>>> as the basis for a new VM & see if that makes a difference.
>>
>> It did, will follow up later with instruction once I have a working
>> system, I'm now trying to get networking going.
>
> That's great. I am interested in knowing how make options resulted
> in illegal instruction traps in guest userspace.

Previous make.conf

STRIP=
CFLAGS+=-fno-omit-frame-pointer

CFLAGS= -O2 -fno-strict-aliasing -pipe
COPTFLAGS= -O -pipe
CXXFLAGS+= -fconserve-space
CPUTYPE?=corei7-avx

>> If I run tcpdump on guest, I see traffic showing up destined for
>> the host. I do not see the traffic make it back out however.
>>
> Could you share output of "ifconfig" and "netstat -rn" from the
> guest as well as the host?

Problem solved, need to wait for a bit after bringing tap/bridge up 
otherwise traffic wont pass. (see working dmesg at the bottom of the page)

I can now make a guest panic if I assign 4GB or more RAM.

GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #0 r246446: Thu Feb  7 12:21:37 UTC 2013
     :/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3399.72-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x306a9  Family = 0x6  Model = 0x3a 
Stepping = 9

Features=0x8fa3ab7f<FPU,VME,DE,PSE,TSC,MSR,PAE,CX8,APIC,SEP,PGE,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,PBE>

Features2=0xe3bae257<SSE3,PCLMULQDQ,DTES64,DS_CPL,SMX,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,F16C,RDRAND,HV>
   AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant
real memory  = 4294967296 (4096 MB)
avail memory = 4114264064 (3923 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <BHYVE  BVMADT  >
random device not loaded; using insecure entropy
ioapic0: Changing APIC ID to 1
ioapic0 <Version 0.0> irqs 0-0 on motherboard
module_register_init: MOD_LOAD (vesa, 0xffffffff80c28300, 0) error 19
kbd0 at kbdmux0
ctl: CAM Target Layer loaded
acpi0: <BHYVE BVXSDT> on motherboard
acpi0: could not allocate interrupt
ACPI Exception: AE_ALREADY_EXISTS, Unable to install System Control 
Interrupt handler (20130117/evevent-152)
acpi0: Could not enable ACPI: AE_ALREADY_EXISTS
device_attach: acpi0 attach returned 6

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x60
fault code              = supervisor write data, page not present
instruction pointer     = 0x20:0xffffffff80c9c813
stack pointer           = 0x28:0xffffffff8182ab70
frame pointer           = 0x28:0xffffffff8182aba0
code segment            = base 0x0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 0 (swapper)
[ thread pid 0 tid 100000 ]
Stopped at      acpi_install_wakeup_handler+0x113:      movq 
%r14,0x60(%r15)

Tracing pid 0 tid 100000 td 0xffffffff814f8ff0
acpi_install_wakeup_handler() at acpi_install_wakeup_handler+0x113/frame 
0xffffffff8182aba0
nexus_acpi_attach() at nexus_acpi_attach+0x8c/frame 0xffffffff8182abd0
device_attach() at device_attach+0x396/frame 0xffffffff8182ac20
bus_generic_new_pass() at bus_generic_new_pass+0x116/frame 
0xffffffff8182ac50
bus_set_pass() at bus_set_pass+0x8f/frame 0xffffffff8182ac80
configure() at configure+0xa/frame 0xffffffff8182ac90
mi_startup() at mi_startup+0x118/frame 0xffffffff8182acb0
btext() at btext+0x2c

   pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
     2     0     0     0  RL                                  [ctl_thrd]
    13     0     0     0  RL      (threaded)                  [geom]
100009                   RunQ                                [g_down]
100008                   RunQ                                [g_up]
100007                   RunQ                                [g_event]
    12     0     0     0  WL      (threaded)                  [intr]
100019                   I                                   [swi5: fast 
taskq]
100017                   I                                   [swi6: 
Giant taskq]
100016                   I                                   [swi6: task 
queue]
100012                   I                                   [swi2: cambio]
100006                   I                                   [swi3: vm]
100005                   I                                   [swi4: clock]
100004                   I                                   [swi1: 
netisr 0]
    11     0     0     0  RL                                  [idle: cpu0]
     1     0     0     0  ?L                                  [kernel]
    10     0     0     0  RL                                  [audit]
     0     0     0     0  RLs     (threaded)                  [kernel]
100020                   RunQ                                [kqueue taskq]
100018                   RunQ                                [thread taskq]
100015                   RunQ                                [acpi_task_2]
100014                   RunQ                                [acpi_task_1]
100013                   RunQ                                [acpi_task_0]
100011                   RunQ                                [ffs_trim 
taskq]
100010                   RunQ                                [firmware 
taskq]
100000                   Run     CPU 0                       [swapper]

GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #0 r246446: Thu Feb  7 12:21:37 UTC 2013
     :/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3399.75-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x306a9  Family = 0x6  Model = 0x3a 
Stepping = 9

Features=0x8fa3ab7f<FPU,VME,DE,PSE,TSC,MSR,PAE,CX8,APIC,SEP,PGE,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,PBE>

Features2=0xe3bae257<SSE3,PCLMULQDQ,DTES64,DS_CPL,SMX,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,F16C,RDRAND,HV>
   AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant
real memory  = 3221225472 (3072 MB)
avail memory = 3078328320 (2935 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <BHYVE  BVMADT  >
random device not loaded; using insecure entropy
ioapic0: Changing APIC ID to 1
ioapic0 <Version 1.1> irqs 0-16 on motherboard
module_register_init: MOD_LOAD (vesa, 0xffffffff80c28300, 0) error 19
kbd0 at kbdmux0
ctl: CAM Target Layer loaded
acpi0: <BHYVE BVXSDT> on motherboard
atrtc0: <AT realtime clock> port 0x70-0x71,0x72-0x77 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib0: no PRT entry for 0.31.INTA
virtio_pci0: <VirtIO PCI Network adapter> port 0x2000-0x201f mem 
0xc0000000-0xc0001fff at device 1.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci0
virtio_pci0: host features: 0x18020 <Status,MrgRxBuf,MacAddress>
virtio_pci0: negotiated features: 0x18020 <Status,MrgRxBuf,MacAddress>
vtnet0: Ethernet address: 00:a0:98:a0:46:9c
virtio_pci1: host features: 0x10000004 <RingIndirect,MaxNumSegs>
virtio_pci1: negotiated features: 0x10000004 <RingIndirect,MaxNumSegs>
vtblk0: 4096MB (8388608 512 byte sectors)
uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on pci0
uart2: console (9600,n,8,1)
Timecounters tick every 10.000 msec
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
Timecounter "TSC-low" frequency 1699875246 Hz quality 1000
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/vtbd0s1a [rw]...
Setting hostuuid: 04124b58-712c-11e2-b131-cdebce931f3f.
Setting hostid: 0x44378452.
No suitable dump device was found.
Entropy harvesting:.
Starting file system checks:
/dev/vtbd0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/vtbd0s1a: clean, 647080 free (760 frags, 80790 blocks, 0.1% 
fragmentation)
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
g_handleattr: vtbd0 bio_length 24 len 28 -> EFAULT
Mounting local file systems:.