[Bug 225791] ena driver causing kernel panics on AWS EC2
bugzilla-noreply at freebsd.org
bugzilla-noreply at freebsd.org
Fri Feb 9 15:38:48 UTC 2018
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225791
Bug ID: 225791
Summary: ena driver causing kernel panics on AWS EC2
Product: Base System
Version: 11.1-RELEASE
Hardware: amd64
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: freebsd-bugs at FreeBSD.org
Reporter: mwalker at carbonhouse.com
Overview:
On some of our more heavily-trafficked c5.large and c5.xlarge servers we're
seeing kernel panics & reboots as often as 2-3 times per day.
Steps to Reproduce:
Have a server with a good amount of network traffic (~3000pps) using the ena
network adapter. Wait a day or less.
Additional Information:
Here's one of the panics:
Feb 5 20:58:51 awesomehost kernel: ena0: Found a Tx that wasn't completed on
time, qid 0, index 0.
Feb 6 03:48:11 awesomehost kernel: ena0: Found a Tx that wasn't completed on
time, qid 1, index 203.
Feb 6 04:25:46 awesomehost syslogd: kernel boot file is /boot/kernel/kernel
Feb 6 04:25:46 awesomehost kernel: [zone: mbuf_jumbo_16k] kern.ipc.nmbjumbo16
limit reached
Feb 6 04:25:46 awesomehost kernel:
Feb 6 04:25:46 awesomehost kernel:
Feb 6 04:25:46 awesomehost kernel: Fatal trap 12: page fault while in kernel
mode
Feb 6 04:25:46 awesomehost kernel: cpuid = 0; apic id = 00
Feb 6 04:25:46 awesomehost kernel: fault virtual address = 0x1c
Feb 6 04:25:46 awesomehost kernel: fault code = supervisor write
data, page not present
Feb 6 04:25:46 awesomehost kernel: instruction pointer =
0x20:0xffffffff8218ef8c
Feb 6 04:25:46 awesomehost kernel: stack pointer =
0x0:0xfffffe0108afe180
Feb 6 04:25:46 awesomehost kernel: frame pointer =
0x0:0xfffffe0108afe260
Feb 6 04:25:46 awesomehost kernel: code segment = base 0x0,
limit 0xfffff, type 0x1b
Feb 6 04:25:46 awesomehost kernel:
Feb 6 04:25:46 awesomehost kernel: = DPL 0, pres 1, long 1, def32 0, gran 1
Feb 6 04:25:46 awesomehost kernel: processor eflags =
Feb 6 04:25:46 awesomehost kernel: Fatal trap 12: page fault while in kernel
mode
Feb 6 04:25:46 awesomehost kernel: cpuid = 1; apic id = 01
Feb 6 04:25:46 awesomehost kernel: fault virtual address = 0x1c
Feb 6 04:25:46 awesomehost kernel: fault code = supervisor write
data, page not present
Feb 6 04:25:46 awesomehost kernel: instruction pointer =
0x20:0xffffffff8218ef8c
Feb 6 04:25:46 awesomehost kernel: interrupt enabled, resume, IOPL = 0
Feb 6 04:25:46 awesomehost kernel: current process = 12 (irq261:
ena0)
Feb 6 04:25:46 awesomehost kernel: trap number = 12
Feb 6 04:25:46 awesomehost kernel: panic: page fault
Feb 6 04:25:46 awesomehost kernel: cpuid = 0
Feb 6 04:25:46 awesomehost kernel: KDB: stack backtrace:
Feb 6 04:25:46 awesomehost kernel: #0 0xffffffff80aadac7 at kdb_backtrace+0x67
Feb 6 04:25:46 awesomehost kernel: #1 0xffffffff80a6bba6 at vpanic+0x186
Feb 6 04:25:46 awesomehost kernel: #2 0xffffffff80a6ba13 at panic+0x43
Feb 6 04:25:46 awesomehost kernel: #3 0xffffffff80edf832 at trap_fatal+0x322
Feb 6 04:25:46 awesomehost kernel: #4 0xffffffff80edf889 at trap_pfault+0x49
Feb 6 04:25:46 awesomehost kernel: #5 0xffffffff80edf0c6 at trap+0x286
Feb 6 04:25:46 awesomehost kernel: #6 0xffffffff80ec36d1 at calltrap+0x8
Feb 6 04:25:46 awesomehost kernel: #7 0xffffffff80a321ec at
intr_event_execute_handlers+0xec
Feb 6 04:25:46 awesomehost kernel: #8 0xffffffff80a324d6 at ithread_loop+0xd6
Feb 6 04:25:46 awesomehost kernel: #9 0xffffffff80a2f845 at fork_exit+0x85
Feb 6 04:25:46 awesomehost kernel: #10 0xffffffff80ec3c0e at
fork_trampoline+0xe
Feb 6 04:25:46 awesomehost kernel: Uptime: 8h48m21s
Feb 6 04:25:46 awesomehost kernel: Rebooting...
Here's a full boot log:
Feb 8 22:57:59 awesomehost kernel: Copyright (c) 1992-2017 The FreeBSD
Project.
[37/2338]
Feb 8 22:57:59 awesomehost kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988,
1989, 1991, 1992, 1993, 1994
Feb 8 22:57:59 awesomehost kernel: The Regents of the University of
California. All rights reserved.
Feb 8 22:57:59 awesomehost kernel: FreeBSD is a registered trademark of The
FreeBSD Foundation.
Feb 8 22:57:59 awesomehost kernel: FreeBSD 11.1-RELEASE-p4 #0: Tue Nov 14
06:12:40 UTC 2017
Feb 8 22:57:59 awesomehost kernel:
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
Feb 8 22:57:59 awesomehost kernel: FreeBSD clang version 4.0.0
(tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
Feb 8 22:57:59 awesomehost kernel: VT(vga): text 80x25
Feb 8 22:57:59 awesomehost kernel: CPU: HammerEM64T (3000.07-MHz K8-class CPU)
Feb 8 22:57:59 awesomehost kernel: Origin="GenuineIntel" Id=0x50653
Family=0x6 Model=0x55 Stepping=3
Feb 8 22:57:59 awesomehost kernel:
Features=0x1f83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS,HTT>
Feb 8 22:57:59 awesomehost kernel:
Features2=0xfffa3203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
Feb 8 22:57:59 awesomehost kernel: AMD
Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
Feb 8 22:57:59 awesomehost kernel: AMD Features2=0x121<LAHF,ABM,Prefetch>
Feb 8 22:57:59 awesomehost kernel: Structured Extended
Features=0xd11f4f3a<TSCADJ,BMI1,HLE,AVX2,BMI2,ERMS,INVPCID,RTM,MPX,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLWB,AVX512CD,AVX512BW>
Feb 8 22:57:59 awesomehost kernel: Structured Extended Features2=0x8<PKU>
Feb 8 22:57:59 awesomehost kernel: XSAVE
Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
Feb 8 22:57:59 awesomehost kernel: TSC: P-state invariant, performance
statistics
Feb 8 22:57:59 awesomehost kernel: Hypervisor: Origin = "KVMKVMKVM"
Feb 8 22:57:59 awesomehost kernel: real memory = 5114953728 (4878 MB)
Feb 8 22:57:59 awesomehost kernel: avail memory = 3845005312 (3666 MB)
Feb 8 22:57:59 awesomehost kernel: Event timer "LAPIC" quality 600
Feb 8 22:57:59 awesomehost kernel: ACPI APIC Table: <AMAZON AMZNAPIC>
Feb 8 22:57:59 awesomehost kernel: FreeBSD/SMP: Multiprocessor System
Detected: 2 CPUs
Feb 8 22:57:59 awesomehost kernel: FreeBSD/SMP: 1 package(s) x 1 core(s) x 2
hardware threads
Feb 8 22:57:59 awesomehost kernel: random: unblocking device.
Feb 8 22:57:59 awesomehost kernel: ioapic0 <Version 1.1> irqs 0-23 on
motherboard
Feb 8 22:57:59 awesomehost kernel: SMP: AP CPU #1 Launched!
Feb 8 22:57:59 awesomehost kernel: random: entropy device external interface
Feb 8 22:57:59 awesomehost kernel: kbd1 at kbdmux0
Feb 8 22:57:59 awesomehost kernel: netmap: loaded module
Feb 8 22:57:59 awesomehost kernel: module_register_init: MOD_LOAD (vesa,
0xffffffff80f5b220, 0) error 19
Feb 8 22:57:59 awesomehost kernel: random: registering fast source Intel
Secure Key RNG
Feb 8 22:57:59 awesomehost kernel: random: fast provider: "Intel Secure Key
RNG"
Feb 8 22:57:59 awesomehost kernel: nexus0
Feb 8 22:57:59 awesomehost kernel: vtvga0: <VT VGA driver> on motherboard
Feb 8 22:57:59 awesomehost kernel: cryptosoft0: <software crypto> on
motherboard
Feb 8 22:57:59 awesomehost kernel: acpi0: <AMAZON AMZNRSDT> on motherboard
Feb 8 22:57:59 awesomehost kernel: acpi0: Power Button (fixed)
Feb 8 22:57:59 awesomehost kernel: cpu0: <ACPI CPU> on acpi0
Feb 8 22:57:59 awesomehost kernel: cpu1: <ACPI CPU> on acpi0
Feb 8 22:57:59 awesomehost kernel: atrtc0: <AT realtime clock> port
0x70-0x71,0x72-0x77 irq 8 on acpi0
Feb 8 22:57:59 awesomehost kernel: Event timer "RTC" frequency 32768 Hz
quality 0
Feb 8 22:57:59 awesomehost kernel: Timecounter "ACPI-fast" frequency 3579545
Hz quality 900
Feb 8 22:57:59 awesomehost kernel: acpi_timer0: <24-bit timer at 3.579545MHz>
port 0xb008-0xb00b on acpi0
Feb 8 22:57:59 awesomehost kernel: pcib0: <ACPI Host-PCI bridge> port
0xcf8-0xcff on acpi0
Feb 8 22:57:59 awesomehost kernel: pci0: <ACPI PCI bus> on pcib0
Feb 8 22:57:59 awesomehost kernel: isab0: <PCI-ISA bridge> at device 1.0 on
pci0
Feb 8 22:57:59 awesomehost kernel: isa0: <ISA bus> on isab0
Feb 8 22:57:59 awesomehost kernel: pci0: <old, non-VGA display device> at
device 1.3 (no driver attached)
Feb 8 22:57:59 awesomehost kernel: vgapci0: <VGA-compatible display> mem
0xfe400000-0xfe7fffff at device 3.0 on pci0
Feb 8 22:57:59 awesomehost kernel: vgapci0: Boot video device
Feb 8 22:57:59 awesomehost kernel: nvme0: <Generic NVMe Device> mem
0xfebf0000-0xfebf3fff irq 11 at device 4.0 on pci0
Feb 8 22:57:59 awesomehost kernel: ena0: <ENA adapter> mem
0xfebf4000-0xfebf7fff at device 5.0 on pci0
Feb 8 22:57:59 awesomehost kernel: ena0: Elastic Network Adapter (ENA)ena
v0.7.0
Feb 8 22:57:59 awesomehost kernel: ena0: initalize 2 io queues
Feb 8 22:57:59 awesomehost kernel: ena0: Ethernet address: 12:3d:00:e2:4f:78
Feb 8 22:57:59 awesomehost kernel: ena0: Allocated msix_entries, vectors (cnt:
3)
Feb 8 22:57:59 awesomehost kernel: nvme1: <Generic NVMe Device> mem
0xfebf8000-0xfebfbfff irq 11 at device 31.0 on pci0
Feb 8 22:57:59 awesomehost kernel: ena0: link is UP
Feb 8 22:57:59 awesomehost kernel: ena0: link state changed to UP
Feb 8 22:57:59 awesomehost kernel: atkbdc0: <Keyboard controller (i8042)> port
0x60,0x64 irq 1 on acpi0
Feb 8 22:57:59 awesomehost kernel: atkbd0: <AT Keyboard> irq 1 on atkbdc0
Feb 8 22:57:59 awesomehost kernel: kbd0 at atkbd0
Feb 8 22:57:59 awesomehost kernel: atkbd0: [GIANT-LOCKED]
Feb 8 22:57:59 awesomehost kernel: uart0: <Non-standard ns8250 class UART with
FIFOs> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
Feb 8 22:57:59 awesomehost kernel: uart0: console (115200,n,8,1)
Feb 8 22:57:59 awesomehost kernel: orm0: <ISA Option ROM> at iomem
0xef000-0xeffff on isa0
Feb 8 22:57:59 awesomehost kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df
iomem 0xa0000-0xbffff on isa0
Feb 8 22:57:59 awesomehost kernel: attimer0: <AT timer> at port 0x40 on isa0
Feb 8 22:57:59 awesomehost kernel: Timecounter "i8254" frequency 1193182 Hz
quality 0
Feb 8 22:57:59 awesomehost kernel: attimer0: Can't map interrupt.
Feb 8 22:57:59 awesomehost kernel: ppc0: cannot reserve I/O port range
Feb 8 22:57:59 awesomehost kernel: Timecounters tick every 1.000 msec
Feb 8 22:57:59 awesomehost kernel: usb_needs_explore_all: no devclass
Feb 8 22:57:59 awesomehost kernel: nvme cam probe device init
Feb 8 22:57:59 awesomehost kernel: nvme0: temperature threshold not supported
Feb 8 22:57:59 awesomehost kernel: nvd0: <Amazon Elastic Block Store> NVMe
namespace
Feb 8 22:57:59 awesomehost kernel: nvd0: 10240MB (20971520 512 byte sectors)
Feb 8 22:57:59 awesomehost kernel: nvme1: temperature threshold not supported
Feb 8 22:57:59 awesomehost kernel: nvd1: <Amazon Elastic Block Store> NVMe
namespace
Feb 8 22:57:59 awesomehost kernel: nvd1: 40960MB (83886080 512 byte sectors)
Feb 8 22:57:59 awesomehost kernel: Trying to mount root from
ufs:/dev/gpt/rootfs [rw]...
Feb 8 22:57:59 awesomehost kernel: WARNING: / was not properly dismounted
Feb 8 22:57:59 awesomehost kernel: WARNING: /: mount pending error: blocks 0
files 2
Feb 8 22:57:59 awesomehost kernel: WARNING: /: reload pending error: blocks 0
files 2
Feb 8 22:57:59 awesomehost kernel: WARNING: / was not properly dismounted
Feb 8 22:57:59 awesomehost kernel: WARNING: /sites was not properly dismounted
Feb 8 22:57:59 awesomehost kernel: ena0: device is going UP
Feb 8 22:57:59 awesomehost kernel: ena0: queue 0 - cpu 0
Feb 8 22:57:59 awesomehost kernel: ena0: queue 1 - cpu 1
Feb 8 22:57:59 awesomehost kernel: pflog0: promiscuous mode enabled
Below are some numbers from one of our hosts experiencing this the most, it's a
c5.large server in us-east-1, booted from the official FreeBSD AMI "FreeBSD
11.1-RELEASE-amd64 (ami-70504266)"
Here's "uname -ar":
FreeBSD awesomehost 11.1-RELEASE-p4 FreeBSD 11.1-RELEASE-p4 #0: Tue Nov 14
06:12:40 UTC 2017
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
Here's "uptime":
3:29PM up 16:31, 2 users, load averages: 0.08, 0.22, 0.23
Here's "sysctl dev.ena":
dev.ena.0.admin_stats.no_completion: 0
dev.ena.0.admin_stats.out_of_space: 0
dev.ena.0.admin_stats.completed_cmd: 58446
dev.ena.0.admin_stats.sumbitted_cmd: 58446
dev.ena.0.admin_stats.aborted_cmd: 0
dev.ena.0.hw_stats.update_stats:
dev.ena.0.hw_stats.rx_drops: 0
dev.ena.0.hw_stats.tx_bytes: 130551171301
dev.ena.0.hw_stats.rx_bytes: 129414445543
dev.ena.0.hw_stats.tx_packets: 175108516
dev.ena.0.hw_stats.rx_packets: 174924189
dev.ena.0.queue1.rx_ring.small_copy_len_pkt: 0
dev.ena.0.queue1.rx_ring.bad_desc_num: 0
dev.ena.0.queue1.rx_ring.dma_mapping_err: 0
dev.ena.0.queue1.rx_ring.mbuf_alloc_fail: 0
dev.ena.0.queue1.rx_ring.page_alloc_fail: 0
dev.ena.0.queue1.rx_ring.bad_csum: 0
dev.ena.0.queue1.rx_ring.refil_partial: 0
dev.ena.0.queue1.rx_ring.bytes: 65125566440
dev.ena.0.queue1.rx_ring.count: 87968075
dev.ena.0.queue1.tx_ring.mbuf_collapse_err: 0
dev.ena.0.queue1.tx_ring.mbuf_collapses: 0
dev.ena.0.queue1.tx_ring.stops: 0
dev.ena.0.queue1.tx_ring.bad_req_id: 0
dev.ena.0.queue1.tx_ring.missing_tx_comp: 0
dev.ena.0.queue1.tx_ring.doorbells: 87925945
dev.ena.0.queue1.tx_ring.tx_poll: 0
dev.ena.0.queue1.tx_ring.napi_comp: 0
dev.ena.0.queue1.tx_ring.unsupported_desc_num: 0
dev.ena.0.queue1.tx_ring.dma_mapping_err: 0
dev.ena.0.queue1.tx_ring.queue_wakeup: 0
dev.ena.0.queue1.tx_ring.prepare_ctx_err: 0
dev.ena.0.queue1.tx_ring.bytes: 65756583294
dev.ena.0.queue1.tx_ring.count: 87999157
dev.ena.0.queue0.rx_ring.small_copy_len_pkt: 0
dev.ena.0.queue0.rx_ring.bad_desc_num: 0
dev.ena.0.queue0.rx_ring.dma_mapping_err: 0
dev.ena.0.queue0.rx_ring.mbuf_alloc_fail: 0
dev.ena.0.queue0.rx_ring.page_alloc_fail: 0
dev.ena.0.queue0.rx_ring.bad_csum: 0
dev.ena.0.queue0.rx_ring.refil_partial: 0
dev.ena.0.queue0.rx_ring.bytes: 64288879103
dev.ena.0.queue0.rx_ring.count: 86956114
dev.ena.0.queue0.tx_ring.mbuf_collapse_err: 0
dev.ena.0.queue0.tx_ring.mbuf_collapses: 0
dev.ena.0.queue0.tx_ring.stops: 0
dev.ena.0.queue0.tx_ring.bad_req_id: 0
dev.ena.0.queue0.tx_ring.missing_tx_comp: 0
dev.ena.0.queue0.tx_ring.doorbells: 87033454
dev.ena.0.queue0.tx_ring.tx_poll: 0
dev.ena.0.queue0.tx_ring.napi_comp: 0
dev.ena.0.queue0.tx_ring.unsupported_desc_num: 0
dev.ena.0.queue0.tx_ring.dma_mapping_err: 0
dev.ena.0.queue0.tx_ring.queue_wakeup: 0
dev.ena.0.queue0.tx_ring.prepare_ctx_err: 0
dev.ena.0.queue0.tx_ring.bytes: 64794589485
dev.ena.0.queue0.tx_ring.count: 87109364
dev.ena.0.admin_q_pause: 0
dev.ena.0.interface_down: 0
dev.ena.0.interface_up: 1
dev.ena.0.wd_expired: 0
dev.ena.0.io_resume: 0
dev.ena.0.io_suspend: 0
dev.ena.0.tx_timeout: 0
dev.ena.0.missing_tx_threshold: 128
dev.ena.0.missing_tx_max_queues: 4
dev.ena.0.missing_tx_timeout: 21474836480
dev.ena.0.keep_alive_timeout: 25769803776
dev.ena.0.wd_active: 1
dev.ena.0.%parent: pci0
dev.ena.0.%pnpinfo: vendor=0x1d0f device=0xec20 subvendor=0x0000
subdevice=0x0000 class=0x020000
dev.ena.0.%location: slot=5 function=0 dbsf=pci0:0:5:0 handle=\_SB_.PCI0.S05_
dev.ena.0.%driver: ena
dev.ena.0.%desc: ENA adapter
dev.ena.%parent:
Here's "vmstat -i":
interrupt total rate
irq1: atkbd0 2 0
irq4: uart0 6 0
cpu0:timer 41324147 695
cpu1:timer 26268292 442
irq256: nvme0 9 0
irq257: nvme0 75412 1
irq258: nvme0 67504 1
irq259: ena0 118511 2
irq260: ena0 101753746 1711
irq261: ena0 102189440 1718
irq262: nvme1 9 0
irq263: nvme1 125224 2
irq264: nvme1 118774 2
Total 272041076 4575
Here's "netstat -in":
Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs
Coll
ena0 1500 <Link#1> 12:3d:00:e2:4f:78 177839782 0 0 178011394
0 0
ena0 - 192.168.4.0/2 192.168.4.101 61087021 - - 45501246 -
-
lo0 16384 <Link#2> lo0 15856955 0 0 15856955 0
0
lo0 - ::1/128 ::1 0 - - 0 -
-
lo0 - fe80::%lo0/64 fe80::1%lo0 0 - - 0 -
-
lo0 - 127.0.0.0/8 127.0.0.1 61418 - - 15856955 -
-
pflog 33160 <Link#3> pflog0 0 0 0 1797 0
0
Here's "sysctl kern.ipc":
kern.ipc.maxsockbuf: 2097152
kern.ipc.sockbuf_waste_factor: 8
kern.ipc.max_linkhdr: 16
kern.ipc.max_protohdr: 60
kern.ipc.max_hdr: 76
kern.ipc.max_datalen: 92
kern.ipc.maxsockets: 122296
kern.ipc.numopensockets: 2141
kern.ipc.soacceptqueue: 65536
kern.ipc.shm_allow_removed: 1
kern.ipc.shm_use_phys: 0
kern.ipc.shmall: 131072
kern.ipc.shmseg: 128
kern.ipc.shmmni: 192
kern.ipc.shmmin: 1
kern.ipc.shmmax: 536870912
kern.ipc.semaem: 16384
kern.ipc.semvmx: 32767
kern.ipc.semusz: 632
kern.ipc.semume: 50
kern.ipc.semopm: 100
kern.ipc.semmsl: 340
kern.ipc.semmnu: 150
kern.ipc.semmns: 340
kern.ipc.semmni: 50
kern.ipc.msgseg: 2048
kern.ipc.msgssz: 8
kern.ipc.msgtql: 40
kern.ipc.msgmnb: 2048
kern.ipc.msgmni: 40
kern.ipc.msgmax: 16384
kern.ipc.aio.lifetime: 30000
kern.ipc.aio.target_procs: 4
kern.ipc.aio.num_procs: 4
kern.ipc.aio.max_procs: 32
kern.ipc.aio.empty_retries: 0
kern.ipc.aio.empty_results: 0
kern.ipc.piperesizeallowed: 1
kern.ipc.piperesizefail: 0
kern.ipc.pipeallocfail: 0
kern.ipc.pipefragretry: 0
kern.ipc.pipekva: 69632
kern.ipc.maxpipekva: 62611456
kern.ipc.umtx_max_robust: 1000
kern.ipc.umtx_vnode_persistent: 0
kern.ipc.nmbufs: 1514295
kern.ipc.nmbjumbo16: 30000
kern.ipc.nmbjumbo9: 35053
kern.ipc.nmbjumbop: 118304
kern.ipc.nmbclusters: 236608
kern.ipc.maxmbufmem: 1938294784
--
You are receiving this mail because:
You are the assignee for the bug.
More information about the freebsd-bugs
mailing list