From bugmaster at FreeBSD.org Mon Sep 7 11:07:12 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 7 11:10:12 2009 Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org Message-ID: <200909071107.n87B7Bc1010449@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/136251 xen [xen] [patch] xn0 doesn't DHCP o kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. o kern/135179 xen [xen] Xen domU does not properly reboot o kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i o kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all o kern/135008 xen [xen] FreeBSD-current/Xen timecounter jumps o kern/134926 xen [xen] [panic] FreeBSD-current Xen DomU networking pani 7 problems total. From kmacy at freebsd.org Sat Sep 12 06:37:02 2009 From: kmacy at freebsd.org (Kip Macy) Date: Sat Sep 12 06:37:08 2009 Subject: gdbserver-xen updates Message-ID: <3c1674c90909112308u1778ce71ud1198236fe6a675a@mail.gmail.com> In case you're not already familiar with it, gdbserver-xen is something I added to xen several years ago, it makes it possible to debug domU guests as if they were ordinary single or threaded processes (UP or SMP). You can find an updated patch for xen-3.3 and xen-3.4 at: http://people.freebsd.org/~kmacy/xen/ pandemonium is a linux xen host delirium is a freebsd system patch xen-3.3 (or xen-3.4) cd xen-3.3/tools make sudo -c make install cd ../debugger/gdb ./gdbbuild sudo -c cp gdb-6.8-linux-i386-xen/gdb/gdbserver/gdbserver-xen /usr/local/bin [root@pandemonium ~]# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 5093 4 r----- 281.0 ExampleDomain 1 3000 1 -b---- 39.5 [root@pandemonium kmacy]# ip addr show dev eth0 6: eth0: mtu 1500 qdisc noqueue link/ether 00:30:48:94:f5:62 brd ff:ff:ff:ff:ff:ff inet 192.168.1.152/16 brd 192.168.255.255 scope global eth0 inet6 fe80::230:48ff:fe94:f562/64 scope link valid_lft forever preferred_lft forever [root@pandemonium kmacy]# gdbserver-xen 192.168.1.152:5555 --attach 1 xc_ptrace(PTRACE_ATTACH, 1, 0, 0) Attached; pid = 1 Listening on port 5555 % gdb /usr/home/kmacy/svn_checkouts/objdir/i386/usr/home/kmacy/svn_checkouts/head_new/sys/XEN/kernel.debug GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) target remote 192.168.1.152:5555 Remote debugging using 192.168.1.152:5555 [New Thread 1] 0xc003e3a7 in hypercall_page () at /usr/home/kmacy/svn_checkouts/head_new/sys/i386/xen/locore.s:249 249 /* At the end of our stack, we shall have free space - so store it */ warning: Unable to find dynamic linker breakpoint function. GDB will be unable to debug shared library initializers and track explicitly loaded dynamic code. warning: shared library handler failed to enable breakpoint Current language: auto; currently asm (gdb) (gdb) bt #0 0xc003e3a7 in hypercall_page () at /usr/home/kmacy/svn_checkouts/head_new/sys/i386/xen/locore.s:249 #1 0xc032b5df in idle_block () at hypercall.h:189 #2 0xc0314ed7 in cpu_idle_hlt (busy=1) at /usr/home/kmacy/svn_checkouts/head_new/sys/i386/i386/machdep.c:1200 #3 0xc03142c2 in cpu_idle (busy=1) at /usr/home/kmacy/svn_checkouts/head_new/sys/i386/i386/machdep.c:1324 #4 0xc00f7d32 in sched_idletd (dummy=0x0) at /usr/home/kmacy/svn_checkouts/head_new/sys/kern/sched_ule.c:2562 #5 0xc00aeed8 in fork_exit (callout=0xc00f7ca0 , arg=0x0, frame=0xc5a4fd38) at /usr/home/kmacy/svn_checkouts/head_new/sys/kern/kern_fork.c:843 #6 0xc030f5b4 in fork_trampoline () at /usr/home/kmacy/svn_checkouts/head_new/sys/i386/xen/exception.s:271 (gdb) To debug from boot pass -p to xm create to pause at the first instruction. -- When harsh accusations depart too far from the truth, they leave bitter consequences. --Tacitus From bugmaster at FreeBSD.org Mon Sep 14 11:07:13 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 14 11:10:04 2009 Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org Message-ID: <200909141107.n8EB7CPk072550@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/136251 xen [xen] [patch] xn0 doesn't DHCP o kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. o kern/135179 xen [xen] Xen domU does not properly reboot o kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i o kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all o kern/135008 xen [xen] FreeBSD-current/Xen timecounter jumps o kern/134926 xen [xen] [panic] FreeBSD-current Xen DomU networking pani 7 problems total. From linimon at FreeBSD.org Wed Sep 16 01:07:04 2009 From: linimon at FreeBSD.org (linimon@FreeBSD.org) Date: Wed Sep 16 01:07:11 2009 Subject: kern/138863: [xen] [panic] pmap_invalidate_cache_range panic on boot in 8.0-BETA4 in XenServer VM Message-ID: <200909160107.n8G174Dn011626@freefall.freebsd.org> Old Synopsis: pmap_invalidate_cache_range panic on boot in 8.0-BETA4 in XenServer VM New Synopsis: [xen] [panic] pmap_invalidate_cache_range panic on boot in 8.0-BETA4 in XenServer VM Responsible-Changed-From-To: freebsd-bugs->freebsd-xen Responsible-Changed-By: linimon Responsible-Changed-When: Wed Sep 16 01:06:34 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=138863 From kib at FreeBSD.org Wed Sep 16 13:03:37 2009 From: kib at FreeBSD.org (kib@FreeBSD.org) Date: Wed Sep 16 13:03:42 2009 Subject: kern/138863: [xen] [panic] pmap_invalidate_cache_range panic on boot in 8.0-BETA4 in XenServer VM Message-ID: <200909161303.n8GD3awB071836@freefall.freebsd.org> Synopsis: [xen] [panic] pmap_invalidate_cache_range panic on boot in 8.0-BETA4 in XenServer VM Responsible-Changed-From-To: freebsd-xen->kib Responsible-Changed-By: kib Responsible-Changed-When: Wed Sep 16 13:00:49 UTC 2009 Responsible-Changed-Why: I assume you use GENERIC i386 with full system virtualization (or whatever it is called in XEN). I am interested in the backtrace for !acpi case too. http://www.freebsd.org/cgi/query-pr.cgi?pr=138863 From bugmaster at FreeBSD.org Mon Sep 21 11:07:11 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 21 11:10:02 2009 Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org Message-ID: <200909211107.n8LB7AOR030490@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/136251 xen [xen] [patch] xn0 doesn't DHCP o kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. o kern/135179 xen [xen] Xen domU does not properly reboot o kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i o kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all o kern/135008 xen [xen] FreeBSD-current/Xen timecounter jumps o kern/134926 xen [xen] [panic] FreeBSD-current Xen DomU networking pani 7 problems total. From lab at gta.com Tue Sep 22 13:00:43 2009 From: lab at gta.com (Larry Baird) Date: Tue Sep 22 13:00:56 2009 Subject: XEN 5.5.0 and clflush Message-ID: <20090922123401.GB29391@gta.com> I originally sent this message to freebsd-current. Got no response, perhaps freebsd-xen is a better mailing list. (-: Since the end of August I have been unable to boot a generic kernel from FreeBSD current or 8 under XEN 5.5.0. Finally had a chance to briefly look at the problem. If I apply attached patch to remove calls to clflush() I am able to boot current. Hopefully somebody can shed some light. Is XEN incorrecty reporting CPUID_CLFSH or is XEN not correctly virtualizing this option. Or is the issue someplace else? I have also attached the dmesg from a successful boot. This issue seems to be same as http://www.freebsd.org/cgi/query-pr.cgi?pr=138863 Here is an attempt to type backtrace from non-booting kernel: pmap_invalidate_cache_range(c3252000,c3253000,c3253000,0,fee00000,...) at +pamp_invalidate_cache_range+0x60 pmap_mapdev_attr(fee00000,400,0,c1420d34,c0ba7a72,...) at pmap_mapdev_attr+0xec pmap_mapdev() at pmap_mapdev+0x20 lapic_init() at lapic_init+0x32 madt_setup_local() at madt_setup_local+0x2c apic_init() at apic_init+0x11a mistartup() at mi_startup+0x96 begin() at begin+0x2c Larry -- ------------------------------------------------------------------------ Larry Baird | http://www.gta.com Global Technology Associates, Inc. | Orlando, FL Email: lab@gta.com | TEL 407-380-0220, FAX 407-380-6080 -------------- next part -------------- A non-text attachment was scrubbed... Name: pmap.patch Type: text/x-diff Size: 657 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-xen/attachments/20090922/20acf7ca/pmap.bin -------------- next part -------------- Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #2: Mon Sep 21 11:00:46 EDT 2009 lab@sw-xenoss.gta.com:/usr/src/sys/i386/compile/GENERIC WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU 5110 @ 1.60GHz (1691.83-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6fb Stepping = 11 Features=0x789fbff Features2=0x80002201> AMD Features=0x20000000 AMD Features2=0x1 TSC: P-state invariant real memory = 536870912 (512 MB) avail memory = 493977600 (471 MB) ACPI APIC Table: ioapic0: Changing APIC ID to 1 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 62500000 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 1.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc220-0xc22f at device 1.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] uhci0: port 0xc200-0xc21f irq 23 at device 1.2 on pci0 uhci0: [ITHREAD] uhci0: LegSup = 0x0000 usbus0: controller did not stop usbus0: on uhci0 pci0: at device 1.3 (no driver attached) vgapci0: mem 0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff at device 2.0 on pci0 pci0: at device 3.0 (no driver attached) re0: port 0xc100-0xc1ff mem 0xf3001000-0xf30010ff irq 32 at device 4.0 on pci0 re0: Chip rev. 0x74800000 re0: MAC rev. 0x00000000 miibus0: on re0 rlphy0: PHY 0 on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto re0: Ethernet address: 4e:c4:60:ce:d0:72 re0: [FILTER] atrtc0: port 0x70-0x71 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse Explorer, device ID 4 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 uart0: port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 cpu0: on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd7fff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1691834274 Hz quality 800 Timecounters tick every 10.000 msec usbus0: 12Mbps Full Speed USB v1.0 ad0: 20480MB at ata0-master WDMA2 ugen0.1: at usbus0 uhub0: on usbus0 GEOM: ad0s1: geometry does not match label (255h,63s != 16h,63s). uhub0: 2 ports with 2 removable, self powered ugen0.2: at usbus0 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 acd0: CDROM at ata1-slave WDMA2 WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ad0s1a lock order reversal: (sleepable after non-sleepable) 1st 0xc390a058 rtentry (rtentry) @ net/route.c:1409 2nd 0xc0f433b8 ifnet_sx (ifnet_sx) @ netinet/sctp_bsd_addr.c:211 KDB: stack backtrace: db_trace_self_wrapper(c0c828bc,d6957788,c08c2725,c08b354b,c0c85715,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08b354b,c0c85715,c35286e8,c352c8b8,d69577e4,...) at kdb_backtrace+0x29 _witness_debugger(c0c85715,c0f433b8,c0c8e50e,c352c8b8,c0c9865e,...) at _witness_debugger+0x25 witness_checkorder(c0f433b8,1,c0c98655,d3,0,...) at witness_checkorder+0x839 _sx_slock(c0f433b8,0,c0c98655,d3,d6957878,...) at _sx_slock+0x85 sctp_init_ifns_for_vrf(3,c39018c0,d69578c0,c08c3178,c3901964,...) at sctp_init_ifns_for_vrf+0x30 sctp_addr_change(c38ffa00,1,d69578c0,c08c256c,d695791c,...) at sctp_addr_change+0x2c rt_newaddrmsg(1,c38ffa00,0,c390a000,c38ffa00,...) at rt_newaddrmsg+0x3f rtinit(c38ffa00,1,5,c0de9cfc,51573592,...) at rtinit+0x381 in_ifinit(0,c0c96ee7,1aa,1a6,c3814800,...) at in_ifinit+0x8f6 in_control(c38f7000,8040691a,c3907dc0,c3814800,c39018c0,...) at in_control+0xccb ifioctl(c38f7000,8040691a,c3907dc0,c39018c0,c38ff200,...) at ifioctl+0x14f0 soo_ioctl(c385f540,8040691a,c3907dc0,c356e100,c39018c0,...) at soo_ioctl+0x415 kern_ioctl(c39018c0,3,8040691a,c3907dc0,8bc060,...) at kern_ioctl+0x1fd ioctl(c39018c0,d6957cf8,c,c0c96f81,c0d665c8,...) at ioctl+0x134 syscall(d6957d38) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x281bf513, esp = 0xbfbfe61c, ebp = 0xbfbfe658 --- From kostikbel at gmail.com Tue Sep 22 13:31:14 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue Sep 22 13:31:20 2009 Subject: XEN 5.5.0 and clflush In-Reply-To: <20090922123401.GB29391@gta.com> References: <20090922123401.GB29391@gta.com> Message-ID: <20090922131034.GV47688@deviant.kiev.zoral.com.ua> On Tue, Sep 22, 2009 at 08:34:01AM -0400, Larry Baird wrote: > I originally sent this message to freebsd-current. Got no response, perhaps > freebsd-xen is a better mailing list. (-: > > Since the end of August I have been unable to boot a generic kernel from > FreeBSD current or 8 under XEN 5.5.0. Finally had a chance to briefly look > at the problem. If I apply attached patch to remove calls to clflush() I > am able to boot current. Hopefully somebody can shed some light. Is > XEN incorrecty reporting CPUID_CLFSH or is XEN not correctly virtualizing > this option. Or is the issue someplace else? I have also attached the > dmesg from a successful boot. This issue seems to be same as > http://www.freebsd.org/cgi/query-pr.cgi?pr=138863 > > > Here is an attempt to type backtrace from non-booting kernel: > > pmap_invalidate_cache_range(c3252000,c3253000,c3253000,0,fee00000,...) at > +pamp_invalidate_cache_range+0x60 > pmap_mapdev_attr(fee00000,400,0,c1420d34,c0ba7a72,...) at pmap_mapdev_attr+0xec > pmap_mapdev() at pmap_mapdev+0x20 > lapic_init() at lapic_init+0x32 > madt_setup_local() at madt_setup_local+0x2c > apic_init() at apic_init+0x11a > mistartup() at mi_startup+0x96 > begin() at begin+0x2c I am sorry for delay in answering, you may use this temporal patch until the issue is resolved somehow. I think I will have to disable CLFLUSH support for intel CPUs when self-snoop is not reported. diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 1f37765..7de1ca5 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -997,7 +997,7 @@ pmap_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva) if (cpu_feature & CPUID_SS) ; /* If "Self Snoop" is supported, do nothing. */ - else if (cpu_feature & CPUID_CLFSH) { + else if (0 && cpu_feature & CPUID_CLFSH) { /* * Otherwise, do per-cache line flush. Use the mfence diff --git a/sys/i386/i386/pmap.c b/sys/i386/i386/pmap.c index 7e3bc37..56c6776 100644 --- a/sys/i386/i386/pmap.c +++ b/sys/i386/i386/pmap.c @@ -994,7 +994,7 @@ pmap_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva) if (cpu_feature & CPUID_SS) ; /* If "Self Snoop" is supported, do nothing. */ - else if (cpu_feature & CPUID_CLFSH) { + else if (0 && cpu_feature & CPUID_CLFSH) { /* * Otherwise, do per-cache line flush. Use the mfence diff --git a/sys/i386/xen/pmap.c b/sys/i386/xen/pmap.c index 1d9c9c1..7dc8029 100644 --- a/sys/i386/xen/pmap.c +++ b/sys/i386/xen/pmap.c @@ -994,7 +994,7 @@ pmap_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva) if (cpu_feature & CPUID_SS) ; /* If "Self Snoop" is supported, do nothing. */ - else if (cpu_feature & CPUID_CLFSH) { + else if (0 && cpu_feature & CPUID_CLFSH) { /* * Otherwise, do per-cache line flush. Use the mfence -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-xen/attachments/20090922/a62eb040/attachment.pgp From lab at gta.com Tue Sep 22 13:59:15 2009 From: lab at gta.com (Larry Baird) Date: Tue Sep 22 13:59:32 2009 Subject: XEN 5.5.0 and clflush In-Reply-To: <20090922131034.GV47688@deviant.kiev.zoral.com.ua> References: <20090922123401.GB29391@gta.com> <20090922131034.GV47688@deviant.kiev.zoral.com.ua> Message-ID: <20090922135914.GA44294@gta.com> Kostik, > I am sorry for delay in answering, you may use this temporal patch until > the issue is resolved somehow. > > I think I will have to disable CLFLUSH support for intel CPUs when self-snoop > is not reported. Thanks for the feedback? Have you heard about this being a problem on platforms other than XEN? I am not sure how many FreeBSD users are using XEN. If this is a large base, getting a fix in before FreeBSD 8 is released would be nice. (-: Larry -- ------------------------------------------------------------------------ Larry Baird | http://www.gta.com Global Technology Associates, Inc. | Orlando, FL Email: lab@gta.com | TEL 407-380-0220, FAX 407-380-6080 From kostikbel at gmail.com Tue Sep 22 14:14:43 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue Sep 22 14:14:49 2009 Subject: XEN 5.5.0 and clflush In-Reply-To: <20090922135914.GA44294@gta.com> References: <20090922123401.GB29391@gta.com> <20090922131034.GV47688@deviant.kiev.zoral.com.ua> <20090922135914.GA44294@gta.com> Message-ID: <20090922141440.GX47688@deviant.kiev.zoral.com.ua> On Tue, Sep 22, 2009 at 09:59:14AM -0400, Larry Baird wrote: > Kostik, > > > I am sorry for delay in answering, you may use this temporal patch until > > the issue is resolved somehow. > > > > I think I will have to disable CLFLUSH support for intel CPUs when self-snoop > > is not reported. > Thanks for the feedback? Have you heard about this being a problem on > platforms other than XEN? No, and AMD should have CPUs without self-snoop, but supporting CLFLUSH. This is the main reason for this optimization came in, and no single complain from this problem on AMD (i.e. on native CPU, not under the hypervisor). > I am not sure how many FreeBSD users are using > XEN. If this is a large base, getting a fix in before FreeBSD 8 is > released would be nice. (-: > > Larry > > > -- > ------------------------------------------------------------------------ > Larry Baird | http://www.gta.com > Global Technology Associates, Inc. | Orlando, FL > Email: lab@gta.com | TEL 407-380-0220, FAX 407-380-6080 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-xen/attachments/20090922/a21642da/attachment.pgp From lab at gta.com Tue Sep 22 17:19:20 2009 From: lab at gta.com (Larry Baird) Date: Tue Sep 22 17:19:26 2009 Subject: XEN 5.5.0 and clflush In-Reply-To: <5078471e0909221003g43a125f4s99a1f841616bb184@mail.gmail.com> References: <20090922123401.GB29391@gta.com> <20090922131034.GV47688@deviant.kiev.zoral.com.ua> <5078471e0909221003g43a125f4s99a1f841616bb184@mail.gmail.com> Message-ID: <20090922171918.GA86010@gta.com> > > I think I will have to disable CLFLUSH support for intel CPUs when > > self-snoop > > is not reported. > > > > > That's the kinda weird part about this though... It's not triggering an > Invalid Instruction, but a GPF. Looking at AMD's description of how CLFLUSH > is supposed to work, I don't see why it's faulting with what looks like a > valid address. > > While this is probably far outside the scope of what their entry-level > support techs will understand, I can try raising this as a bug with Citrix > under our support contract if you're confident that this is broken on Xen's > end. I keep wondering about comments having to do with AMD processor. The native processor is an an Intel Xeon. Is this piece of the puzzle important? Larry -- ------------------------------------------------------------------------ Larry Baird | http://www.gta.com Global Technology Associates, Inc. | Orlando, FL Email: lab@gta.com | TEL 407-380-0220, FAX 407-380-6080 From toasty at dragondata.com Tue Sep 22 17:28:10 2009 From: toasty at dragondata.com (Kevin Day) Date: Tue Sep 22 17:28:17 2009 Subject: XEN 5.5.0 and clflush In-Reply-To: <20090922131034.GV47688@deviant.kiev.zoral.com.ua> References: <20090922123401.GB29391@gta.com> <20090922131034.GV47688@deviant.kiev.zoral.com.ua> Message-ID: <5078471e0909221003g43a125f4s99a1f841616bb184@mail.gmail.com> On Tue, Sep 22, 2009 at 8:10 AM, Kostik Belousov wrote: > > I think I will have to disable CLFLUSH support for intel CPUs when > self-snoop > is not reported. > > That's the kinda weird part about this though... It's not triggering an Invalid Instruction, but a GPF. Looking at AMD's description of how CLFLUSH is supposed to work, I don't see why it's faulting with what looks like a valid address. While this is probably far outside the scope of what their entry-level support techs will understand, I can try raising this as a bug with Citrix under our support contract if you're confident that this is broken on Xen's end. From kostikbel at gmail.com Wed Sep 23 11:06:00 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Wed Sep 23 11:06:07 2009 Subject: XEN 5.5.0 and clflush In-Reply-To: <20090922171918.GA86010@gta.com> References: <20090922123401.GB29391@gta.com> <20090922131034.GV47688@deviant.kiev.zoral.com.ua> <5078471e0909221003g43a125f4s99a1f841616bb184@mail.gmail.com> <20090922171918.GA86010@gta.com> Message-ID: <20090923110543.GC47688@deviant.kiev.zoral.com.ua> On Tue, Sep 22, 2009 at 01:19:18PM -0400, Larry Baird wrote: > > > I think I will have to disable CLFLUSH support for intel CPUs when > > > self-snoop > > > is not reported. > > > > > > > > That's the kinda weird part about this though... It's not triggering an > > Invalid Instruction, but a GPF. Looking at AMD's description of how CLFLUSH > > is supposed to work, I don't see why it's faulting with what looks like a > > valid address. > > > > While this is probably far outside the scope of what their entry-level > > support techs will understand, I can try raising this as a bug with Citrix > > under our support contract if you're confident that this is broken on Xen's > > end. > I keep wondering about comments having to do with AMD processor. The > native processor is an an Intel Xeon. Is this piece of the puzzle > important? Ok, the whole story (mostly to be able to point out an URL with description until the issue is sorted out). FreeBSD got ability to specify page caching attributes in the page table entries, so called Page Attribute Table (PAT). The physical to linear page mapping may specify the caching attribute, like WB (write-back), UC (uncacheable), WT (write-through) and so on. The procedure to set the caching attribute for the page involves removing old mapping, flushing TLB, creating new mapping, and then flushing the cache line that could be allocated for the remmapped physical memory. Flushing the cache may be performed by one of the three mechanisms: - do nothing; this is acceptable if CPU has so called Self-Snoop feature. For future discussion it is important to note that the feature is declared in the feature bitmap returned by CPUID instruction. - do the series of CLFLUSH instructions over the linear region. This method should be used if CPU has CLFLUSH support, again reported by CPUID. - do the WBINVD instruction that flushes the whole CPU cache. The methods above are ordered from most efficient (do nothing) to very time-consuming, and FreeBSD falls to the next method if previous is not supported. Real Intel CPUs (almost) all have all three methods implemented, and we do nothing to flush cache since self snoop is there. AMD CPUs, at least older models, do not have self-snoop, I believe, but do support CLFLUSH. I got no reports of problems caused by CLFLUSH on the AMD. If I manually disable self-snoop on Intel CPU, I get a reversed trap when doing CLFLUSH over the mapping for APIC registers page. I do not know the cause for this, my unfounded speculation is that there is some cache line allocated for it, and it is written out on CLFLUSH. Speculation is flimsy since BIOS should have programmed MTRR so that APIC is UC mode. Now, add the XEN to the picture. When whole machine virtualization (how it is called by XEN ?) runs GENERIC/i386 kernel, it uses so called VT feature of recent Intel CPUs, that, among other, gives the hypervisor the ability to intercept CPUID instruction. XEN seems to modify reported CPU features, removing self-snoop. As result. we fall back to CLFLUSH method. XEN seems to translate a fault caused by cache write-back to the APIC registrers, into #gpf. This is the issue that you get on XEN. So, I do not know why Intels cause trap on CLFLUSH, this may be either some stupidness in my clflush code, or an additional factor that I did not taken care of yet. Patch I sent disables CLFLUSH unconditionally, that is bad for performance on AMD. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-xen/attachments/20090923/b7a7ef59/attachment.pgp From o.roeschke at gmx.net Sat Sep 26 13:01:39 2009 From: o.roeschke at gmx.net (Oliver Roeschke) Date: Sat Sep 26 13:01:51 2009 Subject: FreeBSD8-RC1 crashed Message-ID: <1253968468.2211.103.camel@phoenix.blechhirn.net> Hi all, I just noticed that my XEN pv-domU based on FreeBSD8-RC1 (based on SVN r197306) crashed with the following message: [XEN] hypervisor wallclock nudged; nudging TOD. [XEN] hypervisor wallclock nudged; nudging TOD. panic: mtx_lock() of spin mutex (null) @ /test/usr-src/sys/net/netisr.c:830 cpuid = 0 KDB: enter: panic [thread pid 21716 tid 100098 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> bt Tracing pid 21716 tid 100098 td 0xc423c900 kdb_enter(c035ddd5,c035ddd5,c035c7cd,e422d9cc,0,...) at kdb_enter+0x3a panic(c035c7cd,0,c036da84,33e,16d,...) at panic+0x136 _mtx_lock_flags(c082d808,0,c036da84,33e,c3441d00,...) at _mtx_lock_flags+0x9a netisr_clearqdrops(e422da28,c423c9a4,c05129d0,0) at netisr_clearqdrops+0x66e netisr_queue_src(1,0,c3441d00,e422da6c,c01862de,...) at netisr_queue_src+0xa7 netisr_queue(1,c3441d00,c3441d48,e422daf8,e422da80,...) at netisr_queue+0x20 if_simloop(c32e9400,c3441d00,2,0,c01a2a8f,...) at if_simloop+0xfe looutput(c32e9400,c3441d00,e422db00,e422daf8,c031ff74,...) at looutput+0x141 ip_output(c3441d00,0,0,0,0,...) at ip_output+0x9cc tcp_output(c40c09e0,ca870340,1b9,c40be7a8,c40dc338,...) at tcp_output+0x1540 tcp_ctloutput(c40dc338,ca870340,c423c900,25,e422dc70,...) at tcp_ctloutput+0x933 soconnect(c40dc338,ca870340,c423c900,bf7fae20,ca870340,...) at soconnect+0x52 kern_connect(c423c900,6,ca870340,ca870340,ffffffff,...) at kern_connect+0xa6 connect(c423c900,e422dd08,c,c03646ed,c039d8b8,...) at connect+0x46 syscall(e422dd48) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x22 --- syscall (98, FreeBSD ELF32, connect), eip = 0x283b9e5b, esp = 0xbf7facbc, ebp = 0xbf7faee8 --- Is this issue already known? I have reserved the domU state, so further troubleshooting is not a problem. If anyone wonders, '/test/usr-src' is '/usr/src' within a ZFS volume ;-)) Regards, --- Mr. Olli From edwin.shao at gmail.com Sat Sep 26 21:08:06 2009 From: edwin.shao at gmail.com (Edwin Shao) Date: Sat Sep 26 21:08:12 2009 Subject: Kernel trap 9 using 8.0-RC1 compiled with "XEN" configuration Message-ID: As stated in subject, newly compiled 8.0-RC1 source using standard "XEN" configuration fails with the below message. WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-RC1 #0: Sat Sep 26 20:23:41 UTC 2009 root@hyper.nekogiri.com:/usr/obj/usr/src/sys/XEN WARNING: WITNESS option enabled, expect reduced performance. Xen reported: 2311.848 MHz processor. Timecounter "ixen" frequency 1000000000 Hz quality 0 CPU: Quad-Core AMD Opteron(tm) Processor 2376 (2311.85-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x100f42 Stepping = 2 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x37ff TSC: P-state invariant real memory = 671088640 (640 MB) avail memory = 646967296 (616 MB) [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) kbd0 at kbdmux0 xenbus0: on motherboard xc0: on motherboard [XEN] xen_rtc_probe: probing Hypervisor RTC clock rtc0: on motherboard [XEN] xen_rtc_attach: attaching Hypervisor RTC clock Timecounters tick every 10.000 msec kernel trap 9 with interrupts disabled Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x21:0xc0309169 stack pointer = 0x29:0xc2861cb8 frame pointer = 0x29:0xc2861cc8 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 1, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 11 (idle: cpu0) [thread pid 11 tid 100003 ] Stopped at spinlock_enter+0x99: hlt db> bt Tracing pid 11 tid 100003 td 0xc2b08b40 spinlock_enter(1,c2b08b40,c03c8d40,c2861ce8,c0309c63,...) at spinlock_enter+0x99 sched_idletd(0,c2861d38,c2b06aa0,0,0,...) at sched_idletd+0x205 fork_exit(c00f5048,0,c2861d38) at fork_exit+0x76 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc2861d70, ebp = 0 --- From bugmaster at FreeBSD.org Mon Sep 28 11:07:08 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 28 11:09:57 2009 Subject: Current problem reports assigned to freebsd-xen@FreeBSD.org Message-ID: <200909281107.n8SB77Ru064229@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/136251 xen [xen] [patch] xn0 doesn't DHCP o kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc. o kern/135179 xen [xen] Xen domU does not properly reboot o kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i o kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all o kern/135008 xen [xen] FreeBSD-current/Xen timecounter jumps o kern/134926 xen [xen] [panic] FreeBSD-current Xen DomU networking pani 7 problems total. From mister.olli at googlemail.com Tue Sep 29 22:02:53 2009 From: mister.olli at googlemail.com (Mr. Olli) Date: Tue Sep 29 22:02:59 2009 Subject: pygrub status Message-ID: <1254260019.2211.175.camel@phoenix.blechhirn.net> I just wanted to know if someone on this list knows about the pygrub status? Has anyone tested if it boots from UFS2? I've had a look at the code (my python understanding is limited right now) but it does not seem to be very complex. maybe some chance to get in touch with python ;-) has anyone started similar approaches or does know where freebsd slice data structure definition can be found? from what I saw in the code pygrub does understand solaris slices. my guessing is that is shouldn't be that hard to adopt this code to freebsd slices (correct me if I'm wrong ;-)) will be happy about any input and chance to get freebsd PV booting out own disk image :-)) Regards, ==== Mr. Olli