From ivan.anyukov at googlemail.com Thu Oct 1 09:03:19 2009 From: ivan.anyukov at googlemail.com (ivan anyukov) Date: Thu Oct 1 11:35:08 2009 Subject: NMI with 7.2-stable Message-ID: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> Hi guys, I'm running 7.2-STABLE on a Thinkpad T60. When connecting a second monitor to my docking station sometimes my FreeBSD freezes. kgdb on the vmcore-file says "non-maskable interrupt trap" Some details: X.Org 1.5.3 using the radeon-Driver I think the problem appears when moving xterms from the first to the second monitor (or back). The mouse cursor looks _very_ strange then and after some minutes the whole system freezes. Does anyone know about the problem? Is it a hardware-failure for sure? A vmcore is generated, I loaded it into kgdb: This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: <<22>NMI>N MIIS AI SA b0b, 0E,I ESIAS A ffff < 2>R<2 >RA<2M >ApMa rpiatyr ietryr oerr,r olri,k elliyk ehlayr dhwaarrdew afraei lfuariel.ure. FFaattaall ttrraapp 1199:: nnoonn--mmaasskkaabbllee iinntteerrrruupptt ttrraapp wwhhiillee iinn kkeerrnneell mmooddee ccppuuiidd == 01;; aappiicc iidd == 0010 iinnssttrruuccttiioonn ppooiinntteerr == 00xx2200::00xxcc00eeaa7733eecc ssttaacckk ppooiinntteerr == 00xx2288::00xxcc4477aa58cc8888 ffrraammee ppooiinntteerr == 00xx2288::00xxcc4477aa58cc8888 ccooddee sseeggmmeenntt == bbaassee 00xx00,, lliimmiitt 00xxffffffffff,, ttyyppee 00xx11bb == DDPPLL 00,, pprreess 11,, ddeeff3322 11,, ggrraann 11 pprroocceessssoorr eeffllaaggss == iinntteerrrruupptt eennaabblleedd,, IIOOPPLL == 00 ccuurrrreenntt pprroocceessss == 1112 ((iiddllee:: ccppuu10)) ttrraapp nnuummbbeerr == 1199 backtrace says: #0 doadump () at pcpu.h:196 #1 0xc07c5258 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07c5535 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ab2d14 in trap_fatal (frame=0xc47a5c48, eva=0) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ab3a9d in trap (frame=0xc47a5c48) at /usr/src/sys/i386/i386/trap.c:726 #5 0xc0a9910b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #6 0xc0ea73ec in acpi_cpu_c1 () at /usr/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550 #7 0xc0ea0623 in acpi_cpu_idle () at /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943 #8 0xc0aa377b in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1183 #9 0xc07e6bd4 in sched_idletd (dummy=0x0) at /usr/src/sys/kern/sched_ule.c:2681 #10 0xc07a1311 in fork_exit (callout=0xc07e690f , arg=0x0, frame=0xc47a5d38) at /usr/src/sys/kern/kern_fork.c:810 #11 0xc0a99180 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264 Any ideas? Thanks a lot! From attilio at freebsd.org Thu Oct 1 13:27:42 2009 From: attilio at freebsd.org (Attilio Rao) Date: Thu Oct 1 13:27:49 2009 Subject: NMI with 7.2-stable In-Reply-To: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> References: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> Message-ID: <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> Can you send a full dmesg? 2009/10/1 ivan anyukov : > Hi guys, > I'm running 7.2-STABLE on a Thinkpad T60. > When connecting a second monitor to my docking station sometimes my FreeBSD > freezes. > kgdb on the vmcore-file says "non-maskable interrupt trap" > Some details: > X.Org 1.5.3 using the radeon-Driver > I think the problem appears when moving xterms from the first to the second > monitor (or back). The mouse cursor looks _very_ strange then and after some > minutes the whole system freezes. > Does anyone know about the problem? Is it a hardware-failure for sure? > > A vmcore is generated, I loaded it into kgdb: > > This GDB was configured as "i386-marcel-freebsd"... > > Unread portion of the kernel message buffer: > <<22>NMI>N MIIS AI SA b0b, 0E,I ESIAS A ffff > < > 2>R<2 >>RA<2M >ApMa rpiatyr ietryr oerr,r olri,k elliyk ehlayr dhwaarrdew afraei > lfuariel.ure. > > > > FFaattaall ttrraapp 1199:: nnoonn--mmaasskkaabbllee iinntteerrrruupptt > ttrraapp wwhhiillee iinn kkeerrnneell mmooddee > > ccppuuiidd == 01;; aappiicc iidd == 0010 > > iinnssttrruuccttiioonn ppooiinntteerr == > 00xx2200::00xxcc00eeaa7733eecc > > ssttaacckk ppooiinntteerr == > 00xx2288::00xxcc4477aa58cc8888 > > ffrraammee ppooiinntteerr == > 00xx2288::00xxcc4477aa58cc8888 > > ccooddee sseeggmmeenntt == bbaassee > 00xx00,, lliimmiitt 00xxffffffffff,, ttyyppee 00xx11bb > > == DDPPLL 00,, pprreess > 11,, ddeeff3322 11,, ggrraann 11 > > pprroocceessssoorr eeffllaaggss == iinntteerrrruupptt > eennaabblleedd,, IIOOPPLL == 00 > > ccuurrrreenntt pprroocceessss == 1112 > ((iiddllee:: ccppuu10)) > > ttrraapp nnuummbbeerr == 1199 > > backtrace says: > #0 doadump () at pcpu.h:196 > #1 0xc07c5258 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 > #2 0xc07c5535 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xc0ab2d14 in trap_fatal (frame=0xc47a5c48, eva=0) at > /usr/src/sys/i386/i386/trap.c:939 > #4 0xc0ab3a9d in trap (frame=0xc47a5c48) at > /usr/src/sys/i386/i386/trap.c:726 > #5 0xc0a9910b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 > #6 0xc0ea73ec in acpi_cpu_c1 () at > /usr/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550 > #7 0xc0ea0623 in acpi_cpu_idle () at > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943 > #8 0xc0aa377b in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1183 > #9 0xc07e6bd4 in sched_idletd (dummy=0x0) at > /usr/src/sys/kern/sched_ule.c:2681 > #10 0xc07a1311 in fork_exit (callout=0xc07e690f , arg=0x0, > frame=0xc47a5d38) > at /usr/src/sys/kern/kern_fork.c:810 > #11 0xc0a99180 in fork_trampoline () at > /usr/src/sys/i386/i386/exception.s:264 > > Any ideas? > > Thanks a lot! > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Peace can only be achieved by understanding - A. Einstein From ivan.anyukov at googlemail.com Thu Oct 1 15:31:34 2009 From: ivan.anyukov at googlemail.com (ivan anyukov) Date: Thu Oct 1 15:38:25 2009 Subject: NMI with 7.2-stable In-Reply-To: <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> References: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> Message-ID: <962bea8e0910010831o2d39e0ecvb2fef29ed2f20954@mail.gmail.com> dmesg goes here: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.2-STABLE #12: Wed Aug 26 23:21:46 CEST 2009 root@fnord:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM) Duo CPU T2400 @ 1.83GHz (1828.76-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6ec Stepping = 12 Features=0xbfe9fbff Features2=0xc1a9 AMD Features=0x100000 Cores per package: 2 real memory = 1609367552 (1534 MB) avail memory = 1563168768 (1490 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ACPI Warning (tbfadt-0505): Optional field "Gpe1Block" has zero address or length: 0 102C/0 [20070320] ioapic0: Changing APIC ID to 1 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi_ec0: port 0x62,0x66 on acpi0 acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 5ff00000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 acpi_lid0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0x2000-0x20ff mem 0xd8000000-0xdfffffff,0xee100000-0xee10ffff irq 16 at device 0.0 on pci1 hdac0: mem 0xee400000-0xee403fff irq 17 at device 27.0 on pci0 hdac0: HDA Driver Revision: 20090329_0131 hdac0: [ITHREAD] pcib2: irq 20 at device 28.0 on pci0 pci2: on pcib2 em0: port 0x3000-0x301f mem 0xee000000-0xee01ffff irq 16 at device 0.0 on pci2 em0: Using MSI interrupt em0: [FILTER] em0: Ethernet address: 00:1e:37:cb:53:b3 pcib3: irq 21 at device 28.1 on pci0 pci3: on pcib3 wpi0: mem 0xedf00000-0xedf00fff irq 17 at device 0.0 on pci3 wpi0: Ethernet address: 00:18:de:9d:0e:8f wpi0: [ITHREAD] pcib4: irq 22 at device 28.2 on pci0 pci4: on pcib4 pcib5: irq 23 at device 28.3 on pci0 pci12: on pcib5 uhci0: port 0x1800-0x181f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x1820-0x183f irq 17 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0x1860-0x187f irq 19 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xee404000-0xee4043ff irq 19 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered uhub5: on uhub4 uhub5: multiple transaction translators uhub5: 4 ports with 4 removable, self powered ums0: on uhub5 ums0: 3 buttons and Z dir. ukbd0: on uhub5 kbd2 at ukbd0 uhid0: on uhub5 pcib6: at device 30.0 on pci0 pci21: on pcib6 cbb0: mem 0xe4300000-0xe4300fff irq 16 at device 0.0 on pci21 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 cbb0: [ITHREAD] isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1880-0x188f at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] atapci1: port 0x18c8-0x18cf,0x18ac-0x18af,0x18c0-0x18c7,0x18a8-0x18ab,0x18b0-0x18bf mem 0xee404400-0xee4047ff irq 16 at device 31.2 on pci0 atapci1: [ITHREAD] atapci1: AHCI called from vendor specific driver atapci1: AHCI Version 01.10 controller with 4 ports detected ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: port not implemented ata3: [ITHREAD] ata4: on atapci1 ata4: port not implemented ata4: [ITHREAD] ata5: on atapci1 ata5: port not implemented ata5: [ITHREAD] pci0: at device 31.3 (no driver attached) acpi_tz0: on acpi0 acpi_tz1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled battery0: on acpi0 acpi_acad0: on acpi0 cpu0: on acpi0 est0: on cpu0 p4tcc0: on cpu0 cpu1: on acpi0 est1: on cpu1 p4tcc1: on cpu1 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd0fff,0xd1000-0xd1fff,0xdc000-0xdffff,0xe0000-0xeffff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 8250 or not responding sio0: [FILTER] sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ugen0: on uhub3 Timecounters tick every 1.000 msec acd0: CDRW at ata0-master UDMA33 ad4: 476940MB at ata2-master SATA150 hdac0: HDA Codec #0: Analog Devices AD1981HD hdac0: HDA Codec #1: Conexant (Unknown) pcm0: at cad 0 nid 1 on hdac0 pcm1: at cad 0 nid 1 on hdac0 SMP: AP CPU #1 Launched! GEOM_LABEL: Label for provider ad4s1a is ufsid/427627623762bv3. GEOM_LABEL: Label for provider ad4s1d is ufsid/427627623762bv3 GEOM_LABEL: Label for provider ad4s1e is ufsid/427627623762bv3. GEOM_LABEL: Label for provider ad4s1f is ufsid/427627623762bv3. Trying to mount root from ufs:/dev/ad4s1a cryptosoft0: on motherboard GEOM_ELI: Device ad4s1g.eli created. GEOM_ELI: Encryption: AES-CBC 128 GEOM_ELI: Crypto: software GEOM_LABEL: Label for provider ad4s1g.eli is ufsid/49863f. GEOM_LABEL: Label ufsid/45 removed. GEOM_LABEL: Label for provider ad4s1a is ufsid/49835. [...GEOM-stuff] drm0: on vgapci0 info: [drm] MSI enabled 1 message(s) vgapci0: child drm0 requested pci_enable_busmaster info: [drm] Initialized radeon 1.29.0 20080528 info: [drm] Setting GART location based on new memory map info: [drm] Loading R500 Microcode info: [drm] Num pipes: 1 info: [drm] writeback test succeeded in 1 usecs drm0: [ITHREAD] 2009/10/1 Attilio Rao > Can you send a full dmesg? > > 2009/10/1 ivan anyukov : > > Hi guys, > > I'm running 7.2-STABLE on a Thinkpad T60. > > When connecting a second monitor to my docking station sometimes my > FreeBSD > > freezes. > > kgdb on the vmcore-file says "non-maskable interrupt trap" > > Some details: > > X.Org 1.5.3 using the radeon-Driver > > I think the problem appears when moving xterms from the first to the > second > > monitor (or back). The mouse cursor looks _very_ strange then and after > some > > minutes the whole system freezes. > > Does anyone know about the problem? Is it a hardware-failure for sure? > > > > A vmcore is generated, I loaded it into kgdb: > > > > This GDB was configured as "i386-marcel-freebsd"... > > > > Unread portion of the kernel message buffer: > > <<22>NMI>N MIIS AI SA b0b, 0E,I ESIAS A ffff > > < > > 2>R<2 > >>RA<2M >ApMa rpiatyr ietryr oerr,r olri,k elliyk ehlayr dhwaarrdew afraei > > lfuariel.ure. > > > > > > > > FFaattaall ttrraapp 1199:: nnoonn--mmaasskkaabbllee > iinntteerrrruupptt > > ttrraapp wwhhiillee iinn kkeerrnneell mmooddee > > > > ccppuuiidd == 01;; aappiicc iidd == 0010 > > > > iinnssttrruuccttiioonn ppooiinntteerr == > > 00xx2200::00xxcc00eeaa7733eecc > > > > ssttaacckk ppooiinntteerr == > > 00xx2288::00xxcc4477aa58cc8888 > > > > ffrraammee ppooiinntteerr == > > 00xx2288::00xxcc4477aa58cc8888 > > > > ccooddee sseeggmmeenntt == bbaassee > > 00xx00,, lliimmiitt 00xxffffffffff,, ttyyppee 00xx11bb > > > > == DDPPLL 00,, pprreess > > 11,, ddeeff3322 11,, ggrraann 11 > > > > pprroocceessssoorr eeffllaaggss == iinntteerrrruupptt > > eennaabblleedd,, IIOOPPLL == 00 > > > > ccuurrrreenntt pprroocceessss == 1112 > > ((iiddllee:: ccppuu10)) > > > > ttrraapp nnuummbbeerr == 1199 > > > > backtrace says: > > #0 doadump () at pcpu.h:196 > > #1 0xc07c5258 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:418 > > #2 0xc07c5535 in panic (fmt=Variable "fmt" is not available. > > ) at /usr/src/sys/kern/kern_shutdown.c:574 > > #3 0xc0ab2d14 in trap_fatal (frame=0xc47a5c48, eva=0) at > > /usr/src/sys/i386/i386/trap.c:939 > > #4 0xc0ab3a9d in trap (frame=0xc47a5c48) at > > /usr/src/sys/i386/i386/trap.c:726 > > #5 0xc0a9910b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 > > #6 0xc0ea73ec in acpi_cpu_c1 () at > > /usr/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550 > > #7 0xc0ea0623 in acpi_cpu_idle () at > > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943 > > #8 0xc0aa377b in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1183 > > #9 0xc07e6bd4 in sched_idletd (dummy=0x0) at > > /usr/src/sys/kern/sched_ule.c:2681 > > #10 0xc07a1311 in fork_exit (callout=0xc07e690f , arg=0x0, > > frame=0xc47a5d38) > > at /usr/src/sys/kern/kern_fork.c:810 > > #11 0xc0a99180 in fork_trampoline () at > > /usr/src/sys/i386/i386/exception.s:264 > > > > Any ideas? > > > > Thanks a lot! > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > -- > Peace can only be achieved by understanding - A. Einstein > From attilio at freebsd.org Thu Oct 1 16:11:16 2009 From: attilio at freebsd.org (Attilio Rao) Date: Thu Oct 1 16:11:22 2009 Subject: NMI with 7.2-stable In-Reply-To: <962bea8e0910010831o2d39e0ecvb2fef29ed2f20954@mail.gmail.com> References: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> <962bea8e0910010831o2d39e0ecvb2fef29ed2f20954@mail.gmail.com> Message-ID: <3bbf2fe10910010911v4db0fbdar9f79ea712c734a23@mail.gmail.com> 2009/10/1 ivan anyukov : > dmesg goes here: > Yes, I think it is an hw failure. Attilio -- Peace can only be achieved by understanding - A. Einstein From reg at freebsd.org Fri Oct 2 20:34:30 2009 From: reg at freebsd.org (Jeremy Lea) Date: Fri Oct 2 20:34:38 2009 Subject: Distributed SSH attack Message-ID: <20091002201039.GA53034@flint.openpave.org> Hi, This is off topic to this list, but I dont want to subscribe to -chat just to post there... Someone is currently running a distributed SSH attack against one of my boxes - one attempted login for root every minute or so for the last 48 hours. They wont get anywhere, since the box in question has no root password, and doesn't allow root logins via SSH anyway... But I was wondering if there were any security researchers out there that might be interested in the +-800 IPs I've collected from the botnet? The resolvable hostnames mostly appear to be in Eastern Europe and South America - I haven't spotted any that might be 'findable' to get the botnet software. I could switch out the machine for a honeypot in a VM or a jail, by moving the host to a new IP, and if you can think of a way of allowing the next login to succeed with any password, then you could try to see what they delivered... But I don't have a lot of time to help. Regards, -Jeremy -- FreeBSD - Because the best things in life are free... http://www.freebsd.org/ From anders at FreeBSD.org Fri Oct 2 20:45:29 2009 From: anders at FreeBSD.org (Anders Nordby) Date: Fri Oct 2 20:45:37 2009 Subject: No end to swap zone exhausted problems? Message-ID: <20091002202936.GA836@fupp.net> Hi, Better post about my swap zone problems here than tear all my hair out. This concerns me: 1) How can SWAPMETA usage continue increasing, while swap space used is not? Example: http://anders.fupp.net/test/swapmeta.png. In the swap meta graph, used space is USED and free space is LIMIT-USED (FREE numbers are a bit odd, it starts with 0) from the SWAPMETA line when running vmstat -z. In the week graph, you can see that after 3 days swap space usage settles at around 90 GB, while swap meta keeps increasing. Is there a leak? 2) By default, kern.maxswzone is set to 32 MB in FreeBSD: kern.maxswzone: 33554432 This leads to vmstat -z showing: ITEM SIZE LIMIT USED FREE REQUESTS FAILURES SWAPMETA: 288, 116519, 0, 0, 0, 0 According to vmstat man page, memory numbers are printed in kilobytes. Where is the relation between maxswzone and SWAPMETA? By booting with different settings for maxswzone and checking what SWAPMETA LIMIT turns out to be, I get this table: kern.maxswzone SWAPMETA LIMIT ============== ============== 32 MB 116519 64 MB 233025 128 MB 466037 256 MB 932074 512 MB 995410 1 GB 995410 2 GB 995410 How come increasing maxswzone beyond 256 MB does not yield a larger SWAPMETA than 995410? Does it stop around 260 MB maxswzone? How can I increase SWAPMETA further to be able to swap more? When maxswzone is set to 2 GB, the number goes negative: kern.maxswzone: -2147483648 I guess my luck is out? Some additional info: - When LIMIT-USED from SWAPMETA gets to 0, the system freezes with the dreaded "swap zone exhausted, increase kern.maxswzone" message. I monitor this. It's nice to know when your system will freeze. - KMEM settings are: vm.kmem_size_max: 3865468109 vm.kmem_size_min: 0 vm.kmem_size: 2718302208 Had no luck by tuning and increasing kmem_size and kmem_size_max, it mostly leads to crashes. - I am running FreeBSD/amd64 7.2-RELEASE, and I have 8 GB physical RAM. - By lowering kern.maxusers from 386 to 16 and trimming my kernel config down to a bare minimum I got SWAPMETA LIMIT increased to a whopping 995475, an increase of ~64 KB. I need more than this. - Why do I use so much swap space? I am running Varnish with malloc for storage, and I have a rather large dataset (6 million objects for this set of servers) with mostly 1 week expire time. Performance wise it works well, better than mmapping a large file which is/was the most common method for Varnish (this gives occasional ~2 minute freezes). Regards, -- Anders. From glarkin at FreeBSD.org Fri Oct 2 21:38:15 2009 From: glarkin at FreeBSD.org (Greg Larkin) Date: Fri Oct 2 21:38:22 2009 Subject: Distributed SSH attack In-Reply-To: <20091002201039.GA53034@flint.openpave.org> References: <20091002201039.GA53034@flint.openpave.org> Message-ID: <4AC66E07.4030605@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeremy Lea wrote: > Hi, > > This is off topic to this list, but I dont want to subscribe to -chat > just to post there... Someone is currently running a distributed SSH > attack against one of my boxes - one attempted login for root every > minute or so for the last 48 hours. They wont get anywhere, since the > box in question has no root password, and doesn't allow root logins via > SSH anyway... > > But I was wondering if there were any security researchers out there > that might be interested in the +-800 IPs I've collected from the > botnet? The resolvable hostnames mostly appear to be in Eastern Europe > and South America - I haven't spotted any that might be 'findable' to > get the botnet software. > > I could switch out the machine for a honeypot in a VM or a jail, by > moving the host to a new IP, and if you can think of a way of allowing > the next login to succeed with any password, then you could try to see > what they delivered... But I don't have a lot of time to help. > > Regards, > -Jeremy > Hi Jeremy, You could set up DenyHosts and contribute to the pool of IPs that are attempting SSH logins on the Net: http://denyhosts.sourceforge.net/faq.html#4_0 It also looks like there's been quite a spike of SSH login activity recently: http://stats.denyhosts.net/stats.html Hope that helps, Greg - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. http://twitter.com/sourcehosting/ - Follow me, follow you -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKxm4H0sRouByUApARAtnPAKCQuivQdE1s0ZZnUO6qVWA87N8ZKgCgjyYD Tbv+hWI+KoXYsEpt0n4gW5k= =xCz7 -----END PGP SIGNATURE----- From aryeh.friedman at gmail.com Fri Oct 2 23:10:45 2009 From: aryeh.friedman at gmail.com (Aryeh M. Friedman) Date: Fri Oct 2 23:10:52 2009 Subject: Distributed SSH attack In-Reply-To: <4AC66E07.4030605@FreeBSD.org> References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> Message-ID: <4AC6810D.1030106@gmail.com> Greg Larkin wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jeremy Lea wrote: > >> Hi, >> >> This is off topic to this list, but I dont want to subscribe to -chat >> just to post there... Someone is currently running a distributed SSH >> attack against one of my boxes - one attempted login for root every >> minute or so for the last 48 hours. They wont get anywhere, since the >> box in question has no root password, and doesn't allow root logins via >> SSH anyway... >> >> But I was wondering if there were any security researchers out there >> that might be interested in the +-800 IPs I've collected from the >> botnet? The resolvable hostnames mostly appear to be in Eastern Europe >> and South America - I haven't spotted any that might be 'findable' to >> get the botnet software. >> >> I could switch out the machine for a honeypot in a VM or a jail, by >> moving the host to a new IP, and if you can think of a way of allowing >> the next login to succeed with any password, then you could try to see >> what they delivered... But I don't have a lot of time to help. >> >> Regards, >> -Jeremy >> >> > > Hi Jeremy, > > You could set up DenyHosts and contribute to the pool of IPs that are > attempting SSH logins on the Net: > http://denyhosts.sourceforge.net/faq.html#4_0 > > It also looks like there's been quite a spike of SSH login activity > recently: http://stats.denyhosts.net/stats.html > > Hope that helps, > Greg > - -- > Greg Larkin > > http://www.FreeBSD.org/ - The Power To Serve > http://www.sourcehosting.net/ - Ready. Set. Code. > http://twitter.com/sourcehosting/ - Follow me, follow you > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFKxm4H0sRouByUApARAtnPAKCQuivQdE1s0ZZnUO6qVWA87N8ZKgCgjyYD > Tbv+hWI+KoXYsEpt0n4gW5k= > =xCz7 > -----END PGP SIGNATURE----- > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > There seems to be some kind of cordinated attack because I have been seeing different backbones wink in and out (work and home are on completely diff backbones and are having roughly the same intermitten interuptions) From jhell at DataIX.net Fri Oct 2 23:47:46 2009 From: jhell at DataIX.net (jhell) Date: Fri Oct 2 23:47:54 2009 Subject: Distributed SSH attack In-Reply-To: <4AC66E07.4030605@FreeBSD.org> References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 2 Oct 2009 17:17 -0000, glarkin wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jeremy Lea wrote: > > Hi, > > > > This is off topic to this list, but I dont want to subscribe to -chat > > just to post there... Someone is currently running a distributed SSH > > attack against one of my boxes - one attempted login for root every > > minute or so for the last 48 hours. They wont get anywhere, since the > > box in question has no root password, and doesn't allow root logins via > > SSH anyway... > > > > But I was wondering if there were any security researchers out there > > that might be interested in the +-800 IPs I've collected from the > > botnet? The resolvable hostnames mostly appear to be in Eastern Europe > > and South America - I haven't spotted any that might be 'findable' to > > get the botnet software. > > > > I could switch out the machine for a honeypot in a VM or a jail, by > > moving the host to a new IP, and if you can think of a way of allowing > > the next login to succeed with any password, then you could try to see > > what they delivered... But I don't have a lot of time to help. > > > > Regards, > > -Jeremy > > > > Hi Jeremy, > > You could set up DenyHosts and contribute to the pool of IPs that are > attempting SSH logins on the Net: > http://denyhosts.sourceforge.net/faq.html#4_0 > > It also looks like there's been quite a spike of SSH login activity > recently: http://stats.denyhosts.net/stats.html > > Hope that helps, > Greg > - -- > Greg Larkin > > http://www.FreeBSD.org/ - The Power To Serve > http://www.sourcehosting.net/ - Ready. Set. Code. > http://twitter.com/sourcehosting/ - Follow me, follow you > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFKxm4H0sRouByUApARAtnPAKCQuivQdE1s0ZZnUO6qVWA87N8ZKgCgjyYD > Tbv+hWI+KoXYsEpt0n4gW5k= > =xCz7 > -----END PGP SIGNATURE----- > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Another temporary to long term solution might be the following utilities, ports/security/sshguard-pf ports/security/expiretable This is more of a pf based solution so that's up to your policies and decision. Giving thanks to the post about DenyHosts I didn't know that existed till this point. Best regards. - -- %{----------------------------------------------------+ | dataix.net!jhell 2048R/89D8547E 2009-09-30 | | BSD since FreeBSD 4.2 Linux since Slackware 2.1 | | 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E | +----------------------------------------------------%} -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iQEcBAEBAgAGBQJKxoxtAAoJEJBXh4mJ2FR+BLQIAIm4nAh8TinDB/QOI6RX2xxO CSv46ZxoRlr2uv3FF5LmIVhPt0tskSrO+WLP0Xjm2ORB05tiFRpbzMBRawH41J1p 0USI90j+y9UzXinGRX9vt3GAofRkfuQuXXMUMAwTCZY1+EyzOP/K0dfRTSTj24LH 386epgCU3FA8S9UqKSPSdpQNxf+Yq/urd6ykfOTtcMUh/m2bakYIgwtVb4zOe+34 lpTlsXxuPcv9WtcOkqkj8LhZgFYKTRajfiw/G8cCnHqlaKuSDSH1hPEu7ePUAC5o wj6TZWh186astBg2WtfIke5zKKQz2ELyT5a3GvhWxR4/l9QWN5F0ZX7TuzaWK1M= =vtNQ -----END PGP SIGNATURE----- From jruohonen at iki.fi Sat Oct 3 08:37:51 2009 From: jruohonen at iki.fi (Jukka Ruohonen) Date: Sat Oct 3 08:37:59 2009 Subject: Distributed SSH attack In-Reply-To: <4AC66E07.4030605@FreeBSD.org> References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> Message-ID: <20091003081335.GA19914@marx.net.bit> On Fri, Oct 02, 2009 at 05:17:59PM -0400, Greg Larkin wrote: > You could set up DenyHosts and contribute to the pool of IPs that are > attempting SSH logins on the Net: > http://denyhosts.sourceforge.net/faq.html#4_0 While I am well aware that a lot of people use DenyHosts or some equivalent tool, I've always been somewhat skeptical about these tools. Few issues: 1. Firewalls should generally be as static as is possible. There is a reason why high securelevel prevents modifications to firewalls. 2. Generally you do not want some parser to modify your firewall rules. Parsing log entries created by remote unauthenticated users as root is never a good idea. 3. Doing (2) increases the attack surface. 4. There have been well-documented cases where (3) has opened opportunities for both remote and local DoS. Two cents, as they say, Jukka. From rb at gid.co.uk Sat Oct 3 11:43:59 2009 From: rb at gid.co.uk (Bob Bishop) Date: Sat Oct 3 11:44:07 2009 Subject: Distributed SSH attack In-Reply-To: <20091003081335.GA19914@marx.net.bit> References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> <20091003081335.GA19914@marx.net.bit> Message-ID: Hi, On 3 Oct 2009, at 09:13, Jukka Ruohonen wrote: > While I am well aware that a lot of people use DenyHosts or some > equivalent > tool, I've always been somewhat skeptical about these tools. Few > issues: > > 1. Firewalls should generally be as static as is possible. There is > a reason > why high securelevel prevents modifications to firewalls. > > 2. Generally you do not want some parser to modify your firewall > rules. > Parsing log entries created by remote unauthenticated users as > root is > never a good idea. > > 3. Doing (2) increases the attack surface. > > 4. There have been well-documented cases where (3) has opened > opportunities > for both remote and local DoS. > > Two cents, as they say, > > Jukka. Blackhole routes can be added as an alternative to tweaking firewall rules. The other objections (esp. 3) still apply of course, but these attacks are such a PITA (noise in the logs if nothing else) that one has to do something. -- Bob Bishop rb@gid.co.uk From kraduk at googlemail.com Sat Oct 3 10:03:30 2009 From: kraduk at googlemail.com (krad) Date: Sat Oct 3 13:21:12 2009 Subject: Distributed SSH attack In-Reply-To: <20091003081335.GA19914@marx.net.bit> References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> <20091003081335.GA19914@marx.net.bit> Message-ID: 2009/10/3 Jukka Ruohonen > On Fri, Oct 02, 2009 at 05:17:59PM -0400, Greg Larkin wrote: > > You could set up DenyHosts and contribute to the pool of IPs that are > > attempting SSH logins on the Net: > > http://denyhosts.sourceforge.net/faq.html#4_0 > > While I am well aware that a lot of people use DenyHosts or some equivalent > tool, I've always been somewhat skeptical about these tools. Few issues: > > 1. Firewalls should generally be as static as is possible. There is a > reason > why high securelevel prevents modifications to firewalls. > > 2. Generally you do not want some parser to modify your firewall rules. > Parsing log entries created by remote unauthenticated users as root is > never a good idea. > > 3. Doing (2) increases the attack surface. > > 4. There have been well-documented cases where (3) has opened opportunities > for both remote and local DoS. > > Two cents, as they say, > > Jukka. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > simplest this to do is disable password auth, and use key based. From doconnor at gsoft.com.au Sat Oct 3 14:27:09 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Sat Oct 3 14:27:17 2009 Subject: Distributed SSH attack In-Reply-To: References: <20091002201039.GA53034@flint.openpave.org> <20091003081335.GA19914@marx.net.bit> Message-ID: <200910032357.02207.doconnor@gsoft.com.au> On Sat, 3 Oct 2009, krad wrote: > simplest this to do is disable password auth, and use key based. Your logs are still full of crap though. I find sshguard works well, and I am fairly sure you couldn't spoof a valid TCP connection through pf sanitising so it would be difficult (nigh-impossible?) for someone to cause you to block a legit IP. If you can, changing the port sshd runs on is by far the simplest work around. Galling as it is to have to change stuff to work around malicious assholes.. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091003/c8f18f56/attachment.pgp From attilio at freebsd.org Sat Oct 3 20:45:37 2009 From: attilio at freebsd.org (Attilio Rao) Date: Sat Oct 3 20:45:44 2009 Subject: sx locks and memory barriers In-Reply-To: <200909291731.32394.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <200909291953.36373.max@love2party.net> <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> <200909291731.32394.jhb@freebsd.org> Message-ID: <3bbf2fe10910031345n5ea54f40i1ca4dd6a1d42ff13@mail.gmail.com> 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 4:42:13 pm Attilio Rao wrote: >> 2009/9/29 Max Laier : >> > On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: >> >> 2009/9/25 Fabio Checconi : >> >> > Hi all, >> >> > looking at sys/sx.h I have some troubles understanding this comment: >> >> > >> >> > * A note about memory barriers. Exclusive locks need to use the same >> >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> >> > * and _rel when releasing an exclusive lock. On the other side, >> >> > * shared lock needs to use an _acq barrier when acquiring the lock >> >> > * but, since they don't update any locked data, no memory barrier is >> >> > * needed when releasing a shared lock. >> >> > >> >> > In particular, I'm not understanding what prevents the following sequence >> >> > from happening: >> >> > >> >> > CPU A CPU B >> >> > >> >> > sx_slock(&data->lock); >> >> > >> >> > sx_sunlock(&data->lock); >> >> > >> >> > /* reordered after the unlock >> >> > by the cpu */ >> >> > if (data->buffer) >> >> > sx_xlock(&data->lock); >> >> > free(data->buffer); >> >> > data->buffer = NULL; >> >> > sx_xunlock(&data->lock); >> >> > >> >> > a = *data->buffer; >> >> > >> >> > IOW, even if readers do not modify the data protected by the lock, >> >> > without a release barrier a memory access may leak past the unlock (as >> >> > the cpu won't notice any dependency between the unlock and the fetch, >> >> > feeling free to reorder them), thus potentially racing with an exclusive >> >> > writer accessing the data. >> >> > >> >> > On architectures where atomic ops serialize memory accesses this would >> >> > never happen, otherwise the sequence above seems possible; am I missing >> >> > something? >> >> >> >> I think your concerns are right, possibly we need this patch: >> >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> >> >> >> However speaking with John we agreed possibly there is a more serious >> >> breakage. Possibly, memory barriers would also require to ensure the >> >> compiler to not reorder the operation, while right now, in FreeBSD, they >> >> just take care of the reordering from the architecture perspective. >> >> The only way I'm aware of GCC offers that is to clobber memory. >> >> I will provide a patch that address this soon, hoping that GCC will be >> >> smart enough to not overhead too much the memory clobbering but just >> >> try to understand what's our purpose and servers it (I will try to >> >> compare code generated before and after the patch at least for tier-1 >> >> architectures). >> > >> > Does GCC really reorder accesses to volatile objects? The C Standard seems to >> > object: >> > >> > 5.1.2.3 - 2 >> > Accessing a volatile object, modifying an object, modifying a file, or calling >> > a function that does any of those operations are all side effects,11) which >> > are changes in the state of the execution environment. Evaluation of an >> > expression may produce side effects. At certain specified points in the >> > execution sequence called sequence points, all side effects of previous >> > evaluations shall be complete and no side effects of subsequent evaluations >> > shall have taken place. (A summary of the sequence points is given in annex >> > C.) >> >> Very interesting. >> I was thinking about the other operating systems which basically do >> 'memory clobbering' for ensuring a compiler barrier, but actually they >> often forsee such a barrier without the conjuction of a memory >> operand. >> >> I think I will need to speak a bit with a GCC engineer in order to see >> what do they implement in regard of volatile operands. > > GCC can be quite aggressive with reordering even in the face of volatile. I > was recently doing a hack to export some data from the kernel to userland > that used a spin loop to grab a snapshot of the contents of a structure > similar to the method used in the kernel with the timehands structures. It > used a volatile structure exposed from the kernel that looked something > like: > > struct foo { > volatile int gen; > /* other stuff */ > }; > > volatile struct foo *p; > > do { > x = p->gen; > /* read other stuff */ > y = p->gen; > } while (x != y && x != 0); > > GCC moved the 'y = ' up into the middle of the '/* read other stuff */'. > I eventually had to add explicit "memory" clobbers to force GCC to not > move the reads of 'gen' around but do them "around" all the other > operations, so that the working code is: > > do { > x = p->gen; > asm volatile("" ::: "memory"); > /* read other stuff */ > asm volatile("" ::: "memory"); > y = p->gen; > } while (x != y && x != 0); The situation was not so desperate as I first thought. Actually, only ia32 and amd64 seems affected by the missing of memory clobbering because it is already done for all the other platform when using all the memory barriers. On ia32 and amd64 too, the impact is more limited than what I expected. atomic_cmpset_* already clobbers the memory and only atomic_store_rel_* is not adeguately protected among the atomics used in our locking primitives, thus I would really expect a limited performance penalty if any. What was not really protected where the functions defined through ATOMIC_ASM() and that was the larger part to fix. I spoke briefly about the compiler support with Christian Lattner (@Apple, LLVM leader) and he mentioned he was aware of the aggressive behaviour of GCC pointing me in the direction that what the C Standard really mandates is that read/write operations with non-volatile operands can be reordered across a volatile operand but that the read/write of volatile operands are strong ordered in regard of eachother. This however means that we have to fix the 'memory clobber' for GCC-simil compilers and also offer a support for the other that let them specify a memory barrier. I then wrote this patch: http://www.freebsd.org/~attilio/atomic.h.diff That should address all the concern raised and also forsee the possibility for other compiler to specify memory barriers semantics differently from normal atomic. rdivacky@ kindly already tested the patch on LLVM/CLANG reporting no problems. I still didn't compare pickly the produced binaries, but I see a little increase in the binary sizes probabilly caming from the .text growing. Attilio -- Peace can only be achieved by understanding - A. Einstein From killing at multiplay.co.uk Sun Oct 4 02:55:47 2009 From: killing at multiplay.co.uk (Steven Hartland) Date: Sun Oct 4 02:55:53 2009 Subject: How to prevent 64bit library paths being searched for 32bit binaries on amd64? Message-ID: <01693A2B7EE64DA78F373BC113A9CF92@multiplay.co.uk> I've just been trying to get a 32bit binary to work on amd64 7.0-RELEASE, the binary had some qt4 dependencies so I set about copying these into /usr/local/lib32 but it seems that for some reason /usr/local/lib is searched first which obviously causes problems when here is a 64bit library of the same name. /etc/rc.d/ldconfig start shows:- ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/mysql /usr/local/lib/pth 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32 /usr/local/lib32/mysql /usr/local/lib32/qt4 LD_LIBARY_PATH isn't set So the question is why is /usr/local/lib searched for a 32bit binary? Is it possible the binary itself is setting it and if so how to I override it? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From delphij at delphij.net Sun Oct 4 08:35:21 2009 From: delphij at delphij.net (Xin LI) Date: Sun Oct 4 08:35:29 2009 Subject: Distributed SSH attack In-Reply-To: <200910032357.02207.doconnor@gsoft.com.au> References: <20091002201039.GA53034@flint.openpave.org> <20091003081335.GA19914@marx.net.bit> <200910032357.02207.doconnor@gsoft.com.au> Message-ID: <4AC85E3B.4040906@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel O'Connor wrote: > On Sat, 3 Oct 2009, krad wrote: >> simplest this to do is disable password auth, and use key based. > > Your logs are still full of crap though. > > I find sshguard works well, and I am fairly sure you couldn't spoof a > valid TCP connection through pf sanitising so it would be difficult > (nigh-impossible?) for someone to cause you to block a legit IP. > > If you can, changing the port sshd runs on is by far the simplest work > around. Galling as it is to have to change stuff to work around > malicious assholes.. Believe it or not, I find this pf.conf rule very effective to mitigate this type of distributed SSH botnet attack: block in quick proto tcp from any os "Linux" to any port ssh Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkrIXjsACgkQi+vbBBjt66DjhACeOJTIYbDuvAjIgYDrQ41aJcw8 +lsAoJhoUOoSL1k4Y/n/UDwqZNSUxId2 =wdkL -----END PGP SIGNATURE----- From danger at FreeBSD.org Sun Oct 4 14:44:59 2009 From: danger at FreeBSD.org (Daniel Gerzo) Date: Sun Oct 4 14:45:12 2009 Subject: [Fwd: HEADSUP: Call for FreeBSD Status Reports] Message-ID: <4AC8B4E8.7080106@FreeBSD.org> Hello guys, I would like to remind you about the FreeBSD Status Reports. The deadline is set to October 7th, and to date I have received only 5 reports, which is very little (considering the fact we are now covering almost 6 months). If you think you have anything to share with the community through the status report entry, please email us at monthly@freebsd.org as soon as possible. It really doesn't have to be a whole article, just a few lines about what you are working on. Thank you! -------- Original Message -------- Subject: HEADSUP: Call for FreeBSD Status Reports Date: Thu, 24 Sep 2009 17:07:29 +0200 From: Daniel Gerzo Organization: The FreeBSD Project To: current@freebsd.org, hackers@freebsd.org, stable@freebsd.org Dear all, I would like to remind you to submit your status reports as soon as possible. Long time has passed since the last status reports were released; and surely a lot has had happened since then. Our developers are relaxed after DevSummit and EuroBSDCon in Cambridge, which both were great! I believe a lot of stuff has been discussed during these events (I hope we will have reports covering this too) and since the last report a lot of things have happened. During that time, two other conferences have been held (BSDCan and AsiaBSDCon), we have released 7.2, not to mention that 8.0 is behind the door. Google Summer of Code should be finished by now too, and we would like to hear about its results. Surely there are a lot more projects which are currently being worked on, so please do not hesitate and write us a few lines - a short description about what you are working on, what are the plans and goals, so we can inform our community about your great work! It's useful for you as well as our users! Please note, the submissions for this quarter (well...rather halfyear, because we should now cover 4-9/2009) are due by October 7th, 2009. Please post the filled-in XML template to be found at http://www.freebsd.org/news/status/report-sample.xml to monthly@FreeBSD.org, or alternatively use our web based form at http://www.freebsd.org/cgi/monthly.cgi. We are looking forward to see your submissions! -- S pozdravom / Best regards Daniel Gerzo, FreeBSD committer From alex at trull.org Sun Oct 4 14:35:59 2009 From: alex at trull.org (Alex Trull) Date: Sun Oct 4 14:57:20 2009 Subject: (Ab)using rcng's features to keep rc.d-style services running should they fail. Message-ID: <20091004141118.GG95662@syndicate.internationalconspiracy.org> Hi all, I realised that because portupgrade/portmaster don't always cleanly restart processes that have died due to being upgraded (mysqld, often!) that this was something I wanted to fix. However, I'd seen the daemontools and wasn't a fan - too much to configure with weird directories and so forth and while monit is very powerful it also takes too much effort to do what could be so much simpler. So why not just (ab)use the rcng system with a script ? the functionality is all there already to do almost everything needed. To check whether something is running and (if not!) start it. So this is my dirty hack so far - runs out of cron with "2>/dev/null" every few minutes and mails me about attempted startups as they happen : #!/bin/sh # start things that should be running find /usr/local/etc/rc.d/ /etc/rc.d/ -type f | egrep -v '(newsyslog|devd|sendmail)' | awk '{print $0" status| grep \"is not running\" && "$0" start"}' | sh Performance is not stunning, thankfuly my cpus are quite idle. real 0m1.198s user 0m0.610s sys 0m0.877s (devd, newsyslog and sendmail are left out because their scripts don't behave quite right.) Initialy I used it purely for the /usr/local/etc/rc.d but I had a base ntpd die on me one evening so decided to throw in /etc/rc.d/ too. This script has also caught a few other failures in port-installed daemons in addition to the ever-common mysqld-upgraded one. Of course it is relatively inefficient executing all those scripts on a regular basis - but it works - has anyone thought of cleaner/more efficient ways of doing this and getting more out of the rcng framework ? Or simpler for that matter. -- Alex -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091004/a6d7598f/attachment.pgp From to.my.trociny at gmail.com Sun Oct 4 18:40:41 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Sun Oct 4 18:41:15 2009 Subject: crashtar Message-ID: <86pr9380kd.fsf@kopusha.onet> Hi, http://trociny.googlecode.com/files/crashtar This simple script is useful for me and might be useful for other people too. The script creates tar archive that contains all files needed for debugging FreeBSD kernel crash (vmcore, kernel, loaded modules, sources that appear in backtrace). This is useful e.g. for debugging a crash on another host, sending it to developers or if you are going to upgrade the kernel on crashed host but would like to keep crashdump in case the developers ask you to provide additional info. Created tar contains also a script that when being run inside unpacked archive will give kgdb(1) session with crash core loaded in it. The script should be run with root privileges because it does chroot(8) before starting kgdb(1). I think I don't have to warn here that a crashdump may be sent only to person you trust :-). -- Mikolaj Golub From brampton+freebsd-hackers at gmail.com Sun Oct 4 19:16:22 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Sun Oct 4 19:16:29 2009 Subject: Make top display thread IDs Message-ID: Hello, I was using "top -H" to display all the different threads on my system. I then wanted to use cpuset to pin a thread to a particular core, however, I couldn't find the thread ID. So I've hacked top to display thread IDs. Hopefully this patch is useful to something, and perhaps it should be included with FreeBSD. I'd be grateful for any feedback or suggestions. thanks Andrew -------------- next part -------------- Index: usr.bin/top/machine.c =================================================================== --- usr.bin/top/machine.c (revision 197611) +++ usr.bin/top/machine.c (working copy) @@ -108,18 +108,18 @@ static char smp_header_thr[] = " PID%s %-*.*s THR PRI NICE SIZE RES STATE C TIME %6s COMMAND"; static char smp_header[] = - " PID%s %-*.*s " "PRI NICE SIZE RES STATE C TIME %6s COMMAND"; + " PID TID%s %-*.*s PRI NICE SIZE RES STATE C TIME %6s COMMAND"; #define smp_Proc_format \ - "%5d%s %-*.*s %s%3d %4s%7s %6s %-6.6s %2d%7s %5.2f%% %.*s" + "%5d%s%s %-*.*s %s%3d %4s%7s %6s %-6.6s %2d%7s %5.2f%% %.*s" static char up_header_thr[] = " PID%s %-*.*s THR PRI NICE SIZE RES STATE TIME %6s COMMAND"; static char up_header[] = - " PID%s %-*.*s " "PRI NICE SIZE RES STATE TIME %6s COMMAND"; + " PID TID%s %-*.*s PRI NICE SIZE RES STATE TIME %6s COMMAND"; #define up_Proc_format \ - "%5d%s %-*.*s %s%3d %4s%7s %6s %-6.6s%.0d%7s %5.2f%% %.*s" + "%5d%s%s %-*.*s %s%3d %4s%7s %6s %-6.6s%.0d%7s %5.2f%% %.*s" /* process state names for the "STATE" column of the display */ @@ -757,7 +757,7 @@ int state; struct rusage ru, *rup; long p_tot, s_tot; - char *proc_fmt, thr_buf[6], jid_buf[6]; + char *proc_fmt, tid_buf[8], thr_buf[6], jid_buf[6]; char *cmdbuf = NULL; char **args; @@ -942,14 +942,19 @@ /* format this entry */ proc_fmt = smpmode ? smp_Proc_format : up_Proc_format; - if (ps.thread != 0) + if (ps.thread) { thr_buf[0] = '\0'; - else + snprintf(tid_buf, sizeof(tid_buf), "%*d", + sizeof(tid_buf) - 1, pp->ki_tid); + } else { + tid_buf[0] = '\0'; snprintf(thr_buf, sizeof(thr_buf), "%*d ", sizeof(thr_buf) - 2, pp->ki_numthreads); + } sprintf(fmt, proc_fmt, pp->ki_pid, + tid_buf, jid_buf, namelength, namelength, (*get_userid)(pp->ki_ruid), thr_buf, From dougb at FreeBSD.org Sun Oct 4 19:30:58 2009 From: dougb at FreeBSD.org (Doug Barton) Date: Sun Oct 4 19:31:17 2009 Subject: (Ab)using rcng's features to keep rc.d-style services running should they fail. In-Reply-To: <20091004141118.GG95662@syndicate.internationalconspiracy.org> References: <20091004141118.GG95662@syndicate.internationalconspiracy.org> Message-ID: <4AC8F7EF.9010303@FreeBSD.org> Alex Trull wrote: > Hi all, > > I realised that because portupgrade/portmaster don't always > cleanly restart processes that have died due to being > upgraded (mysqld, often!) that this was something I wanted > to fix. I can't speak to portupgrade, however for portmaster there is no such facility whatsoever. The admin is expected to disable things prior to an upgrade and re-enable them when the upgrade is done. I don't feel that this is an overwhelming burden. :) That said I have it in mind to add a facility to handle this feature. Stay tuned for more news about this. > So why not just (ab)use the rcng system with a script ? First, it's rc.d now if you please. Second, I don't think that there is anything wrong with your concept that would classify it as abuse, although I'm not sure I would have implemented it in quite the same way. > the > functionality is all there already to do almost everything > needed. To check whether something is running and (if not!) > start it. > > So this is my dirty hack so far - runs out of cron with > "2>/dev/null" every few minutes and mails me about attempted > startups as they happen : > > #!/bin/sh > # start things that should be running > find /usr/local/etc/rc.d/ /etc/rc.d/ -type f | egrep -v '(newsyslog|devd|sendmail)' | awk '{print $0" status| grep \"is not running\" && "$0" start"}' | sh There are a couple of "problems" with this, although please understand I'm not criticizing, I'm just offering what I hope are constructive suggestions. First, (and I consider this to be a bug) there are several scripts in /etc/rc.d that are not actually 'startup' scripts in the true sense. Therefore I would not attempt to run them all. Personally if I were going to do what you're doing I would make an explicit list of scripts I wanted to test for. If you are going to continue to use awk you might want to learn how to avoid piping it to grep, that's an extra subshell that you don't really need. Finally I would do something like this (untested): for service in ntpd mysqld foo bar; do if [ -x /usr/local/etc/rc.d/$service ]; then service=/usr/local/etc/rc.d/$service elif [ -x /etc/rc.d/$service ]; then service=/etc/rc.d/$service else echo "Cannot find $service in /etc/rc.d or /usr/local/etc/rc.d" exit 1 fi $service start | grep -v 'already running' done > (devd, newsyslog and sendmail are left out because their > scripts don't behave quite right.) I don't see anything wrong with devd's output from the status command, sendmail's is a little hard to parse because it's doing a lot of things in one script. newsyslog is spitting out 'not running' which arguably it should not do since that script is not for starting a persistent service, it's just a 'run at boot' thing. In any case, if you find what you think are bugs in rc.d related stuff feel free to report them to freebsd-rc@freebsd.org. hth, Doug -- This .signature sanitized for your protection From mlfbsd at ci0.org Sun Oct 4 22:32:20 2009 From: mlfbsd at ci0.org (Olivier Houchard) Date: Mon Oct 5 04:12:31 2009 Subject: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' In-Reply-To: <4AC21F44.6060004@tomjudge.com> References: <4AC106AA.9000305@tomjudge.com> <20090928202132.GA15236@ci0.org> <4AC16B5A.8090407@tomjudge.com> <20090929093825.GA26424@ci0.org> <4AC21F44.6060004@tomjudge.com> Message-ID: <20091004223433.GA73189@ci0.org> > Hi Olivier, > > I have tried the patch and here are the boot results: > > > i80321: BAR0 = 20000004.00000000 BAR1 = 40000004.00000000 > i80219: BAR0 = 20000000.00000000 BAR1 = 40000000.00000000 > i80219: I/O Processor, acting as PCI host > i80321: SBDR = 0xa0000000 SBR0 = 0x00000018 SBR1 = 0x00000020 > i80321: BANK0 = 0x10000000 BANK1 = 0x10000000 > i80321: Reserve space for private devices (Inbound Window 1) > hi:0x00000000 lo:0x8000000c xlate:0x80000000 size:0x04000000 > i80321: RAM access (Inbound Window 2) > hi:0x00000000 lo:0xa000000c xlate:0xa0000000 size:0x20000000 > obio0 on iq0 > uart0: <16550 or compatible> on obio0 > uart0: [FILTER] > uart0: console (115200,n,8,1) > itimer0: on iq0 > iopwdog0: on iq0 > pcib0: on iq0 > pci0: on pcib0 > Device 1 routed to irq 27 > Device 2 routed to irq 30 > Device 3 routed to irq 29 > Device 5 routed to irq 30 > Device 5 routed to irq 29 > Device 5 routed to irq 27 > em0: port > 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 27 at device 1.0 > on pci0 > em0: Start: 0x00000000 > em0: End: 0x0001FFFF > em0: Size: 0x00020000 > Fatal kernel mode data abort: 'External Linefetch Abort (P)' > trapframe: 0xc00faad0 > FSR=00000406, FAR=Invalid, spsr=200000d3 > r0 =c00d0400, r1 =cd5bf000, r2 =00000010, r3 =0000000a > r4 =c317e008, r5 =cd5bf000, r6 =c00d0400, r7 =c130212c > r8 =c317e008, r9 =c0071180, r10=c317e000, r11=c00fab40 > r12=c00fab44, ssp=c00fab1c, slr=c106a96c, pc =c106a968 > > [thread pid 0 tid 100000 ] > Stopped at e1000_init_script_state_82541+0x24c: blx r7 > db> > > > > As you can see I added some debug to if_em.c as such: > > Index: sys/dev/e1000/if_em.c > =================================================================== > --- sys/dev/e1000/if_em.c (revision 197472) > +++ sys/dev/e1000/if_em.c (working copy) > @@ -2770,6 +2770,9 @@ > rman_get_bustag(adapter->memory); > adapter->osdep.mem_bus_space_handle = > rman_get_bushandle(adapter->memory); > + device_printf(dev,"Start: 0x%08lX\n", rman_get_start(adapter->memory)); > + device_printf(dev,"End: 0x%08lX\n", rman_get_end(adapter->memory)); > + device_printf(dev,"Size: 0x%08lX\n", rman_get_size(adapter->memory)); > adapter->hw.hw_addr = (u8 *)&adapter->osdep.mem_bus_space_handle; > > /* Only older adapters use IO mapping */ > > > But the memory mapping seems to be missing the most significant 0x8. > I fail to see how it happens. Could you printf the value of sc->sc_mem once set in i80321_pci_attach(), and if it appears to be 0, the value of i80321_softc->sc_owin[0].owin_xlate_lo at the different points it can be set ? Thanks a lot, Olivier From to.my.trociny at gmail.com Mon Oct 5 05:48:11 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Mon Oct 5 05:48:17 2009 Subject: crashinfo: print the content of ddb capture budder Message-ID: <81ws3a1je1.fsf@zhuzha.ua1> Hi, It would be nice if crashinfo(8) were also trying to output the content of ddb capture buffer. Something like in this patch: --- crashinfo.sh.orig 2009-10-05 08:26:26.000000000 +0300 +++ crashinfo.sh 2009-10-05 08:43:56.000000000 +0300 @@ -304,3 +304,18 @@ echo "kernel config" echo config -x $KERNEL + +file=`mktemp /tmp/crashinfo.XXXXXX` +if [ $? -eq 0 ]; then + ddb capture -M $VMCORE -N $KERNEL print > $file 2>/dev/null + if [ -s $file ]; then + echo "------------------------------------------------------------------------" + echo "ddb capture buffer" + echo + cat $file | + sed -e 's/p\{10\}p*//' # XXX: this removes the unfilled part of a capture buffer + echo + fi + rm -f $file +fi + -- Mikolaj Golub From tom at tomjudge.com Mon Oct 5 13:48:28 2009 From: tom at tomjudge.com (Tom Judge) Date: Mon Oct 5 13:48:36 2009 Subject: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' In-Reply-To: <20091004223433.GA73189@ci0.org> References: <4AC106AA.9000305@tomjudge.com> <20090928202132.GA15236@ci0.org> <4AC16B5A.8090407@tomjudge.com> <20090929093825.GA26424@ci0.org> <4AC21F44.6060004@tomjudge.com> <20091004223433.GA73189@ci0.org> Message-ID: <4AC9F906.3040306@tomjudge.com> Olivier Houchard wrote: >> Hi Olivier, >> >> I have tried the patch and here are the boot results: >> >> >> > > I fail to see how it happens. > Could you printf the value of sc->sc_mem once set in i80321_pci_attach(), > and if it appears to be 0, the value of i80321_softc->sc_owin[0].owin_xlate_lo > at the different points it can be set ? > > Thanks a lot, > > Olivier > Hi Olivier, I have been working though this with Mark Tinguely and we came up with this patch that makes it work. Not sure on its correctness but it works. Tom -------------- next part -------------- Index: i80321_pci.c =================================================================== --- i80321_pci.c (revision 197735) +++ i80321_pci.c (working copy) @@ -92,8 +92,7 @@ sc->sc_busno = busno; sc->sc_pciio = &i80321_softc->sc_pci_iot; sc->sc_pcimem = &i80321_softc->sc_pci_memt; - sc->sc_mem = i80321_softc->sc_owin[0].owin_xlate_lo + - VERDE_OUT_XLATE_MEM_WIN_SIZE; + sc->sc_mem = i80321_softc->sc_owin[0].owin_xlate_lo; sc->sc_io = i80321_softc->sc_iow_vaddr; /* Initialize memory and i/o rmans. */ @@ -110,7 +109,8 @@ sc->sc_mem_rman.rm_descr = "I80321 PCI Memory"; if (rman_init(&sc->sc_mem_rman) != 0 || rman_manage_region(&sc->sc_mem_rman, - 0, VERDE_OUT_XLATE_MEM_WIN_SIZE) != 0) { + VERDE_OUT_XLATE_MEM_WIN0_BASE, + VERDE_OUT_XLATE_MEM_WIN0_BASE + VERDE_OUT_XLATE_MEM_WIN_SIZE) != 0) { panic("i80321_pci_probe: failed to set up memory rman"); } sc->sc_irq_rman.rm_type = RMAN_ARRAY; @@ -297,6 +297,9 @@ sc->sc_mem; start &= (0x1000000 - 1); end &= (0x1000000 - 1); + start += 0x80000000; + end += 0x80000000; + device_printf(child, "SYS_RES_MEMORY: start: 0x%08lX end: 0x%08lX\n",start,end); break; case SYS_RES_IOPORT: rm = &sc->sc_io_rman; @@ -312,12 +315,15 @@ } rv = rman_reserve_resource(rm, start, end, count, flags, child); + device_printf(child, "RMAN_RESERVE_RESOURCE: start: 0x%08lX end: 0x%08lX\n", + rman_get_start(rv), + rman_get_end(rv)); if (rv == NULL) return (NULL); rman_set_rid(rv, *rid); if (type != SYS_RES_IRQ) { if (type == SYS_RES_MEMORY) - bh += (rman_get_start(rv)); + bh = (rman_get_start(rv)); rman_set_bustag(rv, bt); rman_set_bushandle(rv, bh); if (flags & RF_ACTIVE) { From jhb at FreeBSD.org Mon Oct 5 13:53:42 2009 From: jhb at FreeBSD.org (John Baldwin) Date: Mon Oct 5 13:53:49 2009 Subject: sx locks and memory barriers In-Reply-To: <3bbf2fe10910031345n5ea54f40i1ca4dd6a1d42ff13@mail.gmail.com> References: <20090924224935.GW473@gandalf.sssup.it> <200909291953.36373.max@love2party.net> <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> <200909291731.32394.jhb@freebsd.org> <3bbf2fe10910031345n5ea54f40i1ca4dd6a1d42ff13@mail.gmail.com> Message-ID: <4AC9FA63.9020606@FreeBSD.org> Attilio Rao wrote: > 2009/9/29 John Baldwin : >> On Tuesday 29 September 2009 4:42:13 pm Attilio Rao wrote: >>> 2009/9/29 Max Laier : >>>> On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: >>>>> 2009/9/25 Fabio Checconi : >>>>>> Hi all, >>>>>> looking at sys/sx.h I have some troubles understanding this comment: >>>>>> >>>>>> * A note about memory barriers. Exclusive locks need to use the same >>>>>> * memory barriers as mutexes: _acq when acquiring an exclusive lock >>>>>> * and _rel when releasing an exclusive lock. On the other side, >>>>>> * shared lock needs to use an _acq barrier when acquiring the lock >>>>>> * but, since they don't update any locked data, no memory barrier is >>>>>> * needed when releasing a shared lock. >>>>>> >>>>>> In particular, I'm not understanding what prevents the following sequence >>>>>> from happening: >>>>>> >>>>>> CPU A CPU B >>>>>> >>>>>> sx_slock(&data->lock); >>>>>> >>>>>> sx_sunlock(&data->lock); >>>>>> >>>>>> /* reordered after the unlock >>>>>> by the cpu */ >>>>>> if (data->buffer) >>>>>> sx_xlock(&data->lock); >>>>>> free(data->buffer); >>>>>> data->buffer = NULL; >>>>>> sx_xunlock(&data->lock); >>>>>> >>>>>> a = *data->buffer; >>>>>> >>>>>> IOW, even if readers do not modify the data protected by the lock, >>>>>> without a release barrier a memory access may leak past the unlock (as >>>>>> the cpu won't notice any dependency between the unlock and the fetch, >>>>>> feeling free to reorder them), thus potentially racing with an exclusive >>>>>> writer accessing the data. >>>>>> >>>>>> On architectures where atomic ops serialize memory accesses this would >>>>>> never happen, otherwise the sequence above seems possible; am I missing >>>>>> something? >>>>> I think your concerns are right, possibly we need this patch: >>>>> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >>>>> >>>>> However speaking with John we agreed possibly there is a more serious >>>>> breakage. Possibly, memory barriers would also require to ensure the >>>>> compiler to not reorder the operation, while right now, in FreeBSD, they >>>>> just take care of the reordering from the architecture perspective. >>>>> The only way I'm aware of GCC offers that is to clobber memory. >>>>> I will provide a patch that address this soon, hoping that GCC will be >>>>> smart enough to not overhead too much the memory clobbering but just >>>>> try to understand what's our purpose and servers it (I will try to >>>>> compare code generated before and after the patch at least for tier-1 >>>>> architectures). >>>> Does GCC really reorder accesses to volatile objects? The C Standard seems to >>>> object: >>>> >>>> 5.1.2.3 - 2 >>>> Accessing a volatile object, modifying an object, modifying a file, or calling >>>> a function that does any of those operations are all side effects,11) which >>>> are changes in the state of the execution environment. Evaluation of an >>>> expression may produce side effects. At certain specified points in the >>>> execution sequence called sequence points, all side effects of previous >>>> evaluations shall be complete and no side effects of subsequent evaluations >>>> shall have taken place. (A summary of the sequence points is given in annex >>>> C.) >>> Very interesting. >>> I was thinking about the other operating systems which basically do >>> 'memory clobbering' for ensuring a compiler barrier, but actually they >>> often forsee such a barrier without the conjuction of a memory >>> operand. >>> >>> I think I will need to speak a bit with a GCC engineer in order to see >>> what do they implement in regard of volatile operands. >> GCC can be quite aggressive with reordering even in the face of volatile. I >> was recently doing a hack to export some data from the kernel to userland >> that used a spin loop to grab a snapshot of the contents of a structure >> similar to the method used in the kernel with the timehands structures. It >> used a volatile structure exposed from the kernel that looked something >> like: >> >> struct foo { >> volatile int gen; >> /* other stuff */ >> }; >> >> volatile struct foo *p; >> >> do { >> x = p->gen; >> /* read other stuff */ >> y = p->gen; >> } while (x != y && x != 0); >> >> GCC moved the 'y = ' up into the middle of the '/* read other stuff */'. >> I eventually had to add explicit "memory" clobbers to force GCC to not >> move the reads of 'gen' around but do them "around" all the other >> operations, so that the working code is: >> >> do { >> x = p->gen; >> asm volatile("" ::: "memory"); >> /* read other stuff */ >> asm volatile("" ::: "memory"); >> y = p->gen; >> } while (x != y && x != 0); > > The situation was not so desperate as I first thought. > Actually, only ia32 and amd64 seems affected by the missing of memory > clobbering because it is already done for all the other platform when > using all the memory barriers. > On ia32 and amd64 too, the impact is more limited than what I > expected. atomic_cmpset_* already clobbers the memory and only > atomic_store_rel_* is not adeguately protected among the atomics used > in our locking primitives, thus I would really expect a limited > performance penalty if any. > What was not really protected where the functions defined through > ATOMIC_ASM() and that was the larger part to fix. > > I spoke briefly about the compiler support with Christian Lattner > (@Apple, LLVM leader) and he mentioned he was aware of the aggressive > behaviour of GCC pointing me in the direction that what the C Standard > really mandates is that read/write operations with non-volatile > operands can be reordered across a volatile operand but that the > read/write of volatile operands are strong ordered in regard of > eachother. This however means that we have to fix the 'memory clobber' > for GCC-simil compilers and also offer a support for the other that > let them specify a memory barrier. > I then wrote this patch: > http://www.freebsd.org/~attilio/atomic.h.diff > > That should address all the concern raised and also forsee the > possibility for other compiler to specify memory barriers semantics > differently from normal atomic. > > rdivacky@ kindly already tested the patch on LLVM/CLANG reporting no problems. > > I still didn't compare pickly the produced binaries, but I see a > little increase in the binary sizes probabilly caming from the .text > growing. This looks good to me. One thing that is missing is that the UP atomic load/store ops still need "memory" clobbers to prevent the compiler from reordering things (you could still have bad juju if you preempted in between the atomic op and the compiler-reordered operation on UP). -- John Baldwin From wxs at FreeBSD.org Mon Oct 5 14:16:41 2009 From: wxs at FreeBSD.org (Wesley Shields) Date: Mon Oct 5 14:16:52 2009 Subject: (Ab)using rcng's features to keep rc.d-style services running should they fail. In-Reply-To: <4AC8F7EF.9010303@FreeBSD.org> References: <20091004141118.GG95662@syndicate.internationalconspiracy.org> <4AC8F7EF.9010303@FreeBSD.org> Message-ID: <20091005135842.GA8629@atarininja.org> On Sun, Oct 04, 2009 at 12:30:55PM -0700, Doug Barton wrote: > Alex Trull wrote: > > Hi all, > > > > I realised that because portupgrade/portmaster don't always > > cleanly restart processes that have died due to being > > upgraded (mysqld, often!) that this was something I wanted > > to fix. > > I can't speak to portupgrade, however for portmaster there is no such > facility whatsoever. The admin is expected to disable things prior to > an upgrade and re-enable them when the upgrade is done. I don't feel > that this is an overwhelming burden. :) There is the @stopdaemon directive in plists (which gets translated into @unexec to forcestop the script). Some ports use it and some do not. Personally I think ports doing this automatically are quite annoying, and would love to rip them all out from the ports. Something like portmaster growing support for it would be welcome provided it does not happen by default. I've always found it funny that there is no @startdaemon directive (rightfully so, as we want people to explicitly turn things on) but it's acceptable if things get turned off via @stopdaemon without explicit permission. If a particular upgrade requires that the thing be not running we should check for that and abort, not go shutting things down. -- WXS From rwmaillists at googlemail.com Mon Oct 5 15:20:49 2009 From: rwmaillists at googlemail.com (RW) Date: Mon Oct 5 15:20:55 2009 Subject: (Ab)using rcng's features to keep rc.d-style services running should they fail. In-Reply-To: <20091004141118.GG95662@syndicate.internationalconspiracy.org> References: <20091004141118.GG95662@syndicate.internationalconspiracy.org> Message-ID: <20091005154729.36044f89@gumby.homeunix.com> On Sun, 4 Oct 2009 15:11:18 +0100 Alex Trull wrote: > Hi all, > > I realised that because portupgrade/portmaster don't always > cleanly restart processes that have died due to being > upgraded (mysqld, often!) that this was something I wanted > to fix. You can configure portupgrade (and FWIW portmanager) to do this. Take look at the sample configuration file. From tom at tomjudge.com Mon Oct 5 17:01:31 2009 From: tom at tomjudge.com (Tom Judge) Date: Mon Oct 5 17:01:51 2009 Subject: Kernel Thread Lock Question Message-ID: <4ACA2645.30801@tomjudge.com> Do I need to hold the per thread lock here? (This is for 7.1) PROC_LOCK(p); //mtx_lock_spin(&sched_lock); breakout = 0; FOREACH_THREAD_IN_PROC(p, td) { thread_lock(td); if (!TD_ON_RUNQ(td) && !TD_IS_RUNNING(td) && !TD_IS_SLEEPING(td)) { breakout = 1; thread_unlock(td); break; } thread_unlock(td); } //mtx_unlock_spin(&sched_lock); PROC_UNLOCK(p); Thanks Tom From julian at elischer.org Mon Oct 5 17:55:26 2009 From: julian at elischer.org (Julian Elischer) Date: Mon Oct 5 17:55:32 2009 Subject: Kernel Thread Lock Question In-Reply-To: <4ACA2645.30801@tomjudge.com> References: <4ACA2645.30801@tomjudge.com> Message-ID: <4ACA330F.7060207@elischer.org> Tom Judge wrote: > Do I need to hold the per thread lock here? (This is for 7.1) > > PROC_LOCK(p); > //mtx_lock_spin(&sched_lock); > breakout = 0; > FOREACH_THREAD_IN_PROC(p, td) { > thread_lock(td); > if (!TD_ON_RUNQ(td) && > !TD_IS_RUNNING(td) && > !TD_IS_SLEEPING(td)) { > breakout = 1; > thread_unlock(td); > break; > } > thread_unlock(td); > } > //mtx_unlock_spin(&sched_lock); > PROC_UNLOCK(p); > "probably not" because the value in the status word can change just after you release it anyhow so your result does not have to be consistent with anything, just with itself. > > Thanks > > Tom > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From das at FreeBSD.ORG Tue Oct 6 19:50:40 2009 From: das at FreeBSD.ORG (David Schultz) Date: Tue Oct 6 19:50:47 2009 Subject: No end to swap zone exhausted problems? In-Reply-To: <20091002202936.GA836@fupp.net> References: <20091002202936.GA836@fupp.net> Message-ID: <20091006193437.GA1196@zim.MIT.EDU> On Fri, Oct 02, 2009, Anders Nordby wrote: > Better post about my swap zone problems here than tear all my hair out. > This concerns me: > > 1) How can SWAPMETA usage continue increasing, while swap space used is > not? Example: http://anders.fupp.net/test/swapmeta.png. In the swap meta > graph, used space is USED and free space is LIMIT-USED (FREE numbers are > a bit odd, it starts with 0) from the SWAPMETA line when running vmstat > -z. In the week graph, you can see that after 3 days swap space usage > settles at around 90 GB, while swap meta keeps increasing. IIRC, the swap pager tracks virtual memory regions that include swapped pages on a granularity of 16 pages. An increase in swapmeta usage could simply mean that swap has become fragmented. It should eventually level out (unless there is actually a leak, but I think we'd know by now). > How come increasing maxswzone beyond 256 MB does not yield a larger > SWAPMETA than 995410? Does it stop around 260 MB maxswzone? How can I > increase SWAPMETA further to be able to swap more? The code appears to limit the number of swapmeta entries (each entry being about 100 bytes) to half the number of pages of physical memory you have (sys/vm/swap_pager.c): n = cnt.v_page_count / 2; if (maxswzone && n > maxswzone / sizeof(struct swblock)) n = maxswzone / sizeof(struct swblock); You could try commenting out the first two of these lines, leaving the third, and see if that lets you raise the limit to something that works better for you. Note that raising it too high is also going to cause problems, as all of your physical memory will be filled with swap metadata. From delphij at delphij.net Tue Oct 6 22:44:52 2009 From: delphij at delphij.net (Xin LI) Date: Tue Oct 6 22:45:09 2009 Subject: (Ab)using rcng's features to keep rc.d-style services running should they fail. In-Reply-To: <20091005135842.GA8629@atarininja.org> References: <20091004141118.GG95662@syndicate.internationalconspiracy.org> <4AC8F7EF.9010303@FreeBSD.org> <20091005135842.GA8629@atarininja.org> Message-ID: <4ACBC857.2030207@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Wesley Shields wrote: > On Sun, Oct 04, 2009 at 12:30:55PM -0700, Doug Barton wrote: >> Alex Trull wrote: >>> Hi all, >>> >>> I realised that because portupgrade/portmaster don't always >>> cleanly restart processes that have died due to being >>> upgraded (mysqld, often!) that this was something I wanted >>> to fix. >> I can't speak to portupgrade, however for portmaster there is no such >> facility whatsoever. The admin is expected to disable things prior to >> an upgrade and re-enable them when the upgrade is done. I don't feel >> that this is an overwhelming burden. :) > > There is the @stopdaemon directive in plists (which gets translated into > @unexec to forcestop the script). Some ports use it and some do not. > Personally I think ports doing this automatically are quite annoying, > and would love to rip them all out from the ports. Something like > portmaster growing support for it would be welcome provided it does not > happen by default. +1 I think this feature should be user-controllable (or, the 'make install' should be 'restart'ing the rc.d script at very least). Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAkrLyFcACgkQi+vbBBjt66DZ5QCfU3LSI+RiZwJv3huFx4wd3QNe UUsAn37vdhs30y+2eE/HLaw424CS7dMh =1FW0 -----END PGP SIGNATURE----- From doconnor at gsoft.com.au Wed Oct 7 02:53:25 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Wed Oct 7 02:53:32 2009 Subject: (Ab)using rcng's features to keep rc.d-style services running should they fail. In-Reply-To: <4ACBC857.2030207@delphij.net> References: <20091004141118.GG95662@syndicate.internationalconspiracy.org> <20091005135842.GA8629@atarininja.org> <4ACBC857.2030207@delphij.net> Message-ID: <200910071322.48738.doconnor@gsoft.com.au> On Wed, 7 Oct 2009, Xin LI wrote: > Wesley Shields wrote: > > On Sun, Oct 04, 2009 at 12:30:55PM -0700, Doug Barton wrote: > >> Alex Trull wrote: > >>> Hi all, > >>> > >>> I realised that because portupgrade/portmaster don't always > >>> cleanly restart processes that have died due to being > >>> upgraded (mysqld, often!) that this was something I wanted > >>> to fix. > >> > >> I can't speak to portupgrade, however for portmaster there is no > >> such facility whatsoever. The admin is expected to disable things > >> prior to an upgrade and re-enable them when the upgrade is done. I > >> don't feel that this is an overwhelming burden. :) > > > > There is the @stopdaemon directive in plists (which gets translated > > into @unexec to forcestop the script). Some ports use it and some > > do not. Personally I think ports doing this automatically are quite > > annoying, and would love to rip them all out from the ports. > > Something like portmaster growing support for it would be welcome > > provided it does not happen by default. > > +1 > > I think this feature should be user-controllable (or, the 'make > install' should be 'restart'ing the rc.d script at very least). It won't actually start anything you haven't enabled in rc.conf though since all ports install rc.d scripts which require FOO_enable to be YES. That said a knob like RESTART_SERVICES or similar would be nice. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091007/84111f61/attachment.pgp From madduck at madduck.net Wed Oct 7 10:06:20 2009 From: madduck at madduck.net (martin f krafft) Date: Wed Oct 7 11:38:12 2009 Subject: Distro Summit 2010: call for papers extended until 18 Oct 2009 Message-ID: <20091007100506.7889434F0@piper.oerlikon.madduck.net> Please redistribute this call for papers. The deadline has been extended to 18 Oct 2009. =============== CALL FOR PAPERS =============== Distro Summit 2010 is a one-day technical conference with a strong focus on collaboration between Free Software distributions hosted at the linux.conf.au 2010. We are looking for proposals from any Free Software distribution, from the typical full distributions (both linux and non-linux) to the niche market derivatives. In spite of the strong focus on collaboration between Free Software distributions, topics may include packaging, maintenance, relationship with upstream developers, release management and QA. For more informtion, please visit: http://distrosummit.org. Important dates =============== * Call for papers ends: Wednesday 18 October 2009 * Announcing the schedule: Friday 2 October 2009 * Conference begins: Monday 18 January 2010 Presentation types ================== We will accept proposals for: * 25 minute standard-length presentations; * 50 minute long presentations. Session lengths include time for audience questions. We intend for standard-length presentation to make up the vast majority of our presentations. If you plan on submitting a proposal for a long presentation, a willingness to present a standard-length presentation will impact positively on your proposal. Submit a proposal ================= To submit your proposal, we'll need the following information: * Your name, contact details and a short biography; * Your proposal title; * Intended audience; * An abstract; * Presentation outline; * Presentation type (standard-length or long). To submit a proposal, or get more information, please write to cfp@distrosummit.org. Unfortunately, we cannot offer sponsorship for speakers. About the Distro Summit ======================= The Distro Summit 2010 is a one-day developer conference with a strong focus on collaboration between free software distributions hosted at the linux.conf.au 2010 (http://www.lca2010.org.nz). In addition to a schedule of technical, social and policy talks, the Distro Summit provides an opportunity for developers, contributors and other interested people to meet in person and work together more closely. Previous similar events have featured speakers from around the world. They have also been extremely beneficial for developing key free software software components and for improving collaboration and sharing between the different distributions. Target Audience =============== The Distro Summit is (mainly) a technical event, but this does not mean that the only target audience are developers and maintainers of free software distributions: the event will feature talks that range from the development to real-world use cases, going through marketing and the social aspects of the maintenance of free software distributions. -- Martin Krafft on the behalf of the Distro Summit organizers From rysto32 at gmail.com Wed Oct 7 17:54:41 2009 From: rysto32 at gmail.com (Ryan Stone) Date: Wed Oct 7 17:54:48 2009 Subject: Make top display thread IDs In-Reply-To: References: Message-ID: If a thread has a name, top -H will display it in parentheses after the executable name. One option would be to print the tid there if the thread has no name. From brampton+freebsd-hackers at gmail.com Wed Oct 7 18:14:33 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Wed Oct 7 18:14:39 2009 Subject: Make top display thread IDs In-Reply-To: References: Message-ID: 2009/10/7 Ryan Stone : > If a thread has a name, top -H will display it in parentheses after > the executable name. ?One option would be to print the tid there if > the thread has no name. > Thanks for your suggestion. I would like the TID always to be displayed, so displaying it when there is no name wouldn't work for me. If you haven't looked at the patch I placed the TID directly after the PID column (when displaying threads in -H mode). thanks Andrew From jandrese at mitre.org Wed Oct 7 21:41:00 2009 From: jandrese at mitre.org (Andresen, Jason R.) Date: Wed Oct 7 21:41:06 2009 Subject: Distributed SSH attack In-Reply-To: <4AC85E3B.4040906@delphij.net> References: <20091002201039.GA53034@flint.openpave.org> <20091003081335.GA19914@marx.net.bit> <200910032357.02207.doconnor@gsoft.com.au> <4AC85E3B.4040906@delphij.net> Message-ID: <600C0C33850FFE49B76BDD81AED4D2580131FCB08C@IMCMBX3.MITRE.ORG> >-----Original Message----- >From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd- >hackers@freebsd.org] On Behalf Of Xin LI >Sent: Sunday, October 04, 2009 4:35 AM >To: Daniel O'Connor >Cc: jruohonen@iki.fi; freebsd-hackers@freebsd.org; krad >Subject: Re: Distributed SSH attack > >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >Daniel O'Connor wrote: >> On Sat, 3 Oct 2009, krad wrote: >>> simplest this to do is disable password auth, and use key based. >> >> Your logs are still full of crap though. >> >> I find sshguard works well, and I am fairly sure you couldn't spoof a >> valid TCP connection through pf sanitising so it would be difficult >> (nigh-impossible?) for someone to cause you to block a legit IP. >> >> If you can, changing the port sshd runs on is by far the simplest work >> around. Galling as it is to have to change stuff to work around >> malicious assholes.. > >Believe it or not, I find this pf.conf rule very effective to mitigate >this type of distributed SSH botnet attack: > >block in quick proto tcp from any os "Linux" to any port ssh How does that work? Does PF do some sort of os fingerprinting on the remote side before allowing the first SYN through? Also, if you have a mix of Linux and FreeBSD boxes, presumably this would not be a great idea right? It's not just getting people who are faking it? >From what I've seen on this attack, it looks like the hosts just send random logins to random IP addresses constantly, so adding an IP address to a blackhole list isn't as effective because you'll be getting hits from thousands of IP addresses, but only a single hit. In fact it looks like this attack is specifically designed to defeat the "I'll add the attacker's IP address to a black hole list" strategy, by coming in on a different address every time. From delphij at delphij.net Wed Oct 7 22:04:52 2009 From: delphij at delphij.net (Xin LI) Date: Wed Oct 7 22:04:58 2009 Subject: Distributed SSH attack In-Reply-To: <600C0C33850FFE49B76BDD81AED4D2580131FCB08C@IMCMBX3.MITRE.ORG> References: <20091002201039.GA53034@flint.openpave.org> <20091003081335.GA19914@marx.net.bit> <200910032357.02207.doconnor@gsoft.com.au> <4AC85E3B.4040906@delphij.net> <600C0C33850FFE49B76BDD81AED4D2580131FCB08C@IMCMBX3.MITRE.ORG> Message-ID: <4ACD107A.5080803@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Anderesen, Andresen, Jason R. wrote: [...] >> Believe it or not, I find this pf.conf rule very effective to mitigate >> this type of distributed SSH botnet attack: >> >> block in quick proto tcp from any os "Linux" to any port ssh > > How does that work? Does PF do some sort of os fingerprinting on the remote side before allowing the first SYN through? Well, this would have pros and cons. pf employs a "fingerprint" mechanism that would passively detect the operating system based on some predefined criteria, and the "Linux" matches several old Linux kernel's TCP fingerprint. Note that with some tweaks to Linux's TCP parameters, or newer Linux kernels, this can be bypassed. However, if the administrator choose to do this, it's not quite likely that their boxes would be part of the botnet. > Also, if you have a mix of Linux and FreeBSD boxes, presumably this > would not be a great idea right? It's not just getting people who > are faking it? Yes and no. Attackers would adopt to whatever defenders trying to stop them, however, for this type of attack (note that blocking Linux from being able to SSH on one system does not mean you would be more safe, it just mitigate the excessive login issue), what the attacker wanted is to have more botnet boxes, and he or she wouldn't care about having 1 more FreeBSD system be there or not, at the expense of faking or tweaking the TCP stack. >> From what I've seen on this attack, it looks like the hosts just >> send random logins to random IP addresses constantly, so adding an >> IP address to a blackhole list isn't as effective because you'll be >> getting hits from thousands of IP addresses, but only a single hit. >> In fact it looks like this attack is specifically designed to >> defeat the "I'll add the attacker's IP address to a black hole >> list" strategy, by coming in on a different address every time. Yes that's right. Since the scan is being done over a large scale of IP address space, it's possible to hide yourself by blocking Linux logins, since these boxes are usually managed by neglecting administrators and tends not to apply security updates from time to time. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAkrNEHkACgkQi+vbBBjt66BFxACfbfrUJnnVM9YGw6bVSo5hnfnO BwwAoKFf8DnRd3suCIYMGhZN6FqlTPrP =NwHo -----END PGP SIGNATURE----- From kostikbel at gmail.com Thu Oct 8 10:02:14 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Thu Oct 8 10:02:20 2009 Subject: sigwait - differences between Linux & FreeBSD In-Reply-To: <6300771b0910071753s6580c099i8c348824a6fe1a72@mail.gmail.com> References: <6300771b0910071753s6580c099i8c348824a6fe1a72@mail.gmail.com> Message-ID: <20091008100209.GG2259@deviant.kiev.zoral.com.ua> On Thu, Oct 08, 2009 at 11:53:21AM +1100, Stephen Hocking wrote: > Hi all, > > In my efforts to make the xrdp port more robust under FreeBSD, I have > discovered that sigwait (kind of an analogue to select(2), but for > signals rather than I/O) re-enables ignored signals in its list under > Linux, but not FreeBSD. The sesman daemon uses SIGCHLD to clean up > after a session has exited. Under Linux this works OK, under FreeSBD > it doesn't. I have worked around it in a very hackish manner (define a > dummy signal handler and enable it using signal, which means that the > sigwait call can then be unblocked by it), but am wondering if anyone > else has run across the same problem, and if so, if they fixed it in > an elegant manner. Also, does anyone know the correct semantics of > sigwait under this situation? ports@ is the wrong list to discuss the issue in the base system. Solaris 10 sigwait(2) manpage says the following: If sigwait() is called on an ignored signal, then the occurrence of the signal will be ignored, unless sigaction() changes the disposition. We have the same behaviour as Solaris, ingored signals are not queued or recorded regardeless of the presence of sigwaiting thread. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091008/efb91be3/attachment.pgp From kostikbel at gmail.com Thu Oct 8 10:27:30 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Thu Oct 8 10:27:41 2009 Subject: sigwait - differences between Linux & FreeBSD In-Reply-To: References: <6300771b0910071753s6580c099i8c348824a6fe1a72@mail.gmail.com> <20091008100209.GG2259@deviant.kiev.zoral.com.ua> Message-ID: <20091008102725.GH2259@deviant.kiev.zoral.com.ua> On Thu, Oct 08, 2009 at 01:23:37PM +0300, Vlad Galu wrote: > On Thu, Oct 8, 2009 at 1:02 PM, Kostik Belousov wrote: > > On Thu, Oct 08, 2009 at 11:53:21AM +1100, Stephen Hocking wrote: > >> Hi all, > >> > >> In my efforts to make the xrdp port more robust under FreeBSD, I have > >> discovered that sigwait (kind of an analogue to select(2), but for > >> signals rather than I/O) re-enables ignored signals in its list under > >> Linux, but not FreeBSD. The sesman daemon uses SIGCHLD to clean up > >> after a session has exited. Under Linux this works OK, under FreeSBD > >> it doesn't. I have worked around it in a very hackish manner (define a > >> dummy signal handler and enable it using signal, which means that the > >> sigwait call can then be unblocked by it), but am wondering if anyone > >> else has run across the same problem, and if so, if they fixed it in > >> an elegant manner. Also, does anyone know the correct semantics of > >> sigwait under this situation? > > > > ports@ is the wrong list to discuss the issue in the base system. > > > > Solaris 10 sigwait(2) manpage says the following: > > If sigwait() is called on an ignored signal, then the occurrence of the > > signal will be ignored, unless sigaction() changes the disposition. > > > > We have the same behaviour as Solaris, ingored signals are not queued or > > recorded regardeless of the presence of sigwaiting thread. > > > > This is a bit confusing. sigwait(2) says: "The signals specified by > set should be blocked at the time of the call to sigwait()."... Blocked and ignored are different attributes of signal. Ignored means that signal disposition is SIG_IGN, that causes the signal delivery event to be ignored. Blocked means that regardeless of signal disposition, signal is not delivered, but it is recorded somewhere and delivery happen when it unblocked. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091008/9e83c612/attachment.pgp From dudu at dudu.ro Thu Oct 8 10:55:43 2009 From: dudu at dudu.ro (Vlad Galu) Date: Thu Oct 8 10:55:49 2009 Subject: sigwait - differences between Linux & FreeBSD In-Reply-To: <20091008100209.GG2259@deviant.kiev.zoral.com.ua> References: <6300771b0910071753s6580c099i8c348824a6fe1a72@mail.gmail.com> <20091008100209.GG2259@deviant.kiev.zoral.com.ua> Message-ID: On Thu, Oct 8, 2009 at 1:02 PM, Kostik Belousov wrote: > On Thu, Oct 08, 2009 at 11:53:21AM +1100, Stephen Hocking wrote: >> Hi all, >> >> In my efforts to make the xrdp port more robust under FreeBSD, I have >> discovered that sigwait (kind of an analogue to select(2), but for >> signals rather than I/O) re-enables ignored signals in its list under >> Linux, but not FreeBSD. The sesman daemon uses SIGCHLD to clean up >> after a session has exited. Under Linux this works OK, under FreeSBD >> it doesn't. I have worked around it in a very hackish manner (define a >> dummy signal handler and enable it using signal, which means that the >> sigwait call can then be unblocked by it), but am wondering if anyone >> else has run across the same problem, and if so, if they fixed it in >> an elegant manner. Also, does anyone know the correct semantics of >> sigwait under this situation? > > ports@ is the wrong list to discuss the issue in the base system. > > Solaris 10 sigwait(2) manpage says the following: > If sigwait() is called on an ignored signal, then the occurrence of the > signal will be ignored, unless sigaction() changes the disposition. > > We have the same behaviour as Solaris, ingored signals are not queued or > recorded regardeless of the presence of sigwaiting thread. > This is a bit confusing. sigwait(2) says: "The signals specified by set should be blocked at the time of the call to sigwait()."... From mel.flynn+fbsd.hackers at mailing.thruhere.net Thu Oct 8 22:15:26 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Thu Oct 8 22:15:33 2009 Subject: Running a program through gdb without "interfering" Message-ID: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> Hi, is there a way to have a program run through gdb and gdb only record a segfault, but otherwise let the program run? Why I'd like this is the following: I've got a i386 jail on an amd64 box, running 7.2-p4. UNAME_p and UNAME_m have been set to i386 as well as ARCH in /etc/make.conf. Running portmaster[1] to build ports under my uid and PM_SU_CMD, sudo *sometimes* segfaults. It's only sudo, so at present I don't have a reason to doubt memory. However, it doesn't dump core, so I'm at a loss what the culprit could be. [1] In order to get this working I had to put a statically compiled ps in the jail, or the uid test would fail. It has the downside that it lists both jail and host processes, but it is acceptable to me as the jail is only accessible from the host (pf enforced). I suspect sudo to have a similar problem or even related to ps returning processes from a uid that doesn't exist in the jail, but without a backtrace I don't know what to fix. -- Mel From mel.flynn+fbsd.hackers at mailing.thruhere.net Thu Oct 8 23:17:02 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Thu Oct 8 23:17:08 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <3a142e750910081538g213eb63cse559b4601e97a3@mail.gmail.com> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <3a142e750910081538g213eb63cse559b4601e97a3@mail.gmail.com> Message-ID: <200910090116.59158.mel.flynn+fbsd.hackers@mailing.thruhere.net> On Friday 09 October 2009 00:38:32 Paul B Mahol wrote: > On 10/9/09, Mel Flynn wrote: > > Hi, > > > > is there a way to have a program run through gdb and gdb only record a > > segfault, but otherwise let the program run? > > > > Why I'd like this is the following: > > I've got a i386 jail on an amd64 box, running 7.2-p4. UNAME_p and UNAME_m > > have > > been set to i386 as well as ARCH in /etc/make.conf. Running portmaster[1] > > to build ports under my uid and PM_SU_CMD, sudo *sometimes* segfaults. > > It's only > > sudo, so at present I don't have a reason to doubt memory. However, it > > doesn't > > dump core, so I'm at a loss what the culprit could be. > > Tried 'sysctl kern.sugid_coredump=1' ? Hmm, no. Enabled now and waiting for the next segfault. I actually looked at the sysctl -d, but it didn't register that this could be the main cause. Perhaps that sentence could be more clear: -kern.sugid_coredump: Enable coredumping set user/group ID processes +kenr.sugid_coredump: Allow setuid/setgid processes to dump core -- Mel From gary.jennejohn at freenet.de Fri Oct 9 09:32:05 2009 From: gary.jennejohn at freenet.de (Gary Jennejohn) Date: Fri Oct 9 09:32:12 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910090116.59158.mel.flynn+fbsd.hackers@mailing.thruhere.net> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <3a142e750910081538g213eb63cse559b4601e97a3@mail.gmail.com> <200910090116.59158.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <20091009113201.0ade498f@ernst.jennejohn.org> On Fri, 9 Oct 2009 01:16:59 +0200 Mel Flynn wrote: > On Friday 09 October 2009 00:38:32 Paul B Mahol wrote: > > On 10/9/09, Mel Flynn wrote: > > > Hi, > > > > > > is there a way to have a program run through gdb and gdb only record a > > > segfault, but otherwise let the program run? > > > > > > Why I'd like this is the following: > > > I've got a i386 jail on an amd64 box, running 7.2-p4. UNAME_p and UNAME_m > > > have > > > been set to i386 as well as ARCH in /etc/make.conf. Running portmaster[1] > > > to build ports under my uid and PM_SU_CMD, sudo *sometimes* segfaults. > > > It's only > > > sudo, so at present I don't have a reason to doubt memory. However, it > > > doesn't > > > dump core, so I'm at a loss what the culprit could be. > > > > Tried 'sysctl kern.sugid_coredump=1' ? > > Hmm, no. Enabled now and waiting for the next segfault. > I actually looked at the sysctl -d, but it didn't register that this could be > the main cause. > Perhaps that sentence could be more clear: > -kern.sugid_coredump: Enable coredumping set user/group ID processes > +kenr.sugid_coredump: Allow setuid/setgid processes to dump core > See the info file for gdb, section 5.3 Signals. It's possible to tell gdb how to handle signals, e.g. stop vs. nostop, etc. --- Gary Jennejohn From des at des.no Fri Oct 9 09:38:31 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri Oct 9 09:38:38 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> (Mel Flynn's message of "Fri, 9 Oct 2009 00:15:24 +0200") References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <86skds7vqi.fsf@ds4.des.no> Mel Flynn writes: > is there a way to have a program run through gdb and gdb only record a > segfault, but otherwise let the program run? Yes, just run "gdb /path/to/program" and type "run". > [...] sudo *sometimes* segfaults [...] However, it doesn't dump core sudo(1) is setuid root. You need to set kern.sugid_coredump to get it to dump core. > [1] In order to get this working I had to put a statically compiled ps in the > jail, or the uid test would fail. It has the downside that it lists both jail > and host processes, [...] Uh, no. Processes outside the jail are not visible inside it, no matter what version of ps(1) or top(1) or any other such program you use. DES -- Dag-Erling Sm?rgrav - des@des.no From stef-list at memberwebs.com Fri Oct 9 14:08:44 2009 From: stef-list at memberwebs.com (Stef Walter) Date: Fri Oct 9 14:08:51 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <4ACF43EA.9050508@memberwebs.com> Mel Flynn wrote: > [1] In order to get this working I had to put a statically compiled ps in the > jail This is a pretty standard practice. I always put these statically built into any jails that don't match the outside system. I use the following crunchgen config to accomplish that. Cheers, Stef # Commands to build in progs ps ipcs netstat pkill top w killall progs systat iostat progs jkill progs kldstat # Link these programs to each other ln pkill pgrep ln w uptime # Libraries which we need libs -lutil -lkvm -ldevstat -lncurses -ldevstat -lm -lnetgraph -lmemstat -lipx From mel.flynn+fbsd.hackers at mailing.thruhere.net Fri Oct 9 14:50:08 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Fri Oct 9 14:50:14 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <86skds7vqi.fsf@ds4.des.no> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skds7vqi.fsf@ds4.des.no> Message-ID: <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> On Friday 09 October 2009 11:38:29 Dag-Erling Sm?rgrav wrote: > Mel Flynn writes: > > is there a way to have a program run through gdb and gdb only record a > > segfault, but otherwise let the program run? > > Yes, just run "gdb /path/to/program" and type "run". Not what I was looking for. The segfaults are random and the only way to somewhat reliably reproduce it is to have portmaster invoke it as it's PM_SU_CMD. And no, running that same command again doesn't trigger the segfault, so it's "something environmental". Hence I'm looking for something like: gdb -batch -x script_with_run_cmd.gdb -exec /usr/local/bin/sudo $argv where somehow I need $argv to be passed as arguments to sudo. I'm thinking i should just wrap it and mktemp(1) a new command script for gdb to use with set args $*, but if anyone has a more clever idea, I'd love to hear it. > > [...] sudo *sometimes* segfaults [...] However, it doesn't dump core > > sudo(1) is setuid root. You need to set kern.sugid_coredump to get it > to dump core. It still segfaults and doesn't dump: Oct 9 04:34:18 smell kernel: pid 39476 (sudo), uid 0: exited on signal 11 Oct 9 04:36:32 smell kernel: pid 79657 (sudo), uid 0: exited on signal 11 Oct 9 04:36:43 smell kernel: pid 82390 (sudo), uid 0: exited on signal 11 Oct 9 04:51:46 smell kernel: pid 3601 (sudo), uid 0: exited on signal 11 find / -name '*.core' in the jail does not yield anything. > > [1] In order to get this working I had to put a statically compiled ps in > > the jail, or the uid test would fail. It has the downside that it lists > > both jail and host processes, [...] > > Uh, no. Processes outside the jail are not visible inside it, no matter > what version of ps(1) or top(1) or any other such program you use. I'll write this off as pilot error, cause I cannot reproduce it. I saw bash as one of the processes listed in a blank ps run, which isn't installed in the jail, but since I don't have the terminal history anymore, it's entirely possible I ran ps on the host. -- Mel From mel.flynn+fbsd.hackers at mailing.thruhere.net Fri Oct 9 15:01:37 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Fri Oct 9 15:01:44 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skds7vqi.fsf@ds4.des.no> <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <200910091701.34788.mel.flynn+fbsd.hackers@mailing.thruhere.net> On Friday 09 October 2009 16:50:04 Mel Flynn wrote: > On Friday 09 October 2009 11:38:29 Dag-Erling Sm?rgrav wrote: > > Mel Flynn writes: > > > [...] sudo *sometimes* segfaults [...] However, it doesn't dump core > > > > sudo(1) is setuid root. You need to set kern.sugid_coredump to get it > > to dump core. > > It still segfaults and doesn't dump: > Oct 9 04:34:18 smell kernel: pid 39476 (sudo), uid 0: exited on signal 11 > Oct 9 04:36:32 smell kernel: pid 79657 (sudo), uid 0: exited on signal 11 > Oct 9 04:36:43 smell kernel: pid 82390 (sudo), uid 0: exited on signal 11 > Oct 9 04:51:46 smell kernel: pid 3601 (sudo), uid 0: exited on signal 11 > > find / -name '*.core' in the jail does not yield anything. FYI, there's one read-only mount into the jail, being /usr/src. I don't see a reason given the commands it segfaults on, for $cwd to be below that.For example it segfaulted on sudo pkg_delete glproto2. Thought I'd mention it to rule it out. -- Mel From jhb at freebsd.org Fri Oct 9 16:08:17 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Oct 9 16:09:02 2009 Subject: crashinfo: print the content of ddb capture budder In-Reply-To: <81ws3a1je1.fsf@zhuzha.ua1> References: <81ws3a1je1.fsf@zhuzha.ua1> Message-ID: <200910091128.11513.jhb@freebsd.org> On Monday 05 October 2009 1:48:06 am Mikolaj Golub wrote: > Hi, > > It would be nice if crashinfo(8) were also trying to output the content of ddb > capture buffer. Something like in this patch: > > --- crashinfo.sh.orig 2009-10-05 08:26:26.000000000 +0300 > +++ crashinfo.sh 2009-10-05 08:43:56.000000000 +0300 > @@ -304,3 +304,18 @@ > echo "kernel config" > echo > config -x $KERNEL > + > +file=`mktemp /tmp/crashinfo.XXXXXX` > +if [ $? -eq 0 ]; then > + ddb capture -M $VMCORE -N $KERNEL print > $file 2>/dev/null > + if [ -s $file ]; then > + echo "------------------------------------------------------------------------" > + echo "ddb capture buffer" > + echo > + cat $file | > + sed -e 's/p\{10\}p*//' # XXX: this removes the unfilled part of a capture buffer > + echo > + fi > + rm -f $file > +fi > + > I'm definitely in favor of this. I assume you have tested it locally? Do you have a sample crash.X.txt file with it enabled? -- John Baldwin From is at rambler-co.ru Fri Oct 9 17:06:29 2009 From: is at rambler-co.ru (Igor Sysoev) Date: Fri Oct 9 17:06:37 2009 Subject: newfs -r 2 Message-ID: <20091009170627.GR14613@rambler-co.ru> I have found that newfs in 8-STABLE has -r switch with zero default value. I think it should be 1 or 2 by default: as I understand, these sectors are not used usually by filesystem anyway since they are not in last cylinder group. Therefore noone would see the difference in usable space, but this reservation will allow to add gjournal to the filesystem later. BTW, could anyone tell how to learn the last sector that filesystem may really use ? -- Igor Sysoev http://sysoev.ru/en/ From nate at thatsmathematics.com Fri Oct 9 17:30:48 2009 From: nate at thatsmathematics.com (Nate Eldredge) Date: Fri Oct 9 17:30:56 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skds7vqi.fsf@ds4.des.no> <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: On Fri, 9 Oct 2009, Mel Flynn wrote: > On Friday 09 October 2009 11:38:29 Dag-Erling Sm?rgrav wrote: >> Mel Flynn writes: >>> is there a way to have a program run through gdb and gdb only record a >>> segfault, but otherwise let the program run? >> >> Yes, just run "gdb /path/to/program" and type "run". > > Not what I was looking for. The segfaults are random and the only way to > somewhat reliably reproduce it is to have portmaster invoke it as it's > PM_SU_CMD. And no, running that same command again doesn't trigger the > segfault, so it's "something environmental". Hence I'm looking for something > like: > gdb -batch -x script_with_run_cmd.gdb -exec /usr/local/bin/sudo $argv > > where somehow I need $argv to be passed as arguments to sudo. I'm thinking i > should just wrap it and mktemp(1) a new command script for gdb to use with set > args $*, but if anyone has a more clever idea, I'd love to hear it. This won't work. You can't debug setuid programs (for reasons which should be obvious). You could do it if you ran everything as root, but it sounds like the bug doesn't occur in that case. -- Nate Eldredge nate@thatsmathematics.com From mel.flynn+fbsd.hackers at mailing.thruhere.net Fri Oct 9 17:32:53 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Fri Oct 9 17:33:00 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skds7vqi.fsf@ds4.des.no> <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <200910091932.49028.mel.flynn+fbsd.hackers@mailing.thruhere.net> On Friday 09 October 2009 16:50:04 Mel Flynn wrote: > On Friday 09 October 2009 11:38:29 Dag-Erling Sm?rgrav wrote: > > Mel Flynn writes: > > > is there a way to have a program run through gdb and gdb only record a > > > segfault, but otherwise let the program run? > > > > Yes, just run "gdb /path/to/program" and type "run". > > Not what I was looking for. The segfaults are random and the only way to > somewhat reliably reproduce it is to have portmaster invoke it as it's > PM_SU_CMD. And no, running that same command again doesn't trigger the > segfault, so it's "something environmental". Hence I'm looking for > something like: > gdb -batch -x script_with_run_cmd.gdb -exec /usr/local/bin/sudo $argv > > where somehow I need $argv to be passed as arguments to sudo. I'm thinking > i should just wrap it and mktemp(1) a new command script for gdb to use > with set args $*, but if anyone has a more clever idea, I'd love to hear > it. Dead end path :/ % bin/gdbsudo echo hi /tmp/gdbsudo.F3kdwJ:1: Error in sourced command file: /usr/local/bin/sudo: Permission denied. % ls -l /usr/local/bin/sudo ---s--x--x 2 root wheel 116380 Oct 8 18:31 /usr/local/bin/sudo % sudo chmod g+r /usr/local/bin/sudo % bin/gdbsudo echo hi (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...sudo: must be setuid root Program exited with code 01. Perhaps the cause of it not dumping core either. Would've been nice to know why it segfaults, but not nice enough to keep digging. -- Mel From jilles at stack.nl Fri Oct 9 18:43:11 2009 From: jilles at stack.nl (Jilles Tjoelker) Date: Fri Oct 9 18:43:23 2009 Subject: sigwait - differences between Linux & FreeBSD In-Reply-To: <20091008100209.GG2259@deviant.kiev.zoral.com.ua> References: <6300771b0910071753s6580c099i8c348824a6fe1a72@mail.gmail.com> <20091008100209.GG2259@deviant.kiev.zoral.com.ua> Message-ID: <20091009184309.GA15210@stack.nl> On Thu, Oct 08, 2009 at 01:02:09PM +0300, Kostik Belousov wrote: > On Thu, Oct 08, 2009 at 11:53:21AM +1100, Stephen Hocking wrote: > > In my efforts to make the xrdp port more robust under FreeBSD, I have > > discovered that sigwait (kind of an analogue to select(2), but for > > signals rather than I/O) re-enables ignored signals in its list under > > Linux, but not FreeBSD. The sesman daemon uses SIGCHLD to clean up > > after a session has exited. Under Linux this works OK, under FreeSBD > > it doesn't. I have worked around it in a very hackish manner (define a > > dummy signal handler and enable it using signal, which means that the > > sigwait call can then be unblocked by it), but am wondering if anyone > > else has run across the same problem, and if so, if they fixed it in > > an elegant manner. Also, does anyone know the correct semantics of > > sigwait under this situation? > ports@ is the wrong list to discuss the issue in the base system. > Solaris 10 sigwait(2) manpage says the following: > If sigwait() is called on an ignored signal, then the occurrence of the > signal will be ignored, unless sigaction() changes the disposition. > We have the same behaviour as Solaris, ingored signals are not queued or > recorded regardeless of the presence of sigwaiting thread. POSIX permits both behaviours here: a blocked and ignored signal may or may not be discarded immediately on generation. Making this depend on whether there is a sigwaiting thread seems broken, and I don't think Linux does that. I think your "very hackish" approach is correct: set up a dummy signal handler after blocking the signal. Additionally, POSIX requires applications to set the SA_SIGINFO flag if they want queuing. This applies even if the signals are blocked and received using sigwaitinfo(2) or sigtimedwait(2). The SA_SIGINFO flag can only be set by setting a handler using sigaction(2). (Note, this does not mean that all signals are queued if SA_SIGINFO is set. It means that signals may not be queued if SA_SIGINFO is not set.) -- Jilles Tjoelker From des at des.no Fri Oct 9 19:27:23 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri Oct 9 19:27:30 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> (Mel Flynn's message of "Fri, 9 Oct 2009 16:50:04 +0200") References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skds7vqi.fsf@ds4.des.no> <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <86skdss6zq.fsf@ds4.des.no> Mel Flynn writes: > Dag-Erling Sm?rgrav writes: > > Yes, just run "gdb /path/to/program" and type "run". > Not what I was looking for. The segfaults are random and the only way to > somewhat reliably reproduce it is to have portmaster invoke it as it's > PM_SU_CMD. And no, running that same command again doesn't trigger the > segfault, so it's "something environmental". Hence I'm looking for something > like: > gdb -batch -x script_with_run_cmd.gdb -exec /usr/local/bin/sudo $argv > > where somehow I need $argv to be passed as arguments to sudo. I'm thinking i > should just wrap it and mktemp(1) a new command script for gdb to use with set > args $*, but if anyone has a more clever idea, I'd love to hear it. Why look for a clever option, when the simple one will do just fine? :>gdb-script-$$ echo "set args $@" >>gdb-script-$$ echo "run" >>gdb-script-$$ gdb -batch -x gdb-script-$$ /usr/local/bin/sudo > It still segfaults and doesn't dump: > Oct 9 04:34:18 smell kernel: pid 39476 (sudo), uid 0: exited on signal 11 > Oct 9 04:36:32 smell kernel: pid 79657 (sudo), uid 0: exited on signal 11 > Oct 9 04:36:43 smell kernel: pid 82390 (sudo), uid 0: exited on signal 11 > Oct 9 04:51:46 smell kernel: pid 3601 (sudo), uid 0: exited on signal 11 > > find / -name '*.core' in the jail does not yield anything. Add 'ulimit -c unlimited' somewhere in the script before it invokes sudo. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Fri Oct 9 19:32:51 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri Oct 9 19:32:57 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: (Nate Eldredge's message of "Fri, 9 Oct 2009 10:30:44 -0700 (PDT)") References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skds7vqi.fsf@ds4.des.no> <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: <86ocogs6qm.fsf@ds4.des.no> Nate Eldredge writes: > This won't work. You can't debug setuid programs (for reasons which > should be obvious). Ah, true, but easily fixable. Add a sysctl for it (just copy-paste the declaration for kern.sugid_coredump and change the name) and check its value in p_candebug() (hint: "if (credentialchanged)"). DES -- Dag-Erling Sm?rgrav - des@des.no From alexbestms at math.uni-muenster.de Fri Oct 9 19:52:45 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 9 19:52:53 2009 Subject: sysinstall colours Message-ID: hi there, sysinstall is probably one of those ancient relics everybody tries to avoid dealing with from a developers point of view but i just found this beautiful screenie of a (probably) ncurse-based installer: http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dresden_7_lrg i was surprised how much better it looks with those nice colours compared to sysinstall. is there any way the sysinstall colours could be adjusted (without a lot of work) to also feature such beautiful colours? i had a quick look at the sysinstall, libdialog and ncurses sources and to me it seems that to change sysinstall's colours the hardcoded values of COLOR_BLACK COLOR_RED COLOR_GREEN COLOR_YELLOW COLOR_BLUE COLOR_MAGENTA COLOR_CYAN COLOR_WHITE have to be changed in contrib/ncurses/ncurses/base/lib_color.c or is there an easier way? because this would of course affect all apps that are linked to ncurses. cheers. alex From to.my.trociny at gmail.com Fri Oct 9 21:25:15 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Fri Oct 9 21:25:21 2009 Subject: crashinfo: print the content of ddb capture budder In-Reply-To: <200910091128.11513.jhb@freebsd.org> (John Baldwin's message of "Fri\, 9 Oct 2009 11\:28\:11 -0400") References: <81ws3a1je1.fsf@zhuzha.ua1> <200910091128.11513.jhb@freebsd.org> Message-ID: <86r5tcmf9m.fsf@kopusha.onet> On Fri, 9 Oct 2009 11:28:11 -0400 John Baldwin wrote: JB> On Monday 05 October 2009 1:48:06 am Mikolaj Golub wrote: >> Hi, >> >> It would be nice if crashinfo(8) were also trying to output the content of ddb >> capture buffer. Something like in this patch: >> >> --- crashinfo.sh.orig 2009-10-05 08:26:26.000000000 +0300 >> +++ crashinfo.sh 2009-10-05 08:43:56.000000000 +0300 >> @@ -304,3 +304,18 @@ >> echo "kernel config" >> echo >> config -x $KERNEL >> + >> +file=`mktemp /tmp/crashinfo.XXXXXX` >> +if [ $? -eq 0 ]; then >> + ddb capture -M $VMCORE -N $KERNEL print > $file 2>/dev/null >> + if [ -s $file ]; then >> + echo "------------------------------------------------------------------------" >> + echo "ddb capture buffer" >> + echo >> + cat $file | >> + sed -e 's/p\{10\}p*//' # XXX: this removes the unfilled part of a capture buffer >> + echo >> + fi >> + rm -f $file >> +fi >> + >> JB> I'm definitely in favor of this. I assume you have tested it locally? Do you have a sample JB> crash.X.txt file with it enabled? I have tested it on 8.0. zhuzha:~% ls -l /var/crash/vmcore.23 -rw------- 1 root wheel 166703104 2009-10-05 08:03 /var/crash/vmcore.23 zhuzha:~% sudo crashinfo Writing crash summary to /var/crash/core.txt.23. zhuzha:~% grep -B5 -A30 'ddb capture buffer' /var/crash/core.txt.23 ------------------------------------------------------------------------ kernel config config: File /boot/kernel.old/kernel doesn't contain configuration file. Either unsupported, or not compiled with INCLUDE_CONFIG_FILE ------------------------------------------------------------------------ ddb capture buffer db:0:kdb.enter.panic> show pcpu cpuid = 0 dynamic pcpu = 0x68ee80 curthread = 0xc4a1ad80: pid 2276 "sysctl" curpcb = 0xe6d44d90 fpcurthread = none idlethread = 0xc4576900: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: db:0:kdb.enter.panic> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x68ee80 curthread = 0xc4a1ad80: pid 2276 "sysctl" curpcb = 0xe6d44d90 fpcurthread = none idlethread = 0xc4576900: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: cpuid = 1 dynamic pcpu = 0x34ffe80 curthread = 0xc5837480: pid 2191 "screen" curpcb = 0xe6e5ed90 fpcurthread = none idlethread = 0xc4576b40: pid 11 "idle: cpu1" zhuzha:~% tail /var/crash/core.txt.23 mi_switch(104,0,c0c798d3,1d6,44,...) at mi_switch+0x200 sleepq_switch(c0dc8190,0,c0c798d3,26e,0,...) at sleepq_switch+0x15f sleepq_timedwait(c0dc7ee0,44,c0c7793c,0,0,...) at sleepq_timedwait+0x6b _sleep(c0dc7ee0,0,44,c0c7793c,2710,...) at _sleep+0x339 scheduler(0,141ec00,141ec00,141e000,1425000,...) at scheduler+0x23e mi_startup() at mi_startup+0x96 begin() at begin+0x2c db:0:kdb.enter.panic> call doadump zhuzha:~% Actually the last echo in the patch looks like is not necessary. Do you want the whole crash.23.txt file for review? Also, I remember I tested it on crashdump of a kernel without ddb support and no issues were noticed too. -- Mikolaj Golub From jhell at DataIX.net Fri Oct 9 21:32:50 2009 From: jhell at DataIX.net (jhell) Date: Fri Oct 9 21:32:57 2009 Subject: sysinstall colours In-Reply-To: References: Message-ID: On Fri, 9 Oct 2009 21:52 +0200, alexbestms@ wrote: > hi there, > > sysinstall is probably one of those ancient relics everybody tries to avoid > dealing with from a developers point of view but i just found this beautiful > screenie of a (probably) ncurse-based installer: > > http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dresden_7_lrg > > i was surprised how much better it looks with those nice colours compared to > sysinstall. > > is there any way the sysinstall colours could be adjusted (without a lot of > work) to also feature such beautiful colours? i had a quick look at the > sysinstall, libdialog and ncurses sources and to me it seems that to change > sysinstall's colours the hardcoded values of > > COLOR_BLACK > COLOR_RED > COLOR_GREEN > COLOR_YELLOW > COLOR_BLUE > COLOR_MAGENTA > COLOR_CYAN > COLOR_WHITE > > have to be changed in contrib/ncurses/ncurses/base/lib_color.c or is there an > easier way? because this would of course affect all apps that are linked to > ncurses. > > cheers. > alex sysinstall looks like this too when it is ran in a gnome terminal or some other terminal that is ran in a X environment. -- ;; dataix.net!jhell 2048R/89D8547E 2009-09-30 ;; BSD since FreeBSD 4.2 Linux since Slackware 2.1 ;; 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E From mail at 0xabadba.be Fri Oct 9 23:10:52 2009 From: mail at 0xabadba.be (james toy) Date: Fri Oct 9 23:11:00 2009 Subject: sysinstall colours In-Reply-To: References: Message-ID: <61c92c7d0910091544g36db4824t40298167fb9275ad@mail.gmail.com> Alexander, ==8<== >> http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dresden_7_lrg ==8<== The head maintainer of Yoper; Tobias G, runs a kernel patchset I work on from http://zen-sources.org as his default kernel. I am sure he would be more than happy to discuss some of their methods if you are interested in learning or modifying the current sysinstall system. Not sure how much this will help; however, it cannot hurt to ask. Taking ideas from Linux distros is rarely a good idea I've found though :) =jt From randi at freebsd.org Sat Oct 10 00:27:55 2009 From: randi at freebsd.org (Randi Harper) Date: Sat Oct 10 00:28:02 2009 Subject: sysinstall colours In-Reply-To: References: Message-ID: On Fri, Oct 9, 2009 at 12:52 PM, Alexander Best < alexbestms@math.uni-muenster.de> wrote: > hi there, > > sysinstall is probably one of those ancient relics everybody tries to avoid > dealing with from a developers point of view but i just found this > beautiful > screenie of a (probably) ncurse-based installer: > > > http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dresden_7_lrg > > i was surprised how much better it looks with those nice colours compared > to > sysinstall. > > is there any way the sysinstall colours could be adjusted (without a lot of > work) to also feature such beautiful colours? i had a quick look at the > sysinstall, libdialog and ncurses sources and to me it seems that to change > sysinstall's colours the hardcoded values of > > COLOR_BLACK > COLOR_RED > COLOR_GREEN > COLOR_YELLOW > COLOR_BLUE > COLOR_MAGENTA > COLOR_CYAN > COLOR_WHITE > > have to be changed in contrib/ncurses/ncurses/base/lib_color.c or is there > an > easier way? because this would of course affect all apps that are linked to > ncurses. > > cheers. > alex > Seriously?!?!?! All the problems with sysinstall, and your idea is to change the color? Are you trying to start a bikeshed? If so, I prefer pink. -- randi From mel.flynn+fbsd.hackers at mailing.thruhere.net Sat Oct 10 01:52:09 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Sat Oct 10 01:52:16 2009 Subject: Running a program through gdb without "interfering" In-Reply-To: <86skdss6zq.fsf@ds4.des.no> References: <200910090015.24175.mel.flynn+fbsd.hackers@mailing.thruhere.net> <200910091650.04231.mel.flynn+fbsd.hackers@mailing.thruhere.net> <86skdss6zq.fsf@ds4.des.no> Message-ID: <200910100352.05524.mel.flynn+fbsd.hackers@mailing.thruhere.net> On Friday 09 October 2009 21:27:21 Dag-Erling Sm?rgrav wrote: > Mel Flynn writes: > > Dag-Erling Sm?rgrav writes: > > > Yes, just run "gdb /path/to/program" and type "run". > > > > Not what I was looking for. The segfaults are random and the only way to > > somewhat reliably reproduce it is to have portmaster invoke it as it's > > PM_SU_CMD. And no, running that same command again doesn't trigger the > > segfault, so it's "something environmental". Hence I'm looking for > > something like: > > gdb -batch -x script_with_run_cmd.gdb -exec /usr/local/bin/sudo $argv > > > > where somehow I need $argv to be passed as arguments to sudo. I'm > > thinking i should just wrap it and mktemp(1) a new command script for gdb > > to use with set args $*, but if anyone has a more clever idea, I'd love > > to hear it. > > Why look for a clever option, when the simple one will do just fine? Cause I don't know how much of the cause of this bug I'm influencing. Even though this is now the simple solution, it would be simpler if gdb (or another debugger) could work similar as sudo, where it would take the first argument as binary and the rest as arguments to the binary. This would do away with some extra IO I'm now creating. Though, it's unlikely it is related to IO, there is no pattern that I've found yet for the segfault, so I'm trying to limit any "extra stuff". I'll patch the kernel tomorrow with the new sysctl and see how far that gets me. > Add 'ulimit -c unlimited' somewhere in the script before it invokes sudo. I'll add it. -- Mel From onemda at gmail.com Sat Oct 10 08:50:22 2009 From: onemda at gmail.com (Paul B Mahol) Date: Sat Oct 10 08:50:29 2009 Subject: sysinstall colours In-Reply-To: References: Message-ID: <3a142e750910100150g7459e037u7b84b4b4bbc2bf8@mail.gmail.com> On 10/9/09, Alexander Best wrote: > hi there, > > sysinstall is probably one of those ancient relics everybody tries to avoid > dealing with from a developers point of view but i just found this beautiful > screenie of a (probably) ncurse-based installer: > > http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dresden_7_lrg > > i was surprised how much better it looks with those nice colours compared to > sysinstall. > > is there any way the sysinstall colours could be adjusted (without a lot of > work) to also feature such beautiful colours? i had a quick look at the > sysinstall, libdialog and ncurses sources and to me it seems that to change > sysinstall's colours the hardcoded values of > > COLOR_BLACK > COLOR_RED > COLOR_GREEN > COLOR_YELLOW > COLOR_BLUE > COLOR_MAGENTA > COLOR_CYAN > COLOR_WHITE > > have to be changed in contrib/ncurses/ncurses/base/lib_color.c or is there > an > easier way? because this would of course affect all apps that are linked to > ncurses. This have nothing to do with ncurses, colors you like simple can not be displayed in current syscons(4) and making support for 256 colors or even true bit color in sysinstall(so that it looks amazing in konsole) is waste of time. From alexbestms at math.uni-muenster.de Sat Oct 10 10:06:42 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Sat Oct 10 10:06:48 2009 Subject: sysinstall colours In-Reply-To: Message-ID: jhell schrieb am 2009-10-09: > On Fri, 9 Oct 2009 21:52 +0200, alexbestms@ wrote: > >hi there, > >sysinstall is probably one of those ancient relics everybody tries > >to avoid > >dealing with from a developers point of view but i just found this > >beautiful > >screenie of a (probably) ncurse-based installer: > >http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dre > >sden_7_lrg > >i was surprised how much better it looks with those nice colours > >compared to > >sysinstall. > >is there any way the sysinstall colours could be adjusted (without > >a lot of > >work) to also feature such beautiful colours? i had a quick look at > >the > >sysinstall, libdialog and ncurses sources and to me it seems that > >to change > >sysinstall's colours the hardcoded values of > >COLOR_BLACK > >COLOR_RED > >COLOR_GREEN > >COLOR_YELLOW > >COLOR_BLUE > >COLOR_MAGENTA > >COLOR_CYAN > >COLOR_WHITE > >have to be changed in contrib/ncurses/ncurses/base/lib_color.c or > >is there an > >easier way? because this would of course affect all apps that are > >linked to > >ncurses. > >cheers. > >alex > sysinstall looks like this too when it is ran in a gnome terminal or > some other terminal that is ran in a X environment. > -- > ;; dataix.net!jhell 2048R/89D8547E 2009-09-30 > ;; BSD since FreeBSD 4.2 Linux since Slackware 2.1 > ;; 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E i'm running sysinstall in xterm with $TERM set to xterm-color and the colors don't seem any different to me. but you have a point there, because i could edit ~/.Xdefaults and adjust some xterm resources to change the standard colours. alex From alexbestms at math.uni-muenster.de Sat Oct 10 10:21:31 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Sat Oct 10 10:21:38 2009 Subject: sysinstall colours In-Reply-To: Message-ID: Randi Harper schrieb am 2009-10-10: > On Fri, Oct 9, 2009 at 12:52 PM, Alexander Best < > alexbestms@math.uni-muenster.de> wrote: > > hi there, > > sysinstall is probably one of those ancient relics everybody tries > > to avoid > > dealing with from a developers point of view but i just found this > > beautiful > > screenie of a (probably) ncurse-based installer: > > http://www.phoronix.net/image.php?id=yoper_2009_beta&image=yoper_dresden_7_lrg > > i was surprised how much better it looks with those nice colours > > compared > > to > > sysinstall. > > is there any way the sysinstall colours could be adjusted (without > > a lot of > > work) to also feature such beautiful colours? i had a quick look at > > the > > sysinstall, libdialog and ncurses sources and to me it seems that > > to change > > sysinstall's colours the hardcoded values of > > COLOR_BLACK > > COLOR_RED > > COLOR_GREEN > > COLOR_YELLOW > > COLOR_BLUE > > COLOR_MAGENTA > > COLOR_CYAN > > COLOR_WHITE > > have to be changed in contrib/ncurses/ncurses/base/lib_color.c or > > is there > > an > > easier way? because this would of course affect all apps that are > > linked to > > ncurses. > > cheers. > > alex > Seriously?!?!?! All the problems with sysinstall, and your idea is to > change > the color? Are you trying to start a bikeshed? If so, I prefer pink. > -- randi of course sysinstall has a ton of problems and should be replaced. no doubt about it. but look at it from this angle: current developers don't seem to have any interest in improving sysinstall. so it's important to get new people involved in freebsd. and the way to do that is with an attractive looking installer and an easy installation process imo. sure a good installer doesn't make a good os. but let's face it. when you've been running windows for a few years and finally want to switch to something else you're likely looking for an os which doesn't frighten you off right at the start (which freebsd sort of does). so i think having a good looking installer is more than just eyecandy. i don't think hardcore developers who prefer working with a bare X and vi(m)/emacs and so forth should look down on people who are slowly starting to get involved in an os, because without new people freebsd will someday be dead. alex From alexbestms at math.uni-muenster.de Sat Oct 10 10:34:07 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Sat Oct 10 10:34:14 2009 Subject: crashtar Message-ID: thanks. this is a cool script and very useful indeed. only thing you might want to do is check for root privileges at the beginning to avoid nasty error messages like. awk: can't open file /var/crash/info.0 source line number 12 thanks again. alex From jhell at DataIX.net Sat Oct 10 11:07:35 2009 From: jhell at DataIX.net (jhell) Date: Sat Oct 10 11:07:41 2009 Subject: sysinstall colours In-Reply-To: References: Message-ID: On Sat, 10 Oct 2009 06:21, alexbestms@ wrote: > of course sysinstall has a ton of problems and should be replaced. no doubt > about it. but look at it from this angle: > You should really ask what the FreeBSD angle on this is. It is really well versed and well planned out and covers many areas among many people which make the installer a very versatile yet very controversial thing to replace. > current developers don't seem to have any interest in improving sysinstall. so > it's important to get new people involved in freebsd. and the way to do that > is with an attractive looking installer and an easy installation process imo. > There has been a lot of expressed interest in replacing the installed multiple times in the past with unsaid I believe 3 projects going on right now (someone correct me if I'm wrong) everywhere from graphical to script. > sure a good installer doesn't make a good os. but let's face it. when you've > been running windows for a few years and finally want to switch to something > else you're likely looking for an os which doesn't frighten you off right at > the start (which freebsd sort of does). > When you have been running windows for a few years and you have decided to research your options you have kept a open mind and willingness to see what else is out there before you have passed judgment upon something because of its colors. More people search for functionality and use before practical looks. > so i think having a good looking installer is more than just eyecandy. i don't > think hardcore developers who prefer working with a bare X and vi(m)/emacs and > so forth should look down on people who are slowly starting to get involved in > an os, because without new people freebsd will someday be dead. > > alex > Certainly not going to be dead. Forums as of this email have 8,334 registered users alone. That's not counting all the Corporate entities, Yahoo if you may and Google if you will plus uncountable more. -- ;; dataix.net!jhell 2048R/89D8547E 2009-09-30 ;; BSD since FreeBSD 4.2 Linux since Slackware 2.1 ;; 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E From to.my.trociny at gmail.com Sat Oct 10 13:43:07 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Sat Oct 10 13:43:40 2009 Subject: crashtar In-Reply-To: (Alexander Best's message of "Sat\, 10 Oct 2009 12\:34\:05 +0200 \(CEST\)") References: Message-ID: <86d44vqs9l.fsf@kopusha.onet> On Sat, 10 Oct 2009 12:34:05 +0200 (CEST) Alexander Best wrote: AB> thanks. this is a cool script and very useful indeed. only thing you might AB> want to do is check for root privileges at the beginning to avoid nasty error AB> messages like. AB> awk: can't open file /var/crash/info.0 AB> source line number 12 In some cases you might not need root privileges. E.g. on some servers I don't have root but SA gives me read access to crashdumps. In this case if the script had a check for root privileges I would not be able to use it. Actually as for me the message looks informative enough, it says that we have some problems with accessing crash dump files, so permissions should be checked. -- Mikolaj Golub From simon at FreeBSD.org Sat Oct 10 14:28:00 2009 From: simon at FreeBSD.org (Simon L. Nielsen) Date: Sat Oct 10 14:28:52 2009 Subject: Is the FreeBSD ABI compatibility policy documented anywhere In-Reply-To: <4ABBD5FA.5070507@memberwebs.com> References: <4ABBD5FA.5070507@memberwebs.com> Message-ID: <20091010142758.GB1225@arthur.nitro.dk> On 2009.09.24 15:26:34 -0500, Stef Walter wrote: > It seems that FreeBSD has an ABI compatibility policy where major > versions remain ABI and API compatible throughout minor point versions. > That is to say that the kernel interfaces and libraries for (eg) > 7-STABLE, 7.1-RELEASE, 7.2-RELEASE are not supposed to change. It's not entirely that simple. The ABI on a stable branch like 7.x should be backward compatible, but there isn't a guarantee of forward compatibility. IE, 7.0 binary should be able to run on 7.x, but a 7.2 binary might not run on 7.0. It should be more or less the same with the API's. PS. do note that there is no 100% guarantee. At times the defacto policy might be violated if there are very good reasons for doing so. This would e.g. an important fix for something where the changed ABI, more likely K(kernel)BI, change should affect few people and the change is required for fixing some important bug. > Is this a policy of the project? If so, is it documented anywhere? Or is > it just a convention? I don't remember seeing it ever documented, just discussed. What I wrote above is also just my understanding of curreny defact policy. -- Simon L. Nielsen From des at des.no Sat Oct 10 17:11:53 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Sat Oct 10 17:12:00 2009 Subject: Is the FreeBSD ABI compatibility policy documented anywhere In-Reply-To: <20091010142758.GB1225@arthur.nitro.dk> (Simon L. Nielsen's message of "Sat, 10 Oct 2009 16:27:58 +0200") References: <4ABBD5FA.5070507@memberwebs.com> <20091010142758.GB1225@arthur.nitro.dk> Message-ID: <86d44vp415.fsf@ds4.des.no> "Simon L. Nielsen" writes: > It's not entirely that simple. The ABI on a stable branch like 7.x > should be backward compatible, but there isn't a guarantee of forward > compatibility. IE, 7.0 binary should be able to run on 7.x, but a 7.2 > binary might not run on 7.0. It should be more or less the same with > the API's. > > PS. do note that there is no 100% guarantee. Correct, but we're getting closer to that now that we have symbol versioning - although we won't reach 100% until we have versioned symbols in *all* libraries, which is currently not the case. Even then, a developer might break the ABI by mistake, but hopefully we'd catch that before it made it into a release. DES -- Dag-Erling Sm?rgrav - des@des.no From alex-goncharov at comcast.net Sat Oct 10 18:11:10 2009 From: alex-goncharov at comcast.net (Alex Goncharov) Date: Sat Oct 10 18:12:09 2009 Subject: Is the FreeBSD ABI compatibility policy documented anywhere In-Reply-To: <86d44vp415.fsf@ds4.des.no> (message from Dag-Erling Smørgrav on Sat, 10 Oct 2009 19:11:50 +0200) References: <4ABBD5FA.5070507@memberwebs.com> <20091010142758.GB1225@arthur.nitro.dk> <86d44vp415.fsf@ds4.des.no> Message-ID: ,--- You/Dag-Erling (Sat, 10 Oct 2009 19:11:50 +0200) ----* | "Simon L. Nielsen" writes: | > It's not entirely that simple. The ABI on a stable branch like 7.x | > should be backward compatible, but there isn't a guarantee of forward | > compatibility. IE, 7.0 binary should be able to run on 7.x, but a 7.2 | > binary might not run on 7.0. It should be more or less the same with | > the API's. | | Correct, but we're getting closer to that now that we have symbol | versioning - although we won't reach 100% until we have versioned | symbols in *all* libraries, which is currently not the case. Even then, | a developer might break the ABI by mistake, but hopefully we'd catch | that before it made it into a release. It's important to note that symbol compatibility is not the whole thing -- the behaviour on exceptions may be equally important to some applications. E.g. CMUCL code has to handle different ways to notify of memory protection failures this way: #if __FreeBSD_version < 700004 #define PROTECTION_VIOLATION_SIGNAL SIGBUS #define PROTECTION_VIOLATION_CODE BUS_PAGE_FAULT #else #define PROTECTION_VIOLATION_SIGNAL SIGSEGV #define PROTECTION_VIOLATION_CODE SEGV_ACCERR #endif A CMUCL binary built on a pre-7.1 (?) release of FreeBSD, will crash almost immediately when run on 7.1 (well, "if memory serves"). -- Alex -- alex-goncharov@comcast.net -- From kostikbel at gmail.com Sat Oct 10 18:21:08 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat Oct 10 18:21:13 2009 Subject: Is the FreeBSD ABI compatibility policy documented anywhere In-Reply-To: References: <4ABBD5FA.5070507@memberwebs.com> <20091010142758.GB1225@arthur.nitro.dk> <86d44vp415.fsf@ds4.des.no> Message-ID: <20091010182048.GD2259@deviant.kiev.zoral.com.ua> On Sat, Oct 10, 2009 at 02:11:07PM -0400, Alex Goncharov wrote: > ,--- You/Dag-Erling (Sat, 10 Oct 2009 19:11:50 +0200) ----* > | "Simon L. Nielsen" writes: > | > It's not entirely that simple. The ABI on a stable branch like 7.x > | > should be backward compatible, but there isn't a guarantee of forward > | > compatibility. IE, 7.0 binary should be able to run on 7.x, but a 7.2 > | > binary might not run on 7.0. It should be more or less the same with > | > the API's. > | > | Correct, but we're getting closer to that now that we have symbol > | versioning - although we won't reach 100% until we have versioned > | symbols in *all* libraries, which is currently not the case. Even then, > | a developer might break the ABI by mistake, but hopefully we'd catch > | that before it made it into a release. > > It's important to note that symbol compatibility is not the whole > thing -- the behaviour on exceptions may be equally important to some > applications. E.g. CMUCL code has to handle different ways to notify > of memory protection failures this way: > > #if __FreeBSD_version < 700004 > #define PROTECTION_VIOLATION_SIGNAL SIGBUS > #define PROTECTION_VIOLATION_CODE BUS_PAGE_FAULT > #else > #define PROTECTION_VIOLATION_SIGNAL SIGSEGV > #define PROTECTION_VIOLATION_CODE SEGV_ACCERR > #endif > > A CMUCL binary built on a pre-7.1 (?) release of FreeBSD, will crash > almost immediately when run on 7.1 (well, "if memory serves"). This has been an issue for 7.0, and it was explicitely handled, see r174254. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091010/a45c30d0/attachment.pgp From alex-goncharov at comcast.net Sat Oct 10 18:41:06 2009 From: alex-goncharov at comcast.net (Alex Goncharov) Date: Sat Oct 10 18:41:12 2009 Subject: Is the FreeBSD ABI compatibility policy documented anywhere In-Reply-To: <20091010182048.GD2259@deviant.kiev.zoral.com.ua> (message from Kostik Belousov on Sat, 10 Oct 2009 21:20:48 +0300) References: <4ABBD5FA.5070507@memberwebs.com> <20091010142758.GB1225@arthur.nitro.dk> <86d44vp415.fsf@ds4.des.no> <20091010182048.GD2259@deviant.kiev.zoral.com.ua> Message-ID: ,--- You/Kostik (Sat, 10 Oct 2009 21:20:48 +0300) ----* | > A CMUCL binary built on a pre-7.1 (?) release of FreeBSD, will crash | > almost immediately when run on 7.1 (well, "if memory serves"). | | This has been an issue for 7.0, and it was explicitely handled, see r174254. I know it was (I was the one who first ran into the problem, way back then). My point here was to state that the behavior on signals is almost as important as symbols' compatibility. As for "it was", see this few-days-old thread: http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/287d1a57226f0cda/a6805cf10ca0cab9?q=cmucl&lnk=nl& I found an even easier way to crash CMUCL 19c/19f on FreeBSD 7.1 where the last message was posted today. Although most messages there are totally unrelated to CMUCL, FreeBSD (in fact, almost anything relevant), somebody, somewhere still hit the issue a few days ago (I am not inclined to analyze his environment, frankly). The question, "Will a FreeBSD 7.2 (e.g.) build run all right on FreeBSD 8.0 (e.g.)?", is often asked, and I think it is impossible to give any guarantees -- the best one can realistically say, is, "probably" (with compatibility libraries, in some case). At least, I don't give any assurances for the binaries I upload to http://common-lisp.net/project/cmucl/downloads: I build against the latest code on a few branches of FreeBSD (RELENG and CURRENT) -- and if a 7.2 build runs for you on 7.1 and/or 8.0, you are in luck (and I do expect this), but it may not, and who will tell you it will, without trying? -- Alex -- alex-goncharov@comcast.net -- From ed at 80386.nl Sun Oct 11 14:29:25 2009 From: ed at 80386.nl (Ed Schouten) Date: Sun Oct 11 14:29:32 2009 Subject: sysinstall colours In-Reply-To: <3a142e750910100150g7459e037u7b84b4b4bbc2bf8@mail.gmail.com> References: <3a142e750910100150g7459e037u7b84b4b4bbc2bf8@mail.gmail.com> Message-ID: <20091011142923.GT71731@hoeg.nl> * Paul B Mahol wrote: > This have nothing to do with ncurses, colors you like simple can not > be displayed in current syscons(4) and making support for 256 colors > or even true bit color in sysinstall(so that it looks amazing in > konsole) is waste of time. Yes. As of recently, our terminal emulator gained 256 color support, but this gets smashed down to 8 colors, simply because Syscons cannot handle more than 16 foreground and 8 background colors. This is how colors are converted: http://80386.nl/pub/xterm-256.png http://80386.nl/pub/teken-256.png -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091011/67bb04e8/attachment.pgp From uqs at spoerlein.net Sun Oct 11 14:50:23 2009 From: uqs at spoerlein.net (Ulrich =?utf-8?B?U3DDtnJsZWlu?=) Date: Sun Oct 11 14:50:30 2009 Subject: RFC: Big Makefile patch for WARNS settings Message-ID: <20091011145021.GG36937@acme.spoerlein.net> Dear -hackers, I would like you to give me your thoughts on the attached patch. There are no functional changes, what I'm trying to do is introduce WARNS?=6 for all top-level Makefiles and override that on a subdir basis. Why the churn? Because I think it sticks out more, if there's a WARNS=0 in your Makefile rather than the default being WARNS=0. Perhaps this gets more incentive going for fixing these WARNS. There's also a lot of work done by the DragonflyBSD folks which I intend to port peu a peu. Note: There are lots of style changes for games/ and sbin/, which I can seperate out for easier review. But I'd like to make some sweeping patches to bring them more inline with style.Makefile(5) Comments? Committers? Uli -------------- next part -------------- A non-text attachment was scrubbed... Name: warns.diff.gz Type: application/octet-stream Size: 23202 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091011/c7c70c61/warns.diff.obj From ed at 80386.nl Sun Oct 11 17:09:19 2009 From: ed at 80386.nl (Ed Schouten) Date: Sun Oct 11 17:09:27 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091011145021.GG36937@acme.spoerlein.net> References: <20091011145021.GG36937@acme.spoerlein.net> Message-ID: <20091011170918.GU71731@hoeg.nl> Hi Ulrich, * Ulrich Sp?rlein wrote: > Comments? Committers? Wouldn't it better to address the root of the problem while there? ;-) Index: number.c =================================================================== --- number.c (revision 197852) +++ number.c (working copy) @@ -88,9 +88,7 @@ int lflag; int -main(argc, argv) - int argc; - char *argv[]; +main(int argc, char *argv[]) { int ch, first; char line[256]; @@ -275,7 +273,7 @@ pfract(len) int len; { - static char *pref[] = { "", "ten-", "hundred-" }; + static char const * const pref[] = { "", "ten-", "hundred-" }; switch(len) { case 1: -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091011/ea673db5/attachment.pgp From danger at FreeBSD.org Sun Oct 11 17:54:29 2009 From: danger at FreeBSD.org (Daniel Gerzo) Date: Sun Oct 11 17:54:53 2009 Subject: FreeBSD Status Reports April - September, 2009 Message-ID: <20091011175428.GA3626@freefall.freebsd.org> FreeBSD Quarterly Status Report Introduction This report covers FreeBSD related projects between April and September 2009. During that time a lot of work has been done on wide variety of projects, including the Google Summer of Code projects. The BSDCan conference was held in Ottawa, CA, in May. The EuroBSDCon conference was held in Cambridge, UK, in September. Both events were very successful. A new major version of FreeBSD, 8.0 is to be released soon. If you are wondering what's new in this long-awaited release, read Ivan Voras' excellent summary. Thanks to all the reporters for the excellent work! We hope you enjoy the reading. Please note that the next deadline for submissions covering reports between October and December 2009 is January 15th, 2010. __________________________________________________________________ Google Summer of Code * About Google Summer of Code 2009 * BSD-licensed iconv (Summer of Code 2009) * BSD-licensed text-processing tools (Summer of Code 2008) * Ext2fs Status report (Summer of Code 2009) * libnetstat(3) - networking statistics (Summer of Code 2009) * pefs - stacked cryptographic filesystem (Summer of Code 2009) Projects * BSD# Project * Clang replacing GCC in the base system * FreeBSD TDM Framework * Grand Central Dispatch - FreeBSD port * libprocstat(3) - process statistics * New BSD licensed debugger * NFSv4 ACLs * The Newcons project * VirtualBox on FreeBSD FreeBSD Team Reports * FreeBSD Bugbusting Team * FreeBSD KDE Team * FreeBSD Ports Management Team * Release Engineering Status Report * The FreeBSD Foundation Status Report Network Infrastructure * Enhancing the FreeBSD TCP Implementation * Modular Congestion Control * Network Stack Virtualization * Stream Control Transmission Protocol (SCTP) Kernel * FreeBSD/ZFS * hwpmc for MIPS Documentation * The FreeBSD Dutch Documentation Project * The FreeBSD German Documentation Project * The FreeBSD Hungarian Documentation Project * The FreeBSD Spanish Documentation Project Architectures * FreeBSD/sparc64 Ports * FreeBSD Gecko Project * Portmaster - utility to assist users with managing ports * Valgrind suite on FreeBSD Miscellaneous * EuroBSDcon 2009 * FreeBSD Developer Summit, Cambridge UK * New approach to the locale database * The FreeBSD Forums __________________________________________________________________ About Google Summer of Code 2009 URL: http://socghop.appspot.com/org/home/google/gsoc2009/freebsd URL: http://wiki.freebsd.org/SummerOfCode2009Projects Contact: Brooks Davis Contact: Tim Kientzle Contact: Robert Watson 2009 was The FreeBSD Project's fifth year of participation in the Google Summer of Code. We had a total of 17 successful projects. Some GSoC code will be shipping with FreeBSD 8.0-RELEASE and others will be integrated into future releases. The FreeBSD GSoC admin team would like to thank Google and our students and mentors of another great year! __________________________________________________________________ BSD# Project URL: http://code.google.com/p/bsd-sharp/ URL: http://www.mono-project.org/ Contact: Romain Tarti?re The BSD# Project is devoted to porting the Mono .NET framework and applications to the FreeBSD operating system. During the past year, the BSD# Team continued to track the Mono development and the lang/mono port have almost always been up-to-date (we however had to skip mono-2.2 because of some regression issues in this release). Most of our patches have been merged in the mono trunk upstream, and should be included in the upcoming mono-2.6 release. In the meantime, a few more .NET related ports have been updated or added to the FreeBSD ports tree. These ports include: * www/xsp and www/mod_mono that make it possible to use FreeBSD for hosting ASP.NET application; * lang/boo, a CLI-targeted programming language similar to Python; * lang/mono-basic, the Visual Basic .NET Framework for Mono; * devel/monodevelop, an Integrated Development Environment for .NET; * and much more... Open tasks: 1. Test mono ports and send feedback (we are especially interested in tests where NOPORTDOCS / WITH_DEBUG is enabled). 2. Port the mono-debugger to FreeBSD. 3. Build a debug live-image of FreeBSD so that Mono hackers without a FreeBSD box can help us fixing bugs more efficiently. __________________________________________________________________ BSD-licensed iconv (Summer of Code 2009) URL: http://wiki.freebsd.org/G%C3%A1borSoC2009 Contact: G?bor K?vesd?n The code has been extracted from NetBSD and has been transformed into an independent shared library. The basic encodings are well supported. Almost all forward conversions (foo -> UTF-32) are compatible with GNU but the reverse ones are not so accurate because of GNU's advanced transliteration. Some extra encodings have also been added. There are two modules, which segfault; they need some debugging. I can keep working on this project as part of my BSc thesis, so I hope to be able to solve the remaining issues. Improved GNU compatibility is also very desired (extra command line options for iconv(1), iconvctl(), private interfaces, etc.). Open tasks: 1. Fix segfaults in Big5 and HZ modules 2. Improve transliteration in reverse encodings 3. Improve GNU compatibility by implementing extra features 4. Verify POSIX compatibility 5. Verify GNU compatibility 6. Check performance __________________________________________________________________ BSD-licensed text-processing tools (Summer of Code 2008) URL: http://wiki.freebsd.org/G%C3%A1borSoC2008 Contact: G?bor K?vesd?n This project was started as part of Google Summer of Code 2008 but there is still a bit of work to complete some missing parts. The BSD-licensed grep implementation is feature-complete and has a good level of GNU compatibility. Our only current concern about the BSD-licensed version is to improve its performance. The GNU variant is much more complex, has about 8 KSLOC, while BSD grep is tiny, has only 1.5 KSLOC. GNU uses some shortcuts and optimizations to speed-up calls to the regex library; that is why it is significantly faster. My point of view is that such optimizations must be implemented in the regex library, keeping the dependent utilities clean and easy to read. BSD grep is so tiny that there is hardly any optimization opportunity by simplifying the code, so the regex library is the next important TODO. There is another issue with the current regex library. It does not support some invalid regular expressions, which work in GNU. We need to maintain compatibility, so we cannot just drop this feature. Actually, BSD grep is linked to the GNU regex library to maintain this feature but due to the lack of the mentioned shortcuts, it is still slower than GNU. Anyway, if we can live with this little performance hit until we get a modern regex library, I think grep is ready to enter HEAD. As for the regex library, NetBSD's result of the last SoC is worth taking a look. The sort utility has been rewritten from scratch. The existing BSD-licensed implementation could not deal with wide characters by design. The new implementation is still lacking some features but is quite complete. There is a performance issue, though. Sorting is a typical algorithmic subject but I am not an algorithmic expert, so my implementation is not completely optimal. Some help would be welcome with this part. The bc/dc utilities have been ported from OpenBSD. They pass OpenBSD's and GNU's regression tests but they arrived too late to catch 8.X, so they will go to HEAD after the release. Open tasks: 1. Improve sort's sorting and file merging algorithms 2. Complete missing features for sort 3. Get a modern regex library for FreeBSD __________________________________________________________________ Clang replacing GCC in the base system URL: http://wiki.freebsd.org/BuildingFreeBSDWithClang Contact: Ed Schouten Contact: Roman Divacky Contact: Brooks Davis Contact: Pawel Worach The clang@FreeBSD team presents the status of clang/LLVM being able to compile FreeBSD system. The current status is: * i386 - kernel boots, world needs little hacks but works * amd64 - kernel boots, world needs little hacks but works * ppc - broken because of unknown RTLD bug * other - unknown All other platforms are untested. A lot has happened over the spring/summer: amd64 got proper mcmodel=kernel support, compiler-rt has been introduced (paving the way for libgcc replacement), we have run two experimental port builds to see how clang does there. The C++ support is able to parse devd.cc without warnings. We have got the kernel working with -O2. FreeBSD has been promoted to be an officially supported plaform in LLVM. As a result of all this work, many parts of FreeBSD that did not compile before now build without problems. Open tasks: 1. The "ClangBSD" branch of FreeBSD got a little stale and has not been updated for a while. 2. We also need to get some important fixes into LLVM to get libc compiling and some other smaller issues. 3. We can still appreciate more testers on minor platforms (mostly on ARM, PPC and MIPS, but testing on other platforms is also welcome). __________________________________________________________________ Enhancing the FreeBSD TCP Implementation URL: http://caia.swin.edu.au/freebsd/etcp09/ URL: http://caia.swin.edu.au/urp/newtcp/ URL: http://www.freebsdfoundation.org/projects.shtml URL: http://people.freebsd.org/~lstewart/patches/tcp_ffcaia2008/ Contact: Lawrence Stewart TCP appropriate byte counting (RFC 3465) support has been merged into the FreeBSD 8 branch and will ship in FreeBSD 8.0-RELEASE. The reassembly queue auto-tuning and SIFTR work was not ready in time to safely integrate for 8.0-RELEASE. Padding has been added to necessary TCP structs to facilitate MFCing features back to the 8-STABLE branch after 8.0 is released. Candidate patches against FreeBSD-CURRENT will be ready for wider testing in the coming weeks. The freebsd-net mailing list will be solicited for testing/feedback when everything is ready. Open tasks: 1. Solicit review/testing and integrate the ALQ kld and variable length message support patch into FreeBSD-CURRENT. 2. Solicit review/testing and integrate the SIFTR tool into FreeBSD-CURRENT. 3. Complete dynamic reassembly queue auto-tuning patch for FreeBSD-CURRENT. 4. Fix an identified bug in the SACK implementation's fast retransmit/fast recovery behavior. 5. Profit! __________________________________________________________________ EuroBSDcon 2009 URL: http://2009.eurobsdcon.org/ URL: http://2010.eurobsdcon.org/ Contact: Sam Smith Contact: Robert Watson EuroBSDcon 2009 happened in Cambridge, with over 160 users, developers, friends and others. Slides, papers and audio are now up on the website for those who could not make it to Cambridge. Next year's event in 2010 will take place in Karlsruhe from 8 to 10 October 2010. If you are interested in what you missed in 2009, or to join the mailing list so you do not miss out next year, visit http://2009.eurosbsdcon.org. __________________________________________________________________ Ext2fs Status report (Summer of Code 2009) URL: http://wiki.freebsd.org/SOC2009AdityaSarawgi Contact: Aditya Sarawgi FreeBSD's ext2fs had some parts under GPL. The aim of my project was to rewrite those parts and free ext2fs from GPL. I have been successful in rewriting the parts and NetBSD's ext2fs was a great help in this. Certain critical parts under GPL were also removed due to which the write performance suffered. I also implemented Orlov Block Allocator for ext2fs. Currently I am planning to make ext2fs Multiprocessor Safe (MPSAFE). My work resides in truncs_ext2fs branch of Perforce. Open tasks: 1. Ext4 support for FreeBSD 2. Directory indexing for ext2fs 3. Journaling in ext2fs using gjournal __________________________________________________________________ FreeBSD Bugbusting Team URL: http://www.FreeBSD.org/support.html#gnats URL: http://wiki.FreeBSD.org/BugBusting URL: http://people.FreeBSD.org/~linimon/studies/prs/ URL: http://people.FreeBSD.org/~linimon/studies/prs/recommended_prs.html Contact: Gavin Atkinson Contact: Mark Linimon Contact: Remko Lodder Contact: Volker Werth We continue to classify PRs as they arrive, adding 'tags' to the subject lines corresponding to the kernel subsystem involved, or man page references for userland PRs. These tags, in turn, produce lists of PRs sorted both by tag and by manpage. The list of PRs recommended for committer evaluation by the Bugbusting Team continues to receive new additions. This list contains PRs, mostly with patches, that the Bugbusting Team feel are probably ready to be committed as-is, or are probably trivially resolved in the hands of a committer with knowledge of the particular subsystem. All committers are invited to take a look at this list whenever they have a spare 5 minutes and wish to close a PR. A full list of all the automatically generated reports is also available at one of the cited URLs. Any recommendations for reports which not currently exist but which would be beneficial are welcomed. Gavin Atkinson gave a presentation on "The PR Collection Status" at the EuroBSDCon 2009 DevSummit, and discussed with other participants several other ideas to make the PR database more useful and usable. Several good ideas came from this, and will hopefully lead to more useful tools in the near future. Discussions also took place on how it may be possible to automatically classify non-ports PRs with a view towards notifying interested parties, although investigations into this have not yet begun. Mark Linimon also continues attempting to define the general problem and investigating possible new workflow models, and presented work on this at BSDCan 2009. Since the last status report, the number of open bugs has increased to around the 5900 mark, partially because of an increased focus on getting more information into the existing PRs, in an attempt to make sure all the information required is now available. As a result, although the number of open PRs has increased, they are hopefully of better quality. As always, more help is appreciated, and committers and non-committers alike are always invited to join us on #freebsd-bugbusters on EFnet and help close stale PRs or commit patches from valid PRs. Open tasks: 1. Work on suggestions from developers who were at the EuroBSDCon DevSummit. 2. Try to find ways to get more committers helping us with closing the PRs that the team has already analyzed. __________________________________________________________________ FreeBSD Developer Summit, Cambridge UK URL: http://wiki.FreeBSD.org/200909DevSummit Contact: Robert Watson Around 70 FreeBSD developers and guests attended the FreeBSD developer summit prior to EuroBSDCon 2009 in Cambridge, UK. Hosted at the University of Cambridge Computer Laboratory, the workshop-style event consisted of prepared presentations, as well as group hacking and discussion sessions. Talks covered topics including 802.11 mesh networking, virtual network stacks and kernels, a new BSD-licensed debugger, benchmarking, bugbusting, NetFPGA, a port of Apple's GCD (Grand Central Dispatch) to FreeBSD, security policy work, cryptographic signatures, FreeBSD.org system administration, time geeks, a new console driver, and the FreeBSD subversion migration. Slides for many talks are now available on the wiki page. A good time was had by all, including a punting outing on the River Cam! __________________________________________________________________ FreeBSD Gecko Project URL: https://trillian.chruetertee.ch/freebsd-gecko/wiki/TODO Contact: Beat Gaetzi Contact: Martin Wilke Contact: Andreas Tobler Andreas Tobler made the classic mistake of sending us a lot of powerpc and sparc64 related patches. The usual punishment, of giving him a commit bit to the Gecko repository, has been applied. We currently have some old ports in the ports tree: * www/mozilla is 5 year old now, no longer supported upstream, and has a lot of security vulnerabilities. We can use www/seamonkey instead. * www/xulrunner is superseeded by www/libxul. A patch that includes the following changes has been tested on pointyhat and is ready for commit: * Remove references to www/mozilla/Makefile.common and www/mozilla/bsd.gecko.mk * Switch USE_GECKO= xulrunner firefox mozilla to USE_GECKO= libxul and remove www/xulrunner We are also working on Firefox 3.6 (Alpha 2), Thunderbird 3.0 (Beta 4), new libxul 1.9.1.3 and Seamonkey 2.0 (Beta 2) ports. All of them are already committed to our Gecko repository. A current status and todo list can be found at http://trillian.chruetertee.ch/freebsd-gecko/wiki/TODO. Open tasks: 1. Remove mozilla, xulrunner and firefox2 from the ports tree. 2. The www/firefox35 port should be moved to www/firefox. 3. The old (and somewhat stale) Gecko providers mozilla, nvu, xulrunner, flock and firefox also need to be removed. __________________________________________________________________ FreeBSD KDE Team URL: http://freebsd.kde.org URL: http://miwi.bsdcrew.de/category/kde/ URL: http://blogs.freebsdish.org/tabthorpe/category/kde Contact: Thomas Abthorpe Contact: Max Brazhnikov Contact: Martin Wilke Since the spring, the FreeBSD KDE team has been busy upgrading KDE from 4.2.0 up through to 4.3.1. As part of the ongoing maintenance of KDE, the team also updated Qt4 from 4.4.3 through to 4.5.2 We added two new committers/maintainers to the team, Kris Moore (kmoore@) and Dima Panov (fluffy@). We also granted enhanced area51 access to contributors Alberto Villa and Raphael Kubo da Costa. Alberto has been our key contributor updating and testing Qt 4.6.0-tp1. Raphael is a KDE developer, who has become our Gitorious liaison, he has been responsible for getting FreeBSD Qt patches merged in upstream. Markus Br?ffer (markus@) spent a lot of time patching widgets and system plugins so they would work under FreeBSD. We would like to thank him for all his effort! Open tasks: 1. Update to Qt 4.6.0 2. Update to KDE 4.4.0 3. Work with our userbase on fixing an EOL for KDE3 in the ports tree __________________________________________________________________ FreeBSD Ports Management Team URL: http://www.freebsd.org/ports/ URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/ URL: http://portsmon.FreeBSD.org/index.html URL: http://www.freebsd.org/portmgr/index.html URL: http://tinderbox.marcuscom.com Contact: Mark Linimon The ports count has soared to over 20,700. The PR count had been driven below 800 by some extraordinary effort, but once again is back to its usual count of around 900. We are currently building packages for amd64-6, amd64-7, amd64-8, i386-6, i386-7, i386-8, sparc64-7, and sparc64-8. There have been preliminary runs of i386-9; however, to be able to continue builds on -9, we will either need to find places to host a number of new machines, or drop package building for -6. The mailing list discussion of the latter proved quite controversial. We have added some new i386 machines to help speed up the builds, but this only makes up for the disk failures on some of our older, slower, i386 nodes. We also appreciate the loan of more package build machines from several committers, including pgollucci@, gahr@, erwin@, Boris Kochergin, and Craig Butler. The portmgr@ team has also welcomed new members Ion-Mihai Tetcu (itetcu@) and Martin Wilke (miwi@). We also thank departing member Kirill Ponomarew (krion@) for his long service. Ion-Mihai has spent much time working on a system that does automatic Quality Assurance on new commits, called QAT. A second tinderbox called QATty has helped us to fix many problems, especially those involving custom PREFIX and LOCALBASE settings, and documentation inclusion options. Ports conformance to documented features / non-default configuration will follow. Between pav and miwi, over 2 dozen experimental ports runs have been completed and committed. We have added 5 new committers since the last report, and 2 older ones have rejoined. Open tasks: 1. We are currently trying to set up ports tinderboxes that can be made available to committers for pre-testing; those who can loan machines for this should contact Ion-Mihai (itetcu@) with details regarding the hardware and bandwidth. 2. Most of the remaining ports PRs are "existing port/PR assigned to committer". Although the maintainer-timeout policy is helping to keep the backlog down, we are going to need to do more to get the ports in the shape they really need to be in. 3. Although we have added many maintainers, we still have almost 4,700 unmaintained ports (see, for instance, the list on portsmon). (The percentage is down to 22%.) We are always looking for dedicated volunteers to adopt at least a few unmaintained ports. As well, the packages on amd64 and sparc64 lag behind i386, and we need more testers for those. __________________________________________________________________ FreeBSD TDM Framework Contact: Rafal Czubak Contact: Michal Hajduk This work's purpose is a generic and flexible framework for systems equipped with Time Division Multiplexing (TDM) units, often found on embedded telecom chips. The framework is designed to support various controllers and many types of TDM channels e.g. voiceband, sound and miscellaneous data channels. Currently, voiceband infrastructure is being developed on Marvell RD-88F6281 reference board. It will serve as an example of how to use the TDM framework for other channel types. The direct objective of using TDM with voiceband channels is bringing a FreeBSD based VoIP system, capable of bridging analog telephone world with digital IP telephony. Together with third party VoIP software (e.g. Asterisk), the design can serve as VoIP Private Branch Exchange (PBX). Current state highlights: * TDM controller interface * TDM channel interface * TDM channel API for kernel modules * codec interface * voiceband channel character device driver * TDM controller driver for Marvell Kirkwood and Discovery SoCs * Si3215 SLIC driver * Si3050 DAA driver Open tasks: 1. Develop demo application showing example usage of voiceband channel. 2. Integrate voiceband infrastructure with Zaptel/DAHDI telephony hardware drivers. __________________________________________________________________ FreeBSD/sparc64 Contact: Marius Strobl Noteworthy developments regarding FreeBSD/sparc64 since the last Status Reports are: * Cas(4), a driver for Sun Cassini/Cassini+, as well as National Semiconductor DP83065 Saturn Gigabit NICs has been committed and thus will be part of FreeBSD beginning with 8.0-RELEASE and 7.3-RELEASE, respectively. This means that the on-board NICs found in Fire V440, as well as the add-on cards based on these chips, are now supported, including on non-sparc64 machines. Unfortunately, the cas(4) driver triggers what seem to be secondary problems with the on-board NICs found in B100 blades and Fire V480, which due to lack of access to such systems could not be fixed so far. * Initial support for sun4u machines based on the "Fire" Host-PCI-Express bridge like Fire V215, V245, etc. has been completed (including support for the on-board ATA controller, which caused several problems at first, and MSI/MSI-X). Some code like the quirk handling for the ALi/ULi chips found in these machines needs to be revisited though and no stability tests have been conducted so far. If all goes well, the code will hit HEAD some time after FreeBSD 8.0-RELEASE has been released. In theory, machines based on the "Oberon" Host-PCI-Express bridge, at least for the most part, should also be supported with these changes, but due to lack of access to a Mx000 series machine the code could not be tested with these so far. * Some bugs in the snd_t4dwave(4) driver have been fixed, as well as some special handling for sparc64 has been added so it does 32-bit DMA and now generally works with the on-board ALi M5451 found for example in Blade 100 and Blade 1500. Unfortunately, it was only tested to work correctly in two out of three Blade 100. Why it still does not work correctly in the remaining one is currently unknown but at least no longer causes IOMMU-panics so testing snd_t4dwave(4) on sparc64 is no longer harmful. These changes will be part of FreeBSD 8.0-RELEASE and 7.3-RELEASE. * Ata-marvell(4) has been fixed to work on sparc64 (actually also on anything that is not x86 with less than 4GB of RAM). These fixes will be part of FreeBSD 8.0-RELEASE and 7.3-RELEASE. * A proper and machine-independent fix for the old problem that the loader leaves the NIC opened by the firmware, which could lead to panics during boot when netbooting, has been developed but not committed yet. Open tasks: __________________________________________________________________ FreeBSD/ZFS Contact: Pawel Dawidek We believe that the ZFS file system is now production-ready in FreeBSD 8.0. Most (if not all) reported bugs were fixed and ZFS is no longer tagged as experimental. There is also ongoing work in Perforce to bring the latest ZFS version (v19) to FreeBSD. Open tasks: 1. Download 8.0 release candidates and test, test, test and report any problems to the freebsd-fs@FreeBSD.org mailing list. __________________________________________________________________ Grand Central Dispatch - FreeBSD port URL: http://libdispatch.macosforge.org/ Contact: Robert Watson Contact: Stacey Son Contact: libdispatch mailing list We have ported libdispatch, Apple's Grand Central Dispatch event and concurrency framework to FreeBSD: * Added new kqueue primitives required to support GCD, such as EVFILT_USER and EV_TRIGGER * Created autoconf and automake build framework for libdispatch * Modified libdispatch to use POSIX semaphores instead of Mach semaphores * Adapted libdispatch to use portable POSIX time routines Jordan Hubbard has also prepared a blocks-aware clang compiler package for FreeBSD. When compiled with clang, libdispatch provides blocks-based, as well as function-based callbacks. The port was presented at the FreeBSD Developer Summit in Cambridge, UK in September, and slides are online on the devsummit wiki page. A FreeBSD port is now available in the Ports Collection. After FreeBSD 8.0-RELEASE has shipped, the new kqueue primitives will be MFC'd so that libdispatch works out of the box on FreeBSD 8.1-RELEASE. Open tasks: 1. Complete porting of libdispatch test suite to FreeBSD. 2. Investigate pthread work queue implementation for FreeBSD. 3. Evaluate performance impact of some machine-dependent and OS-dependent optimizations present in the Mac OS X version of libdispatch to decide if they should be done for other platforms and OS's. 4. Explore whether FreeBSD base operating system tools would benefit from being modified to use libdispatch. __________________________________________________________________ hwpmc for MIPS URL: http://wiki.freebsd.org/FreeBSD/mips URL: http://wiki.freebsd.org/FreeBSD/mips/UBNT-RouterStationPro Contact: George Neville-Neil Currently working on board bringup. I have looked over the docs for how MIPS provides performance counters and will begin adding code soon. __________________________________________________________________ libnetstat(3) - networking statistics (Summer of Code 2009) URL: http://wiki.FreeBSD.org/PGJSoc2009 URL: http://p4web.freebsd.org/@md=d&cd=//&c=McZ@//depot/projects/soc2009/pgj _libstat/?ac=83 Contact: G?bor P?li The libnetstat(3) project provides a user-space library API to monitor networking functions with the following benefits: * ABI-robust interface making use of accessor functions in order to divorce monitoring applications from kernel or user ABI changes. * Supports running 32-bit monitoring tools on top of a 64-bit kernel. * Improved consistency for both kvm(3) and sysctl(3) when retrieving information. The supported abstractions are as follows: * Active sockets and socket buffers * Network interfaces and multicast interfaces * mbuf(9) statistics * bpf(4) statistics * Routing statistics, routing tables, multicast routing * Protocol-dependent statistics There is a sample application, called nettop(8), which provides a simple ncurses-based top(1)-like interface for monitoring active connections and network buffer allocations via the library. A modified version of netstat(1) has also been created to use libnetstat(3) as much as possible. __________________________________________________________________ libprocstat(3) - process statistics URL: http://svn.freebsd.org/viewvc/base/projects/libprocstat/ Contact: Stanislav Sedov Contact: Ulf Lilleengen The libprocstat project is an ongoing effort to develop a library that can be used to retrieve information about running processes and open files in the uniform and platform-independent way both from a running system or from core files. This will facilitate the implementation of file- or process-monitoring applications like lsof(1), fstat(1), fuser, etc. The libprocstat repository contains a preliminary version of the library. It also includes rewrites of the fstat and the fuser utilities ported to use this library instead of retrieving all the required information via the kvm(3) interface; one of the important advantages of the versions that use libprocstat is that these utilities are ABI independent. Open tasks: 1. Implement KVM-based namecache lookup to retrieve filesystem paths associated with file descriptors and VM objects. 2. Analyze possible ways of exporting file and process information from the kernel in an extensible and ABI-independent way. __________________________________________________________________ Modular Congestion Control URL: http://caia.swin.edu.au/urp/newtcp/ URL: http://svn.freebsd.org/viewvc/base/projects/tcp_cc_8.x/ Contact: Lawrence Stewart The patch has received some significant rototilling in the past few months to prepare it for merging to FreeBSD-CURRENT. Additionally, I completed an implementation of the CUBIC congestion control algorithm to complement the existing NewReno and H-TCP algorithm implementations already available. I have one further intrusive change to make, which will allow congestion control modules to be shared between the TCP and SCTP stacks. Once this is complete, I will be soliciting for review/testing in the hope of committing the patch to FreeBSD-CURRENT in time to be able to backport it for 8.1-RELEASE. Open tasks: 1. Abstract the congestion control specific variables out of the TCP and SCTP control blocks into a new struct that can be passed into the API instead of the control block itself. __________________________________________________________________ Network Stack Virtualization URL: http://wiki.freebsd.org/Image URL: http://wiki.freebsd.org/200909DevSummit Contact: Bjoern A. Zeeb Contact: Marko Zec Contact: Robert Watson The network stack virtualization project aims at extending the FreeBSD kernel to maintain multiple independent instances of networking state. This allows for networking independence between jail environment, each maintaining its private network interfaces, IPv4 and IPv6 network and port address space, routing tables, IPSec configuration, firewalls, and more. During the last months the remaining pieces of the VIMAGE work were merged by Marko, Julian and Bjoern. Robert Watson developed a vnet allocator to overcome ABI issues. Jamie Gritton merged his hierachical jail framework that now also is the management interface for virtual network stacks. During the FreeBSD Developer Summit that took place at EuroBSDCon 2009 in Cambridge, UK, people virtualized more code. As a result SCTP and another accept filter were virtualized and more people became familiar with the design of VImage and the underlying concepts. Finally getting more hands involved was a crucial first step for the long term success of kernel virtualization. The next steps will be to finish the network stack virtualization, generalize the allocator framework before thinking of virtualizing further subsystems and to update the related documentation. Along with that a proper jail management framework will be worked on. Long term goals, amongst others, will be to virtualize more subsystems like SYS-V IPC, better privilege handling, and resource limits. In the upcoming FreeBSD 8.0 Release, vnets are treated as an experimental feature. As a result, they are not yet recommended for use in production environments. There was lots of time spent to finalize the infrastructure for vnets though, so that further changes can be merged and we are aiming to have things production ready for 8.2. In case you want to help to achieve this goal, feel free to contact us and support or help virtualizing outstanding parts like two firewalls, appletalk, netipx, ... as well as generating regression tests. __________________________________________________________________ New approach to the locale database URL: http://wiki.freebsd.org/LocaleNewApproach URL: svn://svn.freebsd.org/base/user/edwin/locale Contact: Edwin Groothuis Contact: i18n mailinglist Problem: Over the years the FreeBSD locale database (share/colldef, share/monetdef, share/msgdef, share/numericdef, share/timedef) has accumulated a total of 165 definitions (language - country-code - character-set triplets). The contents of the files for Western European languages are often low-ASCII but for Eastern European and Asian languages partly or fully high-ASCII. Without knowing how to display or interpret the character-sets, it is difficult to make sure by the general audience that the local language (language - country-code) definitions are displayed properly in various character-sets. Suggested approach: With the combination of the data in the Unicode project (whose goal is to define all the possible written characters and symbols on this planet) and the Common Locale Data Repository (whose goal is to document all the different data and definitions needed for the locale database), we can easily keep track of the data, without the need of being able to display the data in the required character sets or understand them fully when updates are submitted by third parties. Current status: Conversion of share/monetdef, share/msgdef, share/numericdef, share/timedef to the new design is completed. The Makefile infrastructure is converted. Regression checks are done. Most of the tools are in place, waiting on the import of bsdiconv to the base system. Open tasks: 1. At this moment the system is not self-hosted yet, because of the lack of an iconv-kind of program in the base operating system. Gabor@ is working on bsdiconv as a GSoC project and once that has been imported we will be able to perform a clean install from the definitions in Unicode text format to the required formats and character sets. __________________________________________________________________ New BSD licensed debugger URL: http://wiki.freebsd.org/TheBsdDebugger URL: http://people.freebsd.org/~dfr/ngdb.git URL: http://wiki.freebsd.org/200909DevSummit?action=AttachFile&do=view&targe t=NGDB-200909.pdf Contact: Doug Rabson <> I have been working recently on writing a new debugger, primarily for the FreeBSD platform. For various reasons, I have been writing it in a relatively obscure C-like language called D. So far, I have a pretty useful (if a little raw at the edges) command line debugger which supports ELF, Dwarf debugging information and (currently) 32 bit FreeBSD and Linux. The engine includes parsing and evaluation of arbitrary C expressions along with the usual debugging tools such as breakpoints, source code listing, single-step etc. All the code is new and BSD licensed. Currently, the thing supports userland debugging of i386 targets via ptrace and post-mortem core file debugging of the same. I will be adding amd64 support real soon (TM) and maybe support for GDB's remote debugging protocol later. __________________________________________________________________ NFSv4 ACLs URL: http://wiki.freebsd.org/NFSv4_ACLs Contact: Edward Tomasz Napierala During Google Summer of Code 2008, I have implemented native support for NFSv4 ACLs for both ZFS and UFS. Most of the code has already been merged to CURRENT. NFSv4 ACLs are unconditionally enabled in ZFS and the usual tools, like getfacl(1) and setfacl(1) can be used to view and change them. I plan to merge the remaining bits (UFS support) this month. It should be possible to MFC it in order to ship in FreeBSD 8.1-RELEASE. Open tasks: 1. UFS changes review 2. Support for NFSv4 ACLs in tar(1) __________________________________________________________________ pefs - stacked cryptographic filesystem (Summer of Code 2009) URL: http://blogs.freebsdish.org/gleb/ URL: http://wiki.freebsd.org/SOC2009GlebKurtsov Contact: Gleb Kurtsou Contact: Stanislav Sedov Pefs is a kernel level filesystem for transparently encrypting files on top of other filesystems (like zfs or ufs). It adds no extra information into files (unlike others), doesn't require cipher block sized io operations, supports per directory/file keys and key chaining, uses unique per file tweak for encryption. Supported algorithms: AES, Camellia, Salsa20. The code is ready for testing. Open tasks: 1. Implement encrypted name lookup/readir cache 2. Optimize sparse files handling and file resizing __________________________________________________________________ Portmaster - utility to assist users with managing ports URL: http://dougbarton.us/portmaster.html Contact: Doug Barton I am currently seeking funding for further development work on portmaster. There are several features that are regularly requested by the community (such as support for installing packages) that I would very much like to implement but that will take more time than I can reasonably volunteer to implement correctly. There is information about the funding proposal available at the link above. Meanwhile I have recently completed another round of bug fixes and feature enhancements. The often-requested ability to specify the -x (exclude) option more than once on the command line was added in version 2.12. Also in that version I added the --list-origins option to make it easier to reinstall ports after a major version upgrade, or install the same set of ports on another system. Open tasks: 1. See the funding proposal. __________________________________________________________________ Release Engineering Status Report URL: http://www.FreeBSD.org/releng/ Contact: Release Engineering Team The Release Engineering Team continues to work on FreeBSD 8.0-RELEASE. Public testing has turned up quite a few problems, many related to the low-level network (routing/ARP table) changes and their interactions with IPv6. Progress continues to be made on fixing up the issues that have been identified during the public testing. At this point in time we are shooting for two more public test builds (RC2 and RC3) followed by the release late October or early November. __________________________________________________________________ Stream Control Transmission Protocol (SCTP) Contact: Randall Stewart SCTP continues to have minor fixes added to it as well as some new features. First and foremost, we now have VIMAGE and SCTP working and playing together. This goal was accomplished with the help of bz@, my new mentee tuexen@ and myself working together at the FreeBSD DevSummit in Cambridge, UK. Also the non-renegable SACK feature contributed by the university of Delaware was fixed so that now its safe to turn on (its sysctl). If you are using SCTP with CMT (Conncurrent Multipath Transfer) you will want to enable this option (CMT is also a sysctl). With CMT enabled you will be able to send data to all the destinations of an SCTP peer. We welcomed a new mentee (soon to be a commiter) to FreeBSD. Michael Tuexen is now a mentee of rrs@. Michael has been contributing to the SCTP work for quite some time and also moonlights as a Professor at the University of Muenster in Germany (when not doing SCTP coding). __________________________________________________________________ The FreeBSD Dutch Documentation Project URL: http://www.freebsd.org/docproj/translations.html#dutch Contact: Ren? Ladan Contact: Remko Lodder The current translations (Handbook and some articles) are kept up to date with the English versions. Some parts of the website have been translated, more work is in progress. Open tasks: 1. Find more volunteers for translating the remaining parts of the website and the FAQ. __________________________________________________________________ The FreeBSD Forums URL: http://forums.freebsd.org/ Contact: FreeBSD Forums Admins Contact: FreeBSD Forums Moderators Since their public launch in November 2008, the FreeBSD Forums (the most recent addition to the user community and support channels for the FreeBSD Operating System) have witnessed a healthy and steady growth. The user population is now at over 8,000 registered users, who have participated in over 6,000 topics, containing over 40,000 posts in total. The sign-up rate hovers between 50-100 each week. The total number of visitors (including 'guests') is hard to gauge but is likely to be a substantial multiple of the registered userbase. New topics and posts are actively 'pushed out' to search engines. This in turn makes the Forums show up in search results more and more often, making it a valuable and very accessible source of information for the FreeBSD community. One of the contributing factors to the Forums' success is their 'BSD-style' approach when it comes to administration and moderation. The Forums have a strong and unified identity, they are neatly divided into sub-forums (like 'Networking', 'Installing & Upgrading', etc.), very actively moderated, spam-free, and with a core group of very active and helpful members, dispensing many combined decades' worth of knowledge to starting, intermediate and professional users of FreeBSD. We expect the Forums to be, and to remain, a central hub in FreeBSD's community and support efforts. __________________________________________________________________ The FreeBSD Foundation Status Report URL: http://www.freebsdfoundation.org Contact: Deb Goodkin Kicking off our fall fund-raising campaign! Find out more at http://www.freebsdfoundation.org/donate/. We were a sponsor for EuroBSDCon 2009, and provided travel grants to 8 FreeBSD developers and users. We sponsored Kyiv BSD 2009, in Kiev Ukraine. We were also a sponsor of BSDCan, and sponsored 7 developers. We funded three new projects, New Console Driver by Ed Schouten, AVR32 Support by Arnar Mar Sig, and Wireless Mesh Support by Rui Paulo, which has completed. We continued funding a project that is making improvements to the FreeBSD TCP Stack by Lawrence Stewart. The project that made removing disk devices with mounted filesystems on them safe, by Edward Napierala, is now complete. We recognized the following FreeBSD developers at EuroBSDCon 2009: Poul-Henning Kamp, Bjoern Zeeb, and Simon Nielsen. These developers received limited edition FreeBSD Foundation vests. Follow us on Twitter now! __________________________________________________________________ The FreeBSD German Documentation Project URL: https://doc.bsdgroup.de URL: http://code.google.com/p/bsdcg-trans/wiki/BSDPJTAdede Contact: Johann Kois Contact: Benedict Reuschling Contact: Martin Wilke In May 2009, Benedict Reuschling received his commit bit to the www/de and doc/de_DE.ISO8859-1 trees under the mentorship of Johann Kois. Since then, he has been working primarily on the Handbook, updating existing chapters and translating new ones. Most notably, the filesystems and DTrace chapters have been recently translated. Bugs found in the original documents along the way were reported back so that the other translation teams could incorporate them, as well. Christoph Sold has put his time in translating the wiki pages of the BSD Certification Group into the German language. This is very helpful for all German people who want to take the exam and like to read the information about it in their native language. Daniel Seuffert has sent valuable corrections and bugfixes. Thanks to both of them for their time and efforts! The website is translated and updated constantly. Missing parts will be translated as time permits. We appreciate any help from volunteers in proofreading documents, translating new ones and keeping them up to date. Even small error reports are of great help for us. You can find contact information at the above URL. Open tasks: 1. Update the existing documentation set (especially the Handbook). 2. Translate more articles to German. 3. Read the translations. Check for problems and mistakes. Send feedback. __________________________________________________________________ The FreeBSD Hungarian Documentation Project URL: http://www.FreeBSD.org/hu URL: http://www.FreeBSD.org/doc/hu URL: http://wiki.FreeBSD.org/HungarianDocumentationProject URL: http://p4web.freebsd.org/@md=d&cd=//depot/projects/docproj_hu/&c=aXw@// depot/projects/docproj_hu/?ac=83 Contact: G?bor K?vesd?n Contact: G?bor P?li In the last months, we have not added new translations, although we have been working on the existing ones to have them updated. We need more translators and volunteers to keep the amount of the translated documentation growing, so feel free to contribute. Every line of submission or feedback is appreciated and highly welcome. If you want to join our work, please read the introduction to the project as well as the FDP Primer (both of them are available in Hungarian). Open tasks: 1. Translate news entries, press releases 2. Translate Release Notes for -CURRENT and 8.X 3. Translate articles 4. Translate web pages 5. Read the translations, send feedback __________________________________________________________________ The FreeBSD Spanish Documentation Project URL: http://www.FreeBSD.org/es URL: http://www.FreeBSD.org/doc/es URL: http://www.freebsd.org/doc/es/articles/fdp-es/ Contact: Jos? Vicente Carrasco Vay? Contact: G?bor K?vesd?n Recently, we have added one new article translation. The existing translations have not been updated, though. We need more human resources to keep up with the work and keep the translations up-to-date. Open tasks: 1. Update the Handbook translation 2. Update the web page translation __________________________________________________________________ The Newcons project URL: http://wiki.FreeBSD.org/Newcons URL: http://people.freebsd.org/~ed/newcons/patches/ Contact: Ed Schouten Some time ago I started writing a new driver for the FreeBSD kernel called vt(4), which is basically a replacement of syscons. There is still a lot of work that needs to be done but it is probably useful to mention what it does (and what does not). Right now there are just two graphics drivers for vt(4), namely a VGA driver for i386 and amd64 and a Microsoft Xbox graphics driver (because it was so easy to implement). I still have to figure out what I am going to do with VESA, because maybe it is better to just ignore VESA and figure out how hard it is to extend DRM to interact with vt(4). Some random features: it already supports both Unicode (UTF-8) input and output, it is MPSAFE and supports per-window graphical fonts of variable dimensions, containing an almost infinite amount of glyphs (both bold and regular). Open tasks: 1. Research needs to be done on DRM's codebase. 2. Syscons should already be migrated to TERM=xterm to make switching between drivers a bit easier. __________________________________________________________________ Valgrind suite on FreeBSD URL: http://wiki.freebsd.org/Valgrind Contact: Stanislav Sedov The Valgrind suite in the FreeBSD ports collection has been updated to version 3.5.0 (the latest available version). Most of the issues of the previous version should be resolved now: we expect memcheck, callgrind and cachegrind to be fully functional on both i386 and amd64 platforms as well as for i386 binaries running on amd64 system. DRD/hellgrind should work too, though they generate a lot of false-positives for now, so their output is a bit messy. Open tasks: 1. Port exp-ptrcheck valgrind tool and fix outstanding issues that show up in memcheck/helgrind/DRD in the Valgrind regression tests suite. 2. More testing (please, help). 3. Integrate our patches upstream. __________________________________________________________________ VirtualBox on FreeBSD URL: http://wiki.freebsd.org/VirtualBox Contact: Beat Gaetzi Contact: Bernhard Froehlich Contact: Dennis Herrmann Contact: Juergen Lock Contact: Martin Wilke VirtualBox has been committed to the Ports tree and synchronized with the latest trunk version from Sun. Several known problems are already fixed and some new features have been added: * VT-x support * Bridging support (Big Thanks to Fredrik Lindberg) * Host Serial Support * ACPI Support * Host DVD/CD access * SMP Support We would like to say thanks to all the people who helped us by reporting bugs and submitting fixes. We also thank the VirtualBox developers for their help with the ongoing effort to port VirtualBox on FreeBSD. From uqs at spoerlein.net Mon Oct 12 08:43:51 2009 From: uqs at spoerlein.net (Ulrich =?utf-8?B?U3DDtnJsZWlu?=) Date: Mon Oct 12 08:43:58 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091011170918.GU71731@hoeg.nl> References: <20091011145021.GG36937@acme.spoerlein.net> <20091011170918.GU71731@hoeg.nl> Message-ID: <20091012084337.GI36937@acme.spoerlein.net> On Sun, 11.10.2009 at 19:09:18 +0200, Ed Schouten wrote: > Hi Ulrich, > > * Ulrich Sp?rlein wrote: > > Comments? Committers? > > Wouldn't it better to address the root of the problem while there? ;-) It sure would, but someone[TM] would have to fix all problems for a top-level dir in a short time frame before new code hits the repo. Or, we might end up with individual WARNS on 98% of the subdirs and need a big sweeping patch to then flip the default anyway, so why not now? It raises the bar for new code entering the tree and we need individual WARNS lowering anyway due to contrib code. The only question then is, when do we want to churn. Right now when a patch is available or do we wait and hope that someday all will be better? Anyway, interest seems low for such a big patch so I guess I'll have to divvy it up for better digestion. Regards, Uli From gabor at FreeBSD.org Mon Oct 12 09:57:11 2009 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Mon Oct 12 09:57:18 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091011170918.GU71731@hoeg.nl> References: <20091011145021.GG36937@acme.spoerlein.net> <20091011170918.GU71731@hoeg.nl> Message-ID: <4AD2FD6E.8090208@FreeBSD.org> Ed Schouten escribi?: > Hi Ulrich, > > * Ulrich Sp?rlein wrote: > >> Comments? Committers? >> > > Wouldn't it better to address the root of the problem while there? ;-) > What I noticed is that the patch sets WARNS?=0 for a lot of utilities, which actually have higher WARNS-compliance. Even if we don't "address the root of the problem" right now, it would be nice to elaborate and set each WARNS level at the higher value possible.For example, usr.bin/ul is marked as 0 by the patch but it seems to me it is WARNS=6 clean. I don't know the last sources, though, but I'm sure it will compile with at least WARNS=3. -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From des at des.no Mon Oct 12 10:30:38 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 12 10:30:45 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <4AD2FD6E.8090208@FreeBSD.org> (Gabor Kovesdan's message of "Mon, 12 Oct 2009 11:57:02 +0200") References: <20091011145021.GG36937@acme.spoerlein.net> <20091011170918.GU71731@hoeg.nl> <4AD2FD6E.8090208@FreeBSD.org> Message-ID: <8663aksy43.fsf@ds4.des.no> Gabor Kovesdan writes: > What I noticed is that the patch sets WARNS?=0 for a lot of utilities, > which actually have higher WARNS-compliance. WARNS level 0 is the current default. All Ulrich's patch does is reverse the logic so that WARNS is 6 by default and anything that didn't already set WARNS explicitly sets it to 0, so the actual value of WARNS in each Makefile is the same as before. This is orthogonal to actually fixing whatever doesn't currently build at a higher WARNS level. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Mon Oct 12 10:34:42 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 12 10:34:48 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091011145021.GG36937@acme.spoerlein.net> ("Ulrich =?utf-8?Q?Sp=C3=B6rlein=22's?= message of "Sun, 11 Oct 2009 16:50:21 +0200") References: <20091011145021.GG36937@acme.spoerlein.net> Message-ID: <861vl8sxxb.fsf@ds4.des.no> Ulrich Sp?rlein writes: > Comments? Committers? You can set WARNS to 4 for rwhod, since we don't do Alpha any more. (actually, you can set it to 6, but 4 is what was already there) DES -- Dag-Erling Sm?rgrav - des@des.no From gabor at FreeBSD.org Mon Oct 12 10:52:07 2009 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Mon Oct 12 10:52:13 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <8663aksy43.fsf@ds4.des.no> References: <20091011145021.GG36937@acme.spoerlein.net> <20091011170918.GU71731@hoeg.nl> <4AD2FD6E.8090208@FreeBSD.org> <8663aksy43.fsf@ds4.des.no> Message-ID: <4AD30A4B.7060409@FreeBSD.org> Dag-Erling Sm?rgrav escribi?: > Gabor Kovesdan writes: > >> What I noticed is that the patch sets WARNS?=0 for a lot of utilities, >> which actually have higher WARNS-compliance. >> > > WARNS level 0 is the current default. All Ulrich's patch does is > reverse the logic so that WARNS is 6 by default and anything that didn't > already set WARNS explicitly sets it to 0, so the actual value of WARNS > in each Makefile is the same as before. This is orthogonal to actually > fixing whatever doesn't currently build at a higher WARNS level. > Yep, I understand that but what I'm saying is that once we are dealing with such a big patch, it would be nice to elaborate the highest WARNS level of each utility and set them accordingly, which doesn't require too much extra effort as opposed to making all of them WARNS=6 compliant. -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From ivoras at freebsd.org Mon Oct 12 11:29:11 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 12 11:29:18 2009 Subject: "global" TCP_NODELAY? Message-ID: I'm trying to work around some extreme brain damageness in PHP (yes, it sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so I'm wondering what are my other options? Is there a way to set TCP_NODELAY system-wide? From ticso at cicely7.cicely.de Mon Oct 12 11:36:25 2009 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Mon Oct 12 11:36:32 2009 Subject: "global" TCP_NODELAY? In-Reply-To: References: Message-ID: <20091012113619.GN48396@cicely7.cicely.de> On Mon, Oct 12, 2009 at 01:27:28PM +0200, Ivan Voras wrote: > I'm trying to work around some extreme brain damageness in PHP (yes, it > sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so > I'm wondering what are my other options? Is there a way to set > TCP_NODELAY system-wide? net.inet.tcp.delayed_ack net.inet.tcp.delacktime Depending on your application it may be sufficient to just reduce the time. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From ivoras at freebsd.org Mon Oct 12 11:42:35 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 12 11:43:05 2009 Subject: "global" TCP_NODELAY? In-Reply-To: <20091012113619.GN48396@cicely7.cicely.de> References: <20091012113619.GN48396@cicely7.cicely.de> Message-ID: Bernd Walter wrote: > On Mon, Oct 12, 2009 at 01:27:28PM +0200, Ivan Voras wrote: >> I'm trying to work around some extreme brain damageness in PHP (yes, it >> sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so >> I'm wondering what are my other options? Is there a way to set >> TCP_NODELAY system-wide? > > net.inet.tcp.delayed_ack > net.inet.tcp.delacktime > > Depending on your application it may be sufficient to just reduce the > time. Really? Doesn't TCP_NODELAY (Nagle's algorithm) work on buffers to be sent rather than on ACKs on received data? From des at des.no Mon Oct 12 11:50:26 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 12 11:50:32 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <4AD30A4B.7060409@FreeBSD.org> (Gabor Kovesdan's message of "Mon, 12 Oct 2009 12:51:55 +0200") References: <20091011145021.GG36937@acme.spoerlein.net> <20091011170918.GU71731@hoeg.nl> <4AD2FD6E.8090208@FreeBSD.org> <8663aksy43.fsf@ds4.des.no> <4AD30A4B.7060409@FreeBSD.org> Message-ID: <86ws30rfun.fsf@ds4.des.no> Gabor Kovesdan writes: > Yep, I understand that but what I'm saying is that once we are dealing > with such a big patch, it would be nice to elaborate the highest WARNS > level of each utility and set them accordingly, which doesn't require > too much extra effort as opposed to making all of them WARNS=6 > compliant. We can do that in a separate patch / commit. DES -- Dag-Erling Sm?rgrav - des@des.no From hunter at comsys.com.ua Mon Oct 12 11:56:03 2009 From: hunter at comsys.com.ua (Sergey Smitienko) Date: Mon Oct 12 11:56:10 2009 Subject: "global" TCP_NODELAY? In-Reply-To: References: Message-ID: <4AD31431.4060503@comsys.com.ua> Ivan Voras ?????: > I'm trying to work around some extreme brain damageness in PHP (yes, > it sucks) which doesn't have a way to set TCP_NODELAY on stream > sockets so I'm wondering what are my other options? Is there a way to > set TCP_NODELAY system-wide? What's wrong with: From ivoras at freebsd.org Mon Oct 12 12:09:37 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 12 12:09:43 2009 Subject: "global" TCP_NODELAY? In-Reply-To: <4AD31431.4060503@comsys.com.ua> References: <4AD31431.4060503@comsys.com.ua> Message-ID: Sergey Smitienko wrote: > Ivan Voras ?????: >> I'm trying to work around some extreme brain damageness in PHP (yes, >> it sucks) which doesn't have a way to set TCP_NODELAY on stream >> sockets so I'm wondering what are my other options? Is there a way to >> set TCP_NODELAY system-wide? > What's wrong with: > > $socket = socket_create_listen(1223); > socket_set_option($socket, SOL_SOCKET, TCP_NODELAY, 1); > var_dump(socket_get_option($socket, SOL_SOCKET, TCP_NODELAY)); > ?> These "socket objects" are completely different from fsockopen() "stream socket objects", and socket_set_option() doesn't work on those. Consequently, you cannot use fgets() and friends to work with sockets created with socket_*() and there is apparently no way to wrap sockets in streams. It's an already finished application that uses stream-like functions (e.g. fgets() and friends) and rewriting it to use raw socket recv() and send() would be nasty. From ticso at cicely7.cicely.de Mon Oct 12 12:24:28 2009 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Mon Oct 12 12:24:35 2009 Subject: "global" TCP_NODELAY? In-Reply-To: References: <20091012113619.GN48396@cicely7.cicely.de> Message-ID: <20091012122423.GO48396@cicely7.cicely.de> On Mon, Oct 12, 2009 at 01:42:08PM +0200, Ivan Voras wrote: > Bernd Walter wrote: > >On Mon, Oct 12, 2009 at 01:27:28PM +0200, Ivan Voras wrote: > >>I'm trying to work around some extreme brain damageness in PHP (yes, it > >>sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so > >>I'm wondering what are my other options? Is there a way to set > >>TCP_NODELAY system-wide? > > > >net.inet.tcp.delayed_ack > >net.inet.tcp.delacktime > > > >Depending on your application it may be sufficient to just reduce the > >time. > > Really? Doesn't TCP_NODELAY (Nagle's algorithm) work on buffers to be > sent rather than on ACKs on received data? Good point. TCP_NODELAY disables both to my knowledge. And setting delacktime to 1 helped me in one buffer case. But I'm not sure anymore - a non delayed ack might piggypack payload much sooner of course. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From tom at tomjudge.com Mon Oct 12 13:50:26 2009 From: tom at tomjudge.com (Tom Judge) Date: Mon Oct 12 13:50:34 2009 Subject: "global" TCP_NODELAY? In-Reply-To: References: <4AD31431.4060503@comsys.com.ua> Message-ID: <4AD333F4.2020304@tomjudge.com> Ivan Voras wrote: > Sergey Smitienko wrote: >> Ivan Voras ?????: >>> I'm trying to work around some extreme brain damageness in PHP (yes, >>> it sucks) which doesn't have a way to set TCP_NODELAY on stream >>> sockets so I'm wondering what are my other options? Is there a way >>> to set TCP_NODELAY system-wide? >> What's wrong with: >> >> > $socket = socket_create_listen(1223); >> socket_set_option($socket, SOL_SOCKET, TCP_NODELAY, 1); >> var_dump(socket_get_option($socket, SOL_SOCKET, TCP_NODELAY)); >> ?> > > These "socket objects" are completely different from fsockopen() > "stream socket objects", and socket_set_option() doesn't work on > those. Consequently, you cannot use fgets() and friends to work with > sockets created with socket_*() and there is apparently no way to wrap > sockets in streams. It's an already finished application that uses > stream-like functions (e.g. fgets() and friends) and rewriting it to > use raw socket recv() and send() would be nasty. Is this for php java bridge by any chance? If so I believe we have a patch floating around so that it can use UNIX sockets rather than INET ones. TJ From ivoras at freebsd.org Mon Oct 12 13:53:06 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 12 13:53:12 2009 Subject: "global" TCP_NODELAY? In-Reply-To: <4AD333F4.2020304@tomjudge.com> References: <4AD31431.4060503@comsys.com.ua> <4AD333F4.2020304@tomjudge.com> Message-ID: Tom Judge wrote: > Ivan Voras wrote: >> Sergey Smitienko wrote: >>> Ivan Voras ?????: >>>> I'm trying to work around some extreme brain damageness in PHP (yes, >>>> it sucks) which doesn't have a way to set TCP_NODELAY on stream >>>> sockets so I'm wondering what are my other options? Is there a way >>>> to set TCP_NODELAY system-wide? >>> What's wrong with: >>> >>> >> $socket = socket_create_listen(1223); >>> socket_set_option($socket, SOL_SOCKET, TCP_NODELAY, 1); >>> var_dump(socket_get_option($socket, SOL_SOCKET, TCP_NODELAY)); >>> ?> >> >> These "socket objects" are completely different from fsockopen() >> "stream socket objects", and socket_set_option() doesn't work on >> those. Consequently, you cannot use fgets() and friends to work with >> sockets created with socket_*() and there is apparently no way to wrap >> sockets in streams. It's an already finished application that uses >> stream-like functions (e.g. fgets() and friends) and rewriting it to >> use raw socket recv() and send() would be nasty. > Is this for php java bridge by any chance? If so I believe we have a > patch floating around so that it can use UNIX sockets rather than INET > ones. No, something in-house. From alfred at freebsd.org Mon Oct 12 15:22:49 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Mon Oct 12 15:22:55 2009 Subject: "global" TCP_NODELAY? In-Reply-To: References: Message-ID: <20091012152248.GJ79298@elvis.mu.org> * Ivan Voras [091012 04:29] wrote: > I'm trying to work around some extreme brain damageness in PHP (yes, it > sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so > I'm wondering what are my other options? Is there a way to set > TCP_NODELAY system-wide? Ivan, many people write php extensions, maybe you can do that? -- - Alfred Perlstein .- AMA, VMOA #5191, 03 vmax, 92 gs500, 85 ch250 .- FreeBSD committer From uqs at spoerlein.net Mon Oct 12 15:54:27 2009 From: uqs at spoerlein.net (Ulrich =?utf-8?B?U3DDtnJsZWlu?=) Date: Mon Oct 12 15:54:34 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <861vl8sxxb.fsf@ds4.des.no> References: <20091011145021.GG36937@acme.spoerlein.net> <861vl8sxxb.fsf@ds4.des.no> Message-ID: <20091012155421.GJ36937@acme.spoerlein.net> On Mon, 12.10.2009 at 12:34:40 +0200, Dag-Erling Sm?rgrav wrote: > Ulrich Sp?rlein writes: > > Comments? Committers? > > You can set WARNS to 4 for rwhod, since we don't do Alpha any more. > > (actually, you can set it to 6, but 4 is what was already there) Is there some easy way to do cross-compiles (like make universe) in just one of the subdirs? That would help tremendously. Bye, Uli From des at des.no Mon Oct 12 16:37:40 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 12 16:37:46 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091012155421.GJ36937@acme.spoerlein.net> ("Ulrich =?utf-8?Q?Sp=C3=B6rlein=22's?= message of "Mon, 12 Oct 2009 17:54:22 +0200") References: <20091011145021.GG36937@acme.spoerlein.net> <861vl8sxxb.fsf@ds4.des.no> <20091012155421.GJ36937@acme.spoerlein.net> Message-ID: <86zl7wpnzh.fsf@ds4.des.no> Ulrich Sp?rlein writes: > Is there some easy way to do cross-compiles (like make universe) in just > one of the subdirs? That would help tremendously. % cd /usr/src % make toolchain TARGET=powerpc % make buildenv TARGET=powerpc %% cd usr.sbin/rwhod %% make ('make buildenv' starts a subshell) DES -- Dag-Erling Sm?rgrav - des@des.no From ivoras at freebsd.org Mon Oct 12 17:28:48 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 12 17:28:54 2009 Subject: "global" TCP_NODELAY? In-Reply-To: <20091012152248.GJ79298@elvis.mu.org> References: <20091012152248.GJ79298@elvis.mu.org> Message-ID: <9bbcef730910121028q7185cb47sd5780fbb0b8f59ad@mail.gmail.com> 2009/10/12 Alfred Perlstein : > * Ivan Voras [091012 04:29] wrote: >> I'm trying to work around some extreme brain damageness in PHP (yes, it >> sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so >> I'm wondering what are my other options? Is there a way to set >> TCP_NODELAY system-wide? > > Ivan, many people write php extensions, maybe you can do that? While writing PHP extensions isn't hard (I've done it before), I'm not yet convinced it's worth the effort in this case - I don't know if TCP_NODELAY will help at all. I'll think about it if time permits. From rwatson at FreeBSD.org Mon Oct 12 18:15:46 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Mon Oct 12 18:15:52 2009 Subject: "global" TCP_NODELAY? In-Reply-To: <9bbcef730910121028q7185cb47sd5780fbb0b8f59ad@mail.gmail.com> References: <20091012152248.GJ79298@elvis.mu.org> <9bbcef730910121028q7185cb47sd5780fbb0b8f59ad@mail.gmail.com> Message-ID: On Mon, 12 Oct 2009, Ivan Voras wrote: > 2009/10/12 Alfred Perlstein : >> * Ivan Voras [091012 04:29] wrote: >>> I'm trying to work around some extreme brain damageness in PHP (yes, it >>> sucks) which doesn't have a way to set TCP_NODELAY on stream sockets so >>> I'm wondering what are my other options? Is there a way to set TCP_NODELAY >>> system-wide? >> >> Ivan, many people write php extensions, maybe you can do that? > > While writing PHP extensions isn't hard (I've done it before), I'm not yet > convinced it's worth the effort in this case - I don't know if TCP_NODELAY > will help at all. I'll think about it if time permits. Create a libc wrapper that calls setsockopt(2) whenever socket(2) is called to create a TCP socket in php, and inject it using LD_PRELOAD. This is a similar trick to what things like socks proxy library wrappers use, is easy to hack together, and avoids having to modify the kernel. When it doesn't work, move on, and if it does, change php :-). Robert N M Watson Computer Laboratory University of Cambridge From ivoras at freebsd.org Mon Oct 12 18:21:15 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 12 18:21:23 2009 Subject: "global" TCP_NODELAY? In-Reply-To: References: <20091012152248.GJ79298@elvis.mu.org> <9bbcef730910121028q7185cb47sd5780fbb0b8f59ad@mail.gmail.com> Message-ID: <9bbcef730910121119r3f7809dfo23c5f2ecdf7b289c@mail.gmail.com> 2009/10/12 Robert Watson : > Create a libc wrapper that calls setsockopt(2) whenever socket(2) is called > to create a TCP socket in php, and inject it using LD_PRELOAD. ?This is a > similar trick to what things like socks proxy library wrappers use, is easy > to hack together, and avoids having to modify the kernel. This is actually the sanest idea I've yet come across, including modifying PHP :) From dougb at FreeBSD.org Tue Oct 13 00:16:29 2009 From: dougb at FreeBSD.org (Doug Barton) Date: Tue Oct 13 00:16:35 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091011145021.GG36937@acme.spoerlein.net> References: <20091011145021.GG36937@acme.spoerlein.net> Message-ID: <4AD3C09B.70801@FreeBSD.org> Ulrich Sp?rlein wrote: > Dear -hackers, > > I would like you to give me your thoughts on the attached patch. There > are no functional changes, what I'm trying to do is introduce WARNS?=6 > for all top-level Makefiles and override that on a subdir basis. > > Why the churn? Because I think it sticks out more, if there's a WARNS=0 > in your Makefile rather than the default being WARNS=0. Perhaps this > gets more incentive going for fixing these WARNS. I don't see how this provides an incentive at all. I also object to this change on the grounds that down the road debugging is potentially going to be more difficult when someone forgets that there is a default WARNS level of 6. One of the strengths of the BSD-style make is the amount of routine stuff that goes on behind the scenes to make it easier to write good makefiles. However I don't think this should be one of those things. > There's also a lot of > work done by the DragonflyBSD folks which I intend to port peu a peu. Can you elaborate on this? What work are you planning to port over, and how does it depend on this default WARNS level issue? > Note: There are lots of style changes for games/ and sbin/, which I can > seperate out for easier review. But I'd like to make some sweeping > patches to bring them more inline with style.Makefile(5) Putting these two changes in the same patch is not a good thing. It makes it harder to read diffs, and commits that address separate issues should be done separately anyway. That said, I think that the style-compliance issue is a valid one, and I personally would be in favor of that happening after the 8.0-release, FWIW. hth, Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ From uqs at spoerlein.net Tue Oct 13 07:04:20 2009 From: uqs at spoerlein.net (Ulrich =?utf-8?B?U3DDtnJsZWlu?=) Date: Tue Oct 13 07:05:03 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <4AD3C09B.70801@FreeBSD.org> References: <20091011145021.GG36937@acme.spoerlein.net> <4AD3C09B.70801@FreeBSD.org> Message-ID: <20091013070417.GK36937@acme.spoerlein.net> Hi Doug, On Mon, 12.10.2009 at 16:49:47 -0700, Doug Barton wrote: > Ulrich Sp?rlein wrote: > > Dear -hackers, > > > > I would like you to give me your thoughts on the attached patch. There > > are no functional changes, what I'm trying to do is introduce WARNS?=6 > > for all top-level Makefiles and override that on a subdir basis. > > > > Why the churn? Because I think it sticks out more, if there's a WARNS=0 > > in your Makefile rather than the default being WARNS=0. Perhaps this > > gets more incentive going for fixing these WARNS. > > I don't see how this provides an incentive at all. > > I also object to this change on the grounds that down the road > debugging is potentially going to be more difficult when someone > forgets that there is a default WARNS level of 6. The "default" would be the setting inherited by, eg, src/bin/Makefile.inc. This already has a WARNS=6, are you saying that debugging stuff under bin/ has been made more difficult by that change? Why do we want bin/ to be WARNS-clean and not care about usr.bin/? To make things clear, no changes will be made to /usr/share/mk, if that's what you are afraid if. Unless you do ".include ../Makefile.inc" somewhere under src/*, you won't get WARNS at all. > One of the strengths of the BSD-style make is the amount of routine > stuff that goes on behind the scenes to make it easier to write good > makefiles. However I don't think this should be one of those things. One of the strengths of BSD in general that I have come to love is its higher consistency compared to most other systems. With WARNS=6 under bin/ and WARNS=2 under sbin/ this consistency is violated. > > There's also a lot of > > work done by the DragonflyBSD folks which I intend to port peu a peu. > > Can you elaborate on this? What work are you planning to port over, > and how does it depend on this default WARNS level issue? See http://gitweb.dragonflybsd.org/dragonfly.git?a=search&h=HEAD&st=commit&s=WARNS6 It depends in no way on the included WARNS level, but "the big switch" needs to be done anyway, so why not upfront? > > Note: There are lots of style changes for games/ and sbin/, which I can > > seperate out for easier review. But I'd like to make some sweeping > > patches to bring them more inline with style.Makefile(5) > > Putting these two changes in the same patch is not a good thing. It > makes it harder to read diffs, and commits that address separate > issues should be done separately anyway. > > That said, I think that the style-compliance issue is a valid one, and > I personally would be in favor of that happening after the > 8.0-release, FWIW. I will separate those changes out and work some more on them on another branch. I would really like to get the WARNS changes in first though. Thanks for all the comments, Uli From alexbestms at math.uni-muenster.de Tue Oct 13 20:50:47 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Tue Oct 13 20:51:00 2009 Subject: crashtar In-Reply-To: <86d44vqs9l.fsf@kopusha.onet> Message-ID: Mikolaj Golub schrieb am 2009-10-10: > On Sat, 10 Oct 2009 12:34:05 +0200 (CEST) Alexander Best wrote: > AB> thanks. this is a cool script and very useful indeed. only thing > you might > AB> want to do is check for root privileges at the beginning to > avoid nasty error > AB> messages like. > AB> awk: can't open file /var/crash/info.0 > AB> source line number 12 > In some cases you might not need root privileges. E.g. on some > servers I don't > have root but SA gives me read access to crashdumps. In this case if > the > script had a check for root privileges I would not be able to use it. > Actually as for me the message looks informative enough, it says that > we have > some problems with accessing crash dump files, so permissions should > be > checked. alright then. :) again: great script. would be great to have this in the ports dir in the near future. cheers. alex From motoom at xs4all.nl Wed Oct 14 00:50:14 2009 From: motoom at xs4all.nl (Michiel Overtoom) Date: Wed Oct 14 01:38:09 2009 Subject: sysinstall colours In-Reply-To: References: Message-ID: <200910140238.07174.motoom@xs4all.nl> On Saturday 10 October 2009 01:59:32 Randi Harper wrote: > All the problems with sysinstall, and your idea is to > change the color? [...] Fancy colors seems to me like a wrong focus of attention too... but maybe it would be nice to be able to run sysinstall in monochrome mode, for maximum contrast? Is this possible already? Just plain gray text on black background, and inverse for the selection bar, would be useful for those that have trouble reading yellow text on a lightgray background. Anyway, it's great to see that sysinstall gets some attention and useful fixes. What sometimes bothers me a bit while using sysinstall is the long sequence of Yes/No questions at a certain moment ('do you want NFS server? do you want NFS client? do you want anonymous FTP? do you want bridging? etc...). A more logical user interface seems a list of configurable options with their current settings, where you can 'point and click' with a selection bar to toggle Yes/No. Greetings, -- "The ability of the OSS process to collect and harness the collective IQ of thousands of individuals across the Internet is simply amazing." - Vinod Valloppillil http://www.catb.org/~esr/halloween/halloween4.html From dougb at FreeBSD.org Wed Oct 14 04:35:23 2009 From: dougb at FreeBSD.org (Doug Barton) Date: Wed Oct 14 04:35:29 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <20091013070417.GK36937@acme.spoerlein.net> References: <20091011145021.GG36937@acme.spoerlein.net> <4AD3C09B.70801@FreeBSD.org> <20091013070417.GK36937@acme.spoerlein.net> Message-ID: <4AD55506.4000507@FreeBSD.org> Ulrich Sp?rlein wrote: > The "default" would be the setting inherited by, eg, > src/bin/Makefile.inc. This already has a WARNS=6, are you saying that > debugging stuff under bin/ has been made more difficult by that change? It certainly can be, yes. Although admittedly I don't spend a lot of time debugging stuff under /bin. > Why do we want bin/ to be WARNS-clean and not care about usr.bin/? Red herring. I'd like everything to be as warns-clean as possible, I just disagree that this change will do anything to improve it. > One of the strengths of BSD in general that I have come to love is its > higher consistency compared to most other systems. With WARNS=6 under > bin/ and WARNS=2 under sbin/ this consistency is violated. The thing that you're glossing over is that most of the stuff in /bin is our code, and a lot of the stuff in /usr/[s]bin is contrib code. Thus they actually ARE different. Then of course there is the whole "Foolish consistency ...." issue. >>> There's also a lot of >>> work done by the DragonflyBSD folks which I intend to port peu a peu. >> Can you elaborate on this? What work are you planning to port over, >> and how does it depend on this default WARNS level issue? > > See > http://gitweb.dragonflybsd.org/dragonfly.git?a=search&h=HEAD&st=commit&s=WARNS6 > > It depends in no way on the included WARNS level, but "the big switch" > needs to be done anyway, so why not upfront? I disagree with your assertion that "the big switch needs to be done anyway." My personal preference would be to see first how many things will need overrides (WARNS != 6) before deciding whether it's worth setting a default. hth, Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ From jhs at berklix.com Wed Oct 14 12:13:21 2009 From: jhs at berklix.com (Julian H. Stacey) Date: Wed Oct 14 12:13:28 2009 Subject: sysinstall colours In-Reply-To: Your message "Wed, 14 Oct 2009 02:38:07 +0200." <200910140238.07174.motoom@xs4all.nl> Message-ID: <200910141143.n9EBh0Dw058809@fire.js.berklix.net> > What sometimes bothers me a bit while using sysinstall is the long sequence of > Yes/No questions at a certain moment ('do you want NFS server? do you want > NFS client? do you want anonymous FTP? do you want bridging? etc...). A > more logical user interface seems a list of configurable options with their > current settings, where you can 'point and click' with a selection bar to > toggle Yes/No. Would be nice. Cheers, Julian -- Julian Stacey: BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail ASCII plain text not HTML & Base64. http://asciiribbon.org Virused Microsoft PCs cause spam. http://berklix.com/free/ From avg at icyb.net.ua Wed Oct 14 17:12:33 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Oct 14 17:12:42 2009 Subject: heci: a new driver for review and testing Message-ID: <4AD6067E.2010503@icyb.net.ua> Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 I actually got around to implementing it (in initial/basic form): http://people.freebsd.org/~avg/heci.tgz Other drivers are: [Linux] http://www.openamt.org/ [OpenSolaris] http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/intel/io/heci/ An example of what functionality this driver could provide: http://article.gmane.org/gmane.linux.drivers.sensors/20398 This actually was my primary motivation for looking into this driver. Your hardware may be supported by this driver if pciconf -lv has something like the following: none0@pci0:0:3:0: class=0x078000 card=0x50448086 chip=0x29c48086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '(Bearlake) HECI Controller' class = simple comms Here's how successful attachment of this driver looks on my system: heci0: mem 0xe0426100-0xe042610f irq 16 at device 3.0 on pci0 heci0: using MSI heci0: [ITHREAD] heci0: found ME client at address 0x03: heci0: status = 0x00 heci0: protocol_name(guid) = A12FF5CA-FACB-4CB4-A958-19A23B2E6881 heci0: found ME client at address 0x07: heci0: status = 0x00 heci0: protocol_name(guid) = 55213584-9A29-4916-BADF-0FB7ED682AEB heci0: found ME client at address 0x20: heci0: status = 0x00 heci0: protocol_name(guid) = 309DCDE8-CCB1-4062-8F78-600115A34327 heci0: found ME client at address 0x21: heci0: status = 0x00 heci0: protocol_name(guid) = 6B5205B9-8185-4519-B889-D98724B58607 The driver has many functional and programming shortcomings, but I still would like to ask for its review and testing. Please read the following warnings and notices: 1. right now the driver is provided only as a module. 2. the driver recognizes only PCI ID of 0x29c48086, which is what I tested it with; please see hecireg.h for a list of potentially supported IDs and add your ID to heci_probe(). 3. heciio.h does not get installed into /usr/include, so for time being you need to take an extra step to make it available to code that wishes to communicate to heci. 4. this driver has far less capabilities than Linux and OpenSolaris drivers, so don't expect every OpenAMT program to work with it (but it does reliably work for me for the QST thing). Now more details about the code: 1. the code contains some style violations like overly long lines, probably others; but I still would like to hear your comments on its style. 2. there are some bad design decisions like using the same mutex+condvar pair for signaling different events (reception of data from ME, emptying of user buffer); but I still would like to hear your comments about improving the design. 3. only a single connection is allowed for a host client 4. only a single connection is allowed to a ME client (if the client can support more). 5. quite an arbitrary timeout is used in all places where we wait for simething to happen. 6. no power management supported. 7. only messages of size less than 512 bytes are supported. 8. only complete messages are supported, multi-part messages are not. 9. userland is expected to read or write the whole message in one go. 10. poll/select functionality is not tested. 11. non-blocking reads/writes are not supported. 12. some (probably important) properties and statuses of ME clients are not interpreted. 13. no specific support for watchdog and PTHI, only simple HECI/MEI communications. 14. probably many more... I think that ultimately, if complete feature support is needed, it would be easier to port the driver from either OpenSolaris or Linux than to add all the features to the presented driver. This is because documentation/description for many features is severely lacking in public access. I guess that Intel developers that worked on the OpenAMT driver had much more detailed specifications to work with. But reverse-engineering some parts of OpenAMT code is very hard. You can see for yourself, if not for the hacker who decompiled a Windows DLL, there would be no way of obtaining even GUID of QST client from public sources. Thank you very much in advance for any feedback, comments, ideas! P.S. BTW, can/may I drop "alternatively GPL" wording from License block of the files I borrowed from Intel? I.e. can a dual BSD+GPL licensed file be turned into BSD-only? Some additional info. The files contain only some data structure/constants definitions. I guess those are not copyrightable at all? I added a bunch of comments to those file describing the constants and structs. -- Andriy Gapon From rnoland at FreeBSD.org Wed Oct 14 21:25:41 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Wed Oct 14 21:25:54 2009 Subject: heci: a new driver for review and testing In-Reply-To: <4AD6067E.2010503@icyb.net.ua> References: <4AD6067E.2010503@icyb.net.ua> Message-ID: <1255555532.95374.147.camel@balrog.2hip.net> On Wed, 2009-10-14 at 20:12 +0300, Andriy Gapon wrote: > Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: > http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 > > I actually got around to implementing it (in initial/basic form): > http://people.freebsd.org/~avg/heci.tgz Your getting a WITNESS warning from malloc() while holding a lock at like 941 of heci.c mec = (struct me_client *)malloc(sizeof(*mec), M_HECI, M_NOWAIT | M_ZERO); fixes it. It also locked up the machine when I tried to kldunload, but haven't identified a root cause of that. robert. > Other drivers are: > [Linux] http://www.openamt.org/ > [OpenSolaris] > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/intel/io/heci/ > > An example of what functionality this driver could provide: > http://article.gmane.org/gmane.linux.drivers.sensors/20398 > This actually was my primary motivation for looking into this driver. > > Your hardware may be supported by this driver if pciconf -lv has something like > the following: > none0@pci0:0:3:0: class=0x078000 card=0x50448086 chip=0x29c48086 rev=0x02 > hdr=0x00 > vendor = 'Intel Corporation' > device = '(Bearlake) HECI Controller' > class = simple comms > > Here's how successful attachment of this driver looks on my system: > heci0: mem > 0xe0426100-0xe042610f irq 16 at device 3.0 on pci0 > heci0: using MSI > heci0: [ITHREAD] > heci0: found ME client at address 0x03: > heci0: status = 0x00 > heci0: protocol_name(guid) = A12FF5CA-FACB-4CB4-A958-19A23B2E6881 > heci0: found ME client at address 0x07: > heci0: status = 0x00 > heci0: protocol_name(guid) = 55213584-9A29-4916-BADF-0FB7ED682AEB > heci0: found ME client at address 0x20: > heci0: status = 0x00 > heci0: protocol_name(guid) = 309DCDE8-CCB1-4062-8F78-600115A34327 > heci0: found ME client at address 0x21: > heci0: status = 0x00 > heci0: protocol_name(guid) = 6B5205B9-8185-4519-B889-D98724B58607 > > The driver has many functional and programming shortcomings, but I still would > like to ask for its review and testing. > Please read the following warnings and notices: > 1. right now the driver is provided only as a module. > 2. the driver recognizes only PCI ID of 0x29c48086, which is what I tested it > with; please see hecireg.h for a list of potentially supported IDs and add your ID > to heci_probe(). > 3. heciio.h does not get installed into /usr/include, so for time being you need > to take an extra step to make it available to code that wishes to communicate to heci. > 4. this driver has far less capabilities than Linux and OpenSolaris drivers, so > don't expect every OpenAMT program to work with it (but it does reliably work for > me for the QST thing). > > Now more details about the code: > 1. the code contains some style violations like overly long lines, probably > others; but I still would like to hear your comments on its style. > 2. there are some bad design decisions like using the same mutex+condvar pair for > signaling different events (reception of data from ME, emptying of user buffer); > but I still would like to hear your comments about improving the design. > 3. only a single connection is allowed for a host client > 4. only a single connection is allowed to a ME client (if the client can support > more). > 5. quite an arbitrary timeout is used in all places where we wait for simething to > happen. > 6. no power management supported. > 7. only messages of size less than 512 bytes are supported. > 8. only complete messages are supported, multi-part messages are not. > 9. userland is expected to read or write the whole message in one go. > 10. poll/select functionality is not tested. > 11. non-blocking reads/writes are not supported. > 12. some (probably important) properties and statuses of ME clients are not > interpreted. > 13. no specific support for watchdog and PTHI, only simple HECI/MEI communications. > 14. probably many more... > > I think that ultimately, if complete feature support is needed, it would be easier > to port the driver from either OpenSolaris or Linux than to add all the features > to the presented driver. This is because documentation/description for many > features is severely lacking in public access. I guess that Intel developers that > worked on the OpenAMT driver had much more detailed specifications to work with. > But reverse-engineering some parts of OpenAMT code is very hard. > You can see for yourself, if not for the hacker who decompiled a Windows DLL, > there would be no way of obtaining even GUID of QST client from public sources. > > Thank you very much in advance for any feedback, comments, ideas! > > P.S. > BTW, can/may I drop "alternatively GPL" wording from License block of the files I > borrowed from Intel? I.e. can a dual BSD+GPL licensed file be turned into BSD-only? > Some additional info. The files contain only some data structure/constants > definitions. I guess those are not copyrightable at all? I added a bunch of > comments to those file describing the constants and structs. > -- Robert Noland FreeBSD From unixmania at gmail.com Wed Oct 14 22:46:22 2009 From: unixmania at gmail.com (Carlos A. M. dos Santos) Date: Wed Oct 14 22:46:29 2009 Subject: heci: a new driver for review and testing In-Reply-To: <4AD6067E.2010503@icyb.net.ua> References: <4AD6067E.2010503@icyb.net.ua> Message-ID: On Wed, Oct 14, 2009 at 2:12 PM, Andriy Gapon wrote: > BTW, can/may I drop "alternatively GPL" wording from License block of the files I > borrowed from Intel? I.e. can a dual BSD+GPL licensed file be turned into BSD-only? If Intel is the copyright owner then you can not change the license without their permission. -- My preferred quotation of Robert Louis Stevenson is "You cannot make an omelette without breaking eggs". Not because I like the omelettes, but because I like the sound of eggs being broken. From rnoland at FreeBSD.org Wed Oct 14 23:36:09 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Wed Oct 14 23:36:16 2009 Subject: heci: a new driver for review and testing In-Reply-To: <4AD6067E.2010503@icyb.net.ua> References: <4AD6067E.2010503@icyb.net.ua> Message-ID: <1255556129.95374.160.camel@balrog.2hip.net> On Wed, 2009-10-14 at 20:12 +0300, Andriy Gapon wrote: > Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: > http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 > > I actually got around to implementing it (in initial/basic form): > http://people.freebsd.org/~avg/heci.tgz > > Other drivers are: > [Linux] http://www.openamt.org/ > [OpenSolaris] > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/intel/io/heci/ > > An example of what functionality this driver could provide: > http://article.gmane.org/gmane.linux.drivers.sensors/20398 > This actually was my primary motivation for looking into this driver. Could you also post the client code in some form more readily accessible than trying to cut/paste from the web page? robert. > Your hardware may be supported by this driver if pciconf -lv has something like > the following: > none0@pci0:0:3:0: class=0x078000 card=0x50448086 chip=0x29c48086 rev=0x02 > hdr=0x00 > vendor = 'Intel Corporation' > device = '(Bearlake) HECI Controller' > class = simple comms > > Here's how successful attachment of this driver looks on my system: > heci0: mem > 0xe0426100-0xe042610f irq 16 at device 3.0 on pci0 > heci0: using MSI > heci0: [ITHREAD] > heci0: found ME client at address 0x03: > heci0: status = 0x00 > heci0: protocol_name(guid) = A12FF5CA-FACB-4CB4-A958-19A23B2E6881 > heci0: found ME client at address 0x07: > heci0: status = 0x00 > heci0: protocol_name(guid) = 55213584-9A29-4916-BADF-0FB7ED682AEB > heci0: found ME client at address 0x20: > heci0: status = 0x00 > heci0: protocol_name(guid) = 309DCDE8-CCB1-4062-8F78-600115A34327 > heci0: found ME client at address 0x21: > heci0: status = 0x00 > heci0: protocol_name(guid) = 6B5205B9-8185-4519-B889-D98724B58607 > > The driver has many functional and programming shortcomings, but I still would > like to ask for its review and testing. > Please read the following warnings and notices: > 1. right now the driver is provided only as a module. > 2. the driver recognizes only PCI ID of 0x29c48086, which is what I tested it > with; please see hecireg.h for a list of potentially supported IDs and add your ID > to heci_probe(). > 3. heciio.h does not get installed into /usr/include, so for time being you need > to take an extra step to make it available to code that wishes to communicate to heci. > 4. this driver has far less capabilities than Linux and OpenSolaris drivers, so > don't expect every OpenAMT program to work with it (but it does reliably work for > me for the QST thing). > > Now more details about the code: > 1. the code contains some style violations like overly long lines, probably > others; but I still would like to hear your comments on its style. > 2. there are some bad design decisions like using the same mutex+condvar pair for > signaling different events (reception of data from ME, emptying of user buffer); > but I still would like to hear your comments about improving the design. > 3. only a single connection is allowed for a host client > 4. only a single connection is allowed to a ME client (if the client can support > more). > 5. quite an arbitrary timeout is used in all places where we wait for simething to > happen. > 6. no power management supported. > 7. only messages of size less than 512 bytes are supported. > 8. only complete messages are supported, multi-part messages are not. > 9. userland is expected to read or write the whole message in one go. > 10. poll/select functionality is not tested. > 11. non-blocking reads/writes are not supported. > 12. some (probably important) properties and statuses of ME clients are not > interpreted. > 13. no specific support for watchdog and PTHI, only simple HECI/MEI communications. > 14. probably many more... > > I think that ultimately, if complete feature support is needed, it would be easier > to port the driver from either OpenSolaris or Linux than to add all the features > to the presented driver. This is because documentation/description for many > features is severely lacking in public access. I guess that Intel developers that > worked on the OpenAMT driver had much more detailed specifications to work with. > But reverse-engineering some parts of OpenAMT code is very hard. > You can see for yourself, if not for the hacker who decompiled a Windows DLL, > there would be no way of obtaining even GUID of QST client from public sources. > > Thank you very much in advance for any feedback, comments, ideas! > > P.S. > BTW, can/may I drop "alternatively GPL" wording from License block of the files I > borrowed from Intel? I.e. can a dual BSD+GPL licensed file be turned into BSD-only? > Some additional info. The files contain only some data structure/constants > definitions. I guess those are not copyrightable at all? I added a bunch of > comments to those file describing the constants and structs. > -- Robert Noland FreeBSD From avg at icyb.net.ua Thu Oct 15 05:23:54 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Thu Oct 15 05:24:06 2009 Subject: heci: a new driver for review and testing In-Reply-To: <1255555532.95374.147.camel@balrog.2hip.net> References: <4AD6067E.2010503@icyb.net.ua> <1255555532.95374.147.camel@balrog.2hip.net> Message-ID: <4AD6B1D5.3010003@icyb.net.ua> on 15/10/2009 00:25 Robert Noland said the following: > On Wed, 2009-10-14 at 20:12 +0300, Andriy Gapon wrote: >> Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: >> http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 >> >> I actually got around to implementing it (in initial/basic form): >> http://people.freebsd.org/~avg/heci.tgz > > Your getting a WITNESS warning from malloc() while holding a lock at > like 941 of heci.c > > mec = (struct me_client *)malloc(sizeof(*mec), M_HECI, > M_NOWAIT | M_ZERO); > > fixes it. Shame on me! I have to admit that tested/used this module only on stable/7 (amd64) and without WITNESS/INVARIANTS. Thank you for catching this. I guess I should also add a check for malloc returning NULL. > It also locked up the machine when I tried to kldunload, but > haven't identified a root cause of that. Haven't seen it here. I will try to investigate. -- Andriy Gapon From avg at icyb.net.ua Thu Oct 15 05:26:31 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Thu Oct 15 05:27:29 2009 Subject: heci: a new driver for review and testing In-Reply-To: <1255556129.95374.160.camel@balrog.2hip.net> References: <4AD6067E.2010503@icyb.net.ua> <1255556129.95374.160.camel@balrog.2hip.net> Message-ID: <4AD6B274.5050906@icyb.net.ua> on 15/10/2009 00:35 Robert Noland said the following: > On Wed, 2009-10-14 at 20:12 +0300, Andriy Gapon wrote: >> Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: >> http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 >> >> I actually got around to implementing it (in initial/basic form): >> http://people.freebsd.org/~avg/heci.tgz >> >> Other drivers are: >> [Linux] http://www.openamt.org/ >> [OpenSolaris] >> http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/intel/io/heci/ >> >> An example of what functionality this driver could provide: >> http://article.gmane.org/gmane.linux.drivers.sensors/20398 >> This actually was my primary motivation for looking into this driver. > > Could you also post the client code in some form more readily accessible > than trying to cut/paste from the web page? > > robert. Sure: http://people.freebsd.org/~avg/heci-qst.c Sorry for not doing this from the very start. -- Andriy Gapon From des at des.no Thu Oct 15 10:25:41 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Thu Oct 15 10:25:48 2009 Subject: heci: a new driver for review and testing In-Reply-To: <4AD6067E.2010503@icyb.net.ua> (Andriy Gapon's message of "Wed, 14 Oct 2009 20:12:30 +0300") References: <4AD6067E.2010503@icyb.net.ua> Message-ID: <86iqehx8bf.fsf@ds4.des.no> Andriy Gapon writes: > BTW, can / may I drop "alternatively GPL" wording from License block > of the files I borrowed from Intel? I.e. can a dual BSD+GPL licensed > file be turned into BSD-only? No. Why would you want to? > Some additional info. The files contain only some data structure / > constants definitions. I guess those are not copyrightable at all? That's debatable. It is probably safe to assume that they aren't *if* they only describe the interface to another piece of software or to the hardware, but if they represent internal data structures used by the Linux driver, they are definitely copyrightable. DES -- Dag-Erling Sm?rgrav - des@des.no From rnoland at FreeBSD.org Thu Oct 15 14:17:04 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Thu Oct 15 14:17:11 2009 Subject: heci: a new driver for review and testing In-Reply-To: <4AD6B1D5.3010003@icyb.net.ua> References: <4AD6067E.2010503@icyb.net.ua> <1255555532.95374.147.camel@balrog.2hip.net> <4AD6B1D5.3010003@icyb.net.ua> Message-ID: <1255616214.2356.872.camel@balrog.2hip.net> On Thu, 2009-10-15 at 08:23 +0300, Andriy Gapon wrote: > on 15/10/2009 00:25 Robert Noland said the following: > > On Wed, 2009-10-14 at 20:12 +0300, Andriy Gapon wrote: > >> Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: > >> http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 > >> > >> I actually got around to implementing it (in initial/basic form): > >> http://people.freebsd.org/~avg/heci.tgz > > > > Your getting a WITNESS warning from malloc() while holding a lock at > > like 941 of heci.c > > > > mec = (struct me_client *)malloc(sizeof(*mec), M_HECI, > > M_NOWAIT | M_ZERO); > > > > fixes it. > > Shame on me! > I have to admit that tested/used this module only on stable/7 (amd64) and > without WITNESS/INVARIANTS. > Thank you for catching this. > I guess I should also add a check for malloc returning NULL. Yes, with M_NOWAIT you should. Here is the dmesg output. heci0: mem 0xd0526100-0xd052610f irq 16 at device 3.0 on pci0 heci0: using MSI heci0: [ITHREAD] heci0: found ME client at address 0x02: heci0: status = 0x00 heci0: protocol_name(guid) = BB875E12-CB58-4D14-AE93-8566183C66C7 heci0: found ME client at address 0x03: heci0: status = 0x00 heci0: protocol_name(guid) = A12FF5CA-FACB-4CB4-A958-19A23B2E6881 heci0: found ME client at address 0x06: heci0: status = 0x00 heci0: protocol_name(guid) = 9B27FD6D-EF72-4967-BCC2-471A32679620 heci0: found ME client at address 0x07: heci0: status = 0x00 heci0: protocol_name(guid) = 55213584-9A29-4916-BADF-0FB7ED682AEB heci0: found ME client at address 0x20: heci0: status = 0x00 heci0: protocol_name(guid) = 23F27E9B-9D26-4FCE-9227-DE06446E5B06 heci0: found ME client at address 0x21: heci0: status = 0x00 heci0: protocol_name(guid) = 309DCDE8-CCB1-4062-8F78-600115A34327 heci0: found ME client at address 0x22: heci0: status = 0x00 heci0: protocol_name(guid) = 6B5205B9-8185-4519-B889-D98724B58607 heci0: found ME client at address 0x23: heci0: status = 0x00 heci0: protocol_name(guid) = 9D9105BD-C8C6-41CA-AC28-84738E2CE9FD heci0: found ME client at address 0x24: heci0: status = 0x00 heci0: protocol_name(guid) = DC506909-7ADB-4A0D-B109-4E25E38C382C heci0: found ME client at address 0x25: heci0: status = 0x00 heci0: protocol_name(guid) = 6733A4DB-0476-4E7B-B3AF-BCFC29BEE7A7 heci0: found ME client at address 0x26: heci0: status = 0x00 heci0: protocol_name(guid) = 12F80028-B4B7-4B2D-ACA8-46E0FF65814C heci0: found ME client at address 0x27: heci0: status = 0x00 heci0: protocol_name(guid) = 05B79A6F-4628-4D7F-899D-A91514CB32AB robert. > > It also locked up the machine when I tried to kldunload, but > > haven't identified a root cause of that. > > Haven't seen it here. > I will try to investigate. > -- Robert Noland FreeBSD From unixmania at gmail.com Thu Oct 15 15:01:51 2009 From: unixmania at gmail.com (Carlos A. M. dos Santos) Date: Thu Oct 15 15:01:57 2009 Subject: heci: a new driver for review and testing In-Reply-To: <20091014203428.7d84bf96@bhuda.mired.org> References: <4AD6067E.2010503@icyb.net.ua> <20091014203428.7d84bf96@bhuda.mired.org> Message-ID: On Wed, Oct 14, 2009 at 9:34 PM, Mike Meyer wrote: > On Wed, 14 Oct 2009 19:46:20 -0300 > "Carlos A. M. dos Santos" wrote: > >> On Wed, Oct 14, 2009 at 2:12 PM, Andriy Gapon wrote: >> >> > BTW, can/may I drop "alternatively GPL" wording from License block of the files I >> > borrowed from Intel? I.e. can a dual BSD+GPL licensed file be turned into BSD-only? >> >> If Intel is the copyright owner then you can not change the license >> without their permission. > > True. But he *has* their permission. They list the requirements for > use and redistribution in the license block in question: > > ?* Redistribution and use in source and binary forms, with or without > ?* modification, are permitted provided that the following conditions > ?* are met: > ?* 1. Redistributions of source code must retain the above copyright > ?* ? ?notice, this list of conditions, and the following disclaimer, > ?* ? ?without modification. > ?* 2. Redistributions in binary form must reproduce at minimum a disclaimer > ?* ? ?substantially similar to the "NO WARRANTY" disclaimer below > ?* ? ?("Disclaimer") and any redistribution must be conditioned upon > ?* ? ?including a substantially similar Disclaimer requirement for further > ?* ? ?binary redistribution. > ?* 3. Neither the names of the above-listed copyright holders nor the names > ?* ? ?of any contributors may be used to endorse or promote products derived > ?* ? ?from this software without specific prior written permission. > > I don't see anything there about retaining the ability to choose an > alternative license. The paragraph in question says that people can > *choose* to distribute it under the GPLv2, not that it *is* being > distributed under the GPLv2. Conversely, I don't see anything there explicitly *allowing* somebody to remove that paragraph so I'd keep it unless I had express clearance from Intel. Making assumptions can be dangerous, especially when dealing with large corporations. That's the main lesson I've learned after several years dealing with such matters. I don't want to make this discussion longer than necessary, so I stop here. -- My preferred quotation of Robert Louis Stevenson is "You cannot make an omelette without breaking eggs". Not because I like the omelettes, but because I like the sound of eggs being broken. From kraduk at googlemail.com Thu Oct 15 13:58:15 2009 From: kraduk at googlemail.com (krad) Date: Thu Oct 15 15:35:07 2009 Subject: Fwd: zfs root In-Reply-To: References: Message-ID: i got no response on questions, anybody able to answer here? ---------- Forwarded message ---------- From: krad Date: 2009/10/3 Subject: zfs root To: questions@freebsd.org Hi, I have a quick question about freebsd on zfs root. I have built a few test systems all work fine. I have one question though. Does the loader replay any information when it accesses the pool? Basically im interested in how or what is done on reboot after the kernel panic or power loss. Are there any safe gaurds with regard to the zpool integrity? From gallatin at cs.duke.edu Thu Oct 15 21:16:30 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Thu Oct 15 21:16:36 2009 Subject: namei (via firmware_get(9)) from taskq in 7.x Message-ID: <4AD79126.8020104@cs.duke.edu> Hi, I'm trying to re-initialize a NIC which uses firmware(9) after a hardware fault. As part of the process, I need to re-load the firmware using firmware_get(). If the firmware kld is not resident, then the machine will panic like this: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x20 fault code = supervisor read data, page not present instruction pointer = 0x8:0xffffffff805b05d4 stack pointer = 0x10:0xffffff8000080460 frame pointer = 0x10:0xffffff8000080510 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 21 (swi5: +) [thread pid 21 tid 100021 ] Stopped at namei+0x174: movq 0x20(%rbx),%rax db> bt Tracing pid 21 tid 100021 td 0xffffff00013c3ae0 namei() at namei+0x174 vn_open_cred() at vn_open_cred+0x3a4 linker_load_module() at linker_load_module+0x1f2 linker_reference_module() at linker_reference_module+0xae firmware_get() at firmware_get+0x136 mxge_load_firmware() at mxge_load_firmware+0x2d mxge_watchdog_task() at mxge_watchdog_task+0x2f6 taskqueue_run() at taskqueue_run+0x9d ithread_loop() at ithread_loop+0x17d fork_exit() at fork_exit+0x11f fork_trampoline() at fork_trampoline+0xe Looking at it in gdb, it seems like the problem is that namei is trying to use ndp->ni_cnd.cn_thread->td_proc->p_fd->fd_cdir which is null in this context. Can somebody tell me what kernel context it is safe to call firmware_get() (and hence namei) from? Is there a safe way to do it from a taskq? FWIW, this seems to work fine (even from a callout context) in 8 and higher. It is only 7 and earlier where I'm having this problem. Thanks, Drew From decado at gmail.com Thu Oct 15 23:46:36 2009 From: decado at gmail.com (Andrew D. Boyd) Date: Thu Oct 15 23:46:42 2009 Subject: sysinstall colours In-Reply-To: <20091011142923.GT71731@hoeg.nl> References: <3a142e750910100150g7459e037u7b84b4b4bbc2bf8@mail.gmail.com> <20091011142923.GT71731@hoeg.nl> Message-ID: <4AD7C232.3020609@gmail.com> Ed Schouten wrote: > * Paul B Mahol wrote: >> This have nothing to do with ncurses, colors you like simple can not >> be displayed in current syscons(4) and making support for 256 colors >> or even true bit color in sysinstall(so that it looks amazing in >> konsole) is waste of time. > > Yes. As of recently, our terminal emulator gained 256 color support, but > this gets smashed down to 8 colors, simply because Syscons cannot handle > more than 16 foreground and 8 background colors. > > This is how colors are converted: > > http://80386.nl/pub/xterm-256.png > http://80386.nl/pub/teken-256.png > My God that is hideous. Save us Ed! Three cheers for your recent console work. From decado at gmail.com Fri Oct 16 00:07:33 2009 From: decado at gmail.com (Andrew D. Boyd) Date: Fri Oct 16 00:07:39 2009 Subject: crashtar In-Reply-To: <86d44vqs9l.fsf@kopusha.onet> References: <86d44vqs9l.fsf@kopusha.onet> Message-ID: <4AD7BF94.908@gmail.com> Mikolaj Golub wrote: > On Sat, 10 Oct 2009 12:34:05 +0200 (CEST) Alexander Best wrote: > > AB> thanks. this is a cool script and very useful indeed. only thing you might > AB> want to do is check for root privileges at the beginning to avoid nasty error > AB> messages like. > > AB> awk: can't open file /var/crash/info.0 > AB> source line number 12 > > In some cases you might not need root privileges. E.g. on some servers I don't > have root but SA gives me read access to crashdumps. In this case if the > script had a check for root privileges I would not be able to use it. > > Actually as for me the message looks informative enough, it says that we have > some problems with accessing crash dump files, so permissions should be > checked. > Instead of checking for root you could check if the file is readable: if [ -r FILE ] and then print an error message. Although the awk error message is sufficient some people might find this helpful. Regards, Andrew From uqs at spoerlein.net Fri Oct 16 13:15:17 2009 From: uqs at spoerlein.net (Ulrich =?utf-8?B?U3DDtnJsZWlu?=) Date: Fri Oct 16 13:15:24 2009 Subject: RFC: Big Makefile patch for WARNS settings In-Reply-To: <86zl7wpnzh.fsf@ds4.des.no> References: <20091011145021.GG36937@acme.spoerlein.net> <861vl8sxxb.fsf@ds4.des.no> <20091012155421.GJ36937@acme.spoerlein.net> <86zl7wpnzh.fsf@ds4.des.no> Message-ID: <20091016131504.GA5222@elmar.spoerlein.net> On Mon, 12.10.2009 at 18:37:38 +0200, Dag-Erling Sm?rgrav wrote: > Ulrich Sp?rlein writes: > > Is there some easy way to do cross-compiles (like make universe) in just > > one of the subdirs? That would help tremendously. > > % cd /usr/src > % make toolchain TARGET=powerpc > % make buildenv TARGET=powerpc > %% cd usr.sbin/rwhod > %% make > > ('make buildenv' starts a subshell) Excellent, I now smashed together the following expect script, which can be used to compile a single subdir with these toolchains. It requires that you build all toolchains once, upfront. It's a total hack, but hey it works for me, example usage would be: $ cd /usr/src $ universe-build -t (builds toolchain for all targets, needs to be done every once in a while) $ universe-build games/grdc WARNS=6 sparc64 amd64 i386 (build grdc with WARNS=6 for 3 archs specified, using the buildenv target above) Pardon my weak tcl/expect-fu Regards, Uli -------------- next part -------------- #!/usr/local/bin/expect # Does the following for variable subdirs and a given set of target archs, # exits upon first failure. # % cd /usr/src # % make toolchain TARGET=powerpc # % make buildenv TARGET=powerpc # %% cd usr.sbin/rwhod # %% make set timeout -1 set toolchain 0 proc usage {argv0} { send_user "usage: $argv0 -t \[target1 target2 ...\]\n" send_user " $argv0 subdir \[make-args\] \[target1 target2 ...\]\n" exit } while {[llength $argv]>0} { set flag [lindex $argv 0] switch -- $flag \ "-t" { set toolchain 1 set argv [lrange $argv 1 end] set argc [expr ($argc - 1)] } default { break } } if {$toolchain!=1} { if {$argc<1} { usage $argv0 exit 1 } set subdir [lindex $argv 0] set argv [lrange $argv 1 end] set argc [expr ($argc - 1)] # if something is present, use it as make args if {$argc>=1} { set margs [lindex $argv 0] set argv [lrange $argv 1 end] set argc [expr ($argc - 1)] } else { set margs {} } } # if there is still something, use it as target list, # else default to everything if {$argc>=1} { set targets [lrange $argv 0 end] } else { set input [open "|make universe -V TARGETS" r] set targets [split [string trimright [read $input]] " "] close $input } if {$toolchain==1} { foreach target $targets { spawn /bin/sh expect "$ " send "env MAKEOBJDIRPREFIX=/usr/obj/$target make toolchain __MAKE_CONF=/dev/null TARGET=$target\n" expect "$ " send "exit \$?\n" expect eof } puts "\n" exit } foreach target $targets { spawn /bin/sh set sid($target) $spawn_id expect "$ " send "env MAKEOBJDIRPREFIX=/usr/obj/$target make buildenv __MAKE_CONF=/dev/null TARGET=$target\n" expect "$ " send "make -C $subdir clean; make -C $subdir $margs\n" expect "$ " # Stupid buildenv is doing sh || true, so we cannot propagate by doing exit $rc :( # grab echo output instead send "echo rc=\$?\n" expect -re "echo rc=..\r" expect -re "rc=(.*)\n" # TODO: save rc per target, don't exit below but print rcs for all archs on exit set rc $expect_out(1,string) send "exit \$?\n" expect "$ " send "exit \$?\n" expect eof if {$rc!=0} { puts "" exit $rc } ## if we expect eof, the wait below will not return, wtf? ## XXX sid($target) wont work here?? #catch {close -i $spawn_id} ## wait returns PID, TYPE, CODE, grab code #set rc [lindex [wait sid($target)] 3] } From kostikbel at gmail.com Sun Oct 18 13:17:15 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Sun Oct 18 13:17:22 2009 Subject: namei (via firmware_get(9)) from taskq in 7.x In-Reply-To: <4AD79126.8020104@cs.duke.edu> References: <4AD79126.8020104@cs.duke.edu> Message-ID: <20091018131656.GP2160@deviant.kiev.zoral.com.ua> On Thu, Oct 15, 2009 at 05:16:22PM -0400, Andrew Gallatin wrote: > Hi, > > I'm trying to re-initialize a NIC which uses firmware(9) > after a hardware fault. As part of the process, I need > to re-load the firmware using firmware_get(). If the > firmware kld is not resident, then the machine will panic > like this: > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x20 > fault code = supervisor read data, page not present > instruction pointer = 0x8:0xffffffff805b05d4 > stack pointer = 0x10:0xffffff8000080460 > frame pointer = 0x10:0xffffff8000080510 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 21 (swi5: +) > [thread pid 21 tid 100021 ] > Stopped at namei+0x174: movq 0x20(%rbx),%rax > db> bt > Tracing pid 21 tid 100021 td 0xffffff00013c3ae0 > namei() at namei+0x174 > vn_open_cred() at vn_open_cred+0x3a4 > linker_load_module() at linker_load_module+0x1f2 > linker_reference_module() at linker_reference_module+0xae > firmware_get() at firmware_get+0x136 > mxge_load_firmware() at mxge_load_firmware+0x2d > mxge_watchdog_task() at mxge_watchdog_task+0x2f6 > taskqueue_run() at taskqueue_run+0x9d > ithread_loop() at ithread_loop+0x17d > fork_exit() at fork_exit+0x11f > fork_trampoline() at fork_trampoline+0xe > > Looking at it in gdb, it seems like the problem is that namei > is trying to use ndp->ni_cnd.cn_thread->td_proc->p_fd->fd_cdir > which is null in this context. > > Can somebody tell me what kernel context it is safe to > call firmware_get() (and hence namei) from? Is there > a safe way to do it from a taskq? > > FWIW, this seems to work fine (even from a callout context) > in 8 and higher. It is only 7 and earlier where I'm having > this problem. It seems that you want a merge of r178042,183614,184842,188057 (one of which is yours). Property changes on: . ___________________________________________________________________ Modified: svn:mergeinfo Merged /head/sys:r178042,183614,184842,188057 Index: kern/subr_firmware.c =================================================================== --- kern/subr_firmware.c (revision 198202) +++ kern/subr_firmware.c (working copy) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2005, Sam Leffler + * Copyright (c) 2005-2008, Sam Leffler * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -41,7 +41,11 @@ #include #include #include +#include +#include +#include + /* * Loadable firmware support. See sys/sys/firmware.h and firmware(9) * form more details on the subsystem. @@ -89,7 +93,7 @@ /* * 'file' is private info managed by the autoload/unload code. * Set at the end of firmware_get(), cleared only in the - * firmware_task, so the latter can depend on its value even + * firmware_unload_task, so the latter can depend on its value even * while the lock is not held. */ linker_file_t file; /* module file, if autoloaded */ @@ -121,14 +125,16 @@ static struct priv_fw firmware_table[FIRMWARE_MAX]; /* - * module release are handled in a separate task as they might sleep. + * Firmware module operations are handled in a separate task as they + * might sleep and they require directory context to do i/o. */ -struct task firmware_task; +static struct taskqueue *firmware_tq; +static struct task firmware_unload_task; /* * This mutex protects accesses to the firmware table. */ -struct mtx firmware_mtx; +static struct mtx firmware_mtx; MTX_SYSINIT(firmware, &firmware_mtx, "firmware table", MTX_DEF); /* @@ -227,7 +233,7 @@ } else if (fp->refcnt != 0) { /* cannot unregister */ err = EBUSY; } else { - linker_file_t x = fp->file; /* save value */ + linker_file_t x = fp->file; /* save value */ if (fp->parent != NULL) /* release parent reference */ fp->parent->refcnt--; @@ -244,6 +250,47 @@ return err; } +static void +loadimage(void *arg, int npending) +{ + struct thread *td = curthread; + char *imagename = arg; + struct priv_fw *fp; + linker_file_t result; + int error; + + /* synchronize with the thread that dispatched us */ + mtx_lock(&firmware_mtx); + mtx_unlock(&firmware_mtx); + + if (td->td_proc->p_fd->fd_rdir == NULL) { + printf("%s: root not mounted yet, no way to load image\n", + imagename); + goto done; + } + error = linker_reference_module(imagename, NULL, &result); + if (error != 0) { + printf("%s: could not load firmware image, error %d\n", + imagename, error); + goto done; + } + + mtx_lock(&firmware_mtx); + fp = lookup(imagename, NULL); + if (fp == NULL || fp->file != NULL) { + mtx_unlock(&firmware_mtx); + if (fp == NULL) + printf("%s: firmware image loaded, " + "but did not register\n", imagename); + (void) linker_release_module(imagename, NULL, NULL); + goto done; + } + fp->file = result; /* record the module identity */ + mtx_unlock(&firmware_mtx); +done: + wakeup_one(imagename); /* we're done */ +} + /* * Lookup and potentially load the specified firmware image. * If the firmware is not found in the registry, try to load a kernel @@ -254,9 +301,9 @@ const struct firmware * firmware_get(const char *imagename) { + struct task fwload_task; struct thread *td; struct priv_fw *fp; - linker_file_t result; mtx_lock(&firmware_mtx); fp = lookup(imagename, NULL); @@ -265,29 +312,34 @@ /* * Image not present, try to load the module holding it. */ - mtx_unlock(&firmware_mtx); td = curthread; if (priv_check(td, PRIV_FIRMWARE_LOAD) != 0 || securelevel_gt(td->td_ucred, 0) != 0) { + mtx_unlock(&firmware_mtx); printf("%s: insufficient privileges to " "load firmware image %s\n", __func__, imagename); return NULL; } - (void) linker_reference_module(imagename, NULL, &result); + /* + * Defer load to a thread with known context. linker_reference_module + * may do filesystem i/o which requires root & current dirs, etc. + * Also we must not hold any mtx's over this call which is problematic. + */ + if (!cold) { + TASK_INIT(&fwload_task, 0, loadimage, __DECONST(void *, + imagename)); + taskqueue_enqueue(firmware_tq, &fwload_task); + msleep(__DECONST(void *, imagename), &firmware_mtx, 0, + "fwload", 0); + } /* - * After loading the module, see if the image is registered now. + * After attempting to load the module, see if the image is registered. */ - mtx_lock(&firmware_mtx); fp = lookup(imagename, NULL); if (fp == NULL) { mtx_unlock(&firmware_mtx); - printf("%s: failed to load firmware image %s\n", - __func__, imagename); - (void) linker_release_module(imagename, NULL, NULL); return NULL; } - fp->file = result; /* record the module identity */ - found: /* common exit point on success */ fp->refcnt++; mtx_unlock(&firmware_mtx); @@ -300,8 +352,8 @@ * to release the resource, but the flag is only advisory. * * If this is the last reference to the firmware image, and this is an - * autoloaded module, wake up the firmware_task to figure out what to do - * with the associated module. + * autoloaded module, wake up the firmware_unload_task to figure out + * what to do with the associated module. */ void firmware_put(const struct firmware *p, int flags) @@ -314,12 +366,53 @@ if (flags & FIRMWARE_UNLOAD) fp->flags |= FW_UNLOAD; if (fp->file) - taskqueue_enqueue(taskqueue_thread, &firmware_task); + taskqueue_enqueue(firmware_tq, &firmware_unload_task); } mtx_unlock(&firmware_mtx); } /* + * Setup directory state for the firmware_tq thread so we can do i/o. + */ +static void +set_rootvnode(void *arg, int npending) +{ + struct thread *td = curthread; + struct proc *p = td->td_proc; + + FILEDESC_XLOCK(p->p_fd); + if (p->p_fd->fd_cdir == NULL) { + p->p_fd->fd_cdir = rootvnode; + VREF(rootvnode); + } + if (p->p_fd->fd_rdir == NULL) { + p->p_fd->fd_rdir = rootvnode; + VREF(rootvnode); + } + FILEDESC_XUNLOCK(p->p_fd); + + free(arg, M_TEMP); +} + +/* + * Event handler called on mounting of /; bounce a task + * into the task queue thread to setup it's directories. + */ +static void +firmware_mountroot(void *arg) +{ + struct task *setroot_task; + + setroot_task = malloc(sizeof(struct task), M_TEMP, M_NOWAIT); + if (setroot_task != NULL) { + TASK_INIT(setroot_task, 0, set_rootvnode, setroot_task); + taskqueue_enqueue(firmware_tq, setroot_task); + } else + printf("%s: no memory for task!\n", __func__); +} +EVENTHANDLER_DEFINE(mountroot, firmware_mountroot, NULL, 0); + +/* * The body of the task in charge of unloading autoloaded modules * that are not needed anymore. * Images can be cross-linked so we may need to make multiple passes, @@ -383,11 +476,23 @@ firmware_modevent(module_t mod, int type, void *unused) { struct priv_fw *fp; - int i, err = EINVAL; + int i, err; switch (type) { case MOD_LOAD: - TASK_INIT(&firmware_task, 0, unloadentry, NULL); + TASK_INIT(&firmware_unload_task, 0, unloadentry, NULL); + firmware_tq = taskqueue_create("taskqueue_firmware", M_WAITOK, + taskqueue_thread_enqueue, &firmware_tq); + /* NB: use our own loop routine that sets up context */ + (void) taskqueue_start_threads(&firmware_tq, 1, PWAIT, + "firmware taskq"); + if (rootvnode != NULL) { + /* + * Root is already mounted so we won't get an event; + * simulate one here. + */ + firmware_mountroot(NULL); + } return 0; case MOD_UNLOAD: @@ -398,8 +503,9 @@ fp->flags |= FW_UNLOAD;; } mtx_unlock(&firmware_mtx); - taskqueue_enqueue(taskqueue_thread, &firmware_task); - taskqueue_drain(taskqueue_thread, &firmware_task); + taskqueue_enqueue(firmware_tq, &firmware_unload_task); + taskqueue_drain(firmware_tq, &firmware_unload_task); + err = 0; for (i = 0; i < FIRMWARE_MAX; i++) { fp = &firmware_table[i]; if (fp->fw.name != NULL) { @@ -409,6 +515,8 @@ err = EINVAL; } } + if (err == 0) + taskqueue_free(firmware_tq); return err; } return EINVAL; @@ -417,7 +525,7 @@ static moduledata_t firmware_mod = { "firmware", firmware_modevent, - 0 + NULL }; DECLARE_MODULE(firmware, firmware_mod, SI_SUB_DRIVERS, SI_ORDER_FIRST); MODULE_VERSION(firmware, 1); Property changes on: kern/subr_firmware.c ___________________________________________________________________ Modified: cvs2svn:cvs-rev - 1.9 + 1.10 Index: kern/subr_prf.c =================================================================== --- kern/subr_prf.c (revision 198202) +++ kern/subr_prf.c (working copy) @@ -955,7 +955,7 @@ } SYSCTL_PROC(_kern, OID_AUTO, msgbuf, CTLTYPE_STRING | CTLFLAG_RD, - 0, 0, sysctl_kern_msgbuf, "A", "Contents of kernel message buffer"); + NULL, 0, sysctl_kern_msgbuf, "A", "Contents of kernel message buffer"); static int msgbuf_clearflag; Property changes on: contrib/pf ___________________________________________________________________ Modified: svn:mergeinfo Merged /head/sys/contrib/pf:r178042,183614,184842,188057 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091018/ce4ea719/attachment.pgp From to.my.trociny at gmail.com Sun Oct 18 15:59:44 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Sun Oct 18 16:02:46 2009 Subject: crashtar In-Reply-To: (Alexander Best's message of "Tue\, 13 Oct 2009 22\:50\:44 +0200 \(CEST\)") References: Message-ID: <86vdicg0b9.fsf@kopusha.onet> On Tue, 13 Oct 2009 22:50:44 +0200 (CEST) Alexander Best wrote: AB> again: great script. would be great to have this in the ports dir in the near AB> future. I have created separate google project for this script http://code.google.com/p/bsdcrashtar/ And submitted to ports http://www.freebsd.org/cgi/query-pr.cgi?pr=139721 BTW, many things in the script have been improved since I posted it here, and user friendly error output is among them :-). -- Mikolaj Golub From avg at icyb.net.ua Sun Oct 18 20:11:11 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Sun Oct 18 20:12:10 2009 Subject: SB7xx watchdog: new driver for review and testing Message-ID: <4ADB764C.2010900@icyb.net.ua> Please review and/or test a new driver for watchdog driver included into AMD SB7xx: http://people.freebsd.org/~avg/amdsbwd.tgz I have tested this driver only with SB700 on Gigabyte GA-MA780G-UD3H motherboard. ichwd driver was used as a starting point for this driver. This can be seen from some function names, general code organization and some small code snippets. Many thanks to ichwd authors and maintainers! Right now I have infrastructure only for building this driver as a module. Things for which that I need the most feedback/ideas: 1. If the driver actually works on your hardware and the hardware description. The driver can be tested by loading the driver and doing 'watchdog -t '. Having debug.bootverbose=1 may provide additional useful info. And better to test this from single-user mode with filesystems mounted r/o. 2. Better name for the driver. amdsbwd stands for "AMD S(outh)B(ridge) WatchDog", but this abbreviation could be cryptic to decipher. 3. Proper location for this driver. At least on my system this driver needs resources (I/O ports and MEM range) that are claimed by ACPI, thus I've made it a child of acpi bus. But this driver doesn't have anything else ACPI-ish in it, so I decided that it doesn't belong under acpica/ or acpi_support/. Am I correct about this? Anything else you would like to report or comment or advise to me. Thank you very much for your help. -- Andriy Gapon From avg at freebsd.org Sun Oct 18 20:15:10 2009 From: avg at freebsd.org (Andriy Gapon) Date: Sun Oct 18 20:15:21 2009 Subject: SB7xx watchdog: new driver for review and testing In-Reply-To: <4ADB764C.2010900@icyb.net.ua> References: <4ADB764C.2010900@icyb.net.ua> Message-ID: <4ADB773C.8090600@freebsd.org> on 18/10/2009 23:10 Andriy Gapon said the following: > Please review and/or test a new driver for watchdog driver included into AMD ^^^^^^^^^-hardware Oh, and please note things marked with XXX and TODO in the code. -- Andriy Gapon From avg at icyb.net.ua Sun Oct 18 20:28:43 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Sun Oct 18 20:28:50 2009 Subject: special key (combo) for switching to K_XLATE Message-ID: <4ADB7A68.9090906@icyb.net.ua> Guys, please take a look at the following hackish ugly code. This is just a proof of concept. It allows me to use an extended multimedia key (that generates 0xe0 0x32 code sequence) on my keyboard to break out of K_RAW into K_XLATE. The code is slightly tested and seems to work. It allowed me to switch out of X terminal to a normal console terminal when X server hanged. Then I could kill X and relatively easily recover from the situation that previously require either a remote access (e.g. through network) or a reboot. What do you think about having such a capability in syscons? Of course, the mode switch should be triggered by some key combination is possible to produce on any/most supported keyboards and also should be hard to press by accident. Note that I haven't (again) checked this code with WITNESS. diff --git a/sys/dev/syscons/syscons.c b/sys/dev/syscons/syscons.c index d158f85..e543765 100644 --- a/sys/dev/syscons/syscons.c +++ b/sys/dev/syscons/syscons.c @@ -3127,6 +3127,7 @@ scgetc(sc_softc_t *sc, u_int flags) #endif u_int c; int this_scr; + static int e0 = 0; int f; int i; @@ -3159,8 +3160,22 @@ next_code: if (!(flags & SCGETC_CN)) random_harvest(&c, sizeof(c), 1, 0, RANDOM_KEYBOARD); - if (scp->kbd_mode != K_XLATE) + if (scp->kbd_mode != K_XLATE) { + if (scp->kbd_mode == K_RAW) { + if (e0) { + e0 = 0; + if (KEYCHAR(c) == 0x32) { + printf("kbd_mode: %d => %d\n", scp->kbd_mode, K_XLATE); + scp->kbd_mode = K_XLATE; + kbdd_ioctl(scp->sc->kbd, KDSKBMODE, (caddr_t)&scp->kbd_mode); + return NOKEY; + } + } + else if (KEYCHAR(c) == 0xe0) + e0 = 1; + } return KEYCHAR(c); + } /* if scroll-lock pressed allow history browsing */ if (!ISGRAPHSC(scp) && scp->history && scp->status & SLKED) { -- Andriy Gapon From rpaulo at freebsd.org Mon Oct 19 11:17:27 2009 From: rpaulo at freebsd.org (Rui Paulo) Date: Mon Oct 19 11:17:39 2009 Subject: SB7xx watchdog: new driver for review and testing In-Reply-To: <4ADB764C.2010900@icyb.net.ua> References: <4ADB764C.2010900@icyb.net.ua> Message-ID: <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> On 18 Oct 2009, at 21:10, Andriy Gapon wrote: > Please review and/or test a new driver for watchdog driver included > into AMD SB7xx: > http://people.freebsd.org/~avg/amdsbwd.tgz > I have tested this driver only with SB700 on Gigabyte GA-MA780G-UD3H > motherboard. > > ichwd driver was used as a starting point for this driver. This can > be seen from > some function names, general code organization and some small code > snippets. > Many thanks to ichwd authors and maintainers! > > Right now I have infrastructure only for building this driver as a > module. > > Things for which that I need the most feedback/ideas: > 1. If the driver actually works on your hardware and the hardware > description. > The driver can be tested by loading the driver and doing 'watchdog - > t number>'. Having debug.bootverbose=1 may provide additional useful > info. > And better to test this from single-user mode with filesystems > mounted r/o. > > 2. Better name for the driver. amdsbwd stands for "AMD S(outh)B(ridge) > WatchDog", but this abbreviation could be cryptic to decipher. > > 3. Proper location for this driver. > At least on my system this driver needs resources (I/O ports and MEM > range) that > are claimed by ACPI, thus I've made it a child of acpi bus. But this > driver > doesn't have anything else ACPI-ish in it, so I decided that it > doesn't belong > under acpica/ or acpi_support/. Am I correct about this? > > Anything else you would like to report or comment or advise to me. > Thank you very much for your help. The driver looks good in general. A few questions: - Can you make the magic numbers a define ? Where did they come from ? - Are you missing a device_set_desc() call ? - If this is what you want to commit, C++ comments are not allowed per- style Regards, -- Rui Paulo From avg at icyb.net.ua Mon Oct 19 11:23:43 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Oct 19 11:23:56 2009 Subject: SB7xx watchdog: new driver for review and testing In-Reply-To: <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> References: <4ADB764C.2010900@icyb.net.ua> <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> Message-ID: <4ADC4C3C.3000007@icyb.net.ua> on 19/10/2009 14:17 Rui Paulo said the following: > On 18 Oct 2009, at 21:10, Andriy Gapon wrote: > >> Please review and/or test a new driver for watchdog driver included >> into AMD SB7xx: >> http://people.freebsd.org/~avg/amdsbwd.tgz >> I have tested this driver only with SB700 on Gigabyte GA-MA780G-UD3H >> motherboard. >> >> ichwd driver was used as a starting point for this driver. This can be >> seen from >> some function names, general code organization and some small code >> snippets. >> Many thanks to ichwd authors and maintainers! >> >> Right now I have infrastructure only for building this driver as a >> module. >> >> Things for which that I need the most feedback/ideas: >> 1. If the driver actually works on your hardware and the hardware >> description. >> The driver can be tested by loading the driver and doing 'watchdog -t >> > number>'. Having debug.bootverbose=1 may provide additional useful info. >> And better to test this from single-user mode with filesystems mounted >> r/o. >> >> 2. Better name for the driver. amdsbwd stands for "AMD S(outh)B(ridge) >> WatchDog", but this abbreviation could be cryptic to decipher. >> >> 3. Proper location for this driver. >> At least on my system this driver needs resources (I/O ports and MEM >> range) that >> are claimed by ACPI, thus I've made it a child of acpi bus. But this >> driver >> doesn't have anything else ACPI-ish in it, so I decided that it >> doesn't belong >> under acpica/ or acpi_support/. Am I correct about this? >> >> Anything else you would like to report or comment or advise to me. >> Thank you very much for your help. > > The driver looks good in general. A few questions: > - Can you make the magic numbers a define ? Where did they come from ? Yes, will do this. The numbers are from register definitions in AMD SB700/710/750 Register Reference Guide: http://developer.amd.com/assets/43009_sb7xx_rrg_pub_1.00.pdf I will add a link to the document too. > - Are you missing a device_set_desc() call ? Yes, I missed this. Thanks! > - If this is what you want to commit, C++ comments are not allowed > per-style Those lines were a result of quick hacking. I will remove them altogether, -- Andriy Gapon From gallatin at cs.duke.edu Mon Oct 19 12:41:29 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Mon Oct 19 12:41:36 2009 Subject: namei (via firmware_get(9)) from taskq in 7.x In-Reply-To: <20091018131656.GP2160@deviant.kiev.zoral.com.ua> References: <4AD79126.8020104@cs.duke.edu> <20091018131656.GP2160@deviant.kiev.zoral.com.ua> Message-ID: <4ADC5E71.7090501@cs.duke.edu> Kostik Belousov wrote: > It seems that you want a merge of r178042,183614,184842,188057 (one of Yes, I finally figured this out on Fri. I probably should have posted a response to this thread to avoid others wasting time on this. Drew From ivoras at freebsd.org Mon Oct 19 13:52:47 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 19 13:52:54 2009 Subject: Make process title - % complete Message-ID: I have a small patch that makes "make" display percentage complete in process title, which can be retrieved in "top" in the form of: 71466 root 1 76 0 7008K 5696K select 0 0:00 0.00% make: 95% (55 more targets out of 1360) (make) The patch is here and I'm inviting reviews and suggestions: http://people.freebsd.org/~ivoras/diffs/make.c.patch if nobody objects, I'll commit it :) From rink at FreeBSD.org Mon Oct 19 14:24:16 2009 From: rink at FreeBSD.org (Rink Springer) Date: Mon Oct 19 14:24:25 2009 Subject: Make process title - % complete In-Reply-To: References: Message-ID: <20091019140806.GB95902@rink.nu> Hi Ivan, On Mon, Oct 19, 2009 at 03:52:30PM +0200, Ivan Voras wrote: > if nobody objects, I'll commit it :) I seem to recall that setproctitle() is quite expensive to call; perhaps it would make sense offer a flag to prevent make(1) from calling it? [1] Anyway, the feature looks nice! I'd like to have it... [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd expect it's negligable but who knows... -- Rink P.W. Springer - http://rink.nu "Beauty often seduces us on the road to truth." - Dr. Wilson From ivoras at freebsd.org Mon Oct 19 14:35:30 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 19 14:35:37 2009 Subject: Make process title - % complete In-Reply-To: <20091019140806.GB95902@rink.nu> References: <20091019140806.GB95902@rink.nu> Message-ID: <9bbcef730910190735t5847e765j16337092b354ec29@mail.gmail.com> 2009/10/19 Rink Springer : > Hi Ivan, > > On Mon, Oct 19, 2009 at 03:52:30PM +0200, Ivan Voras wrote: >> if nobody objects, I'll commit it :) > > I seem to recall that setproctitle() is quite expensive to call; perhaps > it would make sense offer a flag to prevent make(1) from calling it? [1] > > Anyway, the feature looks nice! I'd like to have it... > > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd > ? ?expect it's negligable but who knows... The loop it's called in is not processed bazillion times per second (though it *is* called surprisingly often; small, fast jobs can result in somewhere in the order of magnitude of 100 iterations per second on a fast CPU). As you said - I expect it's negligable compared to fork() and the work jobs themselves do. From ivoras at freebsd.org Mon Oct 19 14:40:53 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 19 14:41:00 2009 Subject: Make process title - % complete In-Reply-To: References: Message-ID: Ivan Voras wrote: > I have a small patch that makes "make" display percentage complete in > process title, which can be retrieved in "top" in the form of: > > 71466 root 1 76 0 7008K 5696K select 0 0:00 0.00% > make: 95% (55 more targets out of 1360) (make) Also: is there someone here more familiar with "make" who can tell me if the "current" top level target (i.e. the one taken from the command line) is kept track of somewhere? For example "clean" in "make clean install". From spam at rm-rf.kiev.ua Mon Oct 19 15:39:24 2009 From: spam at rm-rf.kiev.ua (Alex Kozlov) Date: Mon Oct 19 15:39:31 2009 Subject: Make process title - % complete Message-ID: <20091019144135.GA91918@ravenloft.kiev.ua> On Mon, Oct 19, 2009 at 04:35:08PM +0200, Ivan Voras wrote: > >> if nobody objects, I'll commit it :) > > > > I seem to recall that setproctitle() is quite expensive to call; perhaps > > it would make sense offer a flag to prevent make(1) from calling it? [1] > > > > Anyway, the feature looks nice! I'd like to have it... > > > > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd > > ? ?expect it's negligable but who knows... > > The loop it's called in is not processed bazillion times per second > (though it *is* called surprisingly often; small, fast jobs can result > in somewhere in the order of magnitude of 100 iterations per second on > a fast CPU). As you said - I expect it's negligable compared to fork() > and the work jobs themselves do. How about add this statistic to make info handler? -- Adios From avg at icyb.net.ua Mon Oct 19 15:41:50 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Oct 19 15:42:03 2009 Subject: SB7xx watchdog: new driver for review and testing In-Reply-To: <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> References: <4ADB764C.2010900@icyb.net.ua> <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> Message-ID: <4ADC88BA.10507@icyb.net.ua> I have put updated version of the driver (C file only) here: http://people.freebsd.org/~avg/amdsbwd.c Please let me know how it looks now. Thank you! -- Andriy Gapon From rpaulo at freebsd.org Mon Oct 19 15:47:38 2009 From: rpaulo at freebsd.org (Rui Paulo) Date: Mon Oct 19 15:48:17 2009 Subject: SB7xx watchdog: new driver for review and testing In-Reply-To: <4ADC88BA.10507@icyb.net.ua> References: <4ADB764C.2010900@icyb.net.ua> <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> <4ADC88BA.10507@icyb.net.ua> Message-ID: On 19 Oct 2009, at 16:41, Andriy Gapon wrote: > > I have put updated version of the driver (C file only) here: > http://people.freebsd.org/~avg/amdsbwd.c Looks good to me. -- Rui Paulo From ivoras at freebsd.org Mon Oct 19 15:52:04 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Oct 19 15:52:10 2009 Subject: Make process title - % complete In-Reply-To: <20091019144135.GA91918@ravenloft.kiev.ua> References: <20091019144135.GA91918@ravenloft.kiev.ua> Message-ID: <9bbcef730910190851m7e82bd1aqc0bc55106c8a5d37@mail.gmail.com> 2009/10/19 Alex Kozlov : > On Mon, Oct 19, 2009 at 04:35:08PM +0200, Ivan Voras wrote: >> >> if nobody objects, I'll commit it :) >> > >> > I seem to recall that setproctitle() is quite expensive to call; perhaps >> > it would make sense offer a flag to prevent make(1) from calling it? [1] >> > >> > Anyway, the feature looks nice! I'd like to have it... >> > >> > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd >> > ? ?expect it's negligable but who knows... >> >> The loop it's called in is not processed bazillion times per second >> (though it *is* called surprisingly often; small, fast jobs can result >> in somewhere in the order of magnitude of 100 iterations per second on >> a fast CPU). As you said - I expect it's negligable compared to fork() >> and the work jobs themselves do. > How about add this statistic to make info handler? You mean SIGINFO? From spam at rm-rf.kiev.ua Mon Oct 19 16:20:18 2009 From: spam at rm-rf.kiev.ua (Alex Kozlov) Date: Mon Oct 19 16:20:25 2009 Subject: Make process title - % complete Message-ID: <20091019162016.GA96201@ravenloft.kiev.ua> On Mon, Oct 19, 2009 at 05:51:42PM +0200, Ivan Voras wrote: > 2009/10/19 Alex Kozlov : > > On Mon, Oct 19, 2009 at 04:35:08PM +0200, Ivan Voras wrote: > >> >> if nobody objects, I'll commit it :) > >> > > >> > I seem to recall that setproctitle() is quite expensive to call; perhaps > >> > it would make sense offer a flag to prevent make(1) from calling it? [1] > >> > > >> > Anyway, the feature looks nice! I'd like to have it... > >> > > >> > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd > >> > ? ?expect it's negligable but who knows... > >> > >> The loop it's called in is not processed bazillion times per second > >> (though it *is* called surprisingly often; small, fast jobs can result > >> in somewhere in the order of magnitude of 100 iterations per second on > >> a fast CPU). As you said - I expect it's negligable compared to fork() > >> and the work jobs themselves do. > > How about add this statistic to make info handler? > You mean SIGINFO? Yes -- Adios From forensixs at gmx.de Mon Oct 19 16:12:35 2009 From: forensixs at gmx.de (Manuel Gebele) Date: Mon Oct 19 16:28:30 2009 Subject: FreeBSD DAQ Card Facility [DCF] Message-ID: <20091019154550.163560@gmx.net> Hi folks, some time ago, I decided to start working on an FreeBSD implementation for data acquisition support. Now I've published a very first Pre-Alpha version of the project. To become a more precisely overview take a look at http://freebsd-dcf.sourceforge.net/ Please notice that the project -as already mentioned- is in an early development stage. For that reason, we've only a skeleton driver which gives a guide about the ``DCF based'' low-level driver development. The ``DCF core'' source code needs a cleanup -I'll do that time permitting. The projects website needs also an update, especially the documentation part. In the foreseeable future I plan to add two USB drivers for two (DAQ) USB plug-in boards. But at this time the main focus is that the DCF project becomes a usable form. I hope that all this would be useful to our beloved FreeBSD =) It would be great to hear about some of other people's opinions. Thanks, Manuel Gebele -- GRATIS f?r alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 From avg at icyb.net.ua Mon Oct 19 19:43:57 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Oct 19 19:44:09 2009 Subject: SB7xx watchdog: new driver for review and testing In-Reply-To: References: <4ADB764C.2010900@icyb.net.ua> <5E0DD277-CAA3-4F01-8561-35CF6C511718@freebsd.org> <4ADC88BA.10507@icyb.net.ua> Message-ID: <4ADCC16A.1080203@icyb.net.ua> on 19/10/2009 18:47 Rui Paulo said the following: > On 19 Oct 2009, at 16:41, Andriy Gapon wrote: > >> >> I have put updated version of the driver (C file only) here: >> http://people.freebsd.org/~avg/amdsbwd.c > > Looks good to me. Thank you for the review and the help! I have now produced a diff against the main tree for full integration of this driver: http://people.freebsd.org/~avg/amdsbwd.diff -- Andriy Gapon From grarpamp at gmail.com Tue Oct 20 02:09:52 2009 From: grarpamp at gmail.com (grarpamp) Date: Tue Oct 20 03:02:50 2009 Subject: vm: kvm_free max_wired Message-ID: Is this telling me I should be able to set kmem_size to around 740MiB before the kernel panics during boot? Any runtime issues with doing that? # sysctl hw.physmem hw.realmem vm.kvm_size vm.kvm_free vm.kmem_size hw.physmem: 1055293440 hw.realmem: 1072627712 vm.kvm_size: 1073737728 vm.kvm_free: 205516800 vm.kmem_size: 536870912 # sysctl -d hw.physmem vm.kvm_size vm.kvm_free vm.kmem_size hw.physmem: hw.realmem: vm.kvm_size: Size of KVM vm.kvm_free: Amount of KVM free vm.kmem_size: Size of kernel memory This doesn't seem to autosize. Is that expected? Should one care about this sysctl? vm.max_wired: System-wide limit to wired page count vm.max_wired: 83211 83211*4096/2^20 = ~325 Also, these are obviously broken / curious: debug.boothowto: -2147481598 net.inet.tcp.inflight.max: 1073725440 Running RELENG_8. From lars.engels at 0x20.net Tue Oct 20 08:24:43 2009 From: lars.engels at 0x20.net (Lars Engels) Date: Tue Oct 20 11:28:35 2009 Subject: Make process title - % complete In-Reply-To: <20091019162016.GA96201@ravenloft.kiev.ua> References: <20091019162016.GA96201@ravenloft.kiev.ua> Message-ID: <20091020100707.60jfc16iskcgcccg@0x20.net> Quoting Alex Kozlov : > On Mon, Oct 19, 2009 at 05:51:42PM +0200, Ivan Voras wrote: >> 2009/10/19 Alex Kozlov : >> > On Mon, Oct 19, 2009 at 04:35:08PM +0200, Ivan Voras wrote: >> >> >> if nobody objects, I'll commit it :) >> >> > >> >> > I seem to recall that setproctitle() is quite expensive to >> call; perhaps >> >> > it would make sense offer a flag to prevent make(1) from >> calling it? [1] >> >> > >> >> > Anyway, the feature looks nice! I'd like to have it... >> >> > >> >> > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd >> >> > expect it's negligable but who knows... >> >> >> >> The loop it's called in is not processed bazillion times per second >> >> (though it *is* called surprisingly often; small, fast jobs can result >> >> in somewhere in the order of magnitude of 100 iterations per second on >> >> a fast CPU). As you said - I expect it's negligable compared to fork() >> >> and the work jobs themselves do. >> > How about add this statistic to make info handler? >> You mean SIGINFO? > Yes Using SIGINFO sounds nice, but make produces so much output that normally you won't see the result because it is scrolled up just after sending the signal. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: PGP Digital Signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091020/8e5ee51c/attachment.pgp From forensixs at gmx.de Tue Oct 20 10:30:00 2009 From: forensixs at gmx.de (Manuel Gebele) Date: Tue Oct 20 11:29:12 2009 Subject: FreeBSD DAQ Card Facility [DCF] Message-ID: <20091020102955.309160@gmx.net> Finally I've updated the documentation part from the FreeBSD DCF project site: http://freebsd-dcf.sourceforge.net/docu.html I hope that helps to get a better overview. Thanks, Manuel Gebele -- Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3.5 - sicherer, schneller und einfacher! http://portal.gmx.net/de/go/atbrowser From des at des.no Tue Oct 20 12:08:13 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue Oct 20 12:08:19 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: <20091020102955.309160@gmx.net> (Manuel Gebele's message of "Tue, 20 Oct 2009 12:29:55 +0200") References: <20091020102955.309160@gmx.net> Message-ID: <86eioytgie.fsf@ds4.des.no> "Manuel Gebele" writes: > Finally I've updated the documentation part from the FreeBSD DCF > project site: What made you choose the name "DCF"? DES -- Dag-Erling Sm?rgrav - des@des.no From spam at rm-rf.kiev.ua Tue Oct 20 12:25:21 2009 From: spam at rm-rf.kiev.ua (Alex Kozlov) Date: Tue Oct 20 12:25:29 2009 Subject: Make process title - % complete Message-ID: <20091020122432.GA50817@ravenloft.kiev.ua> On Tue, Oct 20, 2009 at 10:07:07AM +0200, Lars Engels wrote: > Quoting Alex Kozlov : > > > On Mon, Oct 19, 2009 at 05:51:42PM +0200, Ivan Voras wrote: > >> 2009/10/19 Alex Kozlov : > >> > On Mon, Oct 19, 2009 at 04:35:08PM +0200, Ivan Voras wrote: > >> >> >> if nobody objects, I'll commit it :) > >> >> > > >> >> > I seem to recall that setproctitle() is quite expensive to > >> call; perhaps > >> >> > it would make sense offer a flag to prevent make(1) from > >> calling it? [1] > >> >> > > >> >> > Anyway, the feature looks nice! I'd like to have it... > >> >> > > >> >> > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd > >> >> > expect it's negligable but who knows... > >> >> > >> >> The loop it's called in is not processed bazillion times per second > >> >> (though it *is* called surprisingly often; small, fast jobs can result > >> >> in somewhere in the order of magnitude of 100 iterations per second on > >> >> a fast CPU). As you said - I expect it's negligable compared to fork() > >> >> and the work jobs themselves do. > >> > How about add this statistic to make info handler? > >> You mean SIGINFO? > > Yes > > Using SIGINFO sounds nice, but make produces so much output that > normally you won't see the result because it is scrolled up just after > sending the signal. Of course ps or top output much more convenient, but if setproctitle so expencive and will be called so often, then SIGINFO may be good compromise. -- Adios From ivoras at freebsd.org Tue Oct 20 12:42:55 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Oct 20 12:43:01 2009 Subject: Make process title - % complete In-Reply-To: <20091020122432.GA50817@ravenloft.kiev.ua> References: <20091020122432.GA50817@ravenloft.kiev.ua> Message-ID: Alex Kozlov wrote: > Of course ps or top output much more convenient, but if setproctitle so > expencive and will be called so often, then SIGINFO may be good > compromise. Regarding speed of setproctitle(), here are some microbenchmark results from the attached test source: getpid: 3661124.75 iterations/s setproctitle: 591357.56 iterations/s Meaning, setprocitle() is around 6 times more expensive than getpid(), meaning it can only be pulled off nearly 600,000 calls/s on a 2.3 GHz Core 2 CPU. I really want to be enlightened about how it could affect wallclock time in make(1). ---- #include #include #include #include #include #define NITER 1e7 double now() { struct timeval tp; gettimeofday(&tp, NULL); return tp.tv_sec + (double)tp.tv_usec / 1e6f; } int main() { double t1, t2, t3; int i; t1 = now(); for (i = 0; i < NITER; i++) getpid(); t2 = now() - t1; printf("getpid: %0.2f iterations/s\n", (float)(NITER/t2)); t1 = now(); for (i = 0; i < NITER; i++) setproctitle("t%d", i); t3 = now() - t1; printf("setproctitle: %0.2f iterations/s\n", (float)(NITER/t3)); } From ady at freebsd.ady.ro Tue Oct 20 12:49:16 2009 From: ady at freebsd.ady.ro (Adrian Penisoara) Date: Tue Oct 20 12:49:22 2009 Subject: Make process title - % complete In-Reply-To: References: Message-ID: <78cb3d3f0910200548p60fd32e5tc525899391e37f41@mail.gmail.com> Hi, On Mon, Oct 19, 2009 at 4:40 PM, Ivan Voras wrote: > Ivan Voras wrote: >> >> I have a small patch that makes "make" display percentage complete in >> process title, which can be retrieved in "top" in the form of: >> >> 71466 root ? ? ? ? ? ? 1 ?76 ? ?0 ?7008K ?5696K select ?0 ? 0:00 ?0.00% >> make: 95% (55 more targets out of 1360) (make) > > Also: is there someone here more familiar with "make" who can tell me if the > "current" top level target (i.e. the one taken from the command line) is > kept track of somewhere? For example "clean" in "make clean install". > gmake does show the nesting level in its output and indeed it's a valuable information if setproctitle is to be used... gmake appears to use a MAKELEVEL environment variable to keep track in between parent/child runs. I see a similar mechanism in our make (using the __MKLVL__ environment variable) but it's restricted only to the check_make_level() function that is checking the nesting level, thus no global variable is available to use. Regards, Adrian Penisoara EnterpriseBSD.com From forensixs at gmx.de Tue Oct 20 12:38:23 2009 From: forensixs at gmx.de (Manuel Gebele) Date: Tue Oct 20 12:49:37 2009 Subject: FreeBSD DAQ Card Facility [DCF] Message-ID: <20091020123819.30560@gmx.net> The acronym stands for DAQ (Data AcQuisition) Card Facility MG -- GRATIS f?r alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 From forensixs at gmx.de Tue Oct 20 12:43:09 2009 From: forensixs at gmx.de (Manuel Gebele) Date: Tue Oct 20 12:58:16 2009 Subject: FreeBSD DAQ Card Facility [DCF] Message-ID: <20091020124304.30560@gmx.net> If you take a look at my site on sourceforge you should know why that acronym. But I'm flexible, so maybe there is a better name for that. Thanks, MG -- Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3.5 - sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser From rnoland at FreeBSD.org Tue Oct 20 13:22:36 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Tue Oct 20 13:22:42 2009 Subject: Make process title - % complete In-Reply-To: References: <20091020122432.GA50817@ravenloft.kiev.ua> Message-ID: <1256044946.2386.28.camel@balrog.2hip.net> On Tue, 2009-10-20 at 14:42 +0200, Ivan Voras wrote: > Alex Kozlov wrote: > > > Of course ps or top output much more convenient, but if setproctitle so > > expencive and will be called so often, then SIGINFO may be good > > compromise. > > Regarding speed of setproctitle(), here are some microbenchmark results > from the attached test source: > > getpid: 3661124.75 iterations/s > setproctitle: 591357.56 iterations/s > > Meaning, setprocitle() is around 6 times more expensive than getpid(), > meaning it can only be pulled off nearly 600,000 calls/s on a 2.3 GHz > Core 2 CPU. > > I really want to be enlightened about how it could affect wallclock time > in make(1). What is the relative difference in buildworld time with and without? robert. > ---- > > #include > #include > #include > #include > #include > > #define NITER 1e7 > > double now() { > struct timeval tp; > gettimeofday(&tp, NULL); > return tp.tv_sec + (double)tp.tv_usec / 1e6f; > } > > int main() { > double t1, t2, t3; > int i; > > t1 = now(); > for (i = 0; i < NITER; i++) > getpid(); > t2 = now() - t1; > > printf("getpid: %0.2f iterations/s\n", (float)(NITER/t2)); > > t1 = now(); > for (i = 0; i < NITER; i++) > setproctitle("t%d", i); > t3 = now() - t1; > > printf("setproctitle: %0.2f iterations/s\n", (float)(NITER/t3)); > } > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Robert Noland FreeBSD From des at des.no Tue Oct 20 13:24:57 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue Oct 20 13:25:04 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: <20091020123819.30560@gmx.net> (Manuel Gebele's message of "Tue, 20 Oct 2009 14:38:19 +0200") References: <20091020123819.30560@gmx.net> Message-ID: <863a5etcyg.fsf@ds4.des.no> "Manuel Gebele" writes: > The acronym stands for DAQ (Data AcQuisition) Card Facility [...] If > you take a look at my site on sourceforge you should know why that > acronym. But I'm flexible, so maybe there is a better name for that. First, when you answer a question on a mailing list, it is customary to use your MUA's "reply" function and to quote the question. Otherwise, there is nothing to tie the answer to the question, and nobody will understand anything, except possibly the person who asked the question in the first place. It is also customary to answer once, not twice. That being taken care of - the reason I asked is that to me (and to many other time boffins and NTP fundamentalists), DCF is a longwave radio station in Frankfurt which is commonly used as an external reference for NTP servers. I would have just called it DAQ, which as you know is an established abbreviation for "Data Acquisition". The "card" part is meaningless; FreeBSD runs on plenty of hardware with integrated DAQ facilities, such as the Soekris, or pretty much anything built around a MIPS or ARM or (soon) AVR32 microcontroller. DES -- Dag-Erling Sm?rgrav - des@des.no From Trond.Endrestol at fagskolen.gjovik.no Tue Oct 20 13:33:43 2009 From: Trond.Endrestol at fagskolen.gjovik.no (=?ISO-8859-1?Q?Trond_Endrest=F8l?=) Date: Tue Oct 20 13:33:55 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: <863a5etcyg.fsf@ds4.des.no> References: <20091020123819.30560@gmx.net> <863a5etcyg.fsf@ds4.des.no> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 20 Oct 2009 15:24+0200, Dag-Erling Sm?rgrav wrote: > "Manuel Gebele" writes: > > The acronym stands for DAQ (Data AcQuisition) Card Facility [...] If > > you take a look at my site on sourceforge you should know why that > > acronym. But I'm flexible, so maybe there is a better name for that. [...] > That being taken care of - the reason I asked is that to me (and to many > other time boffins and NTP fundamentalists), DCF is a longwave radio > station in Frankfurt which is commonly used as an external reference for > NTP servers. The call sign is really DCF77. Trond. - -- - ---------------------------------------------------------------------- Trond Endrest?l | Trond.Endrestol@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX | FreeBSD 7.2-STABLE & Alpine 2.00 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkrdvDEACgkQbYWZalUoElugNACbBgWAVm2EUd6QRE7pA07oVrP6 ldcAn1YmYbFfpdu/z+bIT1eyj0cd1KiG =Gjqa -----END PGP SIGNATURE----- From spam at rm-rf.kiev.ua Tue Oct 20 13:33:49 2009 From: spam at rm-rf.kiev.ua (Alex Kozlov) Date: Tue Oct 20 13:33:55 2009 Subject: Make process title - % complete Message-ID: <20091020133343.GA53941@ravenloft.kiev.ua> On Tue, Oct 20, 2009 at 02:42:17PM +0200, Ivan Voras wrote: > Alex Kozlov wrote: > > > Of course ps or top output much more convenient, but if setproctitle so > > expencive and will be called so often, then SIGINFO may be good > > compromise. > > Regarding speed of setproctitle(), here are some microbenchmark results > from the attached test source: > > getpid: 3661124.75 iterations/s > setproctitle: 591357.56 iterations/s > > Meaning, setprocitle() is around 6 times more expensive than getpid(), > meaning it can only be pulled off nearly 600,000 calls/s on a 2.3 GHz > Core 2 CPU. > > I really want to be enlightened about how it could affect wallclock time > in make(1). make universe few times with and without patch? -- Adios From des at des.no Tue Oct 20 13:39:28 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue Oct 20 13:39:35 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: ("Trond =?utf-8?Q?Endrest=C3=B8l=22's?= message of "Tue, 20 Oct 2009 15:33:37 +0200 (CEST)") References: <20091020123819.30560@gmx.net> <863a5etcyg.fsf@ds4.des.no> Message-ID: <86y6n6rxps.fsf@ds4.des.no> Trond Endrest?l writes: > The call sign is really DCF77. ...and it's not really in Frankfurt. True, but irrelevant. See RFC2030. DES -- Dag-Erling Sm?rgrav - des@des.no From rdivacky at freebsd.org Tue Oct 20 17:33:21 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Tue Oct 20 17:33:28 2009 Subject: Make process title - % complete In-Reply-To: References: <20091020122432.GA50817@ravenloft.kiev.ua> Message-ID: <20091020171354.GA92192@freebsd.org> On Tue, Oct 20, 2009 at 02:42:17PM +0200, Ivan Voras wrote: > Alex Kozlov wrote: > > >Of course ps or top output much more convenient, but if setproctitle so > >expencive and will be called so often, then SIGINFO may be good > >compromise. > > Regarding speed of setproctitle(), here are some microbenchmark results > from the attached test source: > > getpid: 3661124.75 iterations/s > setproctitle: 591357.56 iterations/s > > Meaning, setprocitle() is around 6 times more expensive than getpid(), > meaning it can only be pulled off nearly 600,000 calls/s on a 2.3 GHz > Core 2 CPU. what about contention? setproctitle() is an sysctl so it will prevent other sysctl's from working when being executed.. From ivoras at freebsd.org Tue Oct 20 20:29:04 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Oct 20 20:29:17 2009 Subject: Make process title - % complete In-Reply-To: <20091020171354.GA92192@freebsd.org> References: <20091020122432.GA50817@ravenloft.kiev.ua> <20091020171354.GA92192@freebsd.org> Message-ID: <9bbcef730910201327h3bbcc526ja7a8283addfe2667@mail.gmail.com> 2009/10/20 Roman Divacky : > On Tue, Oct 20, 2009 at 02:42:17PM +0200, Ivan Voras wrote: >> Alex Kozlov wrote: >> >> >Of course ps or top output much more convenient, but if setproctitle so >> >expencive and will be called so often, then SIGINFO may be good >> >compromise. >> >> Regarding speed of setproctitle(), here are some microbenchmark results >> from the attached test source: >> >> getpid: 3661124.75 iterations/s >> setproctitle: 591357.56 iterations/s >> >> Meaning, setprocitle() is around 6 times more expensive than getpid(), >> meaning it can only be pulled off nearly 600,000 calls/s on a 2.3 GHz >> Core 2 CPU. > > what about contention? setproctitle() is an sysctl so it will prevent > other sysctl's from working when being executed.. Others sysctls... for that particular process (since it modifies process-global data) which happens to be single-threaded :P From keramida at ceid.upatras.gr Tue Oct 20 23:44:32 2009 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Tue Oct 20 23:44:39 2009 Subject: Make process title - % complete In-Reply-To: <9bbcef730910190851m7e82bd1aqc0bc55106c8a5d37@mail.gmail.com> (Ivan Voras's message of "Mon, 19 Oct 2009 17:51:42 +0200") References: <20091019144135.GA91918@ravenloft.kiev.ua> <9bbcef730910190851m7e82bd1aqc0bc55106c8a5d37@mail.gmail.com> Message-ID: <87tyxtbpgp.fsf@kobe.laptop> On Mon, 19 Oct 2009 17:51:42 +0200, Ivan Voras wrote: >2009/10/19 Alex Kozlov : >> How about add this statistic to make info handler? > > You mean SIGINFO? Yes, that's the ``info handler''. While printing something on SINGINFO arrival is a nice idea, it may not be extremely useful for make(1). With dd(1) this is very useful to see, but with long-running make jobs that write tons of output to stderr any information from SIGINFO may be lost in the noise. From spawk at acm.poly.edu Wed Oct 21 00:57:11 2009 From: spawk at acm.poly.edu (Boris Kochergin) Date: Wed Oct 21 00:57:18 2009 Subject: Make process title - % complete In-Reply-To: <87tyxtbpgp.fsf@kobe.laptop> References: <20091019144135.GA91918@ravenloft.kiev.ua> <9bbcef730910190851m7e82bd1aqc0bc55106c8a5d37@mail.gmail.com> <87tyxtbpgp.fsf@kobe.laptop> Message-ID: <4ADE55B8.3080100@acm.poly.edu> Giorgos Keramidas wrote: > On Mon, 19 Oct 2009 17:51:42 +0200, Ivan Voras wrote: > >> 2009/10/19 Alex Kozlov : >> >>> How about add this statistic to make info handler? >>> >> You mean SIGINFO? >> > > Yes, that's the ``info handler''. > > While printing something on SINGINFO arrival is a nice idea, it may not > be extremely useful for make(1). With dd(1) this is very useful to see, > but with long-running make jobs that write tons of output to stderr any > information from SIGINFO may be lost in the noise. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > The SIGINFO handler could be made to put the make process to sleep for, say, a second to give the user some time to read the statistics, but I'm sure there are lots of objections to be made to that, too. -Boris From max at love2party.net Wed Oct 21 01:11:45 2009 From: max at love2party.net (Max Laier) Date: Wed Oct 21 01:12:18 2009 Subject: Make process title - % complete In-Reply-To: <20091019140806.GB95902@rink.nu> References: <20091019140806.GB95902@rink.nu> Message-ID: <200910210311.54871.max@love2party.net> On Monday 19 October 2009 16:08:06 Rink Springer wrote: > Hi Ivan, > > On Mon, Oct 19, 2009 at 03:52:30PM +0200, Ivan Voras wrote: > > if nobody objects, I'll commit it :) > > I seem to recall that setproctitle() is quite expensive to call; perhaps > it would make sense offer a flag to prevent make(1) from calling it? [1] Just rate-limit the setproctitle() call to once/sec or once/percentage-step and be done with it. I must say that trying it out on a kernel build didn't proof too useful as the targets have vastly different runtimes, but I think it's a good addition nonetheless. So please, go for it Ivan. > Anyway, the feature looks nice! I'd like to have it... > > [1] I'm unsure how expensive it is compared to fork(1)-ing etc; I'd > expect it's negligable but who knows... > -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From keramida at ceid.upatras.gr Wed Oct 21 01:23:21 2009 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Wed Oct 21 01:23:27 2009 Subject: Make process title - % complete In-Reply-To: <4ADE55B8.3080100@acm.poly.edu> (Boris Kochergin's message of "Tue, 20 Oct 2009 20:28:40 -0400") References: <20091019144135.GA91918@ravenloft.kiev.ua> <9bbcef730910190851m7e82bd1aqc0bc55106c8a5d37@mail.gmail.com> <87tyxtbpgp.fsf@kobe.laptop> <4ADE55B8.3080100@acm.poly.edu> Message-ID: <87hbtt4k1o.fsf@kobe.laptop> On Tue, 20 Oct 2009 20:28:40 -0400, Boris Kochergin wrote: >Giorgos Keramidas wrote: >>On Mon, 19 Oct 2009 17:51:42 +0200, Ivan Voras wrote: >>>2009/10/19 Alex Kozlov : >>>> How about add this statistic to make info handler? >>> >>> You mean SIGINFO? >> >> Yes, that's the ``info handler''. >> >> While printing something on SINGINFO arrival is a nice idea, it may >> not be extremely useful for make(1). With dd(1) this is very useful >> to see, but with long-running make jobs that write tons of output to >> stderr any information from SIGINFO may be lost in the noise. > > The SIGINFO handler could be made to put the make process to sleep > for, say, a second to give the user some time to read the statistics, > but I'm sure there are lots of objections to be made to that, too. That would be bad, indeed. David Wolfskill has emailed me, in the meantime, that it's probably ok to `lose' SIGINFO output in the noise of build output, as long as it is easy to grep for it. So I still like the idea, but without a delay please :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091021/8b626c51/attachment.pgp From alexbestms at math.uni-muenster.de Wed Oct 21 01:41:23 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Wed Oct 21 01:41:30 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED Message-ID: hi there, just a little mmap(2) related question. running the following code causes a segfault: mmap( (void*)0x1000, 0x80047000, PROT_NONE, MAP_ANON|MAP_FIXED, -1, 0 ); while the following doesn't: mmap( (void*)0x1000, 0xffffffff, PROT_NONE, MAP_ANON|MAP_FIXED, -1, 0 ); is this a known problem? seems reproducible on all branches. cheers alex -------------- next part -------------- #include #include #include int main() { printf("mmap: %p\n", mmap( (void*)0x1000, 0xffffffff, PROT_NONE, MAP_ANON|MAP_FIXED, -1, 0 )); printf("mmap: %p\n", mmap( (void*)0x1000, 0x80047000, PROT_NONE, MAP_ANON|MAP_FIXED, -1, 0 )); return 0; } From nate at thatsmathematics.com Wed Oct 21 02:38:26 2009 From: nate at thatsmathematics.com (Nate Eldredge) Date: Wed Oct 21 02:38:33 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED In-Reply-To: References: Message-ID: On Wed, 21 Oct 2009, Alexander Best wrote: > hi there, This is on a 32-bit platform I take it? > just a little mmap(2) related question. running the following code causes a > segfault: > > mmap( (void*)0x1000, 0x80047000, PROT_NONE, MAP_ANON|MAP_FIXED, -1, 0 ); I don't doubt it. You mapped over a big chunk of your address space with memory that's inaccessible (PROT_NONE). This probably includes your program's code. So when the mmap call returns from the kernel and tries to execute the next instruction of your program, it finds that the instruction pointer is pointing to inaccessible memory. Result: segfault. This is quite normal. What are you actually trying to accomplish with this? > while the following doesn't: > > mmap( (void*)0x1000, 0xffffffff, PROT_NONE, MAP_ANON|MAP_FIXED, -1, 0 ); Did you check whether the mmap actually succeeded? I bet it didn't. You have a length that isn't a multiple of the page size and wraps around 32 bits. I bet you got an EINVAL, and the mmap call didn't actually do anything. > is this a known problem? seems reproducible on all branches. Not a problem at all, I suspect. -- Nate Eldredge nate@thatsmathematics.com From ady at freebsd.ady.ro Wed Oct 21 09:03:43 2009 From: ady at freebsd.ady.ro (Adrian Penisoara) Date: Wed Oct 21 09:03:51 2009 Subject: Make process title - % complete In-Reply-To: <200910210311.54871.max@love2party.net> References: <20091019140806.GB95902@rink.nu> <200910210311.54871.max@love2party.net> Message-ID: <78cb3d3f0910210203n66b0b08bl1bbfb3db7ec7e024@mail.gmail.com> Hi, On Wed, Oct 21, 2009 at 3:11 AM, Max Laier wrote: > On Monday 19 October 2009 16:08:06 Rink Springer wrote: >> Hi Ivan, >> >> On Mon, Oct 19, 2009 at 03:52:30PM +0200, Ivan Voras wrote: >> > if nobody objects, I'll commit it :) >> >> I seem to recall that setproctitle() is quite expensive to call; perhaps >> it would make sense offer a flag to prevent make(1) from calling it? [1] > > Just rate-limit the setproctitle() call to once/sec or once/percentage-step > and be done with it. Rather try to setproctitle() in the same make process every second with a one second initial delay (so that short lived make processes won't be bogged down by this expensive call). And preferentially do the timing check after returning from exec() of a child make. This way the stats won't be "perturbed" by the short lived make's and only one make process will call setproctitle() at any time (except when running "make -j" ?). > > I must say that trying it out on a kernel build didn't proof too useful as the > targets have vastly different runtimes, but I think it's a good addition > nonetheless. ?So please, go for it Ivan. > If you implement it, please use a control mechanism like, say, an environment variable MAKE_TRACK_PROGRESS, which, for performance and POLA sake, might default to disabled (including when environment is not defined). My 5cents, Adrian Penisoara EnterpriseBSD.com From des at des.no Wed Oct 21 10:20:28 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed Oct 21 10:20:35 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: <20091021100451.98450@gmx.net> (Manuel Gebele's message of "Wed, 21 Oct 2009 12:04:51 +0200") References: <20091021100451.98450@gmx.net> Message-ID: <863a5dt5ec.fsf@ds4.des.no> "Manuel Gebele" writes: > Dag-Erling Sm?rgrav writes: > > I would have just called it DAQ, which as you know is an established > > abbreviation for "Data Acquisition". [...] > I have no objection to that naming. Maybe I should call it just > ``FreeBSD DAQ Facility'' (FDF). I don't understand why it has to be so complicated. What's wrong with just "DAQ"? (not that any of this really matters in the big picture...) DES -- Dag-Erling Sm?rgrav - des@des.no From forensixs at gmx.de Wed Oct 21 12:01:23 2009 From: forensixs at gmx.de (Manuel Gebele) Date: Wed Oct 21 12:01:30 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: <863a5dt5ec.fsf@ds4.des.no> References: <20091021100451.98450@gmx.net> <863a5dt5ec.fsf@ds4.des.no> Message-ID: <20091021120120.163550@gmx.net> on 21/10/2009 12:20 Dag-Erling Sm?rgrav said the following: > I don't understand why it has to be so complicated. > What's wrong with just "DAQ"? [...] What please is complicated in ``FreeBSD DAQ Facility''? Whats about KLD ``Dynamic Kernel Linker Facility''? BTW, if I send you a private email and you answer to the list, it would be great if you could put in all of the email content: on 20/10/2009 15:24 Dag-Erling Sm?rgrav said the following: > First, when you answer a question on a mailing list, > it is customary to use your MUA's "reply" function and > to quote the question. Otherwise, there is nothing to > tie the answer to the question, and nobody will understand > anything, except possibly the person who asked the question > in the first place. [...] Oh, my mistake sorry for that. I'm not an ML expert at all, and that was my first message to a list (in that 'reply' context). > That being taken care of - the reason I asked is that > to me (and to many other time boffins and NTP > fundamentalists), DCF is a longwave radio station in > Frankfurt which is commonly used as an external reference > for NTP servers. DCF(77) is in the near of "Frankfurt am Main (not Oder)" thats true. On the other side, there are so many definitions for DCF; Data Capture Facility, Data Compression Facility and so on (look at http://www.acronymfinder.com/DCF.html). We get problems if we open ``/dev/dcf77'' --maybe =) Whats about the MSF station in Rugby --england? MSF - Microsoft Solutions Framework (please no jokes about $MS) MSF - Metasploit Framework Whats about code with an 'VLF' acronym =) ? --better not But I understand your point, maybe we should exclude any misunderstandings. > I would have just called it DAQ, which as you know is > an established abbreviation for "Data Acquisition". [...] I have no objection to that naming. Maybe I should call it just ``FreeBSD DAQ Facility'' (FDF). Thanks, Manuel Gebele -- Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3.5 - sicherer, schneller und einfacher! http://portal.gmx.net/de/go/atbrowser From alexbestms at math.uni-muenster.de Wed Oct 21 13:53:14 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Wed Oct 21 13:53:20 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED In-Reply-To: Message-ID: Nate Eldredge schrieb am 2009-10-21: > On Wed, 21 Oct 2009, Alexander Best wrote: > >hi there, > This is on a 32-bit platform I take it? yes. > >just a little mmap(2) related question. running the following code > >causes a > >segfault: > >mmap( (void*)0x1000, 0x80047000, PROT_NONE, MAP_ANON|MAP_FIXED, -1, > >0 ); > I don't doubt it. You mapped over a big chunk of your address space > with memory that's inaccessible (PROT_NONE). This probably includes > your program's code. So when the mmap call returns from the kernel > and tries to execute the next instruction of your program, it finds > that the instruction pointer is pointing to inaccessible memory. > Result: segfault. This is quite normal. > What are you actually trying to accomplish with this? this code serves only one purpose: to trigger a segfault. i don't use the code for any other purpose. i was under the impression that mmap() should either succeed or fail (tertium non datur). mmap's manual doesn't say anything about mmap() causing segfaults. from your description of the problem i don't think there's a quick fix to it. so it might be a good idea to add this case to the mmap(2) bug section. > >while the following doesn't: > >mmap( (void*)0x1000, 0xffffffff, PROT_NONE, MAP_ANON|MAP_FIXED, -1, > >0 ); > Did you check whether the mmap actually succeeded? I bet it didn't. > You have a length that isn't a multiple of the page size and wraps > around 32 bits. I bet you got an EINVAL, and the mmap call didn't > actually do anything. > >is this a known problem? seems reproducible on all branches. > Not a problem at all, I suspect. > -- indeed the mmap() call fails with EINVAL but it doesn't segfault. to make this clear: i don't expect either one of the mmap() calls to succeed. > Nate Eldredge > nate@thatsmathematics.com From rwatson at FreeBSD.org Wed Oct 21 15:06:05 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Wed Oct 21 15:06:11 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED In-Reply-To: References: Message-ID: On Wed, 21 Oct 2009, Alexander Best wrote: > this code serves only one purpose: to trigger a segfault. i don't use the > code for any other purpose. i was under the impression that mmap() should > either succeed or fail (tertium non datur). mmap's manual doesn't say > anything about mmap() causing segfaults. Have you tried ktracing the application? I think you'll find that mmap(2) system call succeeded fine, and that the segfault comes from attempting to execute the address in libc on return to userspace, as a result of libc not being at that address anymore (since you removed its mapping). You can use procstat -v to inspect address space use by processes, but as a general rule you don't want to pass anything other than an address of 0x0 to mmap(2) unless you're very carefully managing the address space of the process. Many userspace libraries are involved in using that address space, but especially the runtime linker which begins execution in userspace when a binary is started. Robert N M Watson Computer Laboratory University of Cambridge From alexbestms at math.uni-muenster.de Wed Oct 21 15:30:54 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Wed Oct 21 15:31:00 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED In-Reply-To: Message-ID: Robert Watson schrieb am 2009-10-21: > On Wed, 21 Oct 2009, Alexander Best wrote: > >this code serves only one purpose: to trigger a segfault. i don't > >use the code for any other purpose. i was under the impression that > >mmap() should either succeed or fail (tertium non datur). mmap's > >manual doesn't say anything about mmap() causing segfaults. > Have you tried ktracing the application? I think you'll find that > mmap(2) system call succeeded fine, and that the segfault comes from > attempting to execute the address in libc on return to userspace, as > a result of libc not being at that address anymore (since you > removed its mapping). You can use procstat -v to inspect address > space use by processes, but as a general rule you don't want to pass > anything other than an address of 0x0 to mmap(2) unless you're very > carefully managing the address space of the process. Many userspace > libraries are involved in using that address space, but especially > the runtime linker which begins execution in userspace when a binary > is started. > Robert N M Watson > Computer Laboratory > University of Cambridge you're right. this kdump shows that the segfault isn't being caused by the mmap() call: 88343 mmap_test CALL mmap(0x1000,0x80047000,PROT_NONE,MAP_FIXED|MAP_ANON,0xffffffff,0,0) 88343 mmap_test RET mmap 4096/0x1000 88343 mmap_test PSIG SIGSEGV SIG_DFL 88343 mmap_test NAMI "mmap_test.core" thanks for clearing things up. however i stil think mentioning this situation in the mmap(2) manual (maybe in section MAP_FIXED) would be a good idea. cheers. alex From alexbestms at math.uni-muenster.de Wed Oct 21 15:51:13 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Wed Oct 21 15:51:22 2009 Subject: mmap(2) with MAP_ANON honouring offset although it shouldn't Message-ID: although the mmap(2) manual states in section MAP_ANON: "The offset argument is ignored." this doesn't seem to be true. running printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, 0x12345678)); and printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, 0)); produces different outputs. i've attached a patch to solve the problem. the patch is similar to the one proposed in this PR, but should apply cleanly to CURRENT: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/71258 cheers. alex -------------- next part -------------- --- src/sys/vm/vm_mmap.c 2009-10-21 04:13:24.000000000 +0200 +++ src/sys/vm/vm_mmap.c 2009-10-21 04:13:43.000000000 +0200 @@ -245,15 +245,18 @@ } /* - * Align the file position to a page boundary, - * and save its page offset component. + * Unless the MAP_ANON flag is set, align the file position + * to a page boundary and save its page offset component. */ - pageoff = (pos & PAGE_MASK); - pos -= pageoff; - - /* Adjust size for rounding (on both ends). */ - size += pageoff; /* low end... */ - size = (vm_size_t) round_page(size); /* hi end */ + if (flags & MAP_ANON) { + pageoff = pos = 0; + } else { + pageoff = (pos & PAGE_MASK); + pos -= pageoff; + /* Adjust size for rounding (on both ends). */ + size += pageoff; /* low end... */ + size = (vm_size_t) round_page(size); /* hi end */ + } /* * Check for illegal addresses. Watch out for address wrap... Note @@ -300,7 +303,6 @@ handle = NULL; handle_type = OBJT_DEFAULT; maxprot = VM_PROT_ALL; - pos = 0; } else { /* * Mapping file, get fp for validation and From des at des.no Wed Oct 21 17:16:51 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed Oct 21 17:16:59 2009 Subject: FreeBSD DAQ Card Facility [DCF] In-Reply-To: <20091021120120.163550@gmx.net> (Manuel Gebele's message of "Wed, 21 Oct 2009 14:01:20 +0200") References: <20091021100451.98450@gmx.net> <863a5dt5ec.fsf@ds4.des.no> <20091021120120.163550@gmx.net> Message-ID: <86y6n4sm4d.fsf@ds4.des.no> "Manuel Gebele" writes: > BTW, if I send you a private email and you answer to the list, I answered to an email you sent to the list. DES -- Dag-Erling Sm?rgrav - des@des.no From jason.harmening at gmail.com Wed Oct 21 16:42:48 2009 From: jason.harmening at gmail.com (Jason Harmening) Date: Wed Oct 21 17:29:43 2009 Subject: multi-seg bus_dmamem_alloc? Message-ID: <2d1264630910210921w4a2fabb1h86e12658a3f7c714@mail.gmail.com> Hi everyone, It seems like there are starting to be some drivers that need to allocate large chunks of DMA-able memory, and since bus_dmamem_alloc() on most architectures is always physically contiguous, it may not work for them. It seems like we could use the new sglist routines to help us here: --Define 2 new functions: int bus_dmamem_alloc_sglist(bus_dma_tag_t dmat, size_t size, struct sglist *sg, int flags, bus_dmamap_t *mapp) void bus_dmamem_free_sglist(bus_dma_tag_t dmat, struct sglist *sg, bus_dmamap_t map); --For sparc64 (or anywhere else we want to use an IOMMU): malloc() the buffer, feed it to sglist_build(), program the IOMMU to meet the constraints in dmat--Isn't this what we already do for sparc64, minus the sglist part? --For direct-mapped architectures: If the constraints in dmat are lenient enough, do malloc() and sglist_build(). Otherwise, do contigmalloc(M_NOWAIT) in a loop, in which we try to allocate as much of the buffer as possible. Anytime an allocation fails, we divide the allocation size by (roughly) 2 until the allocation succeeds, and continue allocating until either we've allocated enough space, or the allocation size drops below PAGE_SIZE, or we exceed dmat->maxsegs. --Some other things we'd need: --bus_dmamap_load_sglist()--I think jhb already did this as part of the sglist work, at least for amd64 --Structures in the busdma map to track allocated buffers so we could free them later. --Are there lower-level calls we could make to just allocate the physical pages instead of malloc()/contigmalloc()? The kva mapping for each allocated buffer segment isn't necessary. A lot of drivers would probably just want to mmap the sglist to userspace anyway. --Could we instead just integrate this multi-seg functionality into the default bus_dmamem_alloc()? We'd at least have to be able map the physical segments into a contiguous kva area--we wouldn't necessarily have to use an sglist in this case either. Let me know if this idea has any potential--if it does, I'd love to try implementing it:) --Jason From alan.l.cox at gmail.com Wed Oct 21 17:51:10 2009 From: alan.l.cox at gmail.com (Alan Cox) Date: Wed Oct 21 17:51:29 2009 Subject: mmap(2) with MAP_ANON honouring offset although it shouldn't In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 10:51 AM, Alexander Best < alexbestms@math.uni-muenster.de> wrote: > although the mmap(2) manual states in section MAP_ANON: > > "The offset argument is ignored." > > this doesn't seem to be true. running > > printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, > 0x12345678)); > > and > > printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, 0)); > > produces different outputs. i've attached a patch to solve the problem. the > patch is similar to the one proposed in this PR, but should apply cleanly > to > CURRENT: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/71258 > The standards for mmap(2) actually disallow values of "off" that are not a multiple of the page size. See http://www.opengroup.org/onlinepubs/000095399/functions/mmap.html for the following: [EINVAL]The *addr* argument (if MAP_FIXED was specified) or *off* is not a multiple of the page size as returned by *sysconf*(), or is considered invalid by the implementation.Both Solaris and Linux enforce this restriction. I'm not convinced that the ability to specify a value for "off" that is not a multiple of the page size is a useful differentiating feature of FreeBSD versus Solaris or Linux. Does anyone have a compelling argument (or use case) to motivate us being different in this respect? If you disallow values for "off" that are not a multiple of the page size, then you are effectively ignoring "off" for MAP_ANON. Regards, Alan From jhb at freebsd.org Wed Oct 21 17:51:16 2009 From: jhb at freebsd.org (John Baldwin) Date: Wed Oct 21 17:51:29 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED In-Reply-To: References: Message-ID: <200910211340.39872.jhb@freebsd.org> On Wednesday 21 October 2009 11:30:51 am Alexander Best wrote: > Robert Watson schrieb am 2009-10-21: > > > On Wed, 21 Oct 2009, Alexander Best wrote: > > > >this code serves only one purpose: to trigger a segfault. i don't > > >use the code for any other purpose. i was under the impression that > > >mmap() should either succeed or fail (tertium non datur). mmap's > > >manual doesn't say anything about mmap() causing segfaults. > > > Have you tried ktracing the application? I think you'll find that > > mmap(2) system call succeeded fine, and that the segfault comes from > > attempting to execute the address in libc on return to userspace, as > > a result of libc not being at that address anymore (since you > > removed its mapping). You can use procstat -v to inspect address > > space use by processes, but as a general rule you don't want to pass > > anything other than an address of 0x0 to mmap(2) unless you're very > > carefully managing the address space of the process. Many userspace > > libraries are involved in using that address space, but especially > > the runtime linker which begins execution in userspace when a binary > > is started. > > > Robert N M Watson > > Computer Laboratory > > University of Cambridge > > > you're right. this kdump shows that the segfault isn't being caused by the > mmap() call: > > 88343 mmap_test CALL > mmap(0x1000,0x80047000,PROT_NONE,MAP_FIXED|MAP_ANON,0xffffffff,0,0) > 88343 mmap_test RET mmap 4096/0x1000 > 88343 mmap_test PSIG SIGSEGV SIG_DFL > 88343 mmap_test NAMI "mmap_test.core" > > thanks for clearing things up. > > however i stil think mentioning this situation in the mmap(2) manual (maybe in > section MAP_FIXED) would be a good idea. I'm not sure it is useful to attempt to enumerate all the possible ways one can shoot one's own foot using mmap(2) in the manual page. The list would be quite long and would require a large amount of imagination. In effect, you are asking for a manual page to document all the possible bugs one could have and manual pages in general do not do that. -- John Baldwin From jhb at freebsd.org Wed Oct 21 17:51:17 2009 From: jhb at freebsd.org (John Baldwin) Date: Wed Oct 21 17:51:30 2009 Subject: mmap(2) with MAP_ANON honouring offset although it shouldn't In-Reply-To: References: Message-ID: <200910211349.10174.jhb@freebsd.org> On Wednesday 21 October 2009 11:51:04 am Alexander Best wrote: > although the mmap(2) manual states in section MAP_ANON: > > "The offset argument is ignored." > > this doesn't seem to be true. running > > printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, > 0x12345678)); > > and > > printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, 0)); > > produces different outputs. i've attached a patch to solve the problem. the > patch is similar to the one proposed in this PR, but should apply cleanly to > CURRENT: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/71258 A simpler patch would be to simply set pos = 0 below the MAP_STACK line if MAP_ANON is set. -- John Baldwin From ben.crowhurst at beatsystems.com Thu Oct 22 09:52:39 2009 From: ben.crowhurst at beatsystems.com (Ben Crowhurst) Date: Thu Oct 22 09:52:47 2009 Subject: mmap(2) with MAP_ANON honouring offset although it shouldn't In-Reply-To: References: Message-ID: <4AE02815.2010103@beatsystems.com> Alan Cox wrote: > On Wed, Oct 21, 2009 at 10:51 AM, Alexander Best < > alexbestms@math.uni-muenster.de> wrote: > > >> although the mmap(2) manual states in section MAP_ANON: >> >> "The offset argument is ignored." >> >> this doesn't seem to be true. running >> >> printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, >> 0x12345678)); >> >> and >> >> printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, 0)); >> >> produces different outputs. i've attached a patch to solve the problem. the >> patch is similar to the one proposed in this PR, but should apply cleanly >> to >> CURRENT: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/71258 >> >> > > The standards for mmap(2) actually disallow values of "off" that are not a > multiple of the page size. > > See http://www.opengroup.org/onlinepubs/000095399/functions/mmap.html for > the following: > [EINVAL]The *addr* argument (if MAP_FIXED was specified) or *off* is not a > multiple of the page size as returned by > *sysconf*(), > or is considered invalid by the implementation.Both Solaris and Linux > enforce this restriction. > > I'm not convinced that the ability to specify a value for "off" that is not > a multiple of the page size is a useful differentiating feature of FreeBSD > versus Solaris or Linux. Does anyone have a compelling argument (or use > case) to motivate us being different in this respect? > > If you disallow values for "off" that are not a multiple of the page size, > then you are effectively ignoring "off" for MAP_ANON. > > Regards, > Alan > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > would it be such a bad idea as to round up the addr/off arguments to the next page size? This was most likely the intention of the caller anyway. Cheers, Ben From yeqing.yan at intel.com Thu Oct 22 08:38:45 2009 From: yeqing.yan at intel.com (Yan, Yeqing) Date: Thu Oct 22 11:31:34 2009 Subject: About FreeBSD syscall usage Message-ID: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> Hi: I?m from Intel China. Our project use FreeBSD 7.0. I have some question about syscall usage but I think mail to the wrong address before. Is there having any doc or example about how to use these syscall? kse_exit kse_wakeup kse_create kse_thr_interrupt kse_release kse_switchin I read $man kse, but I can not find any example about how to use it. I write some test codes to call these function but all these codes are failed. mac_syscall I read $man 3 mac, but I can not find the usage about mac_syscall function. thr_create thr_suspend thr_kill2 By the way, it is said ?I think that KSE was used in 5.x and 6.x and then dropped in favor of a 1:1 threading model when 7.0 was released? Does it mean the KSE syscall can be removed from FreeBSD 7.0? Thank you very much! -----Original Message----- From: Kevin Kinsey [mailto:kdk@daleco.biz] Sent: 2009?10?21? 22:18 To: Yan, Yeqing Cc: freebsd-questions@FreeBSD.org Subject: Re: Question about FreeBSD syscall usage Yan, Yeqing wrote: > Hi: > I'm from Intel China. Our project use FreeBSD 7.0 > and I have some questions about the FreeBSD syscall. > I don't know how to use these syscall below. > Is there having some doc or example about how to use these syscall? > > kse_exit > kse_wakeup > kse_create > kse_thr_interrupt > kse_release > kse_switchin > > mac_syscall > > thr_create > thr_suspend > thr_kill2 > > Thank you very much! > > Best Regards > Yan, Yeqing Hello Yeqing, You might want to write to "hackers@freebsd.org" ... ... some of those guys *wrote* these syscalls. However, since it's a question, I'll take a stab at it. Have you read: $man kse $man 3 mac $man libthr ? Also, see www.freebsd.org/kse/ However, I think that KSE was used in 5.x and 6.x and then dropped in favor of a 1:1 threading model when 7.0 was released (I'm sure some "hacker@" can correct this information if I'm wrong). I hope this is helpful to you. Kevin Kinsey Best Regards Yan, Yeqing From stas at deglitch.com Thu Oct 22 12:41:52 2009 From: stas at deglitch.com (Stanislav Sedov) Date: Thu Oct 22 12:42:50 2009 Subject: About FreeBSD syscall usage In-Reply-To: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> References: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> Message-ID: <20091022164144.064b916b.stas@deglitch.com> On Thu, 22 Oct 2009 16:20:32 +0800 "Yan, Yeqing" mentioned: > kse_exit > > kse_wakeup > > kse_create > > kse_thr_interrupt > > kse_release > > kse_switchin > > I read $man kse, but I can not find any example about how to use it. I write some test codes to call these function but all these codes are failed. > > > > mac_syscall > > I read $man 3 mac, but I can not find the usage about mac_syscall function. > > > > thr_create > > thr_suspend > > thr_kill2 > I fear there's no documentation on these syscalls exists. So for use information you'll have to refer to the actual source code of these syscalls/or and libc/libthr source code which makes uses of them. > > By the way, it is said ?I think that KSE was used in 5.x and 6.x and then dropped in favor of a 1:1 threading model when 7.0 was released? > > Does it mean the KSE syscall can be removed from FreeBSD 7.0? > libkse (M:N threading) was default threading library on FreeBSD versions prior to 7.0, and the default has changed to libthr (1:1 threading in FreeBSD 7). libkse was completely removed in FreeBSD 8, but it is still functional on FreeBSD 7.x. -- Stanislav Sedov ST4096-RIPE -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 801 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091022/f21ecb29/attachment.pgp From jhb at freebsd.org Thu Oct 22 12:47:06 2009 From: jhb at freebsd.org (John Baldwin) Date: Thu Oct 22 12:47:29 2009 Subject: About FreeBSD syscall usage In-Reply-To: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> References: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> Message-ID: <200910220834.32082.jhb@freebsd.org> On Thursday 22 October 2009 4:20:32 am Yan, Yeqing wrote: > Hi: > > I?m from Intel China. Our project use FreeBSD 7.0. I have some question about syscall usage but I think mail to the wrong address before. > > Is there having any doc or example about how to use these syscall? > > > > kse_exit > > kse_wakeup > > kse_create > > kse_thr_interrupt > > kse_release > > kse_switchin These are used internally to implement pthreads when using KSE. Probably the only useful documentation would be the source for libkse. > I read $man kse, but I can not find any example about how to use it. I write some test codes to call these function but all these codes are failed. > > > > mac_syscall > > I read $man 3 mac, but I can not find the usage about mac_syscall function. This one I am not sure of. > thr_create > > thr_suspend > > thr_kill2 These are used by libpthread to implement pthreads (libthr in < 7.x). The best documentation for these would be the source to libpthread as well. > By the way, it is said ?I think that KSE was used in 5.x and 6.x and then dropped in favor of a 1:1 threading model when 7.0 was released? > > Does it mean the KSE syscall can be removed from FreeBSD 7.0? It has been removed entirely from 8.0. It is still present in 7, but it is deprecated and not the default. -- John Baldwin From jhb at freebsd.org Thu Oct 22 12:47:06 2009 From: jhb at freebsd.org (John Baldwin) Date: Thu Oct 22 12:47:30 2009 Subject: About FreeBSD syscall usage In-Reply-To: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> References: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> Message-ID: <200910220834.32082.jhb@freebsd.org> On Thursday 22 October 2009 4:20:32 am Yan, Yeqing wrote: > Hi: > > I?m from Intel China. Our project use FreeBSD 7.0. I have some question about syscall usage but I think mail to the wrong address before. > > Is there having any doc or example about how to use these syscall? > > > > kse_exit > > kse_wakeup > > kse_create > > kse_thr_interrupt > > kse_release > > kse_switchin These are used internally to implement pthreads when using KSE. Probably the only useful documentation would be the source for libkse. > I read $man kse, but I can not find any example about how to use it. I write some test codes to call these function but all these codes are failed. > > > > mac_syscall > > I read $man 3 mac, but I can not find the usage about mac_syscall function. This one I am not sure of. > thr_create > > thr_suspend > > thr_kill2 These are used by libpthread to implement pthreads (libthr in < 7.x). The best documentation for these would be the source to libpthread as well. > By the way, it is said ?I think that KSE was used in 5.x and 6.x and then dropped in favor of a 1:1 threading model when 7.0 was released? > > Does it mean the KSE syscall can be removed from FreeBSD 7.0? It has been removed entirely from 8.0. It is still present in 7, but it is deprecated and not the default. -- John Baldwin From julian at elischer.org Thu Oct 22 17:43:51 2009 From: julian at elischer.org (Julian Elischer) Date: Thu Oct 22 17:44:00 2009 Subject: About FreeBSD syscall usage In-Reply-To: <20091022164144.064b916b.stas@deglitch.com> References: <95608CFE3D0C064B8468DB61F8403BE04A0D4A1C5E@PDSMSX501.ccr.corp.intel.com> <20091022164144.064b916b.stas@deglitch.com> Message-ID: <4AE096B6.2080701@elischer.org> Stanislav Sedov wrote: > On Thu, 22 Oct 2009 16:20:32 +0800 > "Yan, Yeqing" mentioned: > >> kse_exit >> >> kse_wakeup >> >> kse_create >> >> kse_thr_interrupt >> >> kse_release >> >> kse_switchin >> >> I read $man kse, but I can not find any example about how to use it. I write some test codes to call these function but all these codes are failed. >> The kse man page documents the syscalls but they are intended to be used only by the libkse library. there was some early test code in /usr/src/tools/KSE but it probably doesn't eve n compile any more. >> >> >> mac_syscall >> >> I read $man 3 mac, but I can not find the usage about mac_syscall function. >> >> >> >> thr_create >> >> thr_suspend >> >> thr_kill2 Once again, these calls are meant to be only accessed from the threading library. (though man pages should be written) >> > > I fear there's no documentation on these syscalls exists. So for use information > you'll have to refer to the actual source code of these syscalls/or and libc/libthr > source code which makes uses of them. > >> By the way, it is said ?I think that KSE was used in 5.x and 6.x and then dropped in favor of a 1:1 threading model when 7.0 was released? >> >> Does it mean the KSE syscall can be removed from FreeBSD 7.0? it will remain oin all 7.x kernels but is removed from 8.x >> > > libkse (M:N threading) was default threading library on FreeBSD versions prior to > 7.0, and the default has changed to libthr (1:1 threading in FreeBSD 7). libkse > was completely removed in FreeBSD 8, but it is still functional on FreeBSD 7.x. KSE based threading, while theoretically useful introduces a number of annoying complications to the kernel and was "holding up" other developement. Since Linux has gone with 1:1 threading, nearly all applications a re now written with 1:1 threading in mind so it made little sense to maintain all the extra complexity for no reason. > From anti_spamsys at yahoo.com Thu Oct 22 19:08:16 2009 From: anti_spamsys at yahoo.com (Trever) Date: Thu Oct 22 19:52:40 2009 Subject: bug in pkg_add ? doesn't fetch dependencies from set path Message-ID: <204698.71349.qm@web113208.mail.gq1.yahoo.com> Does anyone else have this problem? # env | grep PACKAGEPACKAGESITE=ftp://ftp.ourdomain.gov/FBSD/# pkg_add -r subversion-1.6.5Fetching ftp://ftp.ourdomain.gov/FBSD/subversion-1.6.5.tbz... Done.Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/pkg-config-0.23_1.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/sqlite3-3.6.14.2.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/gettext-0.17_1.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/neon28-0.28.4.tbz: File unavailable (e.g., file not found, no access) In plain English:pkg_add -r goes to the correct domain and path (per environment variable I set) to fetch the package I want to install, but when it goes to get the dependencies for the package it just correctly fetched, it subsequently fetches to the wrong path (goes to /All instead of /FBSD), though it does fetch to the correct domain. This is a pain because our ftp server has many uses, and having an "All" directory in the root is ugliness (whether All is a link to FBSD or whatever, I don't want "All", I just want "FBSD"). I have tried various combinations of setting both or one of PACKAGESITE and/or PACKAGEROOT, just in case that would somehow help, but to no avail. ?Of course it would seem that PACKAGESITE alone is what I want (but that and nothing else I have tried works). Thank you much. -T From gallatin at cs.duke.edu Thu Oct 22 20:08:20 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Thu Oct 22 20:08:27 2009 Subject: semaphores between processes Message-ID: <4AE0BBAB.3040807@cs.duke.edu> Hi, We're designing some software which has to lock access to shared memory pages between several processes, and has to run on Linux, Solaris, and FreeBSD. We were planning to have the lock be a pthread_mutex_t residing in the shared memory page. This works well on Linux and Solaris, but FreeBSD (at least 7-stable) does not support PTHREAD_PROCESS_SHARED mutexes. We then moved on to posix semaphores. Using sem_wait/sem_post with the sem_t residing in a shared page seems to work on all 3 platforms. However, the FreeBSD (7-stable) man page for sem_init(3) has this scary text regarding the pshared value: The sem_init() function initializes the unnamed semaphore pointed to by sem to have the value value. A non-zero value for pshared specifies a shared semaphore that can be used by multiple processes, which this implementation is not capable of. Is this text obsolete? Or is my test just "getting lucky"? Is there recommended way to do this? Thanks, Drew From glen.j.barber at gmail.com Thu Oct 22 20:30:29 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Thu Oct 22 20:30:35 2009 Subject: bug in pkg_add ? doesn't fetch dependencies from set path In-Reply-To: <204698.71349.qm@web113208.mail.gq1.yahoo.com> References: <204698.71349.qm@web113208.mail.gq1.yahoo.com> Message-ID: <4ad871310910221257s5533da63l3ed062d9641b36@mail.gmail.com> Hello, On Thu, Oct 22, 2009 at 2:41 PM, Trever wrote: > Does anyone else have this problem? > # env | grep PACKAGEPACKAGESITE=ftp://ftp.ourdomain.gov/FBSD/# pkg_add -r subversion-1.6.5Fetching ftp://ftp.ourdomain.gov/FBSD/subversion-1.6.5.tbz... Done.Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/pkg-config-0.23_1.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/sqlite3-3.6.14.2.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/gettext-0.17_1.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/neon28-0.28.4.tbz: File unavailable (e.g., file not found, no access) > > In plain English:pkg_add -r goes to the correct domain and path (per environment variable I set) to fetch the package I want to install, but when it goes to get the dependencies for the package it just correctly fetched, it subsequently fetches to the wrong path (goes to /All instead of /FBSD), though it does fetch to the correct domain. > This is a pain because our ftp server has many uses, and having an "All" directory in the root is ugliness (whether All is a link to FBSD or whatever, I don't want "All", I just want "FBSD"). > I have tried various combinations of setting both or one of PACKAGESITE and/or PACKAGEROOT, just in case that would somehow help, but to no avail. ?Of course it would seem that PACKAGESITE alone is what I want (but that and nothing else I have tried works). > Thank you much. > -T > pkg_add(1) expects the PACKAGESITE to follow the same hierarchy as a tinderbox. Without a tinderbox, you can 'mkdir /usr/ports/packages' on your local build server. When you 'make package' (or your preferred choice) the packages will be put in /usr/ports/packages with the correct hierarchy, symlinks to PACKAGESITE/All/ etc. HTH. -- Glen Barber From gallatin at cs.duke.edu Thu Oct 22 21:07:39 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Thu Oct 22 21:07:51 2009 Subject: semaphores between processes In-Reply-To: References: <4AE0BBAB.3040807@cs.duke.edu> Message-ID: <4AE0C995.5060303@cs.duke.edu> Daniel Eischen wrote: > On Thu, 22 Oct 2009, Andrew Gallatin wrote: > >> Hi, >> >> We're designing some software which has to lock access to >> shared memory pages between several processes, and has to >> run on Linux, Solaris, and FreeBSD. We were planning to >> have the lock be a pthread_mutex_t residing in the >> shared memory page. This works well on Linux and Solaris, >> but FreeBSD (at least 7-stable) does not support >> PTHREAD_PROCESS_SHARED mutexes. >> >> We then moved on to posix semaphores. Using sem_wait/sem_post >> with the sem_t residing in a shared page seems to work on >> all 3 platforms. However, the FreeBSD (7-stable) man page >> for sem_init(3) has this scary text regarding the pshared >> value: >> >> The sem_init() function initializes the unnamed semaphore pointed >> to by >> sem to have the value value. A non-zero value for pshared >> specifies a >> shared semaphore that can be used by multiple processes, which this >> implementation is not capable of. >> >> Is this text obsolete? Or is my test just "getting lucky"? > > I think you're getting lucky. Yes, after playing with the code some, I now see that. :( >> Is there recommended way to do this? > > I believe the only way to do this is with SYSV semaphores > (semop, semget, semctl). Unfortunately, these are not as > easy to use, IMHO. Yes, they are pretty ugly, and we were hoping to avoid them. Are there any plans to support either PTHREAD_PROCESS_SHARED mutexes, or pshared posix semaphores in FreeBSD? Thanks, Drew From deischen at freebsd.org Thu Oct 22 21:10:51 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Thu Oct 22 21:10:57 2009 Subject: semaphores between processes In-Reply-To: <4AE0BBAB.3040807@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> Message-ID: On Thu, 22 Oct 2009, Andrew Gallatin wrote: > Hi, > > We're designing some software which has to lock access to > shared memory pages between several processes, and has to > run on Linux, Solaris, and FreeBSD. We were planning to > have the lock be a pthread_mutex_t residing in the > shared memory page. This works well on Linux and Solaris, > but FreeBSD (at least 7-stable) does not support > PTHREAD_PROCESS_SHARED mutexes. > > We then moved on to posix semaphores. Using sem_wait/sem_post > with the sem_t residing in a shared page seems to work on > all 3 platforms. However, the FreeBSD (7-stable) man page > for sem_init(3) has this scary text regarding the pshared > value: > > The sem_init() function initializes the unnamed semaphore pointed to by > sem to have the value value. A non-zero value for pshared specifies a > shared semaphore that can be used by multiple processes, which this > implementation is not capable of. > > Is this text obsolete? Or is my test just "getting lucky"? I think you're getting lucky. > Is there recommended way to do this? I believe the only way to do this is with SYSV semaphores (semop, semget, semctl). Unfortunately, these are not as easy to use, IMHO. -- DE From deischen at freebsd.org Thu Oct 22 21:17:20 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Thu Oct 22 21:17:26 2009 Subject: semaphores between processes In-Reply-To: <4AE0C995.5060303@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> Message-ID: On Thu, 22 Oct 2009, Andrew Gallatin wrote: > Daniel Eischen wrote: >> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >> >>> Hi, >>> >>> We're designing some software which has to lock access to >>> shared memory pages between several processes, and has to >>> run on Linux, Solaris, and FreeBSD. We were planning to >>> have the lock be a pthread_mutex_t residing in the >>> shared memory page. This works well on Linux and Solaris, >>> but FreeBSD (at least 7-stable) does not support >>> PTHREAD_PROCESS_SHARED mutexes. >>> >>> We then moved on to posix semaphores. Using sem_wait/sem_post >>> with the sem_t residing in a shared page seems to work on >>> all 3 platforms. However, the FreeBSD (7-stable) man page >>> for sem_init(3) has this scary text regarding the pshared >>> value: >>> >>> The sem_init() function initializes the unnamed semaphore pointed to >>> by >>> sem to have the value value. A non-zero value for pshared specifies a >>> shared semaphore that can be used by multiple processes, which this >>> implementation is not capable of. >>> >>> Is this text obsolete? Or is my test just "getting lucky"? >> >> I think you're getting lucky. > > Yes, after playing with the code some, I now see that. :( > >>> Is there recommended way to do this? >> >> I believe the only way to do this is with SYSV semaphores >> (semop, semget, semctl). Unfortunately, these are not as >> easy to use, IMHO. > > Yes, they are pretty ugly, and we were hoping to avoid them. > Are there any plans to support either PTHREAD_PROCESS_SHARED > mutexes, or pshared posix semaphores in FreeBSD? It's planned, just not (yet) being actively worked on. It's a API change mostly, and then adding in all the compat hooks so we don't break ABI. -- DE From dudu at dudu.ro Thu Oct 22 21:32:36 2009 From: dudu at dudu.ro (Vlad Galu) Date: Thu Oct 22 21:32:43 2009 Subject: semaphores between processes In-Reply-To: <4AE0BBAB.3040807@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> Message-ID: On Thu, Oct 22, 2009 at 11:08 PM, Andrew Gallatin wrote: > Hi, > > We're designing some software which has to lock access to > shared memory pages between several processes, and has to > run on Linux, Solaris, and FreeBSD. ?We were planning to > have the lock be a pthread_mutex_t residing in the > shared memory page. ?This works well on Linux and Solaris, > but FreeBSD (at least 7-stable) does not support > PTHREAD_PROCESS_SHARED mutexes. > > We then moved on to posix semaphores. ?Using sem_wait/sem_post > with the sem_t residing in a shared page seems to work on > all 3 platforms. ?However, the FreeBSD (7-stable) man page > for sem_init(3) has this scary text regarding the pshared > value: > > ? ? The sem_init() function initializes the unnamed semaphore pointed to by > ? ? sem to have the value value. ?A non-zero value for pshared specifies a > ? ? shared semaphore that can be used by multiple processes, which this > ? ? implementation is not capable of. > > Is this text obsolete? ?Or is my test just "getting lucky"? > > Is there recommended way to do this? > > Thanks, > > Drew Hi Andrew, This works in Linux is because Linux defines sem_t as a struct type(or union, IIRC), while our sem_t is a pointer type (with more state kept in the pointed struct). SYSV semaphores seems the way to go... From yuri at rawbw.com Thu Oct 22 21:37:09 2009 From: yuri at rawbw.com (Yuri) Date: Thu Oct 22 21:37:16 2009 Subject: Failure to boot from HD formatted not by FreBSD Message-ID: <4AE0D084.7070108@rawbw.com> I wonder what are the limitations of MBR (/boot/boot0)? I have the OEM-formatted harddrive (160MB), on which I want to add FreeBSD into the free space. I created the FreeBSD partition, set MBR (in 8.0-RC2), but when it displays the boot prompt and I hit some key I only see '#' character printed and nothing happens. When I took MBR from my older FreeBSD HD and placed it on that one, it doesn't print '#' characters but also doesn't boot. When I boot into the healthy system with this new OEM HD attached I also see dmesg messages like this: GEOM: ad4: partition 2 does not start on a track boundary. What could be wrong, why it doesn't boot? Yuri --- partitions in MBR --- Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 7 (0x07),(NTFS, OS/2 HPFS, QNX-2 (16 bit) or Advanced UNIX) start 2079, size 44039457 (21503 Meg), flag 80 (active) beg: cyl 0/ head 33/ sector 1; end: cyl 1023/ head 116/ sector 63 The data for partition 2 is: sysid 15 (0x0f),(Extended DOS (LBA)) start 220416000, size 61222912 (29894 Meg), flag 0 beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 53/ sector 58 The data for partition 3 is: sysid 18 (0x12),(Compaq diagnostics) start 281638912, size 30942896 (15108 Meg), flag 0 beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 80/ sector 63 The data for partition 4 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 44041536, size 176373792 (86120 Meg), flag 0 beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 55/ sector 63 From jilles at stack.nl Thu Oct 22 21:47:35 2009 From: jilles at stack.nl (Jilles Tjoelker) Date: Thu Oct 22 21:47:42 2009 Subject: semaphores between processes In-Reply-To: <4AE0BBAB.3040807@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> Message-ID: <20091022214733.GA32745@stack.nl> On Thu, Oct 22, 2009 at 04:08:11PM -0400, Andrew Gallatin wrote: > We then moved on to posix semaphores. Using sem_wait/sem_post > with the sem_t residing in a shared page seems to work on > all 3 platforms. However, the FreeBSD (7-stable) man page > for sem_init(3) has this scary text regarding the pshared > value: > The sem_init() function initializes the unnamed semaphore pointed > to by > sem to have the value value. A non-zero value for pshared specifies a > shared semaphore that can be used by multiple processes, which this > implementation is not capable of. > Is this text obsolete? Or is my test just "getting lucky"? They work, but only if the processes are related and do not exec and the parent process initializes the semaphores before forking. This is because sem_t is a pointer to a malloc'ed structure. For process-shared semaphores this really only contains an identifier of the kernel semaphore. This also means process-shared semaphores are slower than in-process semaphores (libthr implements those using atomics and umtx so that system calls are only needed if a thread needs to sleep or be awakened). This is documented in comments in the source code, but not in man pages or other documentation. > Is there recommended way to do this? Apart from sysv semaphores, perhaps posix named semaphores (sem_open() etc). These require a 'kldload sem' on older versions though. -- Jilles Tjoelker From anti_spamsys at yahoo.com Thu Oct 22 20:41:01 2009 From: anti_spamsys at yahoo.com (Trever) Date: Thu Oct 22 22:57:18 2009 Subject: bug in pkg_add ? doesn't fetch dependencies from set path In-Reply-To: <4ad871310910221257s5533da63l3ed062d9641b36@mail.gmail.com> Message-ID: <821063.97828.qm@web113202.mail.gq1.yahoo.com> I realized that this was probably the case only after I sent the email, but this keeps me from having to test so thanks. I do think the documentation leads one to think differently about what PACKAGESITE does; perhaps I'll try to submit a clarification. ?As I read it, it talks about "subverting the logic" of the fetches, which led me to think the expected hierarchy would be subverted- something like that. ? -T --- On Thu, 10/22/09, Glen Barber wrote: From: Glen Barber Subject: Re: bug in pkg_add ? doesn't fetch dependencies from set path To: "Trever" Cc: freebsd-hackers@freebsd.org Date: Thursday, October 22, 2009, 12:57 PM Hello, On Thu, Oct 22, 2009 at 2:41 PM, Trever wrote: > Does anyone else have this problem? > # env | grep PACKAGEPACKAGESITE=ftp://ftp.ourdomain.gov/FBSD/# pkg_add -r subversion-1.6.5Fetching ftp://ftp.ourdomain.gov/FBSD/subversion-1.6.5.tbz... Done.Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/pkg-config-0.23_1.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/sqlite3-3.6.14.2.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/gettext-0.17_1.tbz: File unavailable (e.g., file not found, no access)Error: FTP Unable to get ftp://ftp.ourdomain.gov/All/neon28-0.28.4.tbz: File unavailable (e.g., file not found, no access) > > In plain English:pkg_add -r goes to the correct domain and path (per environment variable I set) to fetch the package I want to install, but when it goes to get the dependencies for the package it just correctly fetched, it subsequently fetches to the wrong path (goes to /All instead of /FBSD), though it does fetch to the correct domain. > This is a pain because our ftp server has many uses, and having an "All" directory in the root is ugliness (whether All is a link to FBSD or whatever, I don't want "All", I just want "FBSD"). > I have tried various combinations of setting both or one of PACKAGESITE and/or PACKAGEROOT, just in case that would somehow help, but to no avail. ?Of course it would seem that PACKAGESITE alone is what I want (but that and nothing else I have tried works). > Thank you much. > -T > pkg_add(1) expects the PACKAGESITE to follow the same hierarchy as a tinderbox.? Without a tinderbox, you can 'mkdir /usr/ports/packages' on your local build server.? When you 'make package' (or your preferred choice) the packages will be put in /usr/ports/packages with the correct hierarchy, symlinks to PACKAGESITE/All/ etc. HTH. -- Glen Barber From lankfordandrew at charter.net Fri Oct 23 01:12:30 2009 From: lankfordandrew at charter.net (Andrew Lankford) Date: Fri Oct 23 02:10:42 2009 Subject: Failure to boot from HD formatted not by FreeBSD Message-ID: <4AE0FEA5.9070004@charter.net> Looks to me like you're trying to get your computer to dual-boot Vista and FreeBSD 8.0, something I finally succeeded in doing. If by "MBR" you mean the first-stage boot program (512 bytes), I couldn't get that to work, nor could I use the standard boot0 menu from FreeBSD. I'm using the windows boot program instead. I think what I did was copy "/boot/boot1" from my root partition to my NTFS partition and then added an option to the Windows boot menu to boot with it. I get the GEOM "track boundary" complaint when I boot up as well. The FBSD 8.0 kernel has a new option 'GEOM_PART_MBR" on by default. Vista insisted on partitioning my drive, so if the new partition handler doesn't like it, it can lump it. In order to get the 8.0 kernel to recognise your old partitions, you need the "GEOM_MBR" option activated. That means you need to load "geom_mbr.ko" into memory before you load and boot from the 8.0 kernel. If you're booting from a FreeBSD 8.0 CD directly into sysinstall, you can escape to a shell and kldload geom_mbr.ko, but you have to then restart sysinstall without rebooting the computer in order for your hard disk partitions to show up. The only reliable way I could find to restart systinstall without rebooting was by pressing the power button. Wierd, eh? I added "option GEOM_MBR" back into my kernel, recompiled, fiddled with my network settings, and now everything seems to work alright. Anyway, all this procedure should be 75% correct since I've managed to successfully upgrade to 8.0 from 7-stable this way. For all I know, I might end up with a corrupted partition six months from now. Either that or Marcel Moolenar will get angry at me. Regards, Andrew Lankford From alexbestms at math.uni-muenster.de Fri Oct 23 02:35:10 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 02:35:16 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH Message-ID: hi everyone, together with hugh mahon (the author of ee) i've been trying to fix a nasty bug in ee. for some reason ee exits (not crashes) and leaves the console corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all running ee instances). unfortunately we were unable to find the problem. it seems to be related to ncurses. running the ee linux binary under freebsd causes no problem with SIGWINCH at all. since the linux binary doesn't need to be linked against ncurses (linux has termio.h/sgtty.h) we assume the problem is related to ncurses. right at the beginning of the ee code all signals get set to SIG_IGN: for (counter = 1; counter <= 32; counter++) signal(counter, SIG_IGN); so actually SIGWINCH shouldn't cause any problems since it gets discarded. looking through the src i'm quite sure that SIGWINCH stays set to SIG_IGN the whole time. yet running ee with truss shows this result when doing `grep WINCH`: sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0) = 0 (0x0) sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0) = 0 (0x0) sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0) = 0 (0x0) sigaction(SIGWINCH,{ SIG_IGN SA_RESTART ss_t },{ SIG_DFL 0x0 ss_t }) = 0 (0x0) sigaction(SIGWINCH,0x0,{ SIG_IGN SA_RESTART ss_t }) = 0 (0x0) sigaction(SIGWINCH,{ 0x280bc130 0x0 ss_t },0x0) = 0 (0x0) sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0) = 0 (0x0) sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0) = 0 (0x0) sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0) = 0 (0x0) so SIGWINCH doesn't stay set to SIG_IGN the whole time. it seems the problem is being caused by some ncurses function which gets called in contrib/ee/ee.c. contrib/ee/new_curse* aren't used since ee relies in freebsd's local ncurse implementation. it would be really great if this nasty bug could be fixed. you'll find a problem report here: http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/136223 please don't hesitate to ask for more details. cheers. alex From lists at mawer.org Fri Oct 23 04:02:51 2009 From: lists at mawer.org (Antony Mawer) Date: Fri Oct 23 04:03:00 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best wrote: > hi everyone, > > together with hugh mahon (the author of ee) i've been trying to fix a nasty > bug in ee. for some reason ee exits (not crashes) and leaves the console > corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all > running ee instances). I noticed this the other day when working on a new 8.0-RC1 system... in my case I was using putty (Windows ssh client) to access the system and maximised the window I had ee running in, and noticed ee just dumped me straight to the prompt. I am wondering if this has anything to do with the new tty subsystem in 8.0, as this wasn't a problem I've experienced before under 7.x... Maybe worth cc'ing ed@ to see if he has any thoughts? --Antony From nate at thatsmathematics.com Fri Oct 23 04:10:05 2009 From: nate at thatsmathematics.com (Nate Eldredge) Date: Fri Oct 23 04:10:12 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: On Fri, 23 Oct 2009, Antony Mawer wrote: > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > wrote: >> hi everyone, >> >> together with hugh mahon (the author of ee) i've been trying to fix a nasty >> bug in ee. for some reason ee exits (not crashes) and leaves the console >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all >> running ee instances). > > I noticed this the other day when working on a new 8.0-RC1 system... > in my case I was using putty (Windows ssh client) to access the system > and maximised the window I had ee running in, and noticed ee just > dumped me straight to the prompt. Seems a good start might be to compile ncurses with -g, link ee against it, put a breakpoint on the SIGWINCH handler, and start single stepping... -- Nate Eldredge nate@thatsmathematics.com From joel at FreeBSD.org Fri Oct 23 05:39:28 2009 From: joel at FreeBSD.org (Joel Dahl) Date: Fri Oct 23 05:39:35 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: <4AE13D5A.6040608@FreeBSD.org> Antony Mawer skrev: > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > wrote: >> hi everyone, >> >> together with hugh mahon (the author of ee) i've been trying to fix a nasty >> bug in ee. for some reason ee exits (not crashes) and leaves the console >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all >> running ee instances). > > I noticed this the other day when working on a new 8.0-RC1 system... > in my case I was using putty (Windows ssh client) to access the system > and maximised the window I had ee running in, and noticed ee just > dumped me straight to the prompt. > > I am wondering if this has anything to do with the new tty subsystem > in 8.0, as this wasn't a problem I've experienced before under 7.x... I've been using ee in Mac OSX for a while and I see the exact same behaviour, ee exits when I resize the terminal window. I haven't seen it in FreeBSD yet, but that is probably because I don't use Xorg on any of my FreeBSD systems... -- Joel From pluknet at gmail.com Fri Oct 23 09:22:08 2009 From: pluknet at gmail.com (pluknet) Date: Fri Oct 23 09:22:29 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: 2009/10/23 Antony Mawer : > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > wrote: >> hi everyone, >> >> together with hugh mahon (the author of ee) i've been trying to fix a nasty >> bug in ee. for some reason ee exits (not crashes) and leaves the console >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all >> running ee instances). > > I noticed this the other day when working on a new 8.0-RC1 system... > in my case I was using putty (Windows ssh client) to access the system > and maximised the window I had ee running in, and noticed ee just > dumped me straight to the prompt. > > I am wondering if this has anything to do with the new tty subsystem > in 8.0, as this wasn't a problem I've experienced before under 7.x... > No, that's a regression appeared in (FreeBSD'ish? version of) ee 1.5.0. -- wbr, pluknet From alexbestms at math.uni-muenster.de Fri Oct 23 10:36:38 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 10:36:44 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: Message-ID: Nate Eldredge schrieb am 2009-10-23: > On Fri, 23 Oct 2009, Antony Mawer wrote: > >On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > > wrote: > >>hi everyone, > >>together with hugh mahon (the author of ee) i've been trying to > >>fix a nasty > >>bug in ee. for some reason ee exits (not crashes) and leaves the > >>console > >>corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should > >>exit all > >>running ee instances). > >I noticed this the other day when working on a new 8.0-RC1 system... > >in my case I was using putty (Windows ssh client) to access the > >system > >and maximised the window I had ee running in, and noticed ee just > >dumped me straight to the prompt. > Seems a good start might be to compile ncurses with -g, link ee > against it, put a breakpoint on the SIGWINCH handler, and start > single stepping... > -- > Nate Eldredge > nate@thatsmathematics.com it seems ncurses registers a standard libc function with SIGWINCH. i started gdb, loaded ee and did "handle SIGWINCH stop" when i do "run" and issue a SIGWINCH to ee this is the output: Program received signal SIGWINCH, Window size changed. 0x281a4063 in read () from /lib/libc.so.7 (gdb) nexti 0x281a4048 in write () from /lib/libc.so.7 (gdb) nexti 0x281a4049 in write () from /lib/libc.so.7 (gdb) nexti 0x281a404e in write () from /lib/libc.so.7 (gdb) nexti 0x281a404f in write () from /lib/libc.so.7 (gdb) nexti 0x281a4055 in write () from /lib/libc.so.7 (gdb) nexti Program exited normally. so i guess ee calls some ncurses function right at the beginning. that ncurses function registers a new function to be called upon SIGWINCH (maybe exit(3)). cheers. alex From alexbestms at math.uni-muenster.de Fri Oct 23 10:42:29 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 10:42:36 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: Message-ID: pluknet schrieb am 2009-10-23: > 2009/10/23 Antony Mawer : > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > > wrote: > >> hi everyone, > >> together with hugh mahon (the author of ee) i've been trying to > >> fix a nasty > >> bug in ee. for some reason ee exits (not crashes) and leaves the > >> console > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should > >> exit all > >> running ee instances). > > I noticed this the other day when working on a new 8.0-RC1 > > system... > > in my case I was using putty (Windows ssh client) to access the > > system > > and maximised the window I had ee running in, and noticed ee just > > dumped me straight to the prompt. > > I am wondering if this has anything to do with the new tty > > subsystem > > in 8.0, as this wasn't a problem I've experienced before under > > 7.x... > No, that's a regression appeared in (FreeBSD'ish? version of) ee > 1.5.0. i'm not so sure this is entirely ee's fault. i just grabbed a really ancient version of ee from http://www.anerd.org/sources/ (ee-1.4.2.src.tgz last modified Jan 2001). yet i see the very same behaviour as with ee 1.5.0. right now it looks more like a tty or ncurses bug. cheers. alex From onemda at gmail.com Fri Oct 23 11:08:23 2009 From: onemda at gmail.com (Paul B Mahol) Date: Fri Oct 23 11:08:29 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: <3a142e750910230408q5aa60bcan1d641bd9e69d96af@mail.gmail.com> On 10/23/09, Alexander Best wrote: > Nate Eldredge schrieb am 2009-10-23: >> On Fri, 23 Oct 2009, Antony Mawer wrote: > >> >On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best >> > wrote: >> >>hi everyone, > >> >>together with hugh mahon (the author of ee) i've been trying to >> >>fix a nasty >> >>bug in ee. for some reason ee exits (not crashes) and leaves the >> >>console >> >>corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should >> >>exit all >> >>running ee instances). > >> >I noticed this the other day when working on a new 8.0-RC1 system... >> >in my case I was using putty (Windows ssh client) to access the >> >system >> >and maximised the window I had ee running in, and noticed ee just >> >dumped me straight to the prompt. > >> Seems a good start might be to compile ncurses with -g, link ee >> against it, put a breakpoint on the SIGWINCH handler, and start >> single stepping... > >> -- > >> Nate Eldredge >> nate@thatsmathematics.com > > it seems ncurses registers a standard libc function with SIGWINCH. i started > gdb, loaded ee and did "handle SIGWINCH stop" > > when i do "run" and issue a SIGWINCH to ee this is the output: > > Program received signal SIGWINCH, Window size changed. > 0x281a4063 in read () from /lib/libc.so.7 > > (gdb) nexti > 0x281a4048 in write () from /lib/libc.so.7 > (gdb) nexti > 0x281a4049 in write () from /lib/libc.so.7 > (gdb) nexti > 0x281a404e in write () from /lib/libc.so.7 > (gdb) nexti > 0x281a404f in write () from /lib/libc.so.7 > (gdb) nexti > 0x281a4055 in write () from /lib/libc.so.7 > (gdb) nexti > > Program exited normally. > > so i guess ee calls some ncurses function right at the beginning. that > ncurses > function registers a new function to be called upon SIGWINCH (maybe > exit(3)). Hmm, from my little experience with ncurses, there is no such function bind to SIGWINCH. Even if exit() is called from ncurses that's because ncurses can not handle current state in any reasonable way so exiting is only possible solution, but in such case something should be printed and program should not exit normally. If you start another shell command from ee, and let it be for example vim, once you resize xterm window, vim will still work but ee will start to consume CPU time. From gary.jennejohn at freenet.de Fri Oct 23 11:50:31 2009 From: gary.jennejohn at freenet.de (Gary Jennejohn) Date: Fri Oct 23 11:50:39 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: <20091023135024.377bcfa6@ernst.jennejohn.org> On Fri, 23 Oct 2009 12:58:43 +0400 pluknet wrote: > 2009/10/23 Antony Mawer : > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > > wrote: > >> hi everyone, > >> > >> together with hugh mahon (the author of ee) i've been trying to fix a nasty > >> bug in ee. for some reason ee exits (not crashes) and leaves the console > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all > >> running ee instances). > > > > I noticed this the other day when working on a new 8.0-RC1 system... > > in my case I was using putty (Windows ssh client) to access the system > > and maximised the window I had ee running in, and noticed ee just > > dumped me straight to the prompt. > > > > I am wondering if this has anything to do with the new tty subsystem > > in 8.0, as this wasn't a problem I've experienced before under 7.x... > > > > No, that's a regression appeared in (FreeBSD'ish? version of) ee 1.5.0. > SIGWINCH is handled in new_curse.c, but it's not being compiled/linked. --- Gary Jennejohn From gary.jennejohn at freenet.de Fri Oct 23 12:00:52 2009 From: gary.jennejohn at freenet.de (Gary Jennejohn) Date: Fri Oct 23 12:00:58 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <20091023135024.377bcfa6@ernst.jennejohn.org> References: <20091023135024.377bcfa6@ernst.jennejohn.org> Message-ID: <20091023140049.110d0562@ernst.jennejohn.org> On Fri, 23 Oct 2009 13:50:24 +0200 Gary Jennejohn wrote: > On Fri, 23 Oct 2009 12:58:43 +0400 > pluknet wrote: > > > 2009/10/23 Antony Mawer : > > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > > > wrote: > > >> hi everyone, > > >> > > >> together with hugh mahon (the author of ee) i've been trying to fix a nasty > > >> bug in ee. for some reason ee exits (not crashes) and leaves the console > > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should exit all > > >> running ee instances). > > > > > > I noticed this the other day when working on a new 8.0-RC1 system... > > > in my case I was using putty (Windows ssh client) to access the system > > > and maximised the window I had ee running in, and noticed ee just > > > dumped me straight to the prompt. > > > > > > I am wondering if this has anything to do with the new tty subsystem > > > in 8.0, as this wasn't a problem I've experienced before under 7.x... > > > > > > > No, that's a regression appeared in (FreeBSD'ish? version of) ee 1.5.0. > > > > SIGWINCH is handled in new_curse.c, but it's not being compiled/linked. > Never mind - I see that new_curse.c is supposed to be a substitute for ncurses if it's not available on the system. Sorry for the noise. --- Gary Jennejohn From des at des.no Fri Oct 23 12:02:40 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri Oct 23 12:02:47 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: (Alexander Best's message of "Fri, 23 Oct 2009 12:42:27 +0200 (CEST)") References: Message-ID: <86tyxqqpwh.fsf@ds4.des.no> [cc: ed for ee and rafan for ncurses] Alexander Best writes: > i'm not so sure this is entirely ee's fault. It is *partly* ee's fault. src/usr.bin/ee/ee.c in 7: in = wgetch(text_win); if (in == -1) continue; src/contrib/ee/ee.c in 8: in = wgetch(text_win); if (in == -1) exit(0); /* without this exit ee will go into an infinite loop if the network session detaches */ >From the wgetch() man page: Programmers concerned about portability should be prepared for either of two cases: (a) signal receipt does not interrupt getch; (b) signal receipt interrupts getch and causes it to return ERR with errno set to EINTR. Under the ncurses implementation, handled signals never inter? rupt getch. so ee is not portable (it should not assume that a "handled signal" such as SIGWINCH does not interrupt wgetch()), but that's not the real issue. The real issue, though, is that when a SIGWINCH is caught, wgetch() correctly returns KEY_RESIZE, but the next call to wgetch() returns -1. The next call after that is fine. I suspect the error lies somewhere inside kgetch() in contrib/ncurses/ncurses/base/lib_getch.c. DES -- Dag-Erling Sm?rgrav - des@des.no From alexbestms at math.uni-muenster.de Fri Oct 23 12:08:53 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 12:09:00 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <20091023135024.377bcfa6@ernst.jennejohn.org> Message-ID: Gary Jennejohn schrieb am 2009-10-23: > On Fri, 23 Oct 2009 12:58:43 +0400 > pluknet wrote: > > 2009/10/23 Antony Mawer : > > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > > > wrote: > > >> hi everyone, > > >> together with hugh mahon (the author of ee) i've been trying to > > >> fix a nasty > > >> bug in ee. for some reason ee exits (not crashes) and leaves the > > >> console > > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should > > >> exit all > > >> running ee instances). > > > I noticed this the other day when working on a new 8.0-RC1 > > > system... > > > in my case I was using putty (Windows ssh client) to access the > > > system > > > and maximised the window I had ee running in, and noticed ee just > > > dumped me straight to the prompt. > > > I am wondering if this has anything to do with the new tty > > > subsystem > > > in 8.0, as this wasn't a problem I've experienced before under > > > 7.x... > > No, that's a regression appeared in (FreeBSD'ish? version of) ee > > 1.5.0. > SIGWINCH is handled in new_curse.c, but it's not being > compiled/linked. > --- > Gary Jennejohn i think that file is only used on systems which have termio.h/sgtty.h and ee doesn't get linked against ncurses. on those systems (linux e.g.) new_curse.c is used to handle certain things which ncurses takes care under freebsd. this is under freebsd: `make`: Neither termio.h or sgtty.h are on this system! Relying on local curses implementation. Generating make.local make -f make.local cc ee.c -o ee -ggdb -DDIAG -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB -DHAS_CTYPE -DHAS_SYS_IOCTL -DHAS_SYS_WAIT -DSLCT_HDR -DTERMCAP="\"/usr/share/misc/termcap\"" -lcursesw `ldd ee`: libncursesw.so.8 => /lib/libncursesw.so.8 (0x2809b000) libc.so.7 => /lib/libc.so.7 (0x280e9000) and under linux: `make`: Generating make.local make -f make.local make[1]: Entering directory `/easyedit-test' cc -c ee.c -DSYS5 -DBSD_SELECT -DNCURSE -DHAS_UNISTD -DHAS_STDLIB -DHAS_CTYPE -DHAS_SYS_IOCTL -DHAS_SYS_WAIT -DSLCT_HDR -s cc new_curse.c -c -DSYS5 -DBSD_SELECT -DNCURSE -DHAS_UNISTD -DHAS_STDLIB -DHAS_CTYPE -DHAS_SYS_IOCTL -DHAS_SYS_WAIT -DSLCT_HDR -s cc -o ee ee.o new_curse.o -DHAS_UNISTD -DHAS_STDLIB -DHAS_CTYPE -DHAS_SYS_IOCTL -DHAS_SYS_WAIT -DSLCT_HDR -s make[1]: Leaving directory `/easyedit-test' `ldd ee`: libc.so.6 => /lib/libc.so.6 (0x2807a000) /lib/ld-linux.so.2 (0x2805b000) alex From alexbestms at math.uni-muenster.de Fri Oct 23 12:16:52 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 12:16:59 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <20091023135024.377bcfa6@ernst.jennejohn.org> Message-ID: Gary Jennejohn schrieb am 2009-10-23: > On Fri, 23 Oct 2009 12:58:43 +0400 > pluknet wrote: > > 2009/10/23 Antony Mawer : > > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > > > wrote: > > >> hi everyone, > > >> together with hugh mahon (the author of ee) i've been trying to > > >> fix a nasty > > >> bug in ee. for some reason ee exits (not crashes) and leaves the > > >> console > > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should > > >> exit all > > >> running ee instances). > > > I noticed this the other day when working on a new 8.0-RC1 > > > system... > > > in my case I was using putty (Windows ssh client) to access the > > > system > > > and maximised the window I had ee running in, and noticed ee just > > > dumped me straight to the prompt. > > > I am wondering if this has anything to do with the new tty > > > subsystem > > > in 8.0, as this wasn't a problem I've experienced before under > > > 7.x... > > No, that's a regression appeared in (FreeBSD'ish? version of) ee > > 1.5.0. > SIGWINCH is handled in new_curse.c, but it's not being > compiled/linked. > --- > Gary Jennejohn i just tried building ee under linux without using new_curse.c and linking the executable against ncurses. running the binary is showing the same problems with SIGWINCH. so if this is in fact caused by a ncurses bug the bug appears in linux ncurses too. alex From pluknet at gmail.com Fri Oct 23 12:34:03 2009 From: pluknet at gmail.com (pluknet) Date: Fri Oct 23 12:34:09 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: <20091023135024.377bcfa6@ernst.jennejohn.org> Message-ID: 2009/10/23 Alexander Best : > Gary Jennejohn schrieb am 2009-10-23: >> On Fri, 23 Oct 2009 12:58:43 +0400 >> pluknet wrote: > >> > 2009/10/23 Antony Mawer : >> > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best >> > > wrote: >> > >> hi everyone, > >> > >> together with hugh mahon (the author of ee) i've been trying to >> > >> fix a nasty >> > >> bug in ee. for some reason ee exits (not crashes) and leaves the >> > >> console >> > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` should >> > >> exit all >> > >> running ee instances). > >> > > I noticed this the other day when working on a new 8.0-RC1 >> > > system... >> > > in my case I was using putty (Windows ssh client) to access the >> > > system >> > > and maximised the window I had ee running in, and noticed ee just >> > > dumped me straight to the prompt. > >> > > I am wondering if this has anything to do with the new tty >> > > subsystem >> > > in 8.0, as this wasn't a problem I've experienced before under >> > > 7.x... > > >> > No, that's a regression appeared in (FreeBSD'ish? version of) ee >> > 1.5.0. > > >> SIGWINCH is handled in new_curse.c, but it's not being >> compiled/linked. > >> --- >> Gary Jennejohn > > i think that file is only used on systems which have termio.h/sgtty.h and ee > doesn't get linked against ncurses. on those systems (linux e.g.) new_curse.c > is used to handle certain things which ncurses takes care under freebsd. > > this is under freebsd: > > `make`: > Neither termio.h or sgtty.h are on this system! > Relying on local curses implementation. > Generating make.local > make -f make.local > cc ee.c -o ee -ggdb -DDIAG -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB -DHAS_CTYPE > -DHAS_SYS_IOCTL -DHAS_SYS_WAIT ? -DSLCT_HDR > -DTERMCAP="\"/usr/share/misc/termcap\"" -lcursesw > > `ldd ee`: > libncursesw.so.8 => /lib/libncursesw.so.8 (0x2809b000) > libc.so.7 => /lib/libc.so.7 (0x280e9000) > [Probably already mentioned.] btw, ee compiled under fbsd with new_curse.c (and not linked with curses/cursesw) goes fine with SIGWINCH. --- Makefile.old 2009-10-23 16:13:45.000000000 +0400 +++ Makefile 2009-10-23 16:30:03.000000000 +0400 @@ -3,15 +3,14 @@ .PATH: ${.CURDIR}/../../contrib/ee CFLAGS+= -DHAS_NCURSES -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB \ - -DHAS_SYS_WAIT + -DHAS_SYS_WAIT -DCAP -DNCURSE PROG= ee +SRCS= ee.c new_curse.c LINKS= ${BINDIR}/ee ${BINDIR}/ree ${BINDIR}/ee ${BINDIR}/edit MLINKS= ee.1 ree.1 ee.1 edit.1 -DPADD= ${LIBNCURSES} -LDADD= -lncurses -WARNS?= 2 +WARNS?= 0 NLS= en_US.US-ASCII fr_FR.ISO8859-1 de_DE.ISO8859-1 pl_PL.ISO8859-2 \ uk_UA.KOI8-U ru_RU.KOI8-R hu_HU.ISO8859-2 $ ldd ./ee ./ee: libc.so.6 => /lib/libc.so.6 (0x2808a000) (yes, it's FreeBSD 6.x). -- wbr, pluknet From alexbestms at math.uni-muenster.de Fri Oct 23 13:26:21 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 13:26:28 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: Message-ID: pluknet schrieb am 2009-10-23: > 2009/10/23 Alexander Best : > > Gary Jennejohn schrieb am 2009-10-23: > >> On Fri, 23 Oct 2009 12:58:43 +0400 > >> pluknet wrote: > >> > 2009/10/23 Antony Mawer : > >> > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best > >> > > wrote: > >> > >> hi everyone, > >> > >> together with hugh mahon (the author of ee) i've been trying > >> > >> to > >> > >> fix a nasty > >> > >> bug in ee. for some reason ee exits (not crashes) and leaves > >> > >> the > >> > >> console > >> > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` > >> > >> should > >> > >> exit all > >> > >> running ee instances). > >> > > I noticed this the other day when working on a new 8.0-RC1 > >> > > system... > >> > > in my case I was using putty (Windows ssh client) to access > >> > > the > >> > > system > >> > > and maximised the window I had ee running in, and noticed ee > >> > > just > >> > > dumped me straight to the prompt. > >> > > I am wondering if this has anything to do with the new tty > >> > > subsystem > >> > > in 8.0, as this wasn't a problem I've experienced before under > >> > > 7.x... > >> > No, that's a regression appeared in (FreeBSD'ish? version of) ee > >> > 1.5.0. > >> SIGWINCH is handled in new_curse.c, but it's not being > >> compiled/linked. > >> --- > >> Gary Jennejohn > > i think that file is only used on systems which have > > termio.h/sgtty.h and ee > > doesn't get linked against ncurses. on those systems (linux e.g.) > > new_curse.c > > is used to handle certain things which ncurses takes care under > > freebsd. > > this is under freebsd: > > `make`: > > Neither termio.h or sgtty.h are on this system! > > Relying on local curses implementation. > > Generating make.local > > make -f make.local > > cc ee.c -o ee -ggdb -DDIAG -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB > > -DHAS_CTYPE > > -DHAS_SYS_IOCTL -DHAS_SYS_WAIT ? -DSLCT_HDR > > -DTERMCAP="\"/usr/share/misc/termcap\"" -lcursesw > > `ldd ee`: > > libncursesw.so.8 => /lib/libncursesw.so.8 (0x2809b000) > > libc.so.7 => /lib/libc.so.7 (0x280e9000) > [Probably already mentioned.] > btw, ee compiled under fbsd with new_curse.c > (and not linked with curses/cursesw) goes fine with SIGWINCH. > --- Makefile.old 2009-10-23 16:13:45.000000000 +0400 > +++ Makefile 2009-10-23 16:30:03.000000000 +0400 > @@ -3,15 +3,14 @@ > .PATH: ${.CURDIR}/../../contrib/ee > CFLAGS+= -DHAS_NCURSES -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB \ > - -DHAS_SYS_WAIT > + -DHAS_SYS_WAIT -DCAP -DNCURSE > PROG= ee > +SRCS= ee.c new_curse.c > LINKS= ${BINDIR}/ee ${BINDIR}/ree ${BINDIR}/ee ${BINDIR}/edit > MLINKS= ee.1 ree.1 ee.1 edit.1 > -DPADD= ${LIBNCURSES} > -LDADD= -lncurses > -WARNS?= 2 > +WARNS?= 0 > NLS= en_US.US-ASCII fr_FR.ISO8859-1 de_DE.ISO8859-1 > pl_PL.ISO8859-2 \ > uk_UA.KOI8-U ru_RU.KOI8-R hu_HU.ISO8859-2 > $ ldd ./ee > ./ee: > libc.so.6 => /lib/libc.so.6 (0x2808a000) > (yes, it's FreeBSD 6.x). won't work under CURRENT i'm afraid: Warning: Object directory not changed from original /usr/src/usr.bin/ee cc -O2 -fno-strict-aliasing -pipe -march=native -DHAS_NCURSES -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB -DHAS_SYS_WAIT -DCAP -DNCURSE -std=gnu99 -fstack-protector -Wno-pointer-sign -c /usr/src/usr.bin/ee/../../contrib/ee/ee.c In file included from /usr/src/usr.bin/ee/../../contrib/ee/ee.c:68: /usr/src/usr.bin/ee/../../contrib/ee/new_curse.h:47:19: error: sgtty.h: No such file or directory *** Error code 1 Stop in /usr/src/usr.bin/ee. here's the entry in ObsoleteFiles.inc: # 20080725: sgtty.h removed OLD_FILES+=usr/include/sgtty.h alex From jhb at freebsd.org Fri Oct 23 13:26:29 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Oct 23 13:26:36 2009 Subject: semaphores between processes In-Reply-To: References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> Message-ID: <200910230802.49873.jhb@freebsd.org> On Thursday 22 October 2009 5:17:07 pm Daniel Eischen wrote: > On Thu, 22 Oct 2009, Andrew Gallatin wrote: > > > Daniel Eischen wrote: > >> On Thu, 22 Oct 2009, Andrew Gallatin wrote: > >> > >>> Hi, > >>> > >>> We're designing some software which has to lock access to > >>> shared memory pages between several processes, and has to > >>> run on Linux, Solaris, and FreeBSD. We were planning to > >>> have the lock be a pthread_mutex_t residing in the > >>> shared memory page. This works well on Linux and Solaris, > >>> but FreeBSD (at least 7-stable) does not support > >>> PTHREAD_PROCESS_SHARED mutexes. > >>> > >>> We then moved on to posix semaphores. Using sem_wait/sem_post > >>> with the sem_t residing in a shared page seems to work on > >>> all 3 platforms. However, the FreeBSD (7-stable) man page > >>> for sem_init(3) has this scary text regarding the pshared > >>> value: > >>> > >>> The sem_init() function initializes the unnamed semaphore pointed to > >>> by > >>> sem to have the value value. A non-zero value for pshared specifies a > >>> shared semaphore that can be used by multiple processes, which this > >>> implementation is not capable of. > >>> > >>> Is this text obsolete? Or is my test just "getting lucky"? > >> > >> I think you're getting lucky. > > > > Yes, after playing with the code some, I now see that. :( > > > >>> Is there recommended way to do this? > >> > >> I believe the only way to do this is with SYSV semaphores > >> (semop, semget, semctl). Unfortunately, these are not as > >> easy to use, IMHO. > > > > Yes, they are pretty ugly, and we were hoping to avoid them. > > Are there any plans to support either PTHREAD_PROCESS_SHARED > > mutexes, or pshared posix semaphores in FreeBSD? > > It's planned, just not (yet) being actively worked on. > It's a API change mostly, and then adding in all the > compat hooks so we don't break ABI. There are also an alternate set of patches on threads@ to allow just shared semaphores I think w/o the changes to the pthread types. I can't recall exactly what they did, but I think rrs@ was playing with using umtx directly to implement some sort of process-shared primitive. -- John Baldwin From pluknet at gmail.com Fri Oct 23 13:32:35 2009 From: pluknet at gmail.com (pluknet) Date: Fri Oct 23 13:32:41 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: 2009/10/23 Alexander Best : > pluknet schrieb am 2009-10-23: >> 2009/10/23 Alexander Best : >> > Gary Jennejohn schrieb am 2009-10-23: >> >> On Fri, 23 Oct 2009 12:58:43 +0400 >> >> pluknet wrote: > >> >> > 2009/10/23 Antony Mawer : >> >> > > On Fri, Oct 23, 2009 at 1:35 PM, Alexander Best >> >> > > wrote: >> >> > >> hi everyone, > >> >> > >> together with hugh mahon (the author of ee) i've been trying >> >> > >> to >> >> > >> fix a nasty >> >> > >> bug in ee. for some reason ee exits (not crashes) and leaves >> >> > >> the >> >> > >> console >> >> > >> corrupted when receiving SIGWINCH (`killall -SIGWINCH ee` >> >> > >> should >> >> > >> exit all >> >> > >> running ee instances). > >> >> > > I noticed this the other day when working on a new 8.0-RC1 >> >> > > system... >> >> > > in my case I was using putty (Windows ssh client) to access >> >> > > the >> >> > > system >> >> > > and maximised the window I had ee running in, and noticed ee >> >> > > just >> >> > > dumped me straight to the prompt. > >> >> > > I am wondering if this has anything to do with the new tty >> >> > > subsystem >> >> > > in 8.0, as this wasn't a problem I've experienced before under >> >> > > 7.x... > > >> >> > No, that's a regression appeared in (FreeBSD'ish? version of) ee >> >> > 1.5.0. > > >> >> SIGWINCH is handled in new_curse.c, but it's not being >> >> compiled/linked. > >> >> --- >> >> Gary Jennejohn > >> > i think that file is only used on systems which have >> > termio.h/sgtty.h and ee >> > doesn't get linked against ncurses. on those systems (linux e.g.) >> > new_curse.c >> > is used to handle certain things which ncurses takes care under >> > freebsd. > >> > this is under freebsd: > >> > `make`: >> > Neither termio.h or sgtty.h are on this system! >> > Relying on local curses implementation. >> > Generating make.local >> > make -f make.local >> > cc ee.c -o ee -ggdb -DDIAG -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB >> > -DHAS_CTYPE >> > -DHAS_SYS_IOCTL -DHAS_SYS_WAIT ? -DSLCT_HDR >> > -DTERMCAP="\"/usr/share/misc/termcap\"" -lcursesw > >> > `ldd ee`: >> > libncursesw.so.8 => /lib/libncursesw.so.8 (0x2809b000) >> > libc.so.7 => /lib/libc.so.7 (0x280e9000) > > >> [Probably already mentioned.] > >> btw, ee compiled under fbsd with new_curse.c >> (and not linked with curses/cursesw) goes fine with SIGWINCH. > >> --- Makefile.old ? ? ? ?2009-10-23 16:13:45.000000000 +0400 >> +++ Makefile ? ?2009-10-23 16:30:03.000000000 +0400 >> @@ -3,15 +3,14 @@ >> ?.PATH: ${.CURDIR}/../../contrib/ee > >> ?CFLAGS+= -DHAS_NCURSES -DHAS_UNISTD -DHAS_STDARG -DHAS_STDLIB \ >> - ? ? ? ?-DHAS_SYS_WAIT >> + ? ? ? ?-DHAS_SYS_WAIT -DCAP -DNCURSE > >> ?PROG= ?ee >> +SRCS= ?ee.c new_curse.c >> ?LINKS= ${BINDIR}/ee ${BINDIR}/ree ${BINDIR}/ee ${BINDIR}/edit >> ?MLINKS= ? ? ? ?ee.1 ree.1 ee.1 edit.1 >> -DPADD= ${LIBNCURSES} >> -LDADD= -lncurses > >> -WARNS?= ? ? ? ?2 >> +WARNS?= ? ? ? ?0 > >> ?NLS= ? en_US.US-ASCII fr_FR.ISO8859-1 de_DE.ISO8859-1 >> ?pl_PL.ISO8859-2 \ >> ? ? ? ? uk_UA.KOI8-U ru_RU.KOI8-R hu_HU.ISO8859-2 > >> $ ldd ./ee >> ./ee: >> ? ? ? ? libc.so.6 => /lib/libc.so.6 (0x2808a000) > >> (yes, it's FreeBSD 6.x). > > won't work under CURRENT i'm afraid: > > Warning: Object directory not changed from original /usr/src/usr.bin/ee > cc ?-O2 -fno-strict-aliasing -pipe -march=native -DHAS_NCURSES -DHAS_UNISTD > -DHAS_STDARG -DHAS_STDLIB ?-DHAS_SYS_WAIT -DCAP -DNCURSE -std=gnu99 > -fstack-protector -Wno-pointer-sign -c > /usr/src/usr.bin/ee/../../contrib/ee/ee.c > In file included from /usr/src/usr.bin/ee/../../contrib/ee/ee.c:68: > /usr/src/usr.bin/ee/../../contrib/ee/new_curse.h:47:19: error: sgtty.h: No > such file or directory > *** Error code 1 > > Stop in /usr/src/usr.bin/ee. > > here's the entry in ObsoleteFiles.inc: > > # 20080725: sgtty.h removed > OLD_FILES+=usr/include/sgtty.h > > alex > Ah, OK. Just a stupid and not-tested guess. -- wbr, pluknet From deischen at freebsd.org Fri Oct 23 14:56:11 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Fri Oct 23 14:56:18 2009 Subject: semaphores between processes In-Reply-To: <200910230802.49873.jhb@freebsd.org> References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> Message-ID: On Fri, 23 Oct 2009, John Baldwin wrote: > On Thursday 22 October 2009 5:17:07 pm Daniel Eischen wrote: >> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >> >>> Daniel Eischen wrote: >>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>>> >>>>> Hi, >>>>> >>>>> We're designing some software which has to lock access to >>>>> shared memory pages between several processes, and has to >>>>> run on Linux, Solaris, and FreeBSD. We were planning to >>>>> have the lock be a pthread_mutex_t residing in the >>>>> shared memory page. This works well on Linux and Solaris, >>>>> but FreeBSD (at least 7-stable) does not support >>>>> PTHREAD_PROCESS_SHARED mutexes. >>>>> >>>>> We then moved on to posix semaphores. Using sem_wait/sem_post >>>>> with the sem_t residing in a shared page seems to work on >>>>> all 3 platforms. However, the FreeBSD (7-stable) man page >>>>> for sem_init(3) has this scary text regarding the pshared >>>>> value: >>>>> >>>>> The sem_init() function initializes the unnamed semaphore pointed to >>>>> by >>>>> sem to have the value value. A non-zero value for pshared specifies > a >>>>> shared semaphore that can be used by multiple processes, which this >>>>> implementation is not capable of. >>>>> >>>>> Is this text obsolete? Or is my test just "getting lucky"? >>>> >>>> I think you're getting lucky. >>> >>> Yes, after playing with the code some, I now see that. :( >>> >>>>> Is there recommended way to do this? >>>> >>>> I believe the only way to do this is with SYSV semaphores >>>> (semop, semget, semctl). Unfortunately, these are not as >>>> easy to use, IMHO. >>> >>> Yes, they are pretty ugly, and we were hoping to avoid them. >>> Are there any plans to support either PTHREAD_PROCESS_SHARED >>> mutexes, or pshared posix semaphores in FreeBSD? >> >> It's planned, just not (yet) being actively worked on. >> It's a API change mostly, and then adding in all the >> compat hooks so we don't break ABI. > > There are also an alternate set of patches on threads@ to allow just shared > semaphores I think w/o the changes to the pthread types. I can't recall > exactly what they did, but I think rrs@ was playing with using umtx directly > to implement some sort of process-shared primitive. That's really not the way to go. The structs really need to become public. -- DE From gallatin at cs.duke.edu Fri Oct 23 15:39:38 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Fri Oct 23 15:39:45 2009 Subject: semaphores between processes In-Reply-To: References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> Message-ID: <4AE1CE31.1090206@cs.duke.edu> Daniel Eischen wrote: > On Fri, 23 Oct 2009, John Baldwin wrote: > >> On Thursday 22 October 2009 5:17:07 pm Daniel Eischen wrote: >>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>> >>>> Daniel Eischen wrote: >>>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> We're designing some software which has to lock access to >>>>>> shared memory pages between several processes, and has to >>>>>> run on Linux, Solaris, and FreeBSD. We were planning to >>>>>> have the lock be a pthread_mutex_t residing in the >>>>>> shared memory page. This works well on Linux and Solaris, >>>>>> but FreeBSD (at least 7-stable) does not support >>>>>> PTHREAD_PROCESS_SHARED mutexes. >>>>>> >>>>>> We then moved on to posix semaphores. Using sem_wait/sem_post >>>>>> with the sem_t residing in a shared page seems to work on >>>>>> all 3 platforms. However, the FreeBSD (7-stable) man page >>>>>> for sem_init(3) has this scary text regarding the pshared >>>>>> value: >>>>>> >>>>>> The sem_init() function initializes the unnamed semaphore >>>>>> pointed to >>>>>> by >>>>>> sem to have the value value. A non-zero value for pshared >>>>>> specifies >> a >>>>>> shared semaphore that can be used by multiple processes, which >>>>>> this >>>>>> implementation is not capable of. >>>>>> >>>>>> Is this text obsolete? Or is my test just "getting lucky"? >>>>> >>>>> I think you're getting lucky. >>>> >>>> Yes, after playing with the code some, I now see that. :( >>>> >>>>>> Is there recommended way to do this? >>>>> >>>>> I believe the only way to do this is with SYSV semaphores >>>>> (semop, semget, semctl). Unfortunately, these are not as >>>>> easy to use, IMHO. >>>> >>>> Yes, they are pretty ugly, and we were hoping to avoid them. >>>> Are there any plans to support either PTHREAD_PROCESS_SHARED >>>> mutexes, or pshared posix semaphores in FreeBSD? >>> >>> It's planned, just not (yet) being actively worked on. >>> It's a API change mostly, and then adding in all the >>> compat hooks so we don't break ABI. >> >> There are also an alternate set of patches on threads@ to allow just >> shared >> semaphores I think w/o the changes to the pthread types. I can't recall >> exactly what they did, but I think rrs@ was playing with using umtx >> directly >> to implement some sort of process-shared primitive. > > That's really not the way to go. The structs really need > to become public. > It would be great if they were, but that discussion was 6 months ago, and nothing seems to have happened. Plus we need to support at least 7.X and probably 6, so any changes here might not even help us. What is wrong with just using umtx directly? It seems to do exactly what we need. Thanks, Drew From jhb at freebsd.org Fri Oct 23 15:45:53 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Oct 23 15:45:59 2009 Subject: semaphores between processes In-Reply-To: References: <4AE0BBAB.3040807@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> Message-ID: <200910231136.21837.jhb@freebsd.org> On Friday 23 October 2009 10:56:06 am Daniel Eischen wrote: > On Fri, 23 Oct 2009, John Baldwin wrote: > > > On Thursday 22 October 2009 5:17:07 pm Daniel Eischen wrote: > >> On Thu, 22 Oct 2009, Andrew Gallatin wrote: > >> > >>> Daniel Eischen wrote: > >>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> We're designing some software which has to lock access to > >>>>> shared memory pages between several processes, and has to > >>>>> run on Linux, Solaris, and FreeBSD. We were planning to > >>>>> have the lock be a pthread_mutex_t residing in the > >>>>> shared memory page. This works well on Linux and Solaris, > >>>>> but FreeBSD (at least 7-stable) does not support > >>>>> PTHREAD_PROCESS_SHARED mutexes. > >>>>> > >>>>> We then moved on to posix semaphores. Using sem_wait/sem_post > >>>>> with the sem_t residing in a shared page seems to work on > >>>>> all 3 platforms. However, the FreeBSD (7-stable) man page > >>>>> for sem_init(3) has this scary text regarding the pshared > >>>>> value: > >>>>> > >>>>> The sem_init() function initializes the unnamed semaphore pointed to > >>>>> by > >>>>> sem to have the value value. A non-zero value for pshared specifies > > a > >>>>> shared semaphore that can be used by multiple processes, which this > >>>>> implementation is not capable of. > >>>>> > >>>>> Is this text obsolete? Or is my test just "getting lucky"? > >>>> > >>>> I think you're getting lucky. > >>> > >>> Yes, after playing with the code some, I now see that. :( > >>> > >>>>> Is there recommended way to do this? > >>>> > >>>> I believe the only way to do this is with SYSV semaphores > >>>> (semop, semget, semctl). Unfortunately, these are not as > >>>> easy to use, IMHO. > >>> > >>> Yes, they are pretty ugly, and we were hoping to avoid them. > >>> Are there any plans to support either PTHREAD_PROCESS_SHARED > >>> mutexes, or pshared posix semaphores in FreeBSD? > >> > >> It's planned, just not (yet) being actively worked on. > >> It's a API change mostly, and then adding in all the > >> compat hooks so we don't break ABI. > > > > There are also an alternate set of patches on threads@ to allow just shared > > semaphores I think w/o the changes to the pthread types. I can't recall > > exactly what they did, but I think rrs@ was playing with using umtx directly > > to implement some sort of process-shared primitive. > > That's really not the way to go. The structs really need > to become public. I was mostly suggesting it as a way to use something sooner since I expect it will be a long while before anyone does the pthreads work. -- John Baldwin From deischen at freebsd.org Fri Oct 23 15:47:40 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Fri Oct 23 15:47:50 2009 Subject: semaphores between processes In-Reply-To: <4AE1CE31.1090206@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> <4AE1CE31.1090206@cs.duke.edu> Message-ID: On Fri, 23 Oct 2009, Andrew Gallatin wrote: > Daniel Eischen wrote: >> On Fri, 23 Oct 2009, John Baldwin wrote: >> >>> On Thursday 22 October 2009 5:17:07 pm Daniel Eischen wrote: >>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>>> >>>>> Daniel Eischen wrote: >>>>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> We're designing some software which has to lock access to >>>>>>> shared memory pages between several processes, and has to >>>>>>> run on Linux, Solaris, and FreeBSD. We were planning to >>>>>>> have the lock be a pthread_mutex_t residing in the >>>>>>> shared memory page. This works well on Linux and Solaris, >>>>>>> but FreeBSD (at least 7-stable) does not support >>>>>>> PTHREAD_PROCESS_SHARED mutexes. >>>>>>> >>>>>>> We then moved on to posix semaphores. Using sem_wait/sem_post >>>>>>> with the sem_t residing in a shared page seems to work on >>>>>>> all 3 platforms. However, the FreeBSD (7-stable) man page >>>>>>> for sem_init(3) has this scary text regarding the pshared >>>>>>> value: >>>>>>> >>>>>>> The sem_init() function initializes the unnamed semaphore pointed >>>>>>> to >>>>>>> by >>>>>>> sem to have the value value. A non-zero value for pshared >>>>>>> specifies >>> a >>>>>>> shared semaphore that can be used by multiple processes, which >>>>>>> this >>>>>>> implementation is not capable of. >>>>>>> >>>>>>> Is this text obsolete? Or is my test just "getting lucky"? >>>>>> >>>>>> I think you're getting lucky. >>>>> >>>>> Yes, after playing with the code some, I now see that. :( >>>>> >>>>>>> Is there recommended way to do this? >>>>>> >>>>>> I believe the only way to do this is with SYSV semaphores >>>>>> (semop, semget, semctl). Unfortunately, these are not as >>>>>> easy to use, IMHO. >>>>> >>>>> Yes, they are pretty ugly, and we were hoping to avoid them. >>>>> Are there any plans to support either PTHREAD_PROCESS_SHARED >>>>> mutexes, or pshared posix semaphores in FreeBSD? >>>> >>>> It's planned, just not (yet) being actively worked on. >>>> It's a API change mostly, and then adding in all the >>>> compat hooks so we don't break ABI. >>> >>> There are also an alternate set of patches on threads@ to allow just >>> shared >>> semaphores I think w/o the changes to the pthread types. I can't recall >>> exactly what they did, but I think rrs@ was playing with using umtx >>> directly >>> to implement some sort of process-shared primitive. >> >> That's really not the way to go. The structs really need >> to become public. >> > > It would be great if they were, but that discussion was 6 months > ago, and nothing seems to have happened. Plus we need to support > at least 7.X and probably 6, so any changes here might not even > help us. > > What is wrong with just using umtx directly? It seems to do > exactly what we need. Because you can't do anything more than use umtx directly, like check for mutex types and return appropriate error codes. Just look at other implementations - Solaris, Linux, all have their pthread_*_t as public structs. -- DE From gallatin at cs.duke.edu Fri Oct 23 15:55:07 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Fri Oct 23 15:55:14 2009 Subject: semaphores between processes In-Reply-To: References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> <4AE1CE31.1090206@cs.duke.edu> Message-ID: <4AE1D1D2.1090307@cs.duke.edu> Daniel Eischen wrote: > On Fri, 23 Oct 2009, Andrew Gallatin wrote: > >> Daniel Eischen wrote: >>> On Fri, 23 Oct 2009, John Baldwin wrote: >>> >>>> On Thursday 22 October 2009 5:17:07 pm Daniel Eischen wrote: >>>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>>>> >>>>>> Daniel Eischen wrote: >>>>>>> On Thu, 22 Oct 2009, Andrew Gallatin wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> We're designing some software which has to lock access to >>>>>>>> shared memory pages between several processes, and has to >>>>>>>> run on Linux, Solaris, and FreeBSD. We were planning to >>>>>>>> have the lock be a pthread_mutex_t residing in the >>>>>>>> shared memory page. This works well on Linux and Solaris, >>>>>>>> but FreeBSD (at least 7-stable) does not support >>>>>>>> PTHREAD_PROCESS_SHARED mutexes. >>>>>>>> >>>>>>>> We then moved on to posix semaphores. Using sem_wait/sem_post >>>>>>>> with the sem_t residing in a shared page seems to work on >>>>>>>> all 3 platforms. However, the FreeBSD (7-stable) man page >>>>>>>> for sem_init(3) has this scary text regarding the pshared >>>>>>>> value: >>>>>>>> >>>>>>>> The sem_init() function initializes the unnamed semaphore >>>>>>>> pointed to >>>>>>>> by >>>>>>>> sem to have the value value. A non-zero value for pshared >>>>>>>> specifies >>>> a >>>>>>>> shared semaphore that can be used by multiple processes, >>>>>>>> which this >>>>>>>> implementation is not capable of. >>>>>>>> >>>>>>>> Is this text obsolete? Or is my test just "getting lucky"? >>>>>>> >>>>>>> I think you're getting lucky. >>>>>> >>>>>> Yes, after playing with the code some, I now see that. :( >>>>>> >>>>>>>> Is there recommended way to do this? >>>>>>> >>>>>>> I believe the only way to do this is with SYSV semaphores >>>>>>> (semop, semget, semctl). Unfortunately, these are not as >>>>>>> easy to use, IMHO. >>>>>> >>>>>> Yes, they are pretty ugly, and we were hoping to avoid them. >>>>>> Are there any plans to support either PTHREAD_PROCESS_SHARED >>>>>> mutexes, or pshared posix semaphores in FreeBSD? >>>>> >>>>> It's planned, just not (yet) being actively worked on. >>>>> It's a API change mostly, and then adding in all the >>>>> compat hooks so we don't break ABI. >>>> >>>> There are also an alternate set of patches on threads@ to allow just >>>> shared >>>> semaphores I think w/o the changes to the pthread types. I can't >>>> recall >>>> exactly what they did, but I think rrs@ was playing with using umtx >>>> directly >>>> to implement some sort of process-shared primitive. >>> >>> That's really not the way to go. The structs really need >>> to become public. >>> >> >> It would be great if they were, but that discussion was 6 months >> ago, and nothing seems to have happened. Plus we need to support >> at least 7.X and probably 6, so any changes here might not even >> help us. >> >> What is wrong with just using umtx directly? It seems to do >> exactly what we need. > > Because you can't do anything more than use umtx directly, > like check for mutex types and return appropriate error > codes. Just look at other implementations - Solaris, > Linux, all have their pthread_*_t as public structs. I'm not saying that having pthread*t public, and getting all the features of real PTHREAD_PROCESS_SHARED would not be far better in general. But in this case all we need is a lock around a shared resource. Eg, nothing fance. So our choices seem to be either: 1) use sysv semaphores (ick) 2) use a hand rolled spinlock (ick) 3) use some sort of hack built into our driver (ick, ick) 4) use umtx Is there some bug or limitation in umtx that makes it inappropriate? (beyond the obvious, like the potential to leave a resource locked forever if the lock holder exits). Thanks, Drew From rea-fbsd at codelabs.ru Fri Oct 23 15:56:43 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Fri Oct 23 15:56:51 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <86tyxqqpwh.fsf@ds4.des.no> References: <86tyxqqpwh.fsf@ds4.des.no> Message-ID: <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> Gentlemen, good day. Fri, Oct 23, 2009 at 02:02:38PM +0200, Dag-Erling Sm??rgrav wrote: > src/contrib/ee/ee.c in 8: > > in = wgetch(text_win); > if (in == -1) > exit(0); /* without this exit ee will go into an > infinite loop if the network > session detaches */ > > >From the wgetch() man page: > > Programmers concerned about portability should be prepared for > either of two cases: (a) signal receipt does not interrupt > getch; (b) signal receipt interrupts getch and causes it to > return ERR with errno set to EINTR. Under the ncurses > implementation, handled signals never inter??? rupt getch. Hmm, we can transform this code to the following one: ----- errno = 0; do { in = wgetch(text_win); } while (errno == EINTR); if (in == -1) exit(0); ----- This won't help with FreeBSD's ncurses, but may be other variants will feel much better with such a event loop variant. > The real issue, though, is that when a SIGWINCH is caught, wgetch() > correctly returns KEY_RESIZE, but the next call to wgetch() returns -1. > The next call after that is fine. I suspect the error lies somewhere > inside kgetch() in contrib/ncurses/ncurses/base/lib_getch.c. The problem should be healed with the attached patch. And you're partly right: this is kgetch() that is returning ERR for the second wgetch(), but kgetch() shouldn't be blamed for this -- _nc_wgetch() should. At least in my opinion ;) Any views on this? -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: ncurses-properly-handle-SIGWINCH.diff Type: text/x-diff Size: 1394 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091023/3720a4a7/ncurses-properly-handle-SIGWINCH.bin From deischen at freebsd.org Fri Oct 23 16:07:57 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Fri Oct 23 16:08:04 2009 Subject: semaphores between processes In-Reply-To: <4AE1D1D2.1090307@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> <4AE1CE31.1090206@cs.duke.edu> <4AE1D1D2.1090307@cs.duke.edu> Message-ID: On Fri, 23 Oct 2009, Andrew Gallatin wrote: > Daniel Eischen wrote: >> On Fri, 23 Oct 2009, Andrew Gallatin wrote: >> >>> It would be great if they were, but that discussion was 6 months >>> ago, and nothing seems to have happened. Plus we need to support >>> at least 7.X and probably 6, so any changes here might not even >>> help us. >>> >>> What is wrong with just using umtx directly? It seems to do >>> exactly what we need. >> >> Because you can't do anything more than use umtx directly, >> like check for mutex types and return appropriate error >> codes. Just look at other implementations - Solaris, >> Linux, all have their pthread_*_t as public structs. > > I'm not saying that having pthread*t public, and getting all > the features of real PTHREAD_PROCESS_SHARED would not be far > better in general. But in this case all we need is a lock around > a shared resource. Eg, nothing fance. So our choices seem to be > either: > > 1) use sysv semaphores (ick) > 2) use a hand rolled spinlock (ick) > 3) use some sort of hack built into our driver (ick, ick) > 4) use umtx > > Is there some bug or limitation in umtx that makes it inappropriate? > (beyond the obvious, like the potential to leave a resource locked > forever if the lock holder exits). We already use umtx. This really is a hack and I wouldn't advocate it. I'm not sure how you could make it work and not break existing ability to return appropriate error codes without slowing down the path in the non-shared case. You'd have to check to see if the address space was shared or not, which would require a system call. All our public pthread_foo() symbols are weak. You can easily override them in your application code in the #ifdef freebsd case. What is wrong with providing your own library that overrides them to do what you require - this shouldn't change your application code? -- DE From gallatin at cs.duke.edu Fri Oct 23 16:27:51 2009 From: gallatin at cs.duke.edu (Andrew Gallatin) Date: Fri Oct 23 16:27:57 2009 Subject: semaphores between processes In-Reply-To: References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> <4AE1CE31.1090206@cs.duke.edu> <4AE1D1D2.1090307@cs.duke.edu> Message-ID: <4AE1D97F.6060708@cs.duke.edu> Daniel Eischen wrote: > > We already use umtx. This really is a hack and I wouldn't > advocate it. I'm not sure how you could make it work and > not break existing ability to return appropriate error > codes without slowing down the path in the non-shared > case. You'd have to check to see if the address space > was shared or not, which would require a system call. I'm probably missing something. What does it matter if the address space is shared, as long as the umtx struct is in shared memory? From my quick read, the umtx operations use a lock word in userspace. For uncontested locks, they use atomic ops to flip an id into the lock word. The kernel takes over for contested locks, and does sleeping, wakup, etc. Is this correct? Is there something here that matters if the address space (and not just the lock word) is shared? > All our public pthread_foo() symbols are weak. You > can easily override them in your application code in > the #ifdef freebsd case. What is wrong with providing > your own library that overrides them to do what you > require - this shouldn't change your application code? > For our code, I was thinking of something like: #ifdef FreeBSD #define lock(x) umtx_lock(x, getpid()) #define unlock(x) umtx_unlock(x, getpid()) #else #define lock(x) pthread_mutex_lock(x) #define unlock(x) pthread_mutex_lock(x) #endif I should probably just shut up and try it.. Drew From alexbestms at math.uni-muenster.de Fri Oct 23 16:32:41 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 16:32:48 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> Message-ID: Eygene Ryabinkin schrieb am 2009-10-23: > Gentlemen, good day. > Fri, Oct 23, 2009 at 02:02:38PM +0200, Dag-Erling Sm??rgrav wrote: > > src/contrib/ee/ee.c in 8: > > in = wgetch(text_win); > > if (in == -1) > > exit(0); /* without this exit ee will go > > into an > > infinite loop if the network > > session detaches */ > > >From the wgetch() man page: > > Programmers concerned about portability should be prepared > > for > > either of two cases: (a) signal receipt does not interrupt > > getch; (b) signal receipt interrupts getch and causes it to > > return ERR with errno set to EINTR. Under the ncurses > > implementation, handled signals never inter??? rupt getch. > Hmm, we can transform this code to the following one: > ----- > errno = 0; > do { > in = wgetch(text_win); > } while (errno == EINTR); > if (in == -1) > exit(0); > ----- > This won't help with FreeBSD's ncurses, but may be other variants > will feel much better with such a event loop variant. > > The real issue, though, is that when a SIGWINCH is caught, wgetch() > > correctly returns KEY_RESIZE, but the next call to wgetch() returns > > -1. > > The next call after that is fine. I suspect the error lies > > somewhere > > inside kgetch() in contrib/ncurses/ncurses/base/lib_getch.c. > The problem should be healed with the attached patch. And you're > partly right: this is kgetch() that is returning ERR for the second > wgetch(), but kgetch() shouldn't be blamed for this -- _nc_wgetch() > should. At least in my opinion ;) > Any views on this? thanks a million. for me the patch works great. :) the sooner it gets committed the better. ;) since this problem is not limited to freebsd it would be nice if somebody could send the patch to the ncurses maintainers so all OSes using ncurses can profit from it. cheers. alex From rea-fbsd at codelabs.ru Fri Oct 23 17:01:10 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Fri Oct 23 17:01:29 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> Message-ID: Fri, Oct 23, 2009 at 06:32:38PM +0200, Alexander Best wrote: > thanks a million. for me the patch works great. :) You're welcome ;)) > the sooner it gets committed the better. ;) It may well break something else. I am not 100% sure that this patch is the proper thing -- curses code is a bit new to me, so I would say that the patch will need more thorough review from maintainers. > since this problem is not limited to freebsd it would be nice if > somebody could send the patch to the ncurses maintainers so all OSes > using ncurses can profit from it. Will also send this patch to bug-ncurses@gnu.org. -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From alexbestms at math.uni-muenster.de Fri Oct 23 17:02:35 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 23 17:02:41 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> Message-ID: Eygene Ryabinkin schrieb am 2009-10-23: > Gentlemen, good day. > Fri, Oct 23, 2009 at 02:02:38PM +0200, Dag-Erling Sm??rgrav wrote: > > src/contrib/ee/ee.c in 8: > > in = wgetch(text_win); > > if (in == -1) > > exit(0); /* without this exit ee will go > > into an > > infinite loop if the network > > session detaches */ > > >From the wgetch() man page: > > Programmers concerned about portability should be prepared > > for > > either of two cases: (a) signal receipt does not interrupt > > getch; (b) signal receipt interrupts getch and causes it to > > return ERR with errno set to EINTR. Under the ncurses > > implementation, handled signals never inter??? rupt getch. > Hmm, we can transform this code to the following one: > ----- > errno = 0; > do { > in = wgetch(text_win); > } while (errno == EINTR); > if (in == -1) > exit(0); > ----- > This won't help with FreeBSD's ncurses, but may be other variants > will feel much better with such a event loop variant. > > The real issue, though, is that when a SIGWINCH is caught, wgetch() > > correctly returns KEY_RESIZE, but the next call to wgetch() returns > > -1. > > The next call after that is fine. I suspect the error lies > > somewhere > > inside kgetch() in contrib/ncurses/ncurses/base/lib_getch.c. > The problem should be healed with the attached patch. And you're > partly right: this is kgetch() that is returning ERR for the second > wgetch(), but kgetch() shouldn't be blamed for this -- _nc_wgetch() > should. At least in my opinion ;) > Any views on this? oh...and btw. the PR (http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/136223) hasn't been assigned to anyone yet. could the person who will be committing the patch please take care of that (maybe setting the PR to analysed->patched->closed? ;) would be great if this fix could make it into 8.0-RELEASE in time because it's quite nasty to lose all your editor data due to a SIGWINCH. cheers. alex From ed at 80386.nl Fri Oct 23 17:13:03 2009 From: ed at 80386.nl (Ed Schouten) Date: Fri Oct 23 17:13:11 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> References: <86tyxqqpwh.fsf@ds4.des.no> <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> Message-ID: <20091023171301.GO1293@hoeg.nl> * Eygene Ryabinkin wrote: > The problem should be healed with the attached patch. Ah, thanks. I looked at this some time ago but I also discovered ncurses was to blame. I didn't have any time to look at it back then, so I obviously forgot. Have you sent it to Thomas Dickey as well? -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091023/131abb1c/attachment.pgp From deischen at freebsd.org Fri Oct 23 17:18:05 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Fri Oct 23 17:18:13 2009 Subject: semaphores between processes In-Reply-To: <4AE1D97F.6060708@cs.duke.edu> References: <4AE0BBAB.3040807@cs.duke.edu> <4AE0C995.5060303@cs.duke.edu> <200910230802.49873.jhb@freebsd.org> <4AE1CE31.1090206@cs.duke.edu> <4AE1D1D2.1090307@cs.duke.edu> <4AE1D97F.6060708@cs.duke.edu> Message-ID: On Fri, 23 Oct 2009, Andrew Gallatin wrote: > Daniel Eischen wrote: > >> >> We already use umtx. This really is a hack and I wouldn't >> advocate it. I'm not sure how you could make it work and >> not break existing ability to return appropriate error >> codes without slowing down the path in the non-shared >> case. You'd have to check to see if the address space >> was shared or not, which would require a system call. > > I'm probably missing something. What does it matter if the > address space is shared, as long as the umtx struct is > in shared memory? > > From my quick read, the umtx operations use a lock word > in userspace. For uncontested locks, they use atomic > ops to flip an id into the lock word. The kernel takes > over for contested locks, and does sleeping, wakup, etc. > Is this correct? Is there something here that matters > if the address space (and not just the lock word) is > shared? > >> All our public pthread_foo() symbols are weak. You >> can easily override them in your application code in >> the #ifdef freebsd case. What is wrong with providing >> your own library that overrides them to do what you >> require - this shouldn't change your application code? >> > > For our code, I was thinking of something like: > > #ifdef FreeBSD > #define lock(x) umtx_lock(x, getpid()) > #define unlock(x) umtx_unlock(x, getpid()) > #else > #define lock(x) pthread_mutex_lock(x) > #define unlock(x) pthread_mutex_lock(x) > #endif > > > I should probably just shut up and try it.. My apologies - I thought you were talking about changing our pthread_mutex_* functions in src/lib/... -- DE From lankfordandrew at charter.net Fri Oct 23 21:05:08 2009 From: lankfordandrew at charter.net (Andrew Lankford) Date: Fri Oct 23 21:23:04 2009 Subject: [Fwd: Failure to boot from HD formatted not by FreeBSD] Message-ID: <4AE21A83.2030809@charter.net> I mentioned that I've set up my laptop to boot using the Windows Vista boot menu. I have, and I needed to copy /boot/boot1 from 7-stable to my NTFS partition in order to successfully boot to my FreeBSD 8.0 partition. Last night, I tried replacing the 7-stable boot1 block with a version from 8.0-RC1, and now it doesn't successfully boot. I gather from the 8.0 release notes that there have been some changes to some part of the boot code. In any case, I can boot via the Windows boot menu with the help of 7-stable's /boot/boot1 file. Hoping that helps .... Andrew Lankford -------- Original Message -------- From: - Thu Oct 22 20:54:06 2009 X-Mozilla-Status: 0001 X-Mozilla-Status2: 00800000 X-Mozilla-Keys: Message-ID: <4AE0FEA5.9070004@charter.net> Date: Thu, 22 Oct 2009 20:53:57 -0400 From: Andrew Lankford User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: Yuri , freebsd-hackers@freebsd.org Subject: Failure to boot from HD formatted not by FreeBSD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bi Looks to me like you're trying to get your computer to dual-boot Vista and FreeBSD 8.0, something I finally succeeded in doing. If by "MBR" you mean the first-stage boot program (512 bytes), I couldn't get that to work, nor could I use the standard boot0 menu from FreeBSD. I'm using the windows boot program instead. I think what I did was copy "/boot/boot1" from my root partition to my NTFS partition and then added an option to the Windows boot menu to boot with it. I get the GEOM "track boundary" complaint when I boot up as well. The FBSD 8.0 kernel has a new option 'GEOM_PART_MBR" on by default. Vista insisted on partitioning my drive, so if the new partition handler doesn't like it, it can lump it. In order to get the 8.0 kernel to recognise your old partitions, you need the "GEOM_MBR" option activated. That means you need to load "geom_mbr.ko" into memory before you load and boot from the 8.0 kernel. If you're booting from a FreeBSD 8.0 CD directly into sysinstall, you can escape to a shell and kldload geom_mbr.ko, but you have to then restart sysinstall without rebooting the computer in order for your hard disk partitions to show up. The only reliable way I could find to restart systinstall without rebooting was by pressing the power button. Wierd, eh? I added "option GEOM_MBR" back into my kernel, recompiled, fiddled with my network settings, and now everything seems to work alright. Anyway, all this procedure should be 75% correct since I've managed to successfully upgrade to 8.0 from 7-stable this way. For all I know, I might end up with a corrupted partition six months from now. Either that or Marcel Moolenar will get angry at me. Regards, Andrew Lankford From rea-fbsd at codelabs.ru Sat Oct 24 04:04:17 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Sat Oct 24 04:04:23 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <20091023171301.GO1293@hoeg.nl> References: <86tyxqqpwh.fsf@ds4.des.no> <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> <20091023171301.GO1293@hoeg.nl> Message-ID: Ed, good day. Fri, Oct 23, 2009 at 07:13:01PM +0200, Ed Schouten wrote: > Have you sent it to Thomas Dickey as well? Sent the patch to bug-ncurses@gnu.org. Do you think that I should send it to Thomas directly as well? -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From ed at 80386.nl Sat Oct 24 08:08:49 2009 From: ed at 80386.nl (Ed Schouten) Date: Sat Oct 24 08:08:56 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: <86tyxqqpwh.fsf@ds4.des.no> <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> <20091023171301.GO1293@hoeg.nl> Message-ID: <20091024080845.GP1293@hoeg.nl> Hy Eygene, * Eygene Ryabinkin wrote: > Sent the patch to bug-ncurses@gnu.org. Do you think that I should > send it to Thomas directly as well? Probably not. bug-ncurses@ should be good enough. Thanks! -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091024/593942dc/attachment.pgp From jilles at stack.nl Sat Oct 24 12:04:36 2009 From: jilles at stack.nl (Jilles Tjoelker) Date: Sat Oct 24 12:04:43 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> References: <86tyxqqpwh.fsf@ds4.des.no> <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> Message-ID: <20091024120434.GA27808@stack.nl> On Fri, Oct 23, 2009 at 07:56:35PM +0400, Eygene Ryabinkin wrote: > Hmm, we can transform this code to the following one: > ----- > errno = 0; > do { > in = wgetch(text_win); > } while (errno == EINTR); > if (in == -1) > exit(0); > ----- > This won't help with FreeBSD's ncurses, but may be other variants > will feel much better with such a event loop variant. That should be: ----- do in = wgetch(text_win); while (in == -1 && errno == EINTR); if (in == -1) exit(0); ----- errno should only be checked after failed function calls or for functions where it is documented that errno should be used to check for error conditions (example: readdir). -- Jilles Tjoelker From rea-fbsd at codelabs.ru Sat Oct 24 15:44:07 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Sat Oct 24 15:44:13 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <20091024120434.GA27808@stack.nl> References: <86tyxqqpwh.fsf@ds4.des.no> <8gJJDa6kRsCUwxK/zidtHIOFMRw@LbQSLh99U4wa807TkC1GazBU7WI> <20091024120434.GA27808@stack.nl> Message-ID: Sat, Oct 24, 2009 at 02:04:34PM +0200, Jilles Tjoelker wrote: > That should be: > ----- > do > in = wgetch(text_win); > while (in == -1 && errno == EINTR); > if (in == -1) > exit(0); > ----- > > errno should only be checked after failed function calls or for > functions where it is documented that errno should be used to check for > error conditions (example: readdir). True, thanks for correction! -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From des at des.no Sun Oct 25 15:00:07 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Sun Oct 25 15:00:15 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: (Alexander Best's message of "Fri, 23 Oct 2009 14:16:50 +0200 (CEST)") References: Message-ID: <864opnbjt5.fsf@ds4.des.no> Alexander Best writes: > i just tried building ee under linux without using new_curse.c and linking the > executable against ncurses. running the binary is showing the same problems > with SIGWINCH. so if this is in fact caused by a ncurses bug the bug appears > in linux ncurses too. No surprise, it's the same as ours. DES -- Dag-Erling Sm?rgrav - des@des.no From imp at bsdimp.com Sun Oct 25 19:40:29 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Sun Oct 25 19:40:41 2009 Subject: Help troubleshooting... Message-ID: <20091025.133437.-1844000782.imp@bsdimp.com> I have a usb stick (8GB) on it. This stick has about 5GB of junk on it at this point. I tried to do 'cat * > /dev/null' recently, to measure how fast it goes. It got about 1GB into the drive and then I got device missing messages. Here's the dmesg messages: # Plug it in da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) # mount it, etc # run the cat command Device da0s1 went missing before all of the data could be written to it; expect data loss. # get error messages # Remove the drive ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 1, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry So devfs thinks the device went missing: static int devfs_fsync(struct vop_fsync_args *ap) { ... if (!vn_isdisk(ap->a_vp, &error)) { bo = &ap->a_vp->v_bufobj; de = ap->a_vp->v_data; if (error == ENXIO && bo->bo_dirty.bv_cnt > 0) { printf("Device %s went missing before all of the data " "could be written to it; expect data loss.\n", de->de_dirent->d_name); ... So it thinks that it isn't a disk. vn_isdisk is return ENXIO because either vp->v_rdev == NULL or the v_rdev->si_devsw == NULL. So how the heck can that happen without other warnings? It appears the only place it is set like this is in devfs_reclaim. But I'm having trouble tracking down where *THAT* is called. This is with the following system: FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 Warner From kostikbel at gmail.com Sun Oct 25 19:49:32 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Sun Oct 25 19:49:39 2009 Subject: Help troubleshooting... In-Reply-To: <20091025.133437.-1844000782.imp@bsdimp.com> References: <20091025.133437.-1844000782.imp@bsdimp.com> Message-ID: <20091025194923.GU2160@deviant.kiev.zoral.com.ua> On Sun, Oct 25, 2009 at 01:34:37PM -0600, M. Warner Losh wrote: > I have a usb stick (8GB) on it. This stick has about 5GB of junk on > it at this point. > > I tried to do 'cat * > /dev/null' recently, to measure how fast it > goes. It got about 1GB into the drive and then I got device missing > messages. > > Here's the dmesg messages: > > # Plug it in > da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 > da0: Removable Direct Access SCSI-0 device > da0: 40.000MB/s transfers > da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) > # mount it, etc > # run the cat command > Device da0s1 went missing before all of the data could be written to it; expect data loss. > # get error messages > # Remove the drive > ugen2.2: at usbus2 (disconnected) > umass0: at uhub2, port 1, addr 2 (disconnected) > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): removing device entry > > So devfs thinks the device went missing: > > static int > devfs_fsync(struct vop_fsync_args *ap) > { > ... > if (!vn_isdisk(ap->a_vp, &error)) { > bo = &ap->a_vp->v_bufobj; > de = ap->a_vp->v_data; > if (error == ENXIO && bo->bo_dirty.bv_cnt > 0) { > printf("Device %s went missing before all of the data " > "could be written to it; expect data loss.\n", > de->de_dirent->d_name); > ... > > So it thinks that it isn't a disk. vn_isdisk is return ENXIO because > either vp->v_rdev == NULL or the v_rdev->si_devsw == NULL. si_devsw is cleared in kern_conf.c:destroy_devl, line 835. I think USB subsystem called either destroy_dev() or destroy_dev_sched(). > > So how the heck can that happen without other warnings? It appears > the only place it is set like this is in devfs_reclaim. But I'm > having trouble tracking down where *THAT* is called. Reclaim might be called if devfs mount point is forcibly unmounted special vnode from which you used to mount pendrive. > > This is with the following system: > > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 > > Warner > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091025/54b9a9c0/attachment.pgp From non at ever.sanda.gr.jp Mon Oct 26 00:27:50 2009 From: non at ever.sanda.gr.jp (non@ever.sanda.gr.jp) Date: Mon Oct 26 00:27:57 2009 Subject: Help troubleshooting... In-Reply-To: <20091025.133437.-1844000782.imp@bsdimp.com> References: <20091025.133437.-1844000782.imp@bsdimp.com> Message-ID: <4AE4E7AF.8060103@ever.sanda.gr.jp> M. Warner Losh wrote: > I have a usb stick (8GB) on it. This stick has about 5GB of junk on > it at this point. > > I tried to do 'cat * > /dev/null' recently, to measure how fast it > goes. It got about 1GB into the drive and then I got device missing > messages. : > So devfs thinks the device went missing: Warner-san, maybe it is caused by the hardware problem on the USB flash memory. Some chip on the memory might have too much heat when you access the memory at fast rate. Then it stops working. // Noriaki Mitsunaga From =?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?= Mon Oct 26 08:29:46 2009 From: =?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?= (=?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?=) Date: Mon Oct 26 08:29:53 2009 Subject: make release stop at Creating ISO imagess Message-ID: <20091026143639.13895c34kml7gmmf@webmail.tint.or.th> hi sirs, apologized me for disturbing this list but ireally have problem s. i make my own release by follwoing document in releng articles. here is my command cd /usr/src/release time make CHROOTDIR=/kaitag/KAITAG BUILDNAME=7.2-RELEASE \ CVSROOT=/var/ftp/pub/ncvs RELEASETAG=RELENG_7_2_0_RELEASE \ EXTSRCDIR=/usr/src EXTPORTSDIR=/usr/ports \ DOC_LANG=en_US.ISO8859-1 NODOC=NO NOPORTS=NO \ MAKE_DVD=YES MAKE_ISOS=YES CDROM=YES RELEASEDISTFILES=/var/ftp/pub/distfiles \ release the build process goes well but then fetching for cdrtools.tbz begin and stop. in my machine i have installed cdrtools and there is also mkisofs file so i wonder why the script /usr/src/release/i386/mkisofsimages.sh still fetching for cdrtools. and here is my uname of my machine wmc# uname -a FreeBSD wmc.tint.or.th 7.2-RELEASE FreeBSD KAITAG #0: Wed Oct 21 14:36:40 ICT 2009 root@wmc.tint.or.th:/usr/obj/usr/src/sys/WMC i386 wmc# i set UNAME_r=7.2-RELEASE in /etc/make.conf though. many thanks in advance for any helps and hints. with best regards, psr -- ???? ?????????? ????? ????????? http://makham.blogspot.com ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From hselasky at c2i.net Mon Oct 26 10:00:17 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 10:00:23 2009 Subject: Help troubleshooting... In-Reply-To: <4AE4E7AF.8060103@ever.sanda.gr.jp> References: <20091025.133437.-1844000782.imp@bsdimp.com> <4AE4E7AF.8060103@ever.sanda.gr.jp> Message-ID: <200910260959.20772.hselasky@c2i.net> On Monday 26 October 2009 01:05:03 non@ever.sanda.gr.jp wrote: > M. Warner Losh wrote: > > I have a usb stick (8GB) on it. This stick has about 5GB of junk on > > it at this point. > > > > I tried to do 'cat * > /dev/null' recently, to measure how fast it > > goes. It got about 1GB into the drive and then I got device missing > > messages. > > > > So devfs thinks the device went missing: > > Warner-san, maybe it is caused by the hardware problem on the USB flash > memory. Some chip on the memory might have too much heat when you access > the memory at fast rate. Then it stops working. > What happens if you read from two USB disks at the same time? If the device went missing the USB HUB signalled that. This is maybe an indication that the USB firmware on the device crashed. Maybe this is due to heat, or unhandled race conditions when the load goes high. Try using "dd" and vary the block size from 512 to 65536 bytes. Does it stop working with all block sizes over time? --HPS From =?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?= Mon Oct 26 10:06:17 2009 From: =?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?= (=?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?=) Date: Mon Oct 26 10:06:24 2009 Subject: make release stop at Creating ISO imagess In-Reply-To: <20091026095317.GA72492@ei.bzerk.org> References: <20091026143639.13895c34kml7gmmf@webmail.tint.or.th> <20091026095317.GA72492@ei.bzerk.org> Message-ID: <20091026163950.56716v4m76bon23q@webmail.tint.or.th> Quoting Ruben de Groot : > On Mon, Oct 26, 2009 at 02:36:39PM +0700, ??????????????? > ????????????????????? typed: >> hi sirs, >> >> apologized me for disturbing this list but ireally have problem s. >> i make my own release by follwoing document in releng articles. >> >> here is my command >> >> cd /usr/src/release >> time make CHROOTDIR=/kaitag/KAITAG BUILDNAME=7.2-RELEASE \ >> CVSROOT=/var/ftp/pub/ncvs RELEASETAG=RELENG_7_2_0_RELEASE \ >> EXTSRCDIR=/usr/src EXTPORTSDIR=/usr/ports \ >> DOC_LANG=en_US.ISO8859-1 NODOC=NO NOPORTS=NO \ >> MAKE_DVD=YES MAKE_ISOS=YES CDROM=YES >> RELEASEDISTFILES=/var/ftp/pub/distfiles \ >> release >> >> the build process goes well but then fetching for cdrtools.tbz begin and >> stop. >> in my machine i have installed cdrtools and there is also mkisofs file >> so i wonder why the script /usr/src/release/i386/mkisofsimages.sh >> still fetching for cdrtools. > > Do you have the cdrtools.tbz package in /var/ftp/pub/distfiles ? > thanks for your time but there is only cdrtools-2.01.tar.bz2 which is source files for making cdrtools. i keep all packages in a separate location, /usr/ports/packages/. > > Ruben > > with best regards, -- psr ????? ????????? ???? ?????????? http://makham.blogspot.com ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From imp at bsdimp.com Mon Oct 26 10:24:06 2009 From: imp at bsdimp.com (Warner Losh) Date: Mon Oct 26 10:24:21 2009 Subject: Help troubleshooting... In-Reply-To: <200910260959.20772.hselasky@c2i.net> References: <20091025.133437.-1844000782.imp@bsdimp.com> <4AE4E7AF.8060103@ever.sanda.gr.jp> <200910260959.20772.hselasky@c2i.net> Message-ID: <96830A33-8392-4C05-B073-48026E24C895@bsdimp.com> But there was no indication the device went missing on the console. And it wasn't until I unplugged it that da0 detached. Usually in the past when it has started throwing errors the console was chatty about why. Warner On Oct 26, 2009, at 2:59 AM, Hans Petter Selasky wrote: > On Monday 26 October 2009 01:05:03 non@ever.sanda.gr.jp wrote: >> M. Warner Losh wrote: >>> I have a usb stick (8GB) on it. This stick has about 5GB of junk on >>> it at this point. >>> >>> I tried to do 'cat * > /dev/null' recently, to measure how fast it >>> goes. It got about 1GB into the drive and then I got device missing >>> messages. >>> >>> So devfs thinks the device went missing: >> >> Warner-san, maybe it is caused by the hardware problem on the USB >> flash >> memory. Some chip on the memory might have too much heat when you >> access >> the memory at fast rate. Then it stops working. >> > > What happens if you read from two USB disks at the same time? > > If the device went missing the USB HUB signalled that. This is maybe > an > indication that the USB firmware on the device crashed. Maybe this > is due to > heat, or unhandled race conditions when the load goes high. > > Try using "dd" and vary the block size from 512 to 65536 bytes. Does > it stop > working with all block sizes over time? > > --HPS > > From alexbestms at math.uni-muenster.de Mon Oct 26 10:44:17 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Mon Oct 26 10:44:24 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: Message-ID: Eygene Ryabinkin schrieb am 2009-10-24: > Ed, good day. > Fri, Oct 23, 2009 at 07:13:01PM +0200, Ed Schouten wrote: > > Have you sent it to Thomas Dickey as well? > Sent the patch to bug-ncurses@gnu.org. Do you think that I should > send it to Thomas directly as well? the patch got committed by thomas and is included in ncurses-5.7-20091024.patch.gz. i guess it will be included in our base version of ncurses once 5.8 gets released, but the patch should quickly make it into the port version of ncurses. alex From alexbestms at math.uni-muenster.de Mon Oct 26 10:48:45 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Mon Oct 26 10:48:51 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <864opnbjt5.fsf@ds4.des.no> Message-ID: Dag-Erling Sm?rgrav schrieb am 2009-10-25: > Alexander Best writes: > > i just tried building ee under linux without using new_curse.c and > > linking the > > executable against ncurses. running the binary is showing the same > > problems > > with SIGWINCH. so if this is in fact caused by a ncurses bug the > > bug appears > > in linux ncurses too. > No surprise, it's the same as ours. > DES this was when we weren't sure if the problem was entirely ncurses-related. could have also been that linux was including the ncurses patches which get released by thomas dickey every now and then. i'm not entirely sure but i think the version in the base dir we use is the unpatched 5.7 release of ncurses. alex From mail25 at bzerk.org Mon Oct 26 09:53:26 2009 From: mail25 at bzerk.org (Ruben de Groot) Date: Mon Oct 26 11:23:26 2009 Subject: make release stop at Creating ISO imagess In-Reply-To: <20091026143639.13895c34kml7gmmf@webmail.tint.or.th> References: <20091026143639.13895c34kml7gmmf@webmail.tint.or.th> Message-ID: <20091026095317.GA72492@ei.bzerk.org> On Mon, Oct 26, 2009 at 02:36:39PM +0700, ??????????????? ????????????????????? typed: > hi sirs, > > apologized me for disturbing this list but ireally have problem s. > i make my own release by follwoing document in releng articles. > > here is my command > > cd /usr/src/release > time make CHROOTDIR=/kaitag/KAITAG BUILDNAME=7.2-RELEASE \ > CVSROOT=/var/ftp/pub/ncvs RELEASETAG=RELENG_7_2_0_RELEASE \ > EXTSRCDIR=/usr/src EXTPORTSDIR=/usr/ports \ > DOC_LANG=en_US.ISO8859-1 NODOC=NO NOPORTS=NO \ > MAKE_DVD=YES MAKE_ISOS=YES CDROM=YES > RELEASEDISTFILES=/var/ftp/pub/distfiles \ > release > > the build process goes well but then fetching for cdrtools.tbz begin and > stop. > in my machine i have installed cdrtools and there is also mkisofs file > so i wonder why the script /usr/src/release/i386/mkisofsimages.sh > still fetching for cdrtools. Do you have the cdrtools.tbz package in /var/ftp/pub/distfiles ? Ruben From imp at bsdimp.com Mon Oct 26 11:53:07 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 11:53:14 2009 Subject: Help troubleshooting... In-Reply-To: <200910260959.20772.hselasky@c2i.net> References: <20091025.133437.-1844000782.imp@bsdimp.com> <4AE4E7AF.8060103@ever.sanda.gr.jp> <200910260959.20772.hselasky@c2i.net> Message-ID: <20091026.054816.1631944692.imp@bsdimp.com> In message: <200910260959.20772.hselasky@c2i.net> Hans Petter Selasky writes: : On Monday 26 October 2009 01:05:03 non@ever.sanda.gr.jp wrote: : > M. Warner Losh wrote: : > > I have a usb stick (8GB) on it. This stick has about 5GB of junk on : > > it at this point. : > > : > > I tried to do 'cat * > /dev/null' recently, to measure how fast it : > > goes. It got about 1GB into the drive and then I got device missing : > > messages. : > > : > > So devfs thinks the device went missing: : > : > Warner-san, maybe it is caused by the hardware problem on the USB flash : > memory. Some chip on the memory might have too much heat when you access : > the memory at fast rate. Then it stops working. : > : : What happens if you read from two USB disks at the same time? I know that the august 25th version failed badly when I tried to burn DVDs from a USB drive to a USB attached DVD burner. This used to work flawlessly. : If the device went missing the USB HUB signalled that. This is maybe an : indication that the USB firmware on the device crashed. Maybe this is due to : heat, or unhandled race conditions when the load goes high. This same flash drive will do 20MB sustained on windows without a glitch using similar commands. : Try using "dd" and vary the block size from 512 to 65536 bytes. Does it stop : working with all block sizes over time? Once I get the message I posted, it is lights out for da0. No further access to the drive works at all. Warner From des at des.no Mon Oct 26 11:55:48 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 26 11:55:55 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: (Alexander Best's message of "Mon, 26 Oct 2009 11:44:14 +0100 (CET)") References: Message-ID: <86d44ae5do.fsf@ds4.des.no> Alexander Best writes: > the patch got committed by thomas and is included in > ncurses-5.7-20091024.patch.gz. > > i guess it will be included in our base version of ncurses once 5.8 gets > released, but the patch should quickly make it into the port version of > ncurses. Apply the upstream patch to vendor/ncurses/dist and merge it to head. Subversion will DTRT when you import 5.8. I believe this is documented on the wiki. DES -- Dag-Erling Sm?rgrav - des@des.no From hselasky at c2i.net Mon Oct 26 11:59:04 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 11:59:16 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.054816.1631944692.imp@bsdimp.com> References: <20091025.133437.-1844000782.imp@bsdimp.com> <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> Message-ID: <200910261258.08135.hselasky@c2i.net> On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: > I know that the august 25th version failed badly when I tried to burn > DVDs from a USB drive to a USB attached DVD burner. This used to work > flawlessly. Hi, There has been a recent fix to the EHCI driver, which might affect Mass Storage when short transfers are used. Also someone else has pointed out that certain VIA chipsets have an IRQ "bug" requiring the need for a software callout to restart the EHCI interrupt handler. This is not yet patched, hence I don't know if this is a real issue. http://svn.freebsd.org/viewvc/base?view=revision&revision=197682 Is your code from after 1st of October? --HPS From hselasky at c2i.net Mon Oct 26 12:00:18 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 12:00:35 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.054816.1631944692.imp@bsdimp.com> References: <20091025.133437.-1844000782.imp@bsdimp.com> <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> Message-ID: <200910261259.23500.hselasky@c2i.net> On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: > In message: <200910260959.20772.hselasky@c2i.net> > > Hans Petter Selasky writes: > : On Monday 26 October 2009 01:05:03 non@ever.sanda.gr.jp wrote: > : > M. Warner Losh wrote: > : > > I have a usb stick (8GB) on it. This stick has about 5GB of junk on > : > > it at this point. > : > > > : > > I tried to do 'cat * > /dev/null' recently, to measure how fast it > : > > goes. It got about 1GB into the drive and then I got device missing > : > > messages. > : > > > : > > So devfs thinks the device went missing: > : > > : > Warner-san, maybe it is caused by the hardware problem on the USB flash > : > memory. Some chip on the memory might have too much heat when you > : > access the memory at fast rate. Then it stops working. > : > : What happens if you read from two USB disks at the same time? > > I know that the august 25th version failed badly when I tried to burn > DVDs from a USB drive to a USB attached DVD burner. This used to work > flawlessly. > > : If the device went missing the USB HUB signalled that. This is maybe an > : indication that the USB firmware on the device crashed. Maybe this is due > : to heat, or unhandled race conditions when the load goes high. > > This same flash drive will do 20MB sustained on windows without a > glitch using similar commands. > > : Try using "dd" and vary the block size from 512 to 65536 bytes. Does it > : stop working with all block sizes over time? > > Once I get the message I posted, it is lights out for da0. No further > access to the drive works at all. > Make sure your driver has got sufficiently enough power, if powered over USB. --HPS From hselasky at c2i.net Mon Oct 26 12:00:18 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 12:00:36 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.054816.1631944692.imp@bsdimp.com> References: <20091025.133437.-1844000782.imp@bsdimp.com> <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> Message-ID: <200910261259.23500.hselasky@c2i.net> On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: > In message: <200910260959.20772.hselasky@c2i.net> > > Hans Petter Selasky writes: > : On Monday 26 October 2009 01:05:03 non@ever.sanda.gr.jp wrote: > : > M. Warner Losh wrote: > : > > I have a usb stick (8GB) on it. This stick has about 5GB of junk on > : > > it at this point. > : > > > : > > I tried to do 'cat * > /dev/null' recently, to measure how fast it > : > > goes. It got about 1GB into the drive and then I got device missing > : > > messages. > : > > > : > > So devfs thinks the device went missing: > : > > : > Warner-san, maybe it is caused by the hardware problem on the USB flash > : > memory. Some chip on the memory might have too much heat when you > : > access the memory at fast rate. Then it stops working. > : > : What happens if you read from two USB disks at the same time? > > I know that the august 25th version failed badly when I tried to burn > DVDs from a USB drive to a USB attached DVD burner. This used to work > flawlessly. > > : If the device went missing the USB HUB signalled that. This is maybe an > : indication that the USB firmware on the device crashed. Maybe this is due > : to heat, or unhandled race conditions when the load goes high. > > This same flash drive will do 20MB sustained on windows without a > glitch using similar commands. > > : Try using "dd" and vary the block size from 512 to 65536 bytes. Does it > : stop working with all block sizes over time? > > Once I get the message I posted, it is lights out for da0. No further > access to the drive works at all. > Make sure your driver has got sufficiently enough power, if powered over USB. --HPS From rafan at FreeBSD.ORG Mon Oct 26 12:10:36 2009 From: rafan at FreeBSD.ORG (Rong-En Fan) Date: Mon Oct 26 12:10:43 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: References: Message-ID: <20091026114004.GT79043@svm.csie.ntu.edu.tw> On Mon, Oct 26, 2009 at 11:44:14AM +0100, Alexander Best wrote: > Eygene Ryabinkin schrieb am 2009-10-24: > > Ed, good day. > > > Fri, Oct 23, 2009 at 07:13:01PM +0200, Ed Schouten wrote: > > > Have you sent it to Thomas Dickey as well? > > > Sent the patch to bug-ncurses@gnu.org. Do you think that I should > > send it to Thomas directly as well? > > the patch got committed by thomas and is included in > ncurses-5.7-20091024.patch.gz. > > i guess it will be included in our base version of ncurses once 5.8 gets > released, but the patch should quickly make it into the port version of > ncurses. devel/ncurses-devel will be updated this week. For base's ncurses, I'll check with Thomas Dickey to see if there will be 5.8 soon. If not, we can also import a recent snapshot. Thanks, Rong-En Fan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091026/6761de02/attachment.pgp From des at des.no Mon Oct 26 12:57:28 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 26 12:57:36 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <20091026114004.GT79043@svm.csie.ntu.edu.tw> (Rong-En Fan's message of "Mon, 26 Oct 2009 19:40:04 +0800") References: <20091026114004.GT79043@svm.csie.ntu.edu.tw> Message-ID: <86zl7ecnyg.fsf@ds4.des.no> Rong-En Fan writes: > devel/ncurses-devel will be updated this week. For base's ncurses, I'll > check with Thomas Dickey to see if there will be 5.8 soon. If not, we > can also import a recent snapshot. There is no reason to wait, nor to import an entire snapshot. Se my earlier message to Alexander. DES -- Dag-Erling Sm?rgrav - des@des.no From hselasky at c2i.net Mon Oct 26 12:59:04 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 12:59:11 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.054816.1631944692.imp@bsdimp.com> References: <20091025.133437.-1844000782.imp@bsdimp.com> <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> Message-ID: <200910261258.08135.hselasky@c2i.net> On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: > I know that the august 25th version failed badly when I tried to burn > DVDs from a USB drive to a USB attached DVD burner. This used to work > flawlessly. Hi, There has been a recent fix to the EHCI driver, which might affect Mass Storage when short transfers are used. Also someone else has pointed out that certain VIA chipsets have an IRQ "bug" requiring the need for a software callout to restart the EHCI interrupt handler. This is not yet patched, hence I don't know if this is a real issue. http://svn.freebsd.org/viewvc/base?view=revision&revision=197682 Is your code from after 1st of October? --HPS From imp at bsdimp.com Mon Oct 26 13:08:43 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 13:09:02 2009 Subject: Help troubleshooting... In-Reply-To: <200910261258.08135.hselasky@c2i.net> References: <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> <200910261258.08135.hselasky@c2i.net> Message-ID: <20091026.070117.439503022.imp@bsdimp.com> In message: <200910261258.08135.hselasky@c2i.net> Hans Petter Selasky writes: : On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: : > I know that the august 25th version failed badly when I tried to burn : > DVDs from a USB drive to a USB attached DVD burner. This used to work : > flawlessly. : : Hi, : : There has been a recent fix to the EHCI driver, which might affect Mass : Storage when short transfers are used. : : Also someone else has pointed out that certain VIA chipsets have an IRQ "bug" : requiring the need for a software callout to restart the EHCI interrupt : handler. This is not yet patched, hence I don't know if this is a real issue. : : http://svn.freebsd.org/viewvc/base?view=revision&revision=197682 : : Is your code from after 1st of October? This code is from: FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 so it would have r197682 baked in (the first number in my rev string is a mystery to me). Re another post: This is a 8GB flash, so I'm sure that there's enough power. Looking at the dmesg, this happend the second or third time I'd plugged in this flash drive. Here's a partial dmesg for usb things: CPU: AMD Turion(tm) 64 Mobile Technology ML-37 (1994.21-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x20f42 Stepping = 2 Features=0x78bfbff Features2=0x1 AMD Features=0xe2500800 AMD Features2=0x1 real memory = 2147483648 (2048 MB) avail memory = 2059546624 (1964 MB) ACPI APIC Table: MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard ... pcib2: at device 5.0 on pci0 pci2: on pcib2 ohci0: mem 0xc0000000-0xc0000fff irq 19 at device 19.0 on pci0 Activate PA 0xc0000000 at VA 0xffffff00c0000000 ohci0: [ITHREAD] usbus0: on ohci0 ohci1: mem 0xc0001000-0xc0001fff irq 19 at device 19.1 on pci0 Activate PA 0xc0001000 at VA 0xffffff00c0001000 ohci1: [ITHREAD] usbus1: on ohci1 ehci0: mem 0xc0002000-0xc0002fff irq 19 at device 19.2 on pci0 Activate PA 0xc0002000 at VA 0xffffff00c0002000 ehci0: [ITHREAD] usbus2: EHCI version 1.0 usbus2: on ehci0 ... Timecounter "TSC" frequency 1994209008 Hz quality 800 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 480Mbps High Speed USB v2.0 Status is 0x30000106 ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire Activate i/o 0x8014 Activate i/o 0x8015 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ad0: 114473MB at ata0-master UDMA100 GEOM: ad0s2: geometry does not match label (255h,63s != 16h,63s). uhub0: 4 ports with 4 removable, self powered uhub1: 4 ports with 4 removable, self powered ... Root mount waiting for: usbus2 uhub2: 8 ports with 8 removable, self powered Root mount waiting for: usbus2 Trying to mount root from ufs:/dev/ad0s2a ugen0.2: at usbus0 ubt0: on usbus0 usb_alloc_device:1635: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT! ugen2.2: at usbus2 umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 6, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): Invalidating pack g_vfs_done():da0s1[READ(offset=5298202624, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=5298268160, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=5298333696, length=65536)]error = 6 g_vfs_done():da0s1[WRITE(offset=1976320, length=32768)]error = 6 g_vfs_done():da0s1[READ(offset=5298137088, length=65536)]error = 6 (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi status == 0x0 (da0:umass-sim0:0:0:0): removing device entry ugen2.2: at usbus2 can't re-use a leaf (%desc)! can't re-use a leaf (%driver)! can't re-use a leaf (%location)! can't re-use a leaf (%pnpinfo)! can't re-use a leaf (%parent)! umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 6, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry ugen2.2: at usbus2 can't re-use a leaf (%desc)! can't re-use a leaf (%driver)! can't re-use a leaf (%location)! can't re-use a leaf (%pnpinfo)! can't re-use a leaf (%parent)! umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) ugen1.2: at usbus1 ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 1, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): Invalidating pack g_vfs_done():da0s1[READ(offset=634398720, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=634464256, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=634529792, length=65536)]error = 6 g_vfs_done():da0s1[WRITE(offset=1976320, length=32768)]error = 6 g_vfs_done():da0s1[READ(offset=634333184, length=65536)]error = 6 (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi status == 0x0 (da0:umass-sim0:0:0:0): removing device entry ugen2.2: at usbus2 can't re-use a leaf (%desc)! can't re-use a leaf (%driver)! can't re-use a leaf (%location)! can't re-use a leaf (%pnpinfo)! can't re-use a leaf (%parent)! umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) Device da0s1 went missing before all of the data could be written to it; expect data loss. ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 1, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry pcm0: mem 0xc0003400-0xc00034ff irq 17 at device 20.5 on pci0 Activate PA 0xc0003400 at VA 0xffffff00c0003400 pcm0: [ITHREAD] pcm0: ugen1.2: at usbus1 (disconnected) ugen1.2: at usbus1 ugen1.2: at usbus1 (disconnected) From imp at bsdimp.com Mon Oct 26 13:08:43 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 13:09:03 2009 Subject: Help troubleshooting... In-Reply-To: <200910261258.08135.hselasky@c2i.net> References: <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> <200910261258.08135.hselasky@c2i.net> Message-ID: <20091026.070117.439503022.imp@bsdimp.com> In message: <200910261258.08135.hselasky@c2i.net> Hans Petter Selasky writes: : On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: : > I know that the august 25th version failed badly when I tried to burn : > DVDs from a USB drive to a USB attached DVD burner. This used to work : > flawlessly. : : Hi, : : There has been a recent fix to the EHCI driver, which might affect Mass : Storage when short transfers are used. : : Also someone else has pointed out that certain VIA chipsets have an IRQ "bug" : requiring the need for a software callout to restart the EHCI interrupt : handler. This is not yet patched, hence I don't know if this is a real issue. : : http://svn.freebsd.org/viewvc/base?view=revision&revision=197682 : : Is your code from after 1st of October? This code is from: FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 so it would have r197682 baked in (the first number in my rev string is a mystery to me). Re another post: This is a 8GB flash, so I'm sure that there's enough power. Looking at the dmesg, this happend the second or third time I'd plugged in this flash drive. Here's a partial dmesg for usb things: CPU: AMD Turion(tm) 64 Mobile Technology ML-37 (1994.21-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x20f42 Stepping = 2 Features=0x78bfbff Features2=0x1 AMD Features=0xe2500800 AMD Features2=0x1 real memory = 2147483648 (2048 MB) avail memory = 2059546624 (1964 MB) ACPI APIC Table: MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard ... pcib2: at device 5.0 on pci0 pci2: on pcib2 ohci0: mem 0xc0000000-0xc0000fff irq 19 at device 19.0 on pci0 Activate PA 0xc0000000 at VA 0xffffff00c0000000 ohci0: [ITHREAD] usbus0: on ohci0 ohci1: mem 0xc0001000-0xc0001fff irq 19 at device 19.1 on pci0 Activate PA 0xc0001000 at VA 0xffffff00c0001000 ohci1: [ITHREAD] usbus1: on ohci1 ehci0: mem 0xc0002000-0xc0002fff irq 19 at device 19.2 on pci0 Activate PA 0xc0002000 at VA 0xffffff00c0002000 ehci0: [ITHREAD] usbus2: EHCI version 1.0 usbus2: on ehci0 ... Timecounter "TSC" frequency 1994209008 Hz quality 800 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 480Mbps High Speed USB v2.0 Status is 0x30000106 ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire Activate i/o 0x8014 Activate i/o 0x8015 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ad0: 114473MB at ata0-master UDMA100 GEOM: ad0s2: geometry does not match label (255h,63s != 16h,63s). uhub0: 4 ports with 4 removable, self powered uhub1: 4 ports with 4 removable, self powered ... Root mount waiting for: usbus2 uhub2: 8 ports with 8 removable, self powered Root mount waiting for: usbus2 Trying to mount root from ufs:/dev/ad0s2a ugen0.2: at usbus0 ubt0: on usbus0 usb_alloc_device:1635: getting device descriptor at addr 2 failed, USB_ERR_TIMEOUT! ugen2.2: at usbus2 umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 6, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): Invalidating pack g_vfs_done():da0s1[READ(offset=5298202624, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=5298268160, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=5298333696, length=65536)]error = 6 g_vfs_done():da0s1[WRITE(offset=1976320, length=32768)]error = 6 g_vfs_done():da0s1[READ(offset=5298137088, length=65536)]error = 6 (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi status == 0x0 (da0:umass-sim0:0:0:0): removing device entry ugen2.2: at usbus2 can't re-use a leaf (%desc)! can't re-use a leaf (%driver)! can't re-use a leaf (%location)! can't re-use a leaf (%pnpinfo)! can't re-use a leaf (%parent)! umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 6, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry ugen2.2: at usbus2 can't re-use a leaf (%desc)! can't re-use a leaf (%driver)! can't re-use a leaf (%location)! can't re-use a leaf (%pnpinfo)! can't re-use a leaf (%parent)! umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) ugen1.2: at usbus1 ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 1, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): Invalidating pack g_vfs_done():da0s1[READ(offset=634398720, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=634464256, length=65536)]error = 6 g_vfs_done():da0s1[READ(offset=634529792, length=65536)]error = 6 g_vfs_done():da0s1[WRITE(offset=1976320, length=32768)]error = 6 g_vfs_done():da0s1[READ(offset=634333184, length=65536)]error = 6 (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi status == 0x0 (da0:umass-sim0:0:0:0): removing device entry ugen2.2: at usbus2 can't re-use a leaf (%desc)! can't re-use a leaf (%driver)! can't re-use a leaf (%location)! can't re-use a leaf (%pnpinfo)! can't re-use a leaf (%parent)! umass0: on usbus2 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:2:0:-1: Attached to scbus2 da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) Device da0s1 went missing before all of the data could be written to it; expect data loss. ugen2.2: at usbus2 (disconnected) umass0: at uhub2, port 1, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry pcm0: mem 0xc0003400-0xc00034ff irq 17 at device 20.5 on pci0 Activate PA 0xc0003400 at VA 0xffffff00c0003400 pcm0: [ITHREAD] pcm0: ugen1.2: at usbus1 (disconnected) ugen1.2: at usbus1 ugen1.2: at usbus1 (disconnected) From hselasky at c2i.net Mon Oct 26 13:13:33 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 13:13:45 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.070117.439503022.imp@bsdimp.com> References: <200910260959.20772.hselasky@c2i.net> <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> Message-ID: <200910261412.38685.hselasky@c2i.net> On Monday 26 October 2009 14:01:17 M. Warner Losh wrote: > In message: <200910261258.08135.hselasky@c2i.net> > > Hans Petter Selasky writes: > : On Monday 26 October 2009 12:48:16 M. Warner Losh wrote: > : > I know that the august 25th version failed badly when I tried to burn > : > DVDs from a USB drive to a USB attached DVD burner. This used to work > : > flawlessly. > : > : Hi, > : > : There has been a recent fix to the EHCI driver, which might affect Mass > : Storage when short transfers are used. > : > : Also someone else has pointed out that certain VIA chipsets have an IRQ > : "bug" requiring the need for a software callout to restart the EHCI > : interrupt handler. This is not yet patched, hence I don't know if this is > : a real issue. > : > : http://svn.freebsd.org/viewvc/base?view=revision&revision=197682 > : > : Is your code from after 1st of October? > > This code is from: > > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri > Oct 23 10:08:48 MDT 2009 > imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 > > so it would have r197682 baked in (the first number in my rev string > is a mystery to me). > > Re another post: This is a 8GB flash, so I'm sure that there's enough > power. > > Looking at the dmesg, this happend the second or third time I'd > plugged in this flash drive. > > Here's a partial dmesg for usb things: > > CPU: AMD Turion(tm) 64 Mobile Technology ML-37 (1994.21-MHz K8-class CPU) > Origin = "AuthenticAMD" Id = 0x20f42 Stepping = 2 > > Features=0x78bfbff,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2> Features2=0x1 > AMD Features=0xe2500800 > AMD Features2=0x1 > real memory = 2147483648 (2048 MB) > avail memory = 2059546624 (1964 MB) > ACPI APIC Table: > MADT: Forcing active-low polarity and level trigger for SCI > ioapic0 irqs 0-23 on motherboard > ... > pcib2: at device 5.0 on pci0 > pci2: on pcib2 > ohci0: mem 0xc0000000-0xc0000fff irq 19 at > device 19.0 on pci0 Activate PA 0xc0000000 at VA 0xffffff00c0000000 > ohci0: [ITHREAD] > usbus0: on ohci0 > ohci1: mem 0xc0001000-0xc0001fff irq 19 at > device 19.1 on pci0 Activate PA 0xc0001000 at VA 0xffffff00c0001000 > ohci1: [ITHREAD] > usbus1: on ohci1 > ehci0: mem 0xc0002000-0xc0002fff irq 19 at > device 19.2 on pci0 Activate PA 0xc0002000 at VA 0xffffff00c0002000 > ehci0: [ITHREAD] > usbus2: EHCI version 1.0 > usbus2: on ehci0 > ... > Timecounter "TSC" frequency 1994209008 Hz quality 800 > Timecounters tick every 1.000 msec > usbus0: 12Mbps Full Speed USB v1.0 > usbus1: 12Mbps Full Speed USB v1.0 > usbus2: 480Mbps High Speed USB v2.0 > Status is 0x30000106 > ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire > Activate i/o 0x8014 > Activate i/o 0x8015 > ugen0.1: at usbus0 > uhub0: on usbus0 > ugen1.1: at usbus1 > uhub1: on usbus1 > ugen2.1: at usbus2 > uhub2: on usbus2 > ad0: 114473MB at ata0-master UDMA100 > GEOM: ad0s2: geometry does not match label (255h,63s != 16h,63s). > uhub0: 4 ports with 4 removable, self powered > uhub1: 4 ports with 4 removable, self powered > ... > Root mount waiting for: usbus2 > uhub2: 8 ports with 8 removable, self powered > Root mount waiting for: usbus2 > Trying to mount root from ufs:/dev/ad0s2a > ugen0.2: at usbus0 > ubt0: 2> on usbus0 usb_alloc_device:1635: getting device descriptor at addr 2 > failed, USB_ERR_TIMEOUT! ugen2.2: at usbus2 > umass0: on usbus2 > umass0: SCSI over Bulk-Only; quirks = 0x0000 > umass0:2:0:-1: Attached to scbus2 > (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 > (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error > (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition > (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 > (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have > changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) > da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 > da0: Removable Direct Access SCSI-0 device > da0: 40.000MB/s transfers > da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) > ugen2.2: at usbus2 (disconnected) > umass0: at uhub2, port 6, addr 2 (disconnected) > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): Invalidating pack > g_vfs_done():da0s1[READ(offset=5298202624, length=65536)]error = 6 > g_vfs_done():da0s1[READ(offset=5298268160, length=65536)]error = 6 > g_vfs_done():da0s1[READ(offset=5298333696, length=65536)]error = 6 > g_vfs_done():da0s1[WRITE(offset=1976320, length=32768)]error = 6 > g_vfs_done():da0s1[READ(offset=5298137088, length=65536)]error = 6 > (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi > status == 0x0 (da0:umass-sim0:0:0:0): removing device entry > ugen2.2: at usbus2 > can't re-use a leaf (%desc)! > can't re-use a leaf (%driver)! > can't re-use a leaf (%location)! > can't re-use a leaf (%pnpinfo)! > can't re-use a leaf (%parent)! > umass0: on usbus2 > umass0: SCSI over Bulk-Only; quirks = 0x0000 > umass0:2:0:-1: Attached to scbus2 > da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 > da0: Removable Direct Access SCSI-0 device > da0: 40.000MB/s transfers > da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) > ugen2.2: at usbus2 (disconnected) > umass0: at uhub2, port 6, addr 2 (disconnected) > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): removing device entry > ugen2.2: at usbus2 > can't re-use a leaf (%desc)! > can't re-use a leaf (%driver)! > can't re-use a leaf (%location)! > can't re-use a leaf (%pnpinfo)! > can't re-use a leaf (%parent)! > umass0: on usbus2 > umass0: SCSI over Bulk-Only; quirks = 0x0000 > umass0:2:0:-1: Attached to scbus2 > (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 > (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error > (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition > (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 > (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have > changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) > da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 > da0: Removable Direct Access SCSI-0 device > da0: 40.000MB/s transfers > da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) > ugen1.2: at usbus1 > ugen2.2: at usbus2 (disconnected) > umass0: at uhub2, port 1, addr 2 (disconnected) > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): Invalidating pack > g_vfs_done():da0s1[READ(offset=634398720, length=65536)]error = 6 > g_vfs_done():da0s1[READ(offset=634464256, length=65536)]error = 6 > g_vfs_done():da0s1[READ(offset=634529792, length=65536)]error = 6 > g_vfs_done():da0s1[WRITE(offset=1976320, length=32768)]error = 6 > g_vfs_done():da0s1[READ(offset=634333184, length=65536)]error = 6 > (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi > status == 0x0 (da0:umass-sim0:0:0:0): removing device entry > ugen2.2: at usbus2 > can't re-use a leaf (%desc)! > can't re-use a leaf (%driver)! > can't re-use a leaf (%location)! > can't re-use a leaf (%pnpinfo)! > can't re-use a leaf (%parent)! > umass0: on usbus2 > umass0: SCSI over Bulk-Only; quirks = 0x0000 > umass0:2:0:-1: Attached to scbus2 > da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 > da0: Removable Direct Access SCSI-0 device > da0: 40.000MB/s transfers > da0: 7660MB (15687680 512 byte sectors: 255H 63S/T 976C) > Device da0s1 went missing before all of the data could be written to it; > expect data loss. ugen2.2: at usbus2 (disconnected) > umass0: at uhub2, port 1, addr 2 (disconnected) > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): removing device entry > pcm0: mem 0xc0003400-0xc00034ff irq 17 at device 20.5 on pci0 > Activate PA 0xc0003400 at VA 0xffffff00c0003400 > pcm0: [ITHREAD] > pcm0: > ugen1.2: at usbus1 (disconnected) > ugen1.2: at usbus1 > ugen1.2: at usbus1 (disconnected) Hi, Just an experiment: usbconfig -u 2 -a 2 suspend Then try to access the device. Any difference? --HPS From des at des.no Mon Oct 26 13:26:25 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 26 13:26:38 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.070117.439503022.imp@bsdimp.com> (M. Warner Losh's message of "Mon, 26 Oct 2009 07:01:17 -0600 (MDT)") References: <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> Message-ID: <86skd6cmm8.fsf@ds4.des.no> "M. Warner Losh" writes: > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 > > so it would have r197682 baked in (the first number in my rev string > is a mystery to me). It means you have an inconsistent tree. The first number is the oldest revision in your tree, the second is the newest, and the M means you have local modifications. > Re another post: This is a 8GB flash, so I'm sure that there's enough > power. Non sequitur. Bigger chips draw more power. Is it plugged directly into the computer? If not, is it plugged into a powered hub? How many other devices are connected to the computer or hub? DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Mon Oct 26 13:26:25 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 26 13:26:38 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.070117.439503022.imp@bsdimp.com> (M. Warner Losh's message of "Mon, 26 Oct 2009 07:01:17 -0600 (MDT)") References: <200910260959.20772.hselasky@c2i.net> <20091026.054816.1631944692.imp@bsdimp.com> <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> Message-ID: <86skd6cmm8.fsf@ds4.des.no> "M. Warner Losh" writes: > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 > > so it would have r197682 baked in (the first number in my rev string > is a mystery to me). It means you have an inconsistent tree. The first number is the oldest revision in your tree, the second is the newest, and the M means you have local modifications. > Re another post: This is a 8GB flash, so I'm sure that there's enough > power. Non sequitur. Bigger chips draw more power. Is it plugged directly into the computer? If not, is it plugged into a powered hub? How many other devices are connected to the computer or hub? DES -- Dag-Erling Sm?rgrav - des@des.no From imp at bsdimp.com Mon Oct 26 13:42:26 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 13:42:39 2009 Subject: Help troubleshooting... In-Reply-To: <86skd6cmm8.fsf@ds4.des.no> References: <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> <86skd6cmm8.fsf@ds4.des.no> Message-ID: <20091026.073759.2130803790.imp@bsdimp.com> In message: <86skd6cmm8.fsf@ds4.des.no> Dag-Erling_Sm?rgrav writes: : "M. Warner Losh" writes: : > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 : > : > so it would have r197682 baked in (the first number in my rev string : > is a mystery to me). : : It means you have an inconsistent tree. The first number is the oldest : revision in your tree, the second is the newest, and the M means you : have local modifications. Yes. Of course I have local modifications, but none in the usb stack. But I've also done a svn update from the top of the tree multiple times and this version number persists. : > Re another post: This is a 8GB flash, so I'm sure that there's enough : > power. : : Non sequitur. Bigger chips draw more power. Is it plugged directly : into the computer? If not, is it plugged into a powered hub? How many : other devices are connected to the computer or hub? Not entirely. This flash has worked in this computer in the past without issues (like a year ago when we were first integrating hpsusb into the tree). This flash is plugged directly into the computer. This behavior is consistent across multiple ports on the computer (so it isn't a bad port). While this doesn't prove it isn't a power issue, the odds are stacked against it being one. If there were a way to get the internal hub to tell me how much power it can deliver, and for me to query the flash to see maximum current draws, we could see if we're close to the edge or not... Warner From imp at bsdimp.com Mon Oct 26 13:42:26 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 13:42:39 2009 Subject: Help troubleshooting... In-Reply-To: <86skd6cmm8.fsf@ds4.des.no> References: <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> <86skd6cmm8.fsf@ds4.des.no> Message-ID: <20091026.073759.2130803790.imp@bsdimp.com> In message: <86skd6cmm8.fsf@ds4.des.no> Dag-Erling_Sm?rgrav writes: : "M. Warner Losh" writes: : > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: Fri Oct 23 10:08:48 MDT 2009 imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 : > : > so it would have r197682 baked in (the first number in my rev string : > is a mystery to me). : : It means you have an inconsistent tree. The first number is the oldest : revision in your tree, the second is the newest, and the M means you : have local modifications. Yes. Of course I have local modifications, but none in the usb stack. But I've also done a svn update from the top of the tree multiple times and this version number persists. : > Re another post: This is a 8GB flash, so I'm sure that there's enough : > power. : : Non sequitur. Bigger chips draw more power. Is it plugged directly : into the computer? If not, is it plugged into a powered hub? How many : other devices are connected to the computer or hub? Not entirely. This flash has worked in this computer in the past without issues (like a year ago when we were first integrating hpsusb into the tree). This flash is plugged directly into the computer. This behavior is consistent across multiple ports on the computer (so it isn't a bad port). While this doesn't prove it isn't a power issue, the odds are stacked against it being one. If there were a way to get the internal hub to tell me how much power it can deliver, and for me to query the flash to see maximum current draws, we could see if we're close to the edge or not... Warner From des at des.no Mon Oct 26 14:01:20 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 26 14:01:35 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.073759.2130803790.imp@bsdimp.com> (M. Warner Losh's message of "Mon, 26 Oct 2009 07:37:59 -0600 (MDT)") References: <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> <86skd6cmm8.fsf@ds4.des.no> <20091026.073759.2130803790.imp@bsdimp.com> Message-ID: <86aazecl00.fsf@ds4.des.no> "M. Warner Losh" writes: > Yes. Of course I have local modifications, but none in the usb stack. > But I've also done a svn update from the top of the tree multiple > times and this version number persists. Weird. r185338 was a commit to the old USB stack. Try to run svnversion in sys/dev/usb, and in an unrelated directory such as sys/conf. If there's a difference, run svn stat in sys/dev/usb and see if anything unexpected shows up. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Mon Oct 26 14:01:20 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Oct 26 14:01:36 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.073759.2130803790.imp@bsdimp.com> (M. Warner Losh's message of "Mon, 26 Oct 2009 07:37:59 -0600 (MDT)") References: <200910261258.08135.hselasky@c2i.net> <20091026.070117.439503022.imp@bsdimp.com> <86skd6cmm8.fsf@ds4.des.no> <20091026.073759.2130803790.imp@bsdimp.com> Message-ID: <86aazecl00.fsf@ds4.des.no> "M. Warner Losh" writes: > Yes. Of course I have local modifications, but none in the usb stack. > But I've also done a svn update from the top of the tree multiple > times and this version number persists. Weird. r185338 was a commit to the old USB stack. Try to run svnversion in sys/dev/usb, and in an unrelated directory such as sys/conf. If there's a difference, run svn stat in sys/dev/usb and see if anything unexpected shows up. DES -- Dag-Erling Sm?rgrav - des@des.no From hselasky at c2i.net Mon Oct 26 14:04:28 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Oct 26 14:04:36 2009 Subject: Help troubleshooting... In-Reply-To: <20091026.073759.2130803790.imp@bsdimp.com> References: <200910261258.08135.hselasky@c2i.net> <86skd6cmm8.fsf@ds4.des.no> <20091026.073759.2130803790.imp@bsdimp.com> Message-ID: <200910261503.33747.hselasky@c2i.net> On Monday 26 October 2009 14:37:59 M. Warner Losh wrote: > In message: <86skd6cmm8.fsf@ds4.des.no> > > Dag-Erling_Sm?rgrav writes: > : "M. Warner Losh" writes: > : > FreeBSD lighthouse 9.0-CURRENT FreeBSD 9.0-CURRENT #41 r185338:198411M: > : > Fri Oct 23 10:08:48 MDT 2009 > : > imp@lighthouse:/cache/svn/head/sys/amd64/compile/LIGHTHOUSE amd64 > : > > : > so it would have r197682 baked in (the first number in my rev string > : > is a mystery to me). > : > : It means you have an inconsistent tree. The first number is the oldest > : revision in your tree, the second is the newest, and the M means you > : have local modifications. > > Yes. Of course I have local modifications, but none in the usb stack. > But I've also done a svn update from the top of the tree multiple > times and this version number persists. > > : > Re another post: This is a 8GB flash, so I'm sure that there's enough > : > power. > : > : Non sequitur. Bigger chips draw more power. Is it plugged directly > : into the computer? If not, is it plugged into a powered hub? How many > : other devices are connected to the computer or hub? > Hi, > Not entirely. This flash has worked in this computer in the past > without issues (like a year ago when we were first integrating hpsusb > into the tree). Since then there has been at least one patch to improve performance in the EHCI driver. When the cat command stops, could you try to run: usbconfig -u XXX -a YYY dump_device_desc dump_curr_config_desc On that device. Is usbconfig able to extract the string descriptors in the device and config descriptor? Or do you get timeouts? Also check vmstat -i . > This flash is plugged directly into the computer. > This behavior is consistent across multiple ports on the computer (so > it isn't a bad port). While this doesn't prove it isn't a power > issue, the odds are stacked against it being one. If there were a way > to get the internal hub to tell me how much power it can deliver, and > for me to query the flash to see maximum current draws, we could see > if we're close to the edge or not... Usually the maximum current is given by the device descriptor, but it might now be the actual value. See usbconfig dump_device_desc. --HPS From imp at bsdimp.com Mon Oct 26 14:45:37 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 14:45:44 2009 Subject: Help troubleshooting... In-Reply-To: <86aazecl00.fsf@ds4.des.no> References: <86skd6cmm8.fsf@ds4.des.no> <20091026.073759.2130803790.imp@bsdimp.com> <86aazecl00.fsf@ds4.des.no> Message-ID: <20091026.084316.35218995.imp@bsdimp.com> In message: <86aazecl00.fsf@ds4.des.no> Dag-Erling_Sm?rgrav writes: : "M. Warner Losh" writes: : > Yes. Of course I have local modifications, but none in the usb stack. : > But I've also done a svn update from the top of the tree multiple : > times and this version number persists. : : Weird. r185338 was a commit to the old USB stack. Try to run : svnversion in sys/dev/usb, and in an unrelated directory such as : sys/conf. If there's a difference, run svn stat in sys/dev/usb and see : if anything unexpected shows up. % cd sys/dev/usb % svnversion 198411M % cd ../../conf % svnversion 198411M Some more digging down this line shows that there was a dangling merge conflict in sys/i386/conf/GENERIC Warner From imp at bsdimp.com Mon Oct 26 14:45:37 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Oct 26 14:45:55 2009 Subject: Help troubleshooting... In-Reply-To: <86aazecl00.fsf@ds4.des.no> References: <86skd6cmm8.fsf@ds4.des.no> <20091026.073759.2130803790.imp@bsdimp.com> <86aazecl00.fsf@ds4.des.no> Message-ID: <20091026.084316.35218995.imp@bsdimp.com> In message: <86aazecl00.fsf@ds4.des.no> Dag-Erling_Sm?rgrav writes: : "M. Warner Losh" writes: : > Yes. Of course I have local modifications, but none in the usb stack. : > But I've also done a svn update from the top of the tree multiple : > times and this version number persists. : : Weird. r185338 was a commit to the old USB stack. Try to run : svnversion in sys/dev/usb, and in an unrelated directory such as : sys/conf. If there's a difference, run svn stat in sys/dev/usb and see : if anything unexpected shows up. % cd sys/dev/usb % svnversion 198411M % cd ../../conf % svnversion 198411M Some more digging down this line shows that there was a dangling merge conflict in sys/i386/conf/GENERIC Warner From uqs at spoerlein.net Mon Oct 26 20:09:50 2009 From: uqs at spoerlein.net (Ulrich =?utf-8?B?U3DDtnJsZWlu?=) Date: Mon Oct 26 20:09:56 2009 Subject: in_cksum.h for sparc64 missing ifdef KERNEL? Message-ID: <20091026200935.GA13602@roadrunner.spoerlein.net> Hi, while trying to cleanup some WARNS issues under sbin/, I noticed that natd(8) fails to compile for sparc64 only, due to missing "struct mbuf" declaration in in_cksum.h. Comparing that header to other arch's headers leads me to believe an #ifdef is missing. See attached patch, but please note that I do not have the resources to run this through a make universe right now. Regards, Uli -------------- next part -------------- A non-text attachment was scrubbed... Name: cksum.diff Type: text/x-diff Size: 478 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20091026/ed140cfe/cksum.bin From olivermahmoudi at gmail.com Mon Oct 26 20:36:08 2009 From: olivermahmoudi at gmail.com (Oliver Mahmoudi) Date: Mon Oct 26 20:36:15 2009 Subject: writing a FreeBSD C library Message-ID: <6b4b2d2c0910261308i367569dbg887d7c713bf20ad1@mail.gmail.com> Hello all y'all Kernel Hackers, Trying to get a deeper understanding of the FreeBSD kernel I am currently I am studying C library files. For this reason I wrote a very simple library, just to understand how it works. Below are my source codes for: 1. the header 2. the library file 3. a simple C program with which I intend to test my library function ############## start of simple library header ######################## #ifndef _s_lib_h_ #define _s_lib_h_ #define SOME_INT 0x0001 /* prototype */ void myprnf(char *); #endif /* _s_lib_h_ */ ##################### end of header ############################ ################# start of library file lb.c ########################## #include #include "slib.h" static void _myprf(char *); void myprnf(char *farg) { char *narg = farg; _myprf(narg); } static void _myprf(char *sarg) { char *pstr = sarg; printf("%s\n", pstr); } ############## end of library file ############################### ################ start of source file ############################ #include "slib.h" int main() { char *somestr = "hello world"; myprnf(somestr); return(1-1); } ################# end of source file ########################### I compiled the library file in the following way. % gcc -I../include -Wall -c lb.c % ar rsv mylib.a lb.o The compilation finished with no warnings so I assume that there are no errors in the library itself. Trying to compile my source file - let's call it source.c - I always get the following error message: % gcc -o testfile source.c /var/tmp//ccQoff1S.o(.text+0x19): In function `main': : undefined reference to `myprintf' % In other words, gcc doesn't seem to find my library. I tried to mv my library file in /lib, /libexec /usr/lib, /usr/libdata and /usr/libexec but none of the above directories would directory would let me compile my source file. Now I am wondering where do I need to put mylib.a file to succeed? Also, this is a static library. What do I need to do under FreeBSD to compile this library into a dynamic library and where would I put this file? Lastly, most if not all the system calls can be found under /sys/kern/*.c. When writing a FreeBSD C program gcc makes use of the corresponding headers, e.g. unistd.h. and the standard library libc. Is it possible to peep a look at the source file for libc, i.e. is it included in the source tree? Where? Thanks Oliver There are many many more kernel related questions on my mind but I think that this is enought for one email. From f.loeber at googlemail.com Mon Oct 26 21:04:48 2009 From: f.loeber at googlemail.com (Florian Loeber) Date: Mon Oct 26 21:04:55 2009 Subject: writing a FreeBSD C library In-Reply-To: <6b4b2d2c0910261308i367569dbg887d7c713bf20ad1@mail.gmail.com> References: <6b4b2d2c0910261308i367569dbg887d7c713bf20ad1@mail.gmail.com> Message-ID: Hi, you have to link your executable to your library. The command-line option is -l. % gcc -o testfile -lmylib source.c Without it, your program doesn't know that this library exists (somewhere, /usr/lib, ...) Regards, Florian From gabor at FreeBSD.org Mon Oct 26 21:32:07 2009 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Mon Oct 26 21:32:13 2009 Subject: writing a FreeBSD C library In-Reply-To: <6b4b2d2c0910261308i367569dbg887d7c713bf20ad1@mail.gmail.com> References: <6b4b2d2c0910261308i367569dbg887d7c713bf20ad1@mail.gmail.com> Message-ID: <4AE60F70.9070808@FreeBSD.org> Oliver Mahmoudi escribi?: > I compiled the library file in the following way. > > % gcc -I../include -Wall -c lb.c > % ar rsv mylib.a lb.o > You can study bsd.lib.mk and bsd.prog.mk in /usr/share/mk. With these two includes you can deal easily with your C programs/libraries. It will serve you very well later. > The compilation finished with no warnings so I assume that there are no > errors in the library itself. > > Trying to compile my source file - let's call it source.c - I always get the > following error message: > > % gcc -o testfile source.c > /var/tmp//ccQoff1S.o(.text+0x19): In function `main': > : undefined reference to `myprintf' > % > > That's easy, your library isn't linked to your program. First, compile the source file but _do not link_: gcc -c source.c This will result in a source.o file. Then link the object files (source.o and the static library) with ld: ld -o testfile source.o mylib.a > In other words, gcc doesn't seem to find my library. I tried to mv my > library file in /lib, /libexec > /usr/lib, /usr/libdata and /usr/libexec but none of the above directories > would directory would > let me compile my source file. > > With static libraries you can just specify the full path when linking. > Now I am wondering where do I need to put mylib.a file to succeed? > > Also, this is a static library. What do I need to do under FreeBSD to > compile this library into a dynamic > library and where would I put this file? > Just using the proper parameters when compiling the library. It is a Linux article but you'll find some explications here: http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html Again, the bsd.*.mk Makefile includes are pretty handful. > Lastly, most if not all the system calls can be found under /sys/kern/*.c. > When writing a FreeBSD C program > gcc makes use of the corresponding headers, e.g. unistd.h. and the standard > library libc. Is it possible > to peep a look at the source file for libc, i.e. is it included in the > source tree? Where? > In /usr/src/lib/libc provided you have the source code installed in /usr/src. > There are many many more kernel related questions on my mind but I think > that this is enought for one > email. > I don't want to desperate you but these are very basic things. You should get a deeper knowledge of the basics first, before you try to do some kernel-related things. Otherwise, you will find it way too difficult and you won't enjoy. Anyway, the book of Marshall K. McKusick and George V. Neville-Neil is a good source of learning kernel stuff: http://www.amazon.com/Design-Implementation-FreeBSD-Operating-System/dp/0201702452 Regards, -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From gonzo at bluezbox.com Tue Oct 27 00:26:31 2009 From: gonzo at bluezbox.com (Oleksandr Tymoshenko) Date: Tue Oct 27 00:26:39 2009 Subject: MIPS: bus_dma(9) and cache problems Message-ID: <4AE63A64.7010205@bluezbox.com> This problem haunts for a couple of days and I can't find a nice and clean solution so this email is actually a cry for help. The problem: There is a buffer loaded by bus_dmamap_load for use as a DMA buffer. Right before this buffer resides block of vital data structure. Consider following scenario: 1. code modifies data in block and this modification ends up in cache and is not written back to memory 2. right after this code calls bus_dmamap_sync for this buffer and as a result cache invalidation is performed 3. Cache function operates on cache line size-aligned addresses and the block in question happens to share the same cache line with the buffer. So modification made at step (1) is lost. If busdma code controls allocation (bus_dmamem_alloc) this situation can be avoided by forcing pointer alignment. But when address come from the "outer space" via bus_dmamap_load* there is not much to do. There are two solutions I've figured so far: - Create bounce page for not properly aligned memory. Which would reduce performance a lot. - Remap buffer's page(s) as uncached. I haven't succeeded with this one yet and not sure it's always possible. May be someone can suggest better way to clean this mess? Thanks! -- gonzo From =?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?= Tue Oct 27 03:09:47 2009 From: =?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?= (=?utf-8?b?4LmE4Lie4Lij4Lix4LiKIA==?=) Date: Tue Oct 27 03:09:54 2009 Subject: make release stop at Creating ISO imagess In-Reply-To: <20091026095317.GA72492@ei.bzerk.org> References: <20091026143639.13895c34kml7gmmf@webmail.tint.or.th> <20091026095317.GA72492@ei.bzerk.org> Message-ID: <20091027094315.13736x5smfzo4g1v@webmail.tint.or.th> Quoting Ruben de Groot : > On Mon, Oct 26, 2009 at 02:36:39PM +0700, ??????????????? > ????????????????????? typed: >> hi sirs, >> >> apologized me for disturbing this list but ireally have problem s. >> i make my own release by follwoing document in releng articles. >> >> here is my command >> >> cd /usr/src/release >> time make CHROOTDIR=/kaitag/KAITAG BUILDNAME=7.2-RELEASE \ >> CVSROOT=/var/ftp/pub/ncvs RELEASETAG=RELENG_7_2_0_RELEASE \ >> EXTSRCDIR=/usr/src EXTPORTSDIR=/usr/ports \ >> DOC_LANG=en_US.ISO8859-1 NODOC=NO NOPORTS=NO \ >> MAKE_DVD=YES MAKE_ISOS=YES CDROM=YES >> RELEASEDISTFILES=/var/ftp/pub/distfiles \ >> release >> >> the build process goes well but then fetching for cdrtools.tbz begin and >> stop. >> in my machine i have installed cdrtools and there is also mkisofs file >> so i wonder why the script /usr/src/release/i386/mkisofsimages.sh >> still fetching for cdrtools. > > Do you have the cdrtools.tbz package in /var/ftp/pub/distfiles ? > > Ruben > > i copy cdrtools.tbz to /var/ftp/pub/distfiles while answering your first mail and let the process ran overnight here is a result, from screen shot == begin of last page screen shot touch cdrom.1 Building CDROM disc1 filesystem image 0 blocks 0 blocks Building CDROM disc2 filesystem image Building CDROM disc3 filesystem image 0 blocks 0 blocks touch cdrom.2 Building bootonly CDROM filesystem image touch cdrom.3 Creating ISO images... The cdrtools port is not installed. Trying to get it now. Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Lates t/cdrtools.tbz...pkg_add: warning: error reading from server: Operation timed ou t ^CSignal 2 received, cleaning up.. Could not get it via pkg_add - please go install this from the ports collection and run this script again. 6240.165u 575.390s 20:45:53.90 9.1% -259+1079k 167010+14463io 87062pf+0w maifa [/usr/src/release] # == end of screen shot after interrupt the process i make ... rerelease once again and now it stuck at the same point == begin of screen shot + export RELEASETAG=RELENG_7_2_0_RELEASE + export RELNOTES_LANG=en_US.ISO8859-1 + export SEPARATE_LIVEFS= + export TARGET=i386 + export TARGET_ARCH=i386 + export RELEASEDIR=/R + export PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin + export MANBUILDCAT=YES + umount /dev + true + mount -t devfs devfs /dev + [ ! -c /dev/null ] + [ -x /etc/rc.d/ldconfig ] + /etc/rc.d/ldconfig start ELF ldconfig path: /lib /usr/lib /usr/lib/compat a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout + [ ! -f /tmp/.world_done ] + [ ! -f /tmp/.skip_ports_index ] + cd /usr/src/release + make obj + make doRELEASE Creating ISO images... The cdrtools port is not installed. Trying to get it now. Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Lates t/cdrtools.tbz... == end of screen shot, currently running process i do not know what happend to the sh script mention ealier. anyway thanks indeed for your time. with best regards -- psr ????? ????????? ???? ?????????? http://makham.blogspot.com ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From linda.messerschmidt at gmail.com Tue Oct 27 17:13:24 2009 From: linda.messerschmidt at gmail.com (Linda Messerschmidt) Date: Tue Oct 27 17:13:30 2009 Subject: FreeBSD 7.2 + NFS + nullfs + unlink + fstat = Stale NFS File Handle Message-ID: <237c27100910271013s1d8602d0l74d7d9d2c137adee@mail.gmail.com> We have encountered a problem with a weird behavior when NFS and nullfs are combined and a program creates, unlinks, and then fstats a file in the resulting directory. After encountering this problem in the wild, I wrote a quick little C program. It creates a file, unlinks the file, and then fstat's the open file descriptor. The results: UFS: OK NFS: OK UFS+NULLFS: OK NFS+NULLFS: fstat returns ESTALE The UFS test is just run in /tmp. All others are run in /mnt (with umounts in between). The NFS setup looks like this: client# mount -o tcp server:/export/example /mnt The UFS+NULLFS setup looks like this: client# mount -t nullfs /tmp /mnt The NFS+NULLFS mount setup looks like this: client# mount -o tcp server:/export /export client# mount -t nullfs /export/example /mnt As far as I understand, this behavior should be supported, and the file should be "finally" deleted when the descriptor is closed. Even so, this is obscure enough that the response would be "don't do that" except that the open-unlink-fstat behavior was encountered with /usr/bin/vi, which does exactly that on its temporary files. So, we either fix it or retool everything not to use nullfs. Does anyone know what the likely source of this different behavior is, and whether it is feasible to address? Or is NFS+NULLFS just pushing the envelope a little too far? This is on 7.2-RELEASE-p1 and 7.2-STABLE. Thanks for any advice! Test program source below: #include #include #include #include #include #include int main() { int fd; struct stat st; fd = open("testfile", O_RDWR | O_CREAT | O_TRUNC, 0666); if (!fd) { fprintf(stderr, "open failed: %s\n",strerror(errno)); return 10; } if (unlink("testfile")) { fprintf(stderr, "unlink failed: %s\n",strerror(errno)); return 20; } if (fstat(fd, &st)) { fprintf(stderr, "fstat failed: %s\n",strerror(errno)); return 30; } close(fd); return 0; } From dclark at engr.scu.edu Wed Oct 28 01:16:37 2009 From: dclark at engr.scu.edu (Dorr H. Clark) Date: Wed Oct 28 01:45:07 2009 Subject: ptrace problem 6.x/7.x - can someone explain this? In-Reply-To: Message-ID: We believe ptrace has a problem in 6.3; we have not tried other releases. The same code, however, exists in 7.1. The bug was first encountered in gdb... (gdb) det Detaching from program: /usr/local/bin/emacs, process 66217 (gdb) att 66224 Attaching to program: /usr/local/bin/emacs, process 66224 Error accessing memory address 0x281ba5a4: Device busy. (gdb) det Detaching from program: /usr/local/bin/emacs, process 66224 ptrace: Device busy. (gdb) quit <--- target process 66224 dies here To isolate this problem, a wrote a simple minded test program was written that just attached and detached. This test program found even the very first detach fails with EBUSY (see test source below): $ ./test1 -p 66217 -c 1 -d 10 pid 66217 count 1 delay 10 Start of pass 0 Calling PT_ATTACH pid 66217 addr 0x0 sig 0 Calling PT_DETACH pid 66217 addr 0xffffffff sig 0 Call 0 to PT_DETACH returned -1, errno 16 Once again, the target process died when the ptracing test program exitted, as would be expected if a detach had failed. The failure return was coming from the following test in kern_ptrace() in sys_process.c /* not currently stopped */ if ((p->p_flag & (P_STOPPED_SIG | P_STOPPED_TRACE)) == 0 || p->p_suspcount != p->p_numthreads || (p->p_flag & P_WAITED) == 0) { error = EBUSY; goto fail; } This is applied to all operations except PT_TRACE_ME, PT_ATTACH, and some instances of PT_CLEAR_STEP. P_WAITED is generally not true. In particular, it's not set automatically when a process is PT_ATTACHed. It is cleared by PT_DETACH and again when ptrace sends a signal (PT_CONTINUE, PT_DETACH.) _But_ it's set in only two places, and they aren't in ptrace code. 2 sys/kern/kern_exit.c kern_wait 773 p->p_flag |= P_WAITED; 3 compat/svr4/svr4_misc.c svr4_sys_waitsys 1351 q->p_flag |= P_WAITED; The relevant one is the first one, primarily. Here's the code: mtx_lock_spin(&sched_lock); if ((p->p_flag & P_STOPPED_SIG) && (p->p_suspcount == p->p_numthreads) && (p->p_flag & P_WAITED) == 0 && (p->p_flag & P_TRACED || options & WUNTRACED)) { mtx_unlock_spin(&sched_lock); p->p_flag |= P_WAITED; sx_xunlock(&proctree_lock); td->td_retval[0] = p->p_pid; if (status) *status = W_STOPCODE(p->p_xstat); PROC_UNLOCK(p); return (0); } mtx_unlock_spin(&sched_lock); So it's only set on processes which are already traced. But it's not set until someone calls wait4() on them - or the equivalent sysV compatability routine. Gdb doesn't always wait4() for processes immediately opon tracing them, and the ptrace man page does not imply this is needed. Moreover, it's not clear why it should matter. The process needs to be stopped in order for it to make sense to do most of the things ptrace does. But - why should it need to be waited for? And what kind of sense does this make to someone writing a debugging tool, where the natural logic seems to be: - attach to process - look at some stuff - stick in some kind of breakpoint or similar and start it going again (or 'step' it) - wait for it to stop - look at and modify stuff - detach, or set it moving again By way of experiment, the test for P_WAITED was removed. Gdb no longer had problems, and no new issues with gdb were encountered (although this was just interactive, no "gdb coverage test" was attempted). The test program also stopped having issues. /* not currently stopped */ if ((p->p_flag & (P_STOPPED_SIG | P_STOPPED_TRACE)) == 0 || p->p_suspcount != p->p_numthreads { error = EBUSY; goto fail; } So does anyone know whether it's safe to simply remove that test? Thanks, Arlie Stephens Engineer Dorr H. Clark Advisor Graduate School of Engineering Santa Clara University, Santa Clara, CA --------------------------------------------------------------------- Test program here --------------------------------------------------------------------- /* * experiment with ptrace, try to see which is broken - gdb or ptrace */ #include #include #include #include #include #include void usage(void) { printf("Simple program to play with ptrace\n"); printf("Usage: test1 -p -c -d \n"); printf("Specify -n for no explicit detach\n"); printf("Will attach and detach repeatedly from target process\n"); exit(1); } int main(int argc, char *argv[]) { pid_t pid = -1; int count = 2; int delay = 5; int nodetach = 0; int opt; int ret; int i; while((opt = getopt(argc, argv, ":p:c:d:n")) != -1) { switch(opt) { case 'c': if (sscanf(optarg, "%d", &count) != 1) { printf("Count should be numeric\n"); usage(); } break; case 'd': if (sscanf(optarg, "%d", &delay) != 1) { printf("Delay should be numeric\n"); usage(); } break; case 'n': nodetach = 1; break; case 'p': if (sscanf(optarg, "%d", &pid) != 1) { printf("Pid should be numeric\n"); usage(); } break; default: printf("Illegal option -%c\n", opt); usage(); break; } } printf("pid %d count %d delay %d\n", pid, count, delay); if (pid == -1) { printf("Pid must be specified\n"); usage(); } if (count <= 0) { printf("Count must be positive\n"); usage(); } if (delay < 0) { printf("Delay must not be negative\n"); usage(); } for (i = 0; i < count; i++) { printf("Start of pass %d\n", i); errno = 0; printf("Calling PT_ATTACH pid %d addr 0x%lx sig %d\n", pid, (unsigned long)(caddr_t)NULL, 0); ret = ptrace(PT_ATTACH, pid, NULL, 0); if (ret != 0) { printf("Call %d to PT_ATTACH returned %d, errno %d\n", i, ret, errno); } sleep(delay); if (!nodetach) { errno = 0; printf("Calling PT_DETACH pid %d addr 0x%lx sig %d\n", pid, (unsigned long)(caddr_t)-1, 0); ret = ptrace(PT_DETACH, pid, (caddr_t)-1, 0); if (ret != 0) { printf("Call %d to PT_DETACH returned %d, " "errno %d\n", i, ret, errno); } } sleep(delay); } return 0; } From simon at comsys.ntu-kpi.kiev.ua Wed Oct 28 12:48:47 2009 From: simon at comsys.ntu-kpi.kiev.ua (Andrey Simonenko) Date: Wed Oct 28 12:48:54 2009 Subject: FreeBSD 7.2 + NFS + nullfs + unlink + fstat = Stale NFS File Handle In-Reply-To: <237c27100910271013s1d8602d0l74d7d9d2c137adee@mail.gmail.com> References: <237c27100910271013s1d8602d0l74d7d9d2c137adee@mail.gmail.com> Message-ID: <20091028124832.GA7852@pm513-1.comsys.ntu-kpi.kiev.ua> On Tue, Oct 27, 2009 at 01:13:18PM -0400, Linda Messerschmidt wrote: > > Does anyone know what the likely source of this different behavior is, > and whether it is feasible to address? Or is NFS+NULLFS just pushing > the envelope a little too far? As I understand when a file is opened in NULLFS its vnode gets new reference on 'count of users', but this new reference is not propagated to the lower vnode (vnode that is under NULLFS). When a file is removed NULLFS passes this op to the lower FS (NFS in this example) and that FS sees that its vnode has only a single reference on 'count of users'. In case of NFS when there is a request to remove a vnode it checks that value of 'count of users' for this vnode. If this count is equal to 1, then NFS client code does 'RPC remove'. If this count is greater than 1 (for example when a file is opened), then NFS client code renames pathname to .nfs-file, but does not send 'RPC remove' to the NFS server. > > fd = open("testfile", O_RDWR | O_CREAT | O_TRUNC, 0666); > if (!fd) { ^^^^^ should be (fd < 0) From linda.messerschmidt at gmail.com Wed Oct 28 13:48:49 2009 From: linda.messerschmidt at gmail.com (Linda Messerschmidt) Date: Wed Oct 28 13:48:57 2009 Subject: FreeBSD 7.2 + NFS + nullfs + unlink + fstat = Stale NFS File Handle In-Reply-To: <20091028124832.GA7852@pm513-1.comsys.ntu-kpi.kiev.ua> References: <237c27100910271013s1d8602d0l74d7d9d2c137adee@mail.gmail.com> <20091028124832.GA7852@pm513-1.comsys.ntu-kpi.kiev.ua> Message-ID: <237c27100910280648k597e2159sf4fd663fe7b77b3b@mail.gmail.com> On Wed, Oct 28, 2009 at 8:48 AM, Andrey Simonenko wrote: > As I understand when a file is opened in NULLFS its vnode gets new > reference on 'count of users', but this new reference is not propagated > to the lower vnode (vnode that is under NULLFS). ?When a file is removed > NULLFS passes this op to the lower FS (NFS in this example) and that > FS sees that its vnode has only a single reference on 'count of users'. > > In case of NFS when there is a request to remove a vnode it checks that > value of 'count of users' for this vnode. ?If this count is equal to 1, > then NFS client code does 'RPC remove'. ?If this count is greater than 1 > (for example when a file is opened), then NFS client code renames pathname > to .nfs-file, but does not send 'RPC remove' to the NFS server. That sounds like a pretty reasonable explanation of what's going on. Unfortunately it does sound like this would be tough to fix. Since NFS deletes are a special case, short of making an NFS-aware nullfs, which seems silly, it sounds like the "solution" would be rewriting nullfs to propagate the reference count. I don't know enough about nullfs to know exactly how hard that would be, but I'm guessing it's not the work of a lazy afternoon. :-) >> ? ? ? if (!fd) { > ? ? ? ? ? ^^^^^ should be (fd < 0) Oops, you are right! Thanks very much! From gleb.kurtsou at gmail.com Wed Oct 28 17:39:58 2009 From: gleb.kurtsou at gmail.com (Gleb Kurtsou) Date: Wed Oct 28 17:40:05 2009 Subject: FreeBSD 7.2 + NFS + nullfs + unlink + fstat = Stale NFS File Handle In-Reply-To: <237c27100910280648k597e2159sf4fd663fe7b77b3b@mail.gmail.com> References: <237c27100910271013s1d8602d0l74d7d9d2c137adee@mail.gmail.com> <20091028124832.GA7852@pm513-1.comsys.ntu-kpi.kiev.ua> <237c27100910280648k597e2159sf4fd663fe7b77b3b@mail.gmail.com> Message-ID: <20091028171550.GA1909@tops> On (28/10/2009 09:48), Linda Messerschmidt wrote: > On Wed, Oct 28, 2009 at 8:48 AM, Andrey Simonenko > wrote: > > As I understand when a file is opened in NULLFS its vnode gets new > > reference on 'count of users', but this new reference is not propagated > > to the lower vnode (vnode that is under NULLFS). ?When a file is removed > > NULLFS passes this op to the lower FS (NFS in this example) and that > > FS sees that its vnode has only a single reference on 'count of users'. > > > > In case of NFS when there is a request to remove a vnode it checks that > > value of 'count of users' for this vnode. ?If this count is equal to 1, > > then NFS client code does 'RPC remove'. ?If this count is greater than 1 > > (for example when a file is opened), then NFS client code renames pathname > > to .nfs-file, but does not send 'RPC remove' to the NFS server. > > That sounds like a pretty reasonable explanation of what's going on. > > Unfortunately it does sound like this would be tough to fix. Since > NFS deletes are a special case, short of making an NFS-aware nullfs, > which seems silly, it sounds like the "solution" would be rewriting > nullfs to propagate the reference count. I don't know enough about > nullfs to know exactly how hard that would be, but I'm guessing it's > not the work of a lazy afternoon. :-) I think that's not about nullfs. Nullfs behaves correctly. It grabs reference for a vnode and even tries too release it as soon as possible. NFS logic is wrong here, imho. vrefcnt(vp) == 1 supposed to mean that vnode is going to be reclaimed on next reference release and nothing more. And it doesn't mean that reference is going to be released any time soon. Although network filesystems in the tree abuse it the same way NFS does. Probably what NFS tries to do is to check if file descriptor is opened for a vnode. This assumption holds only for a single code path out of many: syscall by user. But there is plenty of code invoking vnode operations without even allocating file descriptor. Propagating per file descriptor reference counting into filesystem itself is a layer violation and should be avoided in my opinion. There should be a way to fix NFS by replacing vrefcnt check. > >> ? ? ? if (!fd) { > > ? ? ? ? ? ^^^^^ should be (fd < 0) > > Oops, you are right! > > Thanks very much! > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From yuri at rawbw.com Wed Oct 28 19:46:21 2009 From: yuri at rawbw.com (Yuri) Date: Wed Oct 28 19:46:29 2009 Subject: USB floppy fails to attach Message-ID: <4AE89F8C.60000@rawbw.com> My USB floppy dive fails to attach to device (see debug log below). VendorID=0x0409 ProductID=0x0040 not mentioned in /usr/src/sys/dev/usb/storage/umass.c Can this be that some that there is a simple fix for this, like adding a quirk? Yuri -----log------- ugen1.2: at usbus1 umass0: on usbus1 umass0: UFI over CBI with CCI; quirks = 0x0000 umass0:umass_cam_action: 4:-1:-1:XPT_PATH_INQ:. umass0:4:0:-1: Attached to scbus4 umass0:umass_cam_rescan: scbus4: scanning for 4:0:-1 umass0:umass_cam_action: 4:-1:-1:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_SET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x12, flags: 0x40, 6b cmd/36b data/18b sense umass0:umass_attach: Attach finishedumass0:umass_cbi_dump_cmd: cmd = 12b (0x120000002400...), data = 36b, dir = in umass0:umass_transfer_start: transfer index = 4 umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=36 umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=0 umass0:umass_transfer_start: transfer index = 8 umass0:umass_t_cbi_status_callback: UFI CCI, ASC = 0x00, ASCQ = 0x00 umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_SET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x12, flags: 0x40, 6b cmd/255b data/18b sense umass0:umass_cbi_dump_cmd: cmd = 12b (0x12010000ff00...), data = 255b, dir = in umass0:umass_transfer_start: transfer index = 4 umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=255 umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=0 umass0:umass_transfer_start: transfer index = 8 umass0:umass_t_cbi_status_callback: UFI CCI, ASC = 0x00, ASCQ = 0x00 umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_SET_TRAN_SETTINGS:. umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_cbi_dump_cmd: cmd = 12b (0x000000000000...), data = 0b, dir = no data phase umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b cmd/0b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umassX:umass_cam_rescan_callback: xpt0: Rescan succeeded umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset (da0:umass-sim0:0:0:0): got CAM status 0x4 (da0:umass-sim0:0:0:0): fatal error, failed to attach to device (da0:umass-sim0:0:0:0): lost device umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b cmd/8b data/32b sense umass0:umass_t_cbi_reset1_callback: CBI reset! umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset (da0:umass-sim0:0:0:0): removing device entry From hselasky at c2i.net Wed Oct 28 23:52:24 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Wed Oct 28 23:52:31 2009 Subject: USB floppy fails to attach In-Reply-To: <4AE89F8C.60000@rawbw.com> References: <4AE89F8C.60000@rawbw.com> Message-ID: <200910290053.29271.hselasky@c2i.net> On Wednesday 28 October 2009 20:46:20 Yuri wrote: > My USB floppy dive fails to attach to device (see debug log below). > VendorID=0x0409 ProductID=0x0040 not mentioned in > /usr/src/sys/dev/usb/storage/umass.c > > Can this be that some that there is a simple fix for this, like adding a > quirk? > > Yuri > > -----log------- > > ugen1.2: at usbus1 > umass0: on usbus1 > umass0: UFI over CBI with CCI; quirks = 0x0000 > umass0:umass_cam_action: 4:-1:-1:XPT_PATH_INQ:. > umass0:4:0:-1: Attached to scbus4 > umass0:umass_cam_rescan: scbus4: scanning for 4:0:-1 > umass0:umass_cam_action: 4:-1:-1:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_SET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x12, flags: 0x40, 6b > cmd/36b data/18b sense > umass0:umass_attach: Attach finishedumass0:umass_cbi_dump_cmd: cmd = 12b > (0x120000002400...), data = 36b, dir = in > > umass0:umass_transfer_start: transfer index = 4 > umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=36 > umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=0 > umass0:umass_transfer_start: transfer index = 8 > umass0:umass_t_cbi_status_callback: UFI CCI, ASC = 0x00, ASCQ = 0x00 > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_SET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x12, flags: 0x40, 6b > cmd/255b data/18b sense > umass0:umass_cbi_dump_cmd: cmd = 12b (0x12010000ff00...), data = 255b, > dir = in > umass0:umass_transfer_start: transfer index = 4 > umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=255 > umass0:umass_t_cbi_data_read_callback: max_bulk=131072, data_rem=0 > umass0:umass_transfer_start: transfer index = 8 > umass0:umass_t_cbi_status_callback: UFI CCI, ASC = 0x00, ASCQ = 0x00 > umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_GET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_SET_TRAN_SETTINGS:. > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_cbi_dump_cmd: cmd = 12b (0x000000000000...), data = 0b, dir > = no data phase > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x00, flags: 0xc0, 6b > cmd/0b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_PATH_INQ:. > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umassX:umass_cam_rescan_callback: xpt0: Rescan succeeded > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > (da0:umass-sim0:0:0:0): got CAM status 0x4 > (da0:umass-sim0:0:0:0): fatal error, failed to attach to device > (da0:umass-sim0:0:0:0): lost device > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > umass0:umass_cam_action: 4:0:0:XPT_SCSI_IO: cmd: 0x25, flags: 0x40, 10b > cmd/8b data/32b sense > umass0:umass_t_cbi_reset1_callback: CBI reset! > umass0:umass_tr_error: transfer error, USB_ERR_STALLED -> reset > (da0:umass-sim0:0:0:0): removing device entry Hi, Your device is stalling on the CBI reset command. Maybe there is a bug in the umass.c driver. Please investigate/experiment more with umass.c until the device works. Link to PDF specification for CBI is mentioned somewhere inside umass.c. -_HPS From rea-fbsd at codelabs.ru Thu Oct 29 06:50:56 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Thu Oct 29 06:51:03 2009 Subject: ptrace problem 6.x/7.x - can someone explain this? In-Reply-To: References: Message-ID: Dorr, good day. Tue, Oct 27, 2009 at 05:32:34PM -0700, Dorr H. Clark wrote: > We believe ptrace has a problem in 6.3; we have not tried other > releases. The same code, however, exists in 7.1. And in HEAD too. > The bug was first encountered in gdb... > > (gdb) det > Detaching from program: /usr/local/bin/emacs, process 66217 > (gdb) att 66224 > Attaching to program: /usr/local/bin/emacs, process 66224 > Error accessing memory address 0x281ba5a4: Device busy. > (gdb) det > Detaching from program: /usr/local/bin/emacs, process 66224 > ptrace: Device busy. > (gdb) quit <--- target process 66224 dies here > > To isolate this problem, a wrote a simple minded test program was > written that just attached and detached. This test program found > even the very first detach fails with EBUSY (see test source below): > > $ ./test1 -p 66217 -c 1 -d 10 > pid 66217 count 1 delay 10 > Start of pass 0 > Calling PT_ATTACH pid 66217 addr 0x0 sig 0 > Calling PT_DETACH pid 66217 addr 0xffffffff sig 0 > Call 0 to PT_DETACH returned -1, errno 16 > > Once again, the target process died when the ptracing test program > exitted, as would be expected if a detach had failed. > > The failure return was coming from the following test in kern_ptrace() > in sys_process.c > > /* not currently stopped */ > if ((p->p_flag & (P_STOPPED_SIG | P_STOPPED_TRACE)) == 0 || > p->p_suspcount != p->p_numthreads || > (p->p_flag & P_WAITED) == 0) { > error = EBUSY; > goto fail; > } Yes, the ptraced process should have been waited for, even after the PT_ATTACH call. This is somewhat documented in ptrace(2), ----- ----- but I agree that the wording is a bit sloppy. I'll try to produce slightly modified explanation in the manual page and will post the patch here and as the PR. I had modified your example to visually display the results of each wait() call that is made after ptrace() invocation. Here we go: ----- $ ./test -p 45901 pid 45901 count 2 delay 5 Start of pass 0 Calling PT_ATTACH pid 45901 addr 0x0 sig 0 Attached wait() yield 0x117f: stopped by signal 17; <-- after PT_ATTACH wait() yield 0x57f: stopped by signal 5; <-- after PT_STEP Calling PT_DETACH pid 45901 addr 0xffffffffffffffff sig 0 Detached. ----- As you see, the process is stopped just after the PT_ATTACH with the signal 17, SIGSTOP. PT_STEP follows with the delivery of the SIGTRAP. Both of these signals should be processed by the parent's wait(). And PT_DETACH works, apart from one thing: on my 8.0 PT_DETACH leads to the segfault of the traced program. I hadn't yet tried it on the other versions, so may be there is some bug in the code of test.c, or some bug in the ptrace() implementation -- can't say for sure. If anyone knows why the program segfaults -- please, speak up. The modified source of the test.s is attached. > This is applied to all operations except PT_TRACE_ME, PT_ATTACH, and > some instances of PT_CLEAR_STEP. > > P_WAITED is generally not true. In particular, it's not set > automatically when a process is PT_ATTACHed. It is cleared by > PT_DETACH and again when ptrace sends a signal (PT_CONTINUE, > PT_DETACH.) _But_ it's set in only two places, and they aren't in > ptrace code. > > 2 sys/kern/kern_exit.c kern_wait 773 p->p_flag |= P_WAITED; > 3 compat/svr4/svr4_misc.c svr4_sys_waitsys 1351 q->p_flag |= P_WAITED; > > The relevant one is the first one, primarily. Here's the code: > > mtx_lock_spin(&sched_lock); > if ((p->p_flag & P_STOPPED_SIG) && > (p->p_suspcount == p->p_numthreads) && > (p->p_flag & P_WAITED) == 0 && > (p->p_flag & P_TRACED || options & WUNTRACED)) { > mtx_unlock_spin(&sched_lock); > p->p_flag |= P_WAITED; > sx_xunlock(&proctree_lock); > td->td_retval[0] = p->p_pid; > if (status) > *status = W_STOPCODE(p->p_xstat); > PROC_UNLOCK(p); > return (0); > } > mtx_unlock_spin(&sched_lock); > > So it's only set on processes which are already traced. But it's not > set until someone calls wait4() on them - or the equivalent sysV > compatability routine. > > Gdb doesn't always wait4() for processes immediately opon tracing > them, and the ptrace man page does not imply this is needed. Hmm, there is at least one thread on the simular matter, http://sourceware.org/ml/gdb/2008-12/msg00041.html and people are saying that wait() still should be present. > Moreover, it's not clear why it should matter. The process > needs to be stopped in order for it to make sense to do most > of the things ptrace does. But - why should it need to be waited for? To see if it was really stopped, I presume. > And what kind of sense does this make to someone writing a debugging > tool, where the natural logic seems to be: > - attach to process - wait for the process' attachment by doing wait(). > - look at some stuff > - stick in some kind of breakpoint or similar and start it going again > (or 'step' it) > - wait for it to stop > - look at and modify stuff > - detach, or set it moving again > > By way of experiment, the test for P_WAITED was removed. Gdb no longer had > problems, and no new issues with gdb were encountered (although this > was just interactive, no "gdb coverage test" was attempted). By the way, I can't reproduce gdb faults with the 8.0 sources. Will try 7.x, but I think that I have no 6.x handy. -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From jason.harmening at gmail.com Thu Oct 29 15:49:57 2009 From: jason.harmening at gmail.com (Jason Harmening) Date: Thu Oct 29 20:28:10 2009 Subject: MIPS: bus_dma(9) and cache problems Message-ID: <2d1264630910290849k2ca29237ubb25cc3b7313ec26@mail.gmail.com> > 1. code modifies data in block and this modification ends up in > cache and is not written back to memory > 2. right after this code calls bus_dmamap_sync for this buffer > and as a result cache invalidation is performed > 3. Cache function operates on cache line size-aligned addresses > and the block in question happens to share the same cache line > with the buffer. So modification made at step (1) is lost. What sync operation are you doing? At least for PREREAD or PREWRITE, I'd expect any dirty cache lines to be flushed to RAM. If this isn't happening, then you may want to submit a bug report. BTW, if you haven't already found it the MIPS sync code for 9-CURRENT is here: http://fxr.watson.org/fxr/source/mips/mips/busdma_machdep.c#L760 From alexbestms at math.uni-muenster.de Fri Oct 30 18:05:49 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Fri Oct 30 18:05:56 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <86zl7ecnyg.fsf@ds4.des.no> Message-ID: Dag-Erling Sm?rgrav schrieb am 2009-10-26: > Rong-En Fan writes: > > devel/ncurses-devel will be updated this week. For base's ncurses, > > I'll > > check with Thomas Dickey to see if there will be 5.8 soon. If not, > > we > > can also import a recent snapshot. > There is no reason to wait, nor to import an entire snapshot. Se my > earlier message to Alexander. > DES rafan just mfc'ed the patch. could we also have the fix in 8.0-RELEASE? should get approved by re@. alex From alexbestms at math.uni-muenster.de Sat Oct 31 02:00:00 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Sat Oct 31 02:00:06 2009 Subject: mmap(2) segaults with certain len values and MAP_ANON|MAP_FIXED In-Reply-To: <200910211340.39872.jhb@freebsd.org> Message-ID: John Baldwin schrieb am 2009-10-21: > On Wednesday 21 October 2009 11:30:51 am Alexander Best wrote: > > Robert Watson schrieb am 2009-10-21: > > > On Wed, 21 Oct 2009, Alexander Best wrote: > > > >this code serves only one purpose: to trigger a segfault. i > > > >don't > > > >use the code for any other purpose. i was under the impression > > > >that > > > >mmap() should either succeed or fail (tertium non datur). mmap's > > > >manual doesn't say anything about mmap() causing segfaults. > > > Have you tried ktracing the application? I think you'll find > > > that > > > mmap(2) system call succeeded fine, and that the segfault comes > > > from > > > attempting to execute the address in libc on return to userspace, > > > as > > > a result of libc not being at that address anymore (since you > > > removed its mapping). You can use procstat -v to inspect address > > > space use by processes, but as a general rule you don't want to > > > pass > > > anything other than an address of 0x0 to mmap(2) unless you're > > > very > > > carefully managing the address space of the process. Many > > > userspace > > > libraries are involved in using that address space, but > > > especially > > > the runtime linker which begins execution in userspace when a > > > binary > > > is started. > > > Robert N M Watson > > > Computer Laboratory > > > University of Cambridge > > you're right. this kdump shows that the segfault isn't being caused > > by the > > mmap() call: > > 88343 mmap_test CALL > > mmap(0x1000,0x80047000,PROT_NONE,MAP_FIXED|MAP_ANON,0xffffffff,0,0) > > 88343 mmap_test RET mmap 4096/0x1000 > > 88343 mmap_test PSIG SIGSEGV SIG_DFL > > 88343 mmap_test NAMI "mmap_test.core" > > thanks for clearing things up. > > however i stil think mentioning this situation in the mmap(2) > > manual (maybe in > > section MAP_FIXED) would be a good idea. > I'm not sure it is useful to attempt to enumerate all the possible > ways one > can shoot one's own foot using mmap(2) in the manual page. The list > would be > quite long and would require a large amount of imagination. In > effect, you > are asking for a manual page to document all the possible bugs one > could have > and manual pages in general do not do that. you're probably right. documenting all things one can do wrong using mmap isn't what the manual is supposed to do. thanks for the help. alex From alexbestms at math.uni-muenster.de Sat Oct 31 02:38:33 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Sat Oct 31 02:38:40 2009 Subject: mmap(2) with MAP_ANON honouring offset although it shouldn't In-Reply-To: <200910211349.10174.jhb@freebsd.org> Message-ID: John Baldwin schrieb am 2009-10-21: > On Wednesday 21 October 2009 11:51:04 am Alexander Best wrote: > > although the mmap(2) manual states in section MAP_ANON: > > "The offset argument is ignored." > > this doesn't seem to be true. running > > printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, > > 0x12345678)); > > and > > printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1, > > 0)); > > produces different outputs. i've attached a patch to solve the > > problem. the > > patch is similar to the one proposed in this PR, but should apply > > cleanly to > > CURRENT: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/71258 > A simpler patch would be to simply set pos = 0 below the MAP_STACK > line if > MAP_ANON is set. how about the following patch. problem seems to be that pos = 0 needs to be set before pageoff is being calculated. i've tested mmap with MAP_STACK and the offset gets discarded just as documented in mmap(2). with the patch the offset handling with MAP_ANON and MAP_STACK (implies MAP_ANON) are the same. another short question: why does the second call when doing printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_READ|PROT_WRITE, MAP_STACK, -1, 0x0)); printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_READ|PROT_WRITE, MAP_STACK, -1, 0x0)); fail? doesn't MAP_STACK allow mapping the same region twice? printf("%p\n", mmap((void*)0x1000, 0x1000, PROT_READ|PROT_WRITE, MAP_STACK, -1, 0x0)); printf("%p\n", mmap((void*)0x2000, 0x1000, PROT_READ|PROT_WRITE, MAP_STACK, -1, 0x0)); works just as expected. cheers. alex -------------- next part -------------- --- /usr/src/sys/vm/vm_mmap.c 2009-10-28 21:37:53.000000000 +0100 +++ ./vm_mmap.c 2009-10-31 03:22:44.000000000 +0100 @@ -241,9 +241,11 @@ ((prot & (PROT_READ | PROT_WRITE)) != (PROT_READ | PROT_WRITE))) return (EINVAL); flags |= MAP_ANON; - pos = 0; } + if (flags & MAP_ANON) + pos = 0; + /* * Align the file position to a page boundary, * and save its page offset component. @@ -300,7 +302,6 @@ handle = NULL; handle_type = OBJT_DEFAULT; maxprot = VM_PROT_ALL; - pos = 0; } else { /* * Mapping file, get fp for validation and From alexbestms at math.uni-muenster.de Sat Oct 31 13:30:50 2009 From: alexbestms at math.uni-muenster.de (Alexander Best) Date: Sat Oct 31 13:30:58 2009 Subject: help needed to fix contrib/ee crash/exit when receiving SIGWINCH In-Reply-To: <86d44ae5do.fsf@ds4.des.no> Message-ID: Dag-Erling Sm?rgrav schrieb am 2009-10-26: > Alexander Best writes: > > the patch got committed by thomas and is included in > > ncurses-5.7-20091024.patch.gz. > > i guess it will be included in our base version of ncurses once 5.8 > > gets > > released, but the patch should quickly make it into the port > > version of > > ncurses. > Apply the upstream patch to vendor/ncurses/dist and merge it to head. > Subversion will DTRT when you import 5.8. I believe this is > documented > on the wiki. > DES great news. so should the PR be closed or should it remain in patched state in order for 7.x to get patched? another question: how about our ncurses base version in general? should it remain the ncurses release version with our own patchset or would it be better to update it more frequently with the official ncurses patchsets? i guess this is the way acpi in the base dir is being handled. vendor updates get integrated on the fly into the base dir. on the other hand acpi isn't in the ports dir. if one needs cutting edge ncurses there's devel/ncurses-devel. so there's a reason to only to update the base version of ncurses when a new release is happening (5.8 e.g.). what's your opinion? alex