From pollo.es.pollo at gmail.com Sun Feb 1 03:24:43 2009 From: pollo.es.pollo at gmail.com (Omar Lopez Limonta) Date: Sun Feb 1 03:24:49 2009 Subject: =?utf-8?q?Invitaci=C3=B3n_a_conectarnos_en_LinkedIn?= Message-ID: <343448111.4883170.1233485776136.JavaMail.app@ech3-cdn07.prod> LinkedIn ------------ Omar Lopez Limonta requested to add you as a connection on LinkedIn: ------------------------------------------ Dikshie, Me gustar?a a?adirte a mi red profesional en LinkedIn. - Omar View invitation from Omar Lopez Limonta http://www.linkedin.com/e/Xz7E5xovV_nlOsWcP87EwRr4q0Tlr40Qed7EZzBQBu/blk/975134672_2/cBYOdPoQcP4RdPALqnpPbOYWrSlI/svi/ ------------------------------------------ Why might connecting with Omar Lopez Limonta be a good idea? Omar Lopez Limonta's connections could be useful to you: After accepting Omar Lopez Limonta's invitation, check Omar Lopez Limonta's connections to see who else you may know and who you might want an introduction to. Building these connections can create opportunities in the future. ------ (c) 2009, LinkedIn Corporation From rysto32 at gmail.com Sun Feb 1 07:51:38 2009 From: rysto32 at gmail.com (Ryan Stone) Date: Sun Feb 1 07:51:44 2009 Subject: gcc 4.3.2 libgcc_s.so exception handling broken? Message-ID: I saw a very similar thing happen when we moved to gcc 4.3 on a 6.1 system. The problem ended up being that we were linking everything, including shared objects, against a static libgcc. This meant that when a C++ program loaded a C++ shared object, there'd be two copies of the exception handling code in the process. What happened was the the executable registered all its exception handlers with the copy in the excecutable, and then loaded the shared object. When it raised an exception, it called the exception handling code in the shared object, which didn't know anything about the exception handlers in the executable, so it couldn't find them, so it called terminate(). The solution was to link everything against a dynamic libgcc, so there'd only be one copy of the exception handling code. Your problem may be similar. I doubt that you're statically linking libgcc in, but if you are, that's you're problem. My guess is that either your executable, or another shared library that your exectuable uses, is linking against a different libgcc_s.so, which would cause the same problem(multiple copies of the exception handling code). ldd should be able to tell you if this is the case. Ryan Stone From sepotvin at videotron.ca Sun Feb 1 10:58:14 2009 From: sepotvin at videotron.ca (Stephane E. Potvin) Date: Sun Feb 1 10:58:21 2009 Subject: telnetd[20170]: ttloop: peer died: Resource temporarily unavailable In-Reply-To: References: Message-ID: <4985EE67.8090807@videotron.ca> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 vasanth raonaik wrote: > Hello Hackers, > > I am seeing this message continuously in syslog for every 60 secs. what > could be the possible reasons for this error messages. Are you using an amd64 kernel? If so, you could try the attached patch. I've encountered a similar problem today (in my case the problem was with dbus) where select would fail due to a 64->32 bit truncation in the kernel. Steph -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmF7mAACgkQmdOXtTCX/ntNWACeJ9TYzbl9hW5aheJP/RtLm2J0 /aMAoIUHbBpsD1v6O30yXpuXkRZWaEfN =4JD4 -----END PGP SIGNATURE----- -------------- next part -------------- Index: kern/sys_generic.c =================================================================== --- kern/sys_generic.c (revision 187983) +++ kern/sys_generic.c (working copy) @@ -903,7 +903,7 @@ * bit position in the fd_mask array. */ static __inline int -selflags(fd_mask **ibits, int idx, int bit) +selflags(fd_mask **ibits, int idx, fd_mask bit) { int flags; int msk; @@ -912,7 +912,7 @@ for (msk = 0; msk < 3; msk++) { if (ibits[msk] == NULL) continue; - if ((ibits[msk][idx] & (fd_mask)bit) == 0) + if ((ibits[msk][idx] & bit) == 0) continue; flags |= select_flags[msk]; } -------------- next part -------------- A non-text attachment was scrubbed... Name: lp64_select_fix.diff.sig Type: application/octet-stream Size: 72 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090201/349ced84/lp64_select_fix.diff.obj From bsd.quest at googlemail.com Sun Feb 1 12:37:56 2009 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Sun Feb 1 12:38:03 2009 Subject: bus_dma (9). What exactly means "Loading of memory allocation" ? Message-ID: <671bb5fc0902011237j19dc7fc5o881649228c121bf2@mail.gmail.com> Hi, at first the cut of text from man (9) bus_dma: bus_dmamap_t A machine-dependent opaque type describing an individual mapping. One map is used for each memory allocation that will be loaded. Maps can be reused once they have been unloaded... Question: What exactly means "Loading of memory allocation" in this context ? Could anyone explain it or give me some little example with DMA functions for understanding it. Thanks a lot, Alexej From babkin at verizon.net Sun Feb 1 18:57:01 2009 From: babkin at verizon.net (Sergey Babkin) Date: Sun Feb 1 19:48:28 2009 Subject: bus_dma (9). What exactly means "Loading of memory allocation" ? Message-ID: <6699015.16785.1233539817447.JavaMail.root@vms063.mailsrvcs.net> If I remember correctly, loading means that the pages become mapped and address space loaded, for any pages pages are allocated and the d machines, like I think DEC Alpha, the the devices are not the same as seen by the CPU translated. And so on. I think my real old articl the Daemonnews site seems to be re http://ezine.daemonnews.org/200008/isa.html -SB (sorry a provider suppo Feb 1, 2009 03:38:27 PM, [1]bsd.quest@googlemail.com at first the cut of text from man (9) bus_dma: bus_dmamap_t mapp One map is used for each memory allocation that will b Maps can be reused once they have been unloaded.. Question: What exactly means "Loading of memory allocation" in thi s context ? Could anyone explain it or give me some little example wi functions for understanding it. References 1. 3D"mailto:bsd.quest@googlemail From yanefbsd at gmail.com Mon Feb 2 00:45:20 2009 From: yanefbsd at gmail.com (Garrett Cooper) Date: Mon Feb 2 00:45:27 2009 Subject: bus_dma (9). What exactly means "Loading of memory allocation" ? In-Reply-To: <6699015.16785.1233539817447.JavaMail.root@vms063.mailsrvcs.net> References: <6699015.16785.1233539817447.JavaMail.root@vms063.mailsrvcs.net> Message-ID: <7d6fde3d0902020045h55255cane0d6dd9f64d23f5@mail.gmail.com> On Sun, Feb 1, 2009 at 5:56 PM, Sergey Babkin wrote: > > If I remember correctly, loading means that the pages become mapped > and visible to the devices. Some buses can access only a limited > address space , like ISA has only a 24-bit address. When a map gets > loaded, for any pages outside of this range the temporary in-ramge > pages are allocated and the d ata gets moved through them. On some > machines, like I think DEC Alpha, the physicall addresses seen by > the devices are not the same as seen by the CPU , these need to be > translated. And so on. > I think my real old articl e had some of these explanations but now > the Daemonnews site seems to be re al slow: > http://ezine.daemonnews.org/200008/isa.html > -SB > (sorry a bout top quoting, it's the only kind the web interface of my > provider suppo rts) > Feb 1, 2009 03:38:27 PM, [1]bsd.quest@googlemail.com wrote: > > Hi, > at first the cut of text from man (9) bus_dma: > bus_dmamap_t > A machine-dependent opaque type describing an individual > mapp ing. > One map is used for each memory allocation that will b e loaded. > Maps can be reused once they have been unloaded.. . > Question: What exactly means "Loading of memory allocation" in thi s context > ? > Could anyone explain it or give me some little example wi th DMA > functions > for understanding it. Unfortunately it's bad English, so that might be where some of the confusion is stemming from. I'll send a doc's PR request after this to fix it. -Garrett From yanefbsd at gmail.com Mon Feb 2 00:46:12 2009 From: yanefbsd at gmail.com (Garrett Cooper) Date: Mon Feb 2 00:46:20 2009 Subject: bus_dma (9). What exactly means "Loading of memory allocation" ? In-Reply-To: <7d6fde3d0902020045h55255cane0d6dd9f64d23f5@mail.gmail.com> References: <6699015.16785.1233539817447.JavaMail.root@vms063.mailsrvcs.net> <7d6fde3d0902020045h55255cane0d6dd9f64d23f5@mail.gmail.com> Message-ID: <7d6fde3d0902020046o7640f217ye88336b53920a538@mail.gmail.com> On Mon, Feb 2, 2009 at 12:45 AM, Garrett Cooper wrote: > On Sun, Feb 1, 2009 at 5:56 PM, Sergey Babkin wrote: >> >> If I remember correctly, loading means that the pages become mapped >> and visible to the devices. Some buses can access only a limited >> address space , like ISA has only a 24-bit address. When a map gets >> loaded, for any pages outside of this range the temporary in-ramge >> pages are allocated and the d ata gets moved through them. On some >> machines, like I think DEC Alpha, the physicall addresses seen by >> the devices are not the same as seen by the CPU , these need to be >> translated. And so on. >> I think my real old articl e had some of these explanations but now >> the Daemonnews site seems to be re al slow: >> http://ezine.daemonnews.org/200008/isa.html >> -SB >> (sorry a bout top quoting, it's the only kind the web interface of my >> provider suppo rts) >> Feb 1, 2009 03:38:27 PM, [1]bsd.quest@googlemail.com wrote: >> >> Hi, >> at first the cut of text from man (9) bus_dma: >> bus_dmamap_t >> A machine-dependent opaque type describing an individual >> mapp ing. >> One map is used for each memory allocation that will b e loaded. >> Maps can be reused once they have been unloaded.. . >> Question: What exactly means "Loading of memory allocation" in thi s context >> ? >> Could anyone explain it or give me some little example wi th DMA >> functions >> for understanding it. > > Unfortunately it's bad English, so that might be where some of the > confusion is stemming from. I'll send a doc's PR request after this to > fix it. > -Garrett Ugh. Nevermind. The question was written improperly -- the manpage wasn't ><. -Garrett From avg at icyb.net.ua Mon Feb 2 03:38:22 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Feb 2 03:38:28 2009 Subject: INTR_FILTER? In-Reply-To: References: <49819757.2010002@icyb.net.ua> <8F669786-30A2-458C-8A6B-3272297ADE14@freebsd.org> <4981EC95.1090002@icyb.net.ua> Message-ID: <4986DB28.6080503@icyb.net.ua> on 30/01/2009 00:30 Rui Paulo said the following: > On 29 Jan 2009, at 17:51, Andriy Gapon wrote: >> BTW, INTR_FILTER seems quite useful. Why, then, it is not the default? > > The drivers would have to be ported to INTR_FILTER. Right now, only asmc > is using INTR_FILTER, so I don't think there is much gain in making it > the default. I am not sure about this part. From the code it seems that INTR_FILTER is backward-compatible, i.e. it gives something and doesn't take away anything. The API and conventions seems to be the same too. There could be some edge cases, of course. -- Andriy Gapon From avg at icyb.net.ua Mon Feb 2 03:45:18 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Feb 2 03:45:24 2009 Subject: INTR_FILTER? In-Reply-To: <4982FBC7.1080100@oltrelinux.com> References: <49819757.2010002@icyb.net.ua> <4982FBC7.1080100@oltrelinux.com> Message-ID: <4986DCC9.8080500@icyb.net.ua> on 30/01/2009 15:08 Paolo Pisati said the following: > Andriy Gapon wrote: >> INTR_FILTER - what does it do? >> It doesn't seem to be documented anywhere, but seems to affect interrupt >> code. >> >> > for a bit more information about interrupt filtering, see here: > > http://lists.freebsd.org/pipermail/freebsd-hackers/2007-February/019704.html Paolo, so would it be correct to say that INTR_FILTER is designed to make life much easier in view of shared interrupts. >From the code it seems that there would be virtually no difference when there are no shared interrupt, but with shared interrupts INTR_FILTER removes all the churn and makes interrupt handling nice and clean (from device drivers point of view). -- Andriy Gapon From rpaulo at freebsd.org Mon Feb 2 03:53:38 2009 From: rpaulo at freebsd.org (Rui Paulo) Date: Mon Feb 2 03:53:45 2009 Subject: INTR_FILTER? In-Reply-To: <4986DB28.6080503@icyb.net.ua> References: <49819757.2010002@icyb.net.ua> <8F669786-30A2-458C-8A6B-3272297ADE14@freebsd.org> <4981EC95.1090002@icyb.net.ua> <4986DB28.6080503@icyb.net.ua> Message-ID: <3EAA1D8D-606B-4F59-81B6-644B56AE4831@freebsd.org> On 2 Feb 2009, at 11:38, Andriy Gapon wrote: > on 30/01/2009 00:30 Rui Paulo said the following: >> On 29 Jan 2009, at 17:51, Andriy Gapon wrote: >>> BTW, INTR_FILTER seems quite useful. Why, then, it is not the >>> default? >> >> The drivers would have to be ported to INTR_FILTER. Right now, only >> asmc >> is using INTR_FILTER, so I don't think there is much gain in making >> it >> the default. > > I am not sure about this part. From the code it seems that INTR_FILTER > is backward-compatible, i.e. it gives something and doesn't take away > anything. The API and conventions seems to be the same too. > There could be some edge cases, of course. Ok, but why enable it in GENERIC right now if the only driver that uses INTR_FILTER is asmc? There's not much point in enabling it now. Maybe in the future. Regards, -- Rui Paulo -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090202/3bbff6a0/PGP.pgp From avg at icyb.net.ua Mon Feb 2 04:01:30 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Feb 2 04:01:36 2009 Subject: INTR_FILTER? In-Reply-To: <3EAA1D8D-606B-4F59-81B6-644B56AE4831@freebsd.org> References: <49819757.2010002@icyb.net.ua> <8F669786-30A2-458C-8A6B-3272297ADE14@freebsd.org> <4981EC95.1090002@icyb.net.ua> <4986DB28.6080503@icyb.net.ua> <3EAA1D8D-606B-4F59-81B6-644B56AE4831@freebsd.org> Message-ID: <4986E08F.2010305@icyb.net.ua> on 02/02/2009 13:53 Rui Paulo said the following: > > On 2 Feb 2009, at 11:38, Andriy Gapon wrote: > >> on 30/01/2009 00:30 Rui Paulo said the following: >>> On 29 Jan 2009, at 17:51, Andriy Gapon wrote: >>>> BTW, INTR_FILTER seems quite useful. Why, then, it is not the default? >>> >>> The drivers would have to be ported to INTR_FILTER. Right now, only asmc >>> is using INTR_FILTER, so I don't think there is much gain in making it >>> the default. >> >> I am not sure about this part. From the code it seems that INTR_FILTER >> is backward-compatible, i.e. it gives something and doesn't take away >> anything. The API and conventions seems to be the same too. >> There could be some edge cases, of course. > > Ok, but why enable it in GENERIC right now if the only driver that uses > INTR_FILTER is asmc? > There's not much point in enabling it now. Maybe in the future. I may be wrong but this could auto-magically improve some cases where there are shared interrupts between drivers with ithreads. In this case, I think, their interrupt handler would be run "in parallel" instead of sequentially. Also, it would make it easier to write new drivers - one would not have to code for !INTR_FILTER case. -- Andriy Gapon From rpaulo at freebsd.org Mon Feb 2 04:17:40 2009 From: rpaulo at freebsd.org (Rui Paulo) Date: Mon Feb 2 04:17:47 2009 Subject: INTR_FILTER? In-Reply-To: <4986E08F.2010305@icyb.net.ua> References: <49819757.2010002@icyb.net.ua> <8F669786-30A2-458C-8A6B-3272297ADE14@freebsd.org> <4981EC95.1090002@icyb.net.ua> <4986DB28.6080503@icyb.net.ua> <3EAA1D8D-606B-4F59-81B6-644B56AE4831@freebsd.org> <4986E08F.2010305@icyb.net.ua> Message-ID: <86915A93-6023-465F-B937-E9905AFDA4B0@freebsd.org> On 2 Feb 2009, at 12:01, Andriy Gapon wrote: > on 02/02/2009 13:53 Rui Paulo said the following: >> >> On 2 Feb 2009, at 11:38, Andriy Gapon wrote: >> >>> on 30/01/2009 00:30 Rui Paulo said the following: >>>> On 29 Jan 2009, at 17:51, Andriy Gapon wrote: >>>>> BTW, INTR_FILTER seems quite useful. Why, then, it is not the >>>>> default? >>>> >>>> The drivers would have to be ported to INTR_FILTER. Right now, >>>> only asmc >>>> is using INTR_FILTER, so I don't think there is much gain in >>>> making it >>>> the default. >>> >>> I am not sure about this part. From the code it seems that >>> INTR_FILTER >>> is backward-compatible, i.e. it gives something and doesn't take >>> away >>> anything. The API and conventions seems to be the same too. >>> There could be some edge cases, of course. >> >> Ok, but why enable it in GENERIC right now if the only driver that >> uses >> INTR_FILTER is asmc? >> There's not much point in enabling it now. Maybe in the future. > > I may be wrong but this could auto-magically improve some cases where > there are shared interrupts between drivers with ithreads. In this > case, > I think, their interrupt handler would be run "in parallel" instead of > sequentially. I haven't read the details of the implementation yet, but how does that work? > Also, it would make it easier to write new drivers - one would not > have > to code for !INTR_FILTER case. Yes, but essentially, backporting needs the !INTR_FILTER case. And I don't know about !i386 && !amd64 archs. -- Rui Paulo -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090202/f858cd9d/PGP.pgp From avg at icyb.net.ua Mon Feb 2 04:24:18 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Feb 2 04:24:25 2009 Subject: INTR_FILTER? In-Reply-To: <86915A93-6023-465F-B937-E9905AFDA4B0@freebsd.org> References: <49819757.2010002@icyb.net.ua> <8F669786-30A2-458C-8A6B-3272297ADE14@freebsd.org> <4981EC95.1090002@icyb.net.ua> <4986DB28.6080503@icyb.net.ua> <3EAA1D8D-606B-4F59-81B6-644B56AE4831@freebsd.org> <4986E08F.2010305@icyb.net.ua> <86915A93-6023-465F-B937-E9905AFDA4B0@freebsd.org> Message-ID: <4986E5EE.1070302@icyb.net.ua> on 02/02/2009 14:17 Rui Paulo said the following: > > On 2 Feb 2009, at 12:01, Andriy Gapon wrote: > > >> I may be wrong but this could auto-magically improve some cases where >> there are shared interrupts between drivers with ithreads. In this case, >> I think, their interrupt handler would be run "in parallel" instead of >> sequentially. > > I haven't read the details of the implementation yet, but how does that > work? As I understand, in non-INTR_FILTER case there is one ithread per interrupt vector ("IRQ"), in INTR_FILTER case there is an ithread per handler ("driver"). This is a little bit simplified. -- Andriy Gapon From laladelausanne at gmail.com Mon Feb 2 04:33:12 2009 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Mon Feb 2 04:33:20 2009 Subject: blockable sleep lock (sleep mutex) 16 In-Reply-To: <02026848-7F83-405C-B4F3-EDD8B47DA294@gmail.com> References: <02026848-7F83-405C-B4F3-EDD8B47DA294@gmail.com> Message-ID: <32679C0A-28C1-4D7A-950C-580787F3971D@gmail.com> On 30 Jan 2009, at 18:11 , Nikola Kne?evi? wrote: > This is the message buffer: > Unread portion of the kernel message buffer: > panic: blockable sleep lock (sleep mutex) 16 @ /usr/src/sys/vm/ > uma_core.c:1834 > Any hints where I should search for the cause? Ok, I solved this problem. I had a critical_enter/exit surrounding code which was calling a lot of mallocs. Now, I'm getting another message, which doesn't make any sense: ---8<--- --- trap 0, rip = 0, rsp = 0xffffffff87834d30, rbp = 0 --- uma_zalloc_arg: zone "256" with the following non-sleepable locks held: exclusive sleep mutex click_instance r = 0 (0xffffff00051b4540) locked @ sched.cc:441 --->8--- It says "non-sleepable locks", yet it classifies click_instance as sleep mutex. I think witness code should emit messages which are more clear. Cheers, Nikola From jhb at freebsd.org Mon Feb 2 06:56:31 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon Feb 2 06:56:42 2009 Subject: blockable sleep lock (sleep mutex) 16 In-Reply-To: <32679C0A-28C1-4D7A-950C-580787F3971D@gmail.com> References: <02026848-7F83-405C-B4F3-EDD8B47DA294@gmail.com> <32679C0A-28C1-4D7A-950C-580787F3971D@gmail.com> Message-ID: <200902020845.21773.jhb@freebsd.org> On Monday 02 February 2009 7:33:08 am Nikola Kne?evi? wrote: > On 30 Jan 2009, at 18:11 , Nikola Kne?evi? wrote: > > > This is the message buffer: > > Unread portion of the kernel message buffer: > > panic: blockable sleep lock (sleep mutex) 16 @ /usr/src/sys/vm/ > > uma_core.c:1834 > > > Any hints where I should search for the cause? > > > Ok, I solved this problem. I had a critical_enter/exit surrounding > code which was calling a lot of mallocs. Now, I'm getting another > message, which doesn't make any sense: > > ---8<--- > --- trap 0, rip = 0, rsp = 0xffffffff87834d30, rbp = 0 --- > uma_zalloc_arg: zone "256" with the following non-sleepable locks held: > exclusive sleep mutex click_instance r = 0 (0xffffff00051b4540) locked > @ sched.cc:441 > --->8--- > > It says "non-sleepable locks", yet it classifies click_instance as > sleep mutex. I think witness code should emit messages which are more > clear. It is confusing, but you can't do an M_WAITOK malloc while holding a mutex. Basically, sleeping actually means calling "*sleep() (such as mtx_sleep()) or cv_*wait*()". Blocking on a mutex is not sleeping, it's "blocking". Some locks (such as sx(9)) do "sleep" when you contest them. In the scheduler, sleeping and blocking are actually quite different (blocking uses turnstiles that handle priority inversions via priority propagation, sleeping uses sleep queues which do not do any of that). The underyling idea is that mutexes should be held for "short" periods of time, and that any sleeps are potentially unbounded. Holding a mutex while sleeping could result in a mutex being held for a long time. -- John Baldwin From jhb at freebsd.org Mon Feb 2 06:56:35 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon Feb 2 06:56:43 2009 Subject: Dynamic ddb commands In-Reply-To: <1233339897.13748.106.camel@amaretto> References: <1233337318.13748.101.camel@amaretto> <1233339897.13748.106.camel@amaretto> Message-ID: <200902020846.33072.jhb@freebsd.org> On Friday 30 January 2009 1:24:57 pm Matthew Fleming wrote: > Just an addenda: > > > I see that BSD 7.1 has dynamic commands using sysinits and sysuninit's to call a new > > db_[un]register_cmd. > > I was looking at HEAD, not RELENG_7_1. The remainder of my questions as > to why this mechanism are the same, though. In general it is far easier to just add sysinit's than to hack directly on the kernel linker. There are very few ddb commands, so one extra pointer or two per command is not a lot of space. -- John Baldwin From matthew.fleming at isilon.com Mon Feb 2 08:17:43 2009 From: matthew.fleming at isilon.com (Matthew Fleming) Date: Mon Feb 2 08:17:49 2009 Subject: Dynamic ddb commands In-Reply-To: <200902020846.33072.jhb@freebsd.org> References: <1233337318.13748.101.camel@amaretto> <1233339897.13748.106.camel@amaretto> <200902020846.33072.jhb@freebsd.org> Message-ID: <1233591461.7789.25.camel@amaretto> > In general it is far easier to just add sysinit's than to hack directly on the > kernel linker. There are very few ddb commands, so one extra pointer or two > per command is not a lot of space. Respectfully, I disagree, for several reasons. First, in order to make sysinit and sysctl work, the kernel linker needed to know that there are a set of elf sections that have special meaning. Yes, using sysinits means that there are still only two elf sections of interest. Second, as I mentioned before, having ddb commands added mixed in with sysinits means that, if I have a bug in my sysinit I may not be able to use some of my ddb commands to debug it. Even if DB_*COMMAND used SI_ORDER_FIRST, any sysinit with the same priority may come first. Next, if you want commands sorted globally, it could be done with either implementation. But I think that commands that are defined by a module should be listed with others from that module. Last, changing struct command introduces a binary compatibility issue. Any older driver that had a ddb command (even if they never realized they couldn't access it) would need to be recompiled. I am not sure of FreeBSD's binary compatibility policy, though, and it would presumably be across a major OS revision number. So this is not a very compelling argument. Thanks, matthew From bsd.quest at googlemail.com Mon Feb 2 08:23:36 2009 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Mon Feb 2 08:23:42 2009 Subject: bus_dma (9). What exactly means "Loading of memory allocation" ? In-Reply-To: <7d6fde3d0902020046o7640f217ye88336b53920a538@mail.gmail.com> References: <6699015.16785.1233539817447.JavaMail.root@vms063.mailsrvcs.net> <7d6fde3d0902020045h55255cane0d6dd9f64d23f5@mail.gmail.com> <7d6fde3d0902020046o7640f217ye88336b53920a538@mail.gmail.com> Message-ID: <671bb5fc0902020823p3ee8d9edl715a074603ab97de@mail.gmail.com> Hi, thanx for your answer. I checked the source code of the *dma() functions. If I understand it correctly, "loading of memory allocation" means the following: 1. At first memory allocation should be done: bufp = *alloc(sizeof ....) 2. then in ... _bus_dmamap_load_buffer() we get physical addres of allocated buffer: if (pmap) curaddr = pmap_extract(pmap, vaddr); else curaddr = pmap_kextract(vaddr); ... then some "magic" with bouncing 3. then physical address will passed to dmat->segments segs[seg].ds_addr = curaddr; segs[seg].ds_len = sgsize; Ok, it all means: getting of physical address of allocated buffer. If physical space not accessble for device, allocating bounce buffers. Getting of physical addresses of allocated buffers. And then put these physical addresses and sizes of buffers in dmat->segments array. <- loading of memory allocation (-: right ? Thanx, Alexej 2009/2/2 Garrett Cooper > On Mon, Feb 2, 2009 at 12:45 AM, Garrett Cooper > wrote: > > On Sun, Feb 1, 2009 at 5:56 PM, Sergey Babkin > wrote: > >> > >> If I remember correctly, loading means that the pages become mapped > >> and visible to the devices. Some buses can access only a limited > >> address space , like ISA has only a 24-bit address. When a map gets > >> loaded, for any pages outside of this range the temporary in-ramge > >> pages are allocated and the d ata gets moved through them. On some > >> machines, like I think DEC Alpha, the physicall addresses seen by > >> the devices are not the same as seen by the CPU , these need to be > >> translated. And so on. > >> I think my real old articl e had some of these explanations but now > >> the Daemonnews site seems to be re al slow: > >> http://ezine.daemonnews.org/200008/isa.html > >> -SB > >> (sorry a bout top quoting, it's the only kind the web interface of my > >> provider suppo rts) > >> Feb 1, 2009 03:38:27 PM, [1]bsd.quest@googlemail.com wrote: > >> > >> Hi, > >> at first the cut of text from man (9) bus_dma: > >> bus_dmamap_t > >> A machine-dependent opaque type describing an individual > >> mapp ing. > >> One map is used for each memory allocation that will b e loaded. > >> Maps can be reused once they have been unloaded.. . > >> Question: What exactly means "Loading of memory allocation" in thi > s context > >> ? > >> Could anyone explain it or give me some little example wi th DMA > >> functions > >> for understanding it. > > > > Unfortunately it's bad English, so that might be where some of the > > confusion is stemming from. I'll send a doc's PR request after this to > > fix it. > > -Garrett > > Ugh. Nevermind. The question was written improperly -- the manpage wasn't > ><. > -Garrett > From kamikaze at bsdforen.de Mon Feb 2 08:49:28 2009 From: kamikaze at bsdforen.de (Dominic Fandrey) Date: Mon Feb 2 08:49:37 2009 Subject: Fw: request responsibility timeout In-Reply-To: <200902010926.00728.fbsd.questions@rachie.is-a-geek.net> References: <4985D26E.9000604@bsdforen.de> <200902010926.00728.fbsd.questions@rachie.is-a-geek.net> Message-ID: <49871CA2.4040606@bsdforen.de> Mel wrote: > On Sunday 01 February 2009 07:48:46 Dominic Fandrey wrote: >> I want to request a responsibility timeout for bin/120784, I have >> submitted a patch matching the previously discussed criteria for >> a commit a couple of weeks ago and I would like to receive some kind >> of reaction. >> >> What is the appropriate channel to do so? > > -hackers > (forwarded to hackers) From sam at freebsd.org Mon Feb 2 09:23:19 2009 From: sam at freebsd.org (Sam Leffler) Date: Mon Feb 2 09:23:25 2009 Subject: Dynamic ddb commands In-Reply-To: <1233591461.7789.25.camel@amaretto> References: <1233337318.13748.101.camel@amaretto> <1233339897.13748.106.camel@amaretto> <200902020846.33072.jhb@freebsd.org> <1233591461.7789.25.camel@amaretto> Message-ID: <49872C05.9010302@freebsd.org> Matthew Fleming wrote: >> In general it is far easier to just add sysinit's than to hack directly on the >> kernel linker. There are very few ddb commands, so one extra pointer or two >> per command is not a lot of space. >> > > Respectfully, I disagree, for several reasons. > > First, in order to make sysinit and sysctl work, the kernel linker > needed to know that there are a set of elf sections that have special > meaning. Yes, using sysinits means that there are still only two elf > sections of interest. > > Second, as I mentioned before, having ddb commands added mixed in with > sysinits means that, if I have a bug in my sysinit I may not be able to > use some of my ddb commands to debug it. Even if DB_*COMMAND used > SI_ORDER_FIRST, any sysinit with the same priority may come first. > > Next, if you want commands sorted globally, it could be done with either > implementation. But I think that commands that are defined by a module > should be listed with others from that module. > > Last, changing struct command introduces a binary compatibility issue. > Any older driver that had a ddb command (even if they never realized > they couldn't access it) would need to be recompiled. I am not sure of > FreeBSD's binary compatibility policy, though, and it would presumably > be across a major OS revision number. So this is not a very compelling > argument. > I'm not sure I buy any of these arguments (well maybe the 2nd one :)) but I'm still open to your changes. I just haven't had time to look at your patch; hope to this week (I have several other patches in my q for review ahead of yours). Sam From julian at elischer.org Mon Feb 2 10:20:42 2009 From: julian at elischer.org (Julian Elischer) Date: Mon Feb 2 10:20:49 2009 Subject: blockable sleep lock (sleep mutex) 16 In-Reply-To: <200902020845.21773.jhb@freebsd.org> References: <02026848-7F83-405C-B4F3-EDD8B47DA294@gmail.com> <32679C0A-28C1-4D7A-950C-580787F3971D@gmail.com> <200902020845.21773.jhb@freebsd.org> Message-ID: <498736C2.3040207@elischer.org> John Baldwin wrote: > On Monday 02 February 2009 7:33:08 am Nikola Kne?evi? wrote: >> On 30 Jan 2009, at 18:11 , Nikola Kne?evi? wrote: >> >>> This is the message buffer: >>> Unread portion of the kernel message buffer: >>> panic: blockable sleep lock (sleep mutex) 16 @ /usr/src/sys/vm/ >>> uma_core.c:1834 >>> Any hints where I should search for the cause? >> >> Ok, I solved this problem. I had a critical_enter/exit surrounding >> code which was calling a lot of mallocs. Now, I'm getting another >> message, which doesn't make any sense: >> >> ---8<--- >> --- trap 0, rip = 0, rsp = 0xffffffff87834d30, rbp = 0 --- >> uma_zalloc_arg: zone "256" with the following non-sleepable locks held: >> exclusive sleep mutex click_instance r = 0 (0xffffff00051b4540) locked >> @ sched.cc:441 >> --->8--- >> >> It says "non-sleepable locks", yet it classifies click_instance as >> sleep mutex. I think witness code should emit messages which are more >> clear. > > It is confusing, but you can't do an M_WAITOK malloc while holding a mutex. > Basically, sleeping actually means calling "*sleep() (such as mtx_sleep()) or > cv_*wait*()". Blocking on a mutex is not sleeping, it's "blocking". Some > locks (such as sx(9)) do "sleep" when you contest them. In the scheduler, > sleeping and blocking are actually quite different (blocking uses turnstiles > that handle priority inversions via priority propagation, sleeping uses sleep > queues which do not do any of that). The underyling idea is that mutexes > should be held for "short" periods of time, and that any sleeps are > potentially unbounded. Holding a mutex while sleeping could result in a > mutex being held for a long time. > the locking overview page man 9 locking tries to explain this.. I've been pestering John to proofread it and make suggestiosn for a while now. (nag nag) From jhb at freebsd.org Mon Feb 2 10:44:46 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon Feb 2 10:44:52 2009 Subject: Dynamic ddb commands In-Reply-To: <1233591461.7789.25.camel@amaretto> References: <1233337318.13748.101.camel@amaretto> <200902020846.33072.jhb@freebsd.org> <1233591461.7789.25.camel@amaretto> Message-ID: <200902021152.48020.jhb@freebsd.org> On Monday 02 February 2009 11:17:41 am Matthew Fleming wrote: > > In general it is far easier to just add sysinit's than to hack directly on the > > kernel linker. There are very few ddb commands, so one extra pointer or two > > per command is not a lot of space. > > Respectfully, I disagree, for several reasons. > > First, in order to make sysinit and sysctl work, the kernel linker > needed to know that there are a set of elf sections that have special > meaning. Yes, using sysinits means that there are still only two elf > sections of interest. Yes, that is precisely the point, to keep the special knowledge in the kernel linker to a minimum. This is why kernel modules use sysinit's as well, and thus device drivers, etc. The kernel linker is already a bit complex, and I'd much prefer to keep non-linker related knowledge out of it as much as possible. > Second, as I mentioned before, having ddb commands added mixed in with > sysinits means that, if I have a bug in my sysinit I may not be able to > use some of my ddb commands to debug it. Even if DB_*COMMAND used > SI_ORDER_FIRST, any sysinit with the same priority may come first. You can still debug it, just not using your new commands. But actually, as long as your sysinit's are after SI_SUB_KLD (as most of them are), then you can still use your debugging commands just fine. > Next, if you want commands sorted globally, it could be done with either > implementation. But I think that commands that are defined by a module > should be listed with others from that module. Well, one of the "features" of ddb (IIRC) is that it allows "automatic" nicknames in that if you have a 'reallylongname' command you can just use 're' as an alias for it if 'reallylongname' is the only command that starts with 're'. Handling that basically requires a sorted list. Also, if you want to group commands in your module, that is easily accomplished by using an appropriate namespace. E.g., prior to this when I used to use 'call foo()' as a poor-man's substitute, I added commands to debug de(4) and had them all start with 'tulip_*'. You could use a similar practice to group your commands if desired. > Last, changing struct command introduces a binary compatibility issue. > Any older driver that had a ddb command (even if they never realized > they couldn't access it) would need to be recompiled. I am not sure of > FreeBSD's binary compatibility policy, though, and it would presumably > be across a major OS revision number. So this is not a very compelling > argument. Correct, it is across major OS revisions, so it is ok to change it for 8.0. -- John Baldwin From christoph.mallon at gmx.de Mon Feb 2 11:42:41 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Mon Feb 2 11:42:50 2009 Subject: write-only variables in src/sys/ - possible bugs Message-ID: <49874CA8.5090605@gmx.de> Hi, I compiled a list of all local variables in src/sys/ (r188000), which are only written to, but never read. This is more than the GCC warning, which only complains about variables, which are only declared (and maybe initialised) and not used otherwise. In contrast this list contains variables with the following usage pattern: int w = 42; // GCC warns about this ... int x; // ... but not this x = 23; x++; return 0; The list contains about 700 entries. About three dozen concern variables named 'error'. Here's one *example* from the list: sys/dev/kbdmux/kbdmux.c:1304 In the function kbdmux_modevent() the variable 'error' is assigned values eight times, but at the end of the function there is just a return 0; and the variable is never read. Probably the value should be returned. You can find the list here: http://tron.homeunix.org/unread_variables.log The list was generated by cparser, a C99 compiler, which uses libFIRM for optimisation and code generation (lang/cparser in the ports). A small disclaimer: There might be some false positives due to errors which are caused by HEAD sources in combination with my installed 7.x headers plus a hacked up build process. Also some warnings are the result from variables, which are only used in debug macros, so td = curthread; KASSERT(td != NULL); provokes a warning (I consider this bad style). Nonetheless the number of false positives should be low. If there is interest, then I can compile a "proper" list. From christoph.mallon at gmx.de Mon Feb 2 12:08:00 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Mon Feb 2 12:08:06 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <20090202195809.GA54528@citylink.fud.org.nz> References: <49874CA8.5090605@gmx.de> <20090202195809.GA54528@citylink.fud.org.nz> Message-ID: <49875293.4050909@gmx.de> Andrew Thompson schrieb: > This is helpful, my only nit would be to run it through sort. :) Fixed (: From thompsa at FreeBSD.org Mon Feb 2 12:23:10 2009 From: thompsa at FreeBSD.org (Andrew Thompson) Date: Mon Feb 2 12:23:17 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <49874CA8.5090605@gmx.de> References: <49874CA8.5090605@gmx.de> Message-ID: <20090202195809.GA54528@citylink.fud.org.nz> On Mon, Feb 02, 2009 at 08:42:32PM +0100, Christoph Mallon wrote: > Hi, > > I compiled a list of all local variables in src/sys/ (r188000), which are > only written to, but never read. This is more than the GCC warning, which > only complains about variables, which are only declared (and maybe > initialised) and not used otherwise. In contrast this list contains > variables with the following usage pattern: > > int w = 42; // GCC warns about this ... > int x; // ... but not this > x = 23; > x++; > return 0; > > The list contains about 700 entries. About three dozen concern variables > named 'error'. Here's one *example* from the list: > > sys/dev/kbdmux/kbdmux.c:1304 > > In the function kbdmux_modevent() the variable 'error' is assigned values > eight times, but at the end of the function there is just a return 0; and > the variable is never read. Probably the value should be returned. > > You can find the list here: > http://tron.homeunix.org/unread_variables.log > > The list was generated by cparser, a C99 compiler, which uses libFIRM for > optimisation and code generation (lang/cparser in the ports). This is helpful, my only nit would be to run it through sort. :) Andrew From max at love2party.net Mon Feb 2 12:47:32 2009 From: max at love2party.net (Max Laier) Date: Mon Feb 2 12:47:39 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <49874CA8.5090605@gmx.de> References: <49874CA8.5090605@gmx.de> Message-ID: <200902022147.22862.max@love2party.net> On Monday 02 February 2009 20:42:32 Christoph Mallon wrote: > Hi, > > I compiled a list of all local variables in src/sys/ (r188000), which > are only written to, but never read. This is more than the GCC warning, > which only complains about variables, which are only declared (and maybe > initialised) and not used otherwise. In contrast this list contains > variables with the following usage pattern: > > int w = 42; // GCC warns about this ... > int x; // ... but not this > x = 23; > x++; > return 0; > > The list contains about 700 entries. About three dozen concern variables > named 'error'. Here's one *example* from the list: > > sys/dev/kbdmux/kbdmux.c:1304 > > In the function kbdmux_modevent() the variable 'error' is assigned > values eight times, but at the end of the function there is just a > return 0; and the variable is never read. Probably the value should be > returned. > > You can find the list here: > http://tron.homeunix.org/unread_variables.log > > The list was generated by cparser, a C99 compiler, which uses libFIRM > for optimisation and code generation (lang/cparser in the ports). > > > A small disclaimer: There might be some false positives due to errors > which are caused by HEAD sources in combination with my installed 7.x > headers plus a hacked up build process. Also some warnings are the > result from variables, which are only used in debug macros, so td = > curthread; KASSERT(td != NULL); provokes a warning (I consider this bad > style). Nonetheless the number of false positives should be low. If > there is interest, then I can compile a "proper" list. Are you interested in false positive reports? If so, I think sys/contrib/pf/net/pf.c:2931 is one. Seems cparser is confused by the union in struct assignment, maybe? Or it suffers from the similar issue with switch/case-statements as gcc. saddr is read from in all but the default case. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From christoph.mallon at gmx.de Mon Feb 2 12:56:52 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Mon Feb 2 12:57:00 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <200902022147.22862.max@love2party.net> References: <49874CA8.5090605@gmx.de> <200902022147.22862.max@love2party.net> Message-ID: <49875E0A.5070209@gmx.de> Max Laier schrieb: > On Monday 02 February 2009 20:42:32 Christoph Mallon wrote: >> A small disclaimer: There might be some false positives due to errors >> which are caused by HEAD sources in combination with my installed 7.x >> headers plus a hacked up build process. Also some warnings are the >> result from variables, which are only used in debug macros, so td = >> curthread; KASSERT(td != NULL); provokes a warning (I consider this bad >> style). Nonetheless the number of false positives should be low. If >> there is interest, then I can compile a "proper" list. > > Are you interested in false positive reports? If so, I think > sys/contrib/pf/net/pf.c:2931 is one. Seems cparser is confused by the union > in struct assignment, maybe? Or it suffers from the similar issue with > switch/case-statements as gcc. saddr is read from in all but the default > case. When neither INET nor INET6 is set, daddr and saddr are only written to. So this part should be enclosed in #if defined INET || defined INET6. Probably this file is not compiled at all, when neither INET nor INET6 are set, so this is certainly the result of the "hacked up build process"-part, sorry. From rizzo at iet.unipi.it Mon Feb 2 13:14:42 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Mon Feb 2 13:14:49 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <49874CA8.5090605@gmx.de> References: <49874CA8.5090605@gmx.de> Message-ID: <20090202210345.GD20288@onelab2.iet.unipi.it> On Mon, Feb 02, 2009 at 08:42:32PM +0100, Christoph Mallon wrote: > Hi, > > I compiled a list of all local variables in src/sys/ (r188000), which > are only written to, but never read. This is more than the GCC warning, interesting list, thanks. Also, 700 entries is not a bad result considering the size of the codebase and the age of parts of it (i am pretty sure there is a lot of code 15+ years old which received little if any mainteinance or use in the past decade). (and i have nothing against old code except that compilers, coding practices and the amount of peer review have improved a lot over time, and so -- with some exceptions -- it is easier to prevent some of these issues with more recent code). cheers luigi From maksim.yevmenkin at gmail.com Mon Feb 2 14:01:53 2009 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Mon Feb 2 14:02:00 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <49874CA8.5090605@gmx.de> References: <49874CA8.5090605@gmx.de> Message-ID: On Mon, Feb 2, 2009 at 11:42 AM, Christoph Mallon wrote: > Hi, > > I compiled a list of all local variables in src/sys/ (r188000), which are > only written to, but never read. This is more than the GCC warning, which > only complains about variables, which are only declared (and maybe > initialised) and not used otherwise. In contrast this list contains > variables with the following usage pattern: > > int w = 42; // GCC warns about this ... > int x; // ... but not this > x = 23; > x++; > return 0; > > The list contains about 700 entries. About three dozen concern variables > named 'error'. Here's one *example* from the list: > > sys/dev/kbdmux/kbdmux.c:1304 > > In the function kbdmux_modevent() the variable 'error' is assigned values > eight times, but at the end of the function there is just a return 0; and > the variable is never read. Probably the value should be returned. fixed. thanks for reporting! thanks, max From phk at phk.freebsd.dk Mon Feb 2 14:41:39 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Feb 2 14:41:49 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: Your message of "Mon, 02 Feb 2009 20:42:32 +0100." <49874CA8.5090605@gmx.de> Message-ID: <7342.1233613563@critter.freebsd.dk> In message <49874CA8.5090605@gmx.de>, Christoph Mallon writes: >I compiled a list of all local variables in src/sys/ (r188000), which >are only written to, but never read. Bravo! -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From linimon at lonesome.com Mon Feb 2 16:15:18 2009 From: linimon at lonesome.com (Mark Linimon) Date: Mon Feb 2 16:38:01 2009 Subject: Fw: request responsibility timeout In-Reply-To: <49871CA2.4040606@bsdforen.de> References: <4985D26E.9000604@bsdforen.de> <200902010926.00728.fbsd.questions@rachie.is-a-geek.net> <49871CA2.4040606@bsdforen.de> Message-ID: <20090202234706.GA7455@soaustin.net> On Mon, Feb 02, 2009 at 05:17:38PM +0100, Dominic Fandrey wrote: > I want to request a responsibility timeout for bin/120784 (with bugmeister hat) AFAIK no one else other than rodrigc has been doing work on the mount utilities, so I don't know who else to assign it to. mcl From gballet at gmail.com Tue Feb 3 05:32:24 2009 From: gballet at gmail.com (Guillaume Ballet) Date: Tue Feb 3 05:32:31 2009 Subject: freebsd-hackers Digest, Vol 305, Issue 6 In-Reply-To: <20090131120022.7519F10656EB@hub.freebsd.org> References: <20090131120022.7519F10656EB@hub.freebsd.org> Message-ID: On Sat, Jan 31, 2009 at 1:00 PM, wrote: > > Message: 4 > Date: Fri, 30 Jan 2009 09:41:58 -0800 > From: "Matthew Fleming" > Subject: Dynamic ddb commands > To: > Message-ID: <1233337318.13748.101.camel@amaretto> > Content-Type: text/plain; charset="iso-8859-1" > > I'm working on BSD 6.x and of course the set of ddb commands is static > to whatever is in the kernel at compile. I see that BSD 7.1 has dynamic > commands using sysinits and sysuninit's to call a new > db_[un]register_cmd. > > I see this, though, only after I have spent a day or so adding a > linker_file_[un]register_ddb() that works similarly to how sysinits are > merged for the boot-time modules (malloc and copy pointers). It seems > to me that this solution (have the linker look for db_set and db_cmd_set > and, if there are any entries, malloc and save pointers to commands) is > more efficient in terms of space than adding a LIST to the command > structure and then forcing sysinits to run. This is what I proposed at first, you can find a patch doing just that following that thread: http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg65165.html But Sam and John suggested to use the sysinit facility instead. I am convinced they are right: - Sysinits are a proven system, this is the least friction path. Also, by adding functions to the linker you still have the (small but existing) risk of adding bugs to that system. - This require adding new sections. Now, on regular x86 machines that doesn't seem too much of a problem. I am however currently working on a port of FreeBSD to some ARM cortex-based board. When debugging using JTAG, the smaller amount of sections whose location in physical memory I have to care about, the simpler for me and those doing the same kind of work. - You are not "forcing" sysinits to run: they are run no matter what when loading a module. - You can still debug sysinits when inserting modules, as the core of the debugger is already running and the list has been loaded in memory. The modules command itself will of course not be available, but the base commands will still be here. Cheers, Guillaume From kamikaze at bsdforen.de Tue Feb 3 07:37:23 2009 From: kamikaze at bsdforen.de (Dominic Fandrey) Date: Tue Feb 3 07:37:55 2009 Subject: Fw: request responsibility timeout In-Reply-To: <20090202234706.GA7455@soaustin.net> References: <4985D26E.9000604@bsdforen.de> <200902010926.00728.fbsd.questions@rachie.is-a-geek.net> <49871CA2.4040606@bsdforen.de> <20090202234706.GA7455@soaustin.net> Message-ID: <4988648D.1090205@bsdforen.de> Mark Linimon wrote: > On Mon, Feb 02, 2009 at 05:17:38PM +0100, Dominic Fandrey wrote: >> I want to request a responsibility timeout for bin/120784 > > (with bugmeister hat) AFAIK no one else other than rodrigc has been > doing work on the mount utilities, so I don't know who else to assign > it to. > > mcl I suppose I'll just have to wait. Thank you for taking a look. Regards From aryeh.friedman at gmail.com Tue Feb 3 11:17:41 2009 From: aryeh.friedman at gmail.com (Aryeh M. Friedman) Date: Tue Feb 3 11:17:49 2009 Subject: usinig cvs diff to make a patch Message-ID: <49889842.9050103@gmail.com> I use a local cvs repo and I have modified a port and which to submit an update for it how do I generate a patch file with cvs (cvs diff seems to give a unusable format)? From dimitry at andric.com Tue Feb 3 11:56:19 2009 From: dimitry at andric.com (Dimitry Andric) Date: Tue Feb 3 11:56:26 2009 Subject: usinig cvs diff to make a patch In-Reply-To: <49889842.9050103@gmail.com> References: <49889842.9050103@gmail.com> Message-ID: <49889CCD.8010609@andric.com> On 2009-02-03 20:17, Aryeh M. Friedman wrote: > I use a local cvs repo and I have modified a port and which to submit an > update for it how do I generate a patch file with cvs (cvs diff seems to > give a unusable format)? Use "cvs diff -up" for unified diff format, with function prototypes. From chuckr at telenix.org Tue Feb 3 12:10:12 2009 From: chuckr at telenix.org (Chuck Robey) Date: Tue Feb 3 12:10:20 2009 Subject: usinig cvs diff to make a patch In-Reply-To: <49889842.9050103@gmail.com> References: <49889842.9050103@gmail.com> Message-ID: <49889BD1.40107@telenix.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Aryeh M. Friedman wrote: > I use a local cvs repo and I have modified a port and which to submit an > update for it how do I generate a patch file with cvs (cvs diff seems to > give a unusable format)? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" cvs diff -u gives the unified format (or cvs diff -c for context, get the pattern?) Or, copying from the web page http://www.eyrie.org/~eagle/notes/cvs/basic-usage.html, which shows how to use the ~/.cvsrc file to make common cvs commands default the way you want them to, you could put into that file the line "diff -u" so it always gives you the unified diff format, which just happens to be the easiest for humans to read, and the format specified in FreeBSD (hint, hint). -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmIm9EACgkQz62J6PPcoOnRlQCeIT8vsLZ6LI764WOaTQy72ym4 Tp0AoJ43TIQYsGFz8YYD2ChhMSUU0Wpc =ta5/ -----END PGP SIGNATURE----- From sullrich at gmail.com Tue Feb 3 12:26:19 2009 From: sullrich at gmail.com (Scott Ullrich) Date: Tue Feb 3 12:26:26 2009 Subject: usinig cvs diff to make a patch In-Reply-To: <49889842.9050103@gmail.com> References: <49889842.9050103@gmail.com> Message-ID: On Tue, Feb 3, 2009 at 2:17 PM, Aryeh M. Friedman wrote: > I use a local cvs repo and I have modified a port and which to submit an > update for it how do I generate a patch file with cvs (cvs diff seems to > give a unusable format)? Maybe try cvs diff -uN ? Scott From drosih at rpi.edu Tue Feb 3 12:48:40 2009 From: drosih at rpi.edu (Garance A Drosihn) Date: Tue Feb 3 12:48:53 2009 Subject: usinig cvs diff to make a patch In-Reply-To: <49889842.9050103@gmail.com> References: <49889842.9050103@gmail.com> Message-ID: At 2:17 PM -0500 2/3/09, Aryeh M. Friedman wrote: >I use a local cvs repo and I have modified a port and which to >submit an update for it how do I generate a patch file with cvs (cvs >diff seems to give a unusable format)? try: cvs diff -u In my case, i have added the following line to my ~/.cvsrc file: diff -uN (-N means "treat absent files as empty") -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu From tevans.uk at googlemail.com Tue Feb 3 13:03:48 2009 From: tevans.uk at googlemail.com (Tom Evans) Date: Tue Feb 3 13:03:55 2009 Subject: usinig cvs diff to make a patch In-Reply-To: <49889BD1.40107@telenix.org> References: <49889842.9050103@gmail.com> <49889BD1.40107@telenix.org> Message-ID: <1233693093.82064.4.camel@jacob.nubtek.com> On Tue, 2009-02-03 at 14:32 -0500, Chuck Robey wrote: > Aryeh M. Friedman wrote: > > I use a local cvs repo and I have modified a port and which to submit an > > update for it how do I generate a patch file with cvs (cvs diff seems to > > give a unusable format)? > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > cvs diff -u gives the unified format (or cvs diff -c for context, get the > pattern?) Or, copying from the web page > http://www.eyrie.org/~eagle/notes/cvs/basic-usage.html, which shows how to use > the ~/.cvsrc file to make common cvs commands default the way you want them to, > you could put into that file the line "diff -u" so it always gives you the > unified diff format, which just happens to be the easiest for humans to read, > and the format specified in FreeBSD (hint, hint). When reading this, I was immediately reminded of this passage from development(7), describing how to set up a local cvs repository for src/ports...: ... you need to set up a ~/.cvsrc (/root/.cvsrc) file, as shown below, for proper cvs(1) operation. Using ~/.cvsrc to specify cvs(1) defaults is an excellent way to ``file and forget'', but you should never forget that you put them in there. # cvs -q diff -u update -Pd checkout -P Cheers Tom From shilp.kamal at yahoo.com Tue Feb 3 13:20:45 2009 From: shilp.kamal at yahoo.com (Kamlesh Patel) Date: Tue Feb 3 13:20:51 2009 Subject: pmap.h: No such file or directory---- Debugging error Message-ID: <12567.73601.qm@web45406.mail.sp1.yahoo.com> Hi All, I have two system one is Debug and another is Target machine. I am debugging using kgdb from Debug machine. Line no: 272 into the function vm_page_startup( vaddr ) of vm_page.c - Freebsd 7.0, mapped=pmap() is there. when i press "S" (step by step debug) then (kgdb)s pmap_map( , , , ,) at /usr/src/sys/i386/i386/pmap.c:1089 1089 va=sva=*virt (kgdb)s 1090 while(start < end){ (kgdb)s 310 pmap.h: No such file or directory Same thing happens with pmap_invalidate_range() pcpu.h: No such file or directory Could anyone help me? Thanks Kamlesh MS CS CSUS From julian at elischer.org Tue Feb 3 13:35:40 2009 From: julian at elischer.org (Julian Elischer) Date: Tue Feb 3 13:35:47 2009 Subject: pmap.h: No such file or directory---- Debugging error In-Reply-To: <12567.73601.qm@web45406.mail.sp1.yahoo.com> References: <12567.73601.qm@web45406.mail.sp1.yahoo.com> Message-ID: <4988B8B1.6040007@elischer.org> Kamlesh Patel wrote: > Hi All, > > I have two system one is Debug and another is Target machine. I am debugging using kgdb from Debug machine. > > Line no: 272 into the function vm_page_startup( vaddr ) of vm_page.c - Freebsd 7.0, > mapped=pmap() is there. when i press "S" (step by step debug) then > > (kgdb)s > pmap_map( , , , ,) at /usr/src/sys/i386/i386/pmap.c:1089 > 1089 va=sva=*virt > (kgdb)s > 1090 while(start < end){ > (kgdb)s > 310 pmap.h: No such file or directory > > > > Same thing happens with pmap_invalidate_range() > pcpu.h: No such file or directory > > Could anyone help me? have you tried: (gdb) help directory Add directory DIR to beginning of search path for source files. Forget cached info on source file locations and line positions. DIR can also be $cwd for the current working directory, or $cdir for the directory in which the source file was compiled into object code. With no argument, reset the search path to $cdir:$cwd, the default. > > Thanks > Kamlesh > MS CS CSUS > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From csjp at freebsd.org Tue Feb 3 15:32:44 2009 From: csjp at freebsd.org (Christian Peron) Date: Tue Feb 3 15:32:57 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <49874CA8.5090605@gmx.de> References: <49874CA8.5090605@gmx.de> Message-ID: <20090203231155.GA69101@jnz.sqrt.ca> I started following up on this and ran into an issue for these: sys/net/bpf_buffer.c:133: warning: variable 'dst' is never read sys/net/bpf_buffer.c:134: warning: variable 'count' is never read sys/net/bpf_buffer.c:142: warning: variable 'dst' is never read /* * Scatter-gather data copy from an mbuf chain to the current kernel buffer. */ void bpf_buffer_append_mbuf(struct bpf_d *d, caddr_t buf, u_int offset, void *src, u_int len) { const struct mbuf *m; u_char *dst; u_int count; m = (struct mbuf *)src; dst = (u_char *)buf + offset; while (len > 0) { if (m == NULL) panic("bpf_mcopy"); count = min(m->m_len, len); bcopy(mtod(m, void *), dst, count); m = m->m_next; [..] Does it not consider being passed as an argument to a function as being read? Cheers From jhb at freebsd.org Wed Feb 4 06:26:29 2009 From: jhb at freebsd.org (John Baldwin) Date: Wed Feb 4 06:26:35 2009 Subject: bus_dma (9). What exactly means "Loading of memory allocation" ? In-Reply-To: <671bb5fc0902020823p3ee8d9edl715a074603ab97de@mail.gmail.com> References: <6699015.16785.1233539817447.JavaMail.root@vms063.mailsrvcs.net> <7d6fde3d0902020046o7640f217ye88336b53920a538@mail.gmail.com> <671bb5fc0902020823p3ee8d9edl715a074603ab97de@mail.gmail.com> Message-ID: <200902040900.24320.jhb@freebsd.org> On Monday 02 February 2009 11:23:33 am Alexej Sokolov wrote: > Hi, > thanx for your answer. I checked the source code of the *dma() functions. > If I understand it correctly, "loading of memory allocation" means the > following: > > 1. At first memory allocation should be done: bufp = *alloc(sizeof ....) > 2. then in ... _bus_dmamap_load_buffer() we get physical addres of allocated > buffer: > if (pmap) > curaddr = pmap_extract(pmap, vaddr); > else > curaddr = pmap_kextract(vaddr); > > ... then some "magic" with bouncing > > 3. then physical address will passed to dmat->segments > segs[seg].ds_addr = curaddr; > segs[seg].ds_len = sgsize; > > Ok, it all means: getting of physical address of allocated buffer. If > physical space not accessble for device, allocating bounce buffers. Getting > of physical addresses of allocated buffers. And then put these physical > addresses and sizes of buffers in dmat->segments array. <- loading of > memory allocation (-: Yes. On architectures with an IOMMU, the load may also program entries into the IOMMU for the specified buffer and then populate the S/G array with the associated DMA addresses (sparc64 uses this). I think the "load" name has more to do with this case in that you are "loading" a buffer into the DMA virtual address space (with IOMMUs you have a separate virtual address space for DMA that is not 1:1 with physical addresses as on i386 machines). If you look at bus_dma as basically implementing an abstract IOMMU on all architectures then it might make a bit more sense. -- John Baldwin From christoph.mallon at gmx.de Wed Feb 4 06:54:44 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Wed Feb 4 06:55:07 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <20090203231155.GA69101@jnz.sqrt.ca> References: <49874CA8.5090605@gmx.de> <20090203231155.GA69101@jnz.sqrt.ca> Message-ID: <4989AC31.6000904@gmx.de> Christian Peron schrieb: > I started following up on this and ran into an issue for these: > > sys/net/bpf_buffer.c:133: warning: variable 'dst' is never read > sys/net/bpf_buffer.c:134: warning: variable 'count' is never read > sys/net/bpf_buffer.c:142: warning: variable 'dst' is never read > > > /* > * Scatter-gather data copy from an mbuf chain to the current kernel buffer. > */ > void > bpf_buffer_append_mbuf(struct bpf_d *d, caddr_t buf, u_int offset, void *src, > u_int len) > { > const struct mbuf *m; > u_char *dst; > u_int count; > > m = (struct mbuf *)src; > dst = (u_char *)buf + offset; > while (len > 0) { > if (m == NULL) > panic("bpf_mcopy"); > count = min(m->m_len, len); > bcopy(mtod(m, void *), dst, count); > m = m->m_next; > [..] > > Does it not consider being passed as an argument to a function as > being read? Yes, function arguments are considered being read. The problem is different here: mtod() should be a macro, but the macro declaration was missing (*cough* hacked build process *cough*). So the parser tried to parse this as function call. Then it hit the "void *", which confused it - it got a type while parsing an expression. I improved the error correction, resolved a few other problems, too, and generated a new list: http://tron.homeunix.org/unread_variables.log (The list has a date at the top, if it is missing, you see the old list in your browser cache) The false positives, which you mentioned, are gone now - thanks for reporting this. The list now contains about 1.000 entries and about 60 concern variables named 'error'. From laladelausanne at gmail.com Wed Feb 4 08:05:06 2009 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Wed Feb 4 08:05:12 2009 Subject: blockable sleep lock (sleep mutex) 16 In-Reply-To: <498736C2.3040207@elischer.org> References: <02026848-7F83-405C-B4F3-EDD8B47DA294@gmail.com> <32679C0A-28C1-4D7A-950C-580787F3971D@gmail.com> <200902020845.21773.jhb@freebsd.org> <498736C2.3040207@elischer.org> Message-ID: On 2 Feb 2009, at 19:09 , Julian Elischer wrote: >>> It says "non-sleepable locks", yet it classifies click_instance >>> as sleep mutex. I think witness code should emit messages which >>> are more clear. >> It is confusing, but you can't do an M_WAITOK malloc while holding >> a mutex. Basically, sleeping actually means calling "*sleep() >> (such as mtx_sleep()) or cv_*wait*()". Blocking on a mutex is not >> sleeping, it's "blocking". Some locks (such as sx(9)) do "sleep" >> when you contest them. In the scheduler, sleeping and blocking are >> actually quite different (blocking uses turnstiles that handle >> priority inversions via priority propagation, sleeping uses sleep >> queues which do not do any of that). The underyling idea is that >> mutexes should be held for "short" periods of time, and that any >> sleeps are potentially unbounded. Holding a mutex while sleeping >> could result in a mutex being held for a long time. > > > the locking overview page > man 9 locking > tries to explain this.. > I've been pestering John to proofread it and make suggestiosn for a > while now. Thanks John and Julian. I agree, man pages should be more clear :) I've switched from using mtx to sx locks, since they offer sleeping while hold. Unfortunately, I've ran into something really weird now, when I unload the module: ---8<--- #0 doadump () at pcpu.h:195 #1 0xffffffff8049ef98 in boot (howto=260) at /usr/src/sys/kern/ kern_shutdown.c:418 #2 0xffffffff8049f429 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xffffffff8075cd26 in trap_fatal (frame=0xc, eva=Variable "eva" is not available. ) at /usr/src/sys/amd64/amd64/trap.c:764 #4 0xffffffff8075da62 in trap (frame=0xffffffff87699940) at /usr/src/ sys/amd64/amd64/trap.c:290 #5 0xffffffff80743bfe in calltrap () at /usr/src/sys/amd64/amd64/ exception.S:209 #6 0xffffffff8052a411 in strcmp (s1=0xffffffff80824a0c "sigacts", s2=0xffffffff877cd3a9
) at /usr/src/sys/libkern/strcmp.c:45 #7 0xffffffff804d7c61 in enroll (description=0xffffffff80824a0c "sigacts", lock_class=0xffffffff80a19fe0) at /usr/src/sys/kern/subr_witness.c:1439 #8 0xffffffff804d7fb1 in witness_init (lock=0xffffff00016f4ca8) at / usr/src/sys/kern/subr_witness.c:618 #9 0xffffffff8049fd31 in sigacts_alloc () at /usr/src/sys/kern/ kern_sig.c:3280 #10 0xffffffff80481121 in fork1 (td=0xffffff0001384a50, flags=20, pages=Variable "pages" is not available. ) at /usr/src/sys/kern/kern_fork.c:453 #11 0xffffffff80481450 in fork (td=0xffffff0001384a50, uap=Variable "uap" is not available. ) at /usr/src/sys/kern/kern_fork.c:106 #12 0xffffffff8075d260 in syscall (frame=0xffffffff87699c80) at /usr/ src/sys/amd64/amd64/trap.c:907 #13 0xffffffff80743e0b in Xfast_syscall () at /usr/src/sys/amd64/amd64/ exception.S:330 #14 0x0000000800ca0a6c in ?? () --->8--- and in fra 7: (kgdb) p *w $5 = {w_name = 0xffffffff877cd3a9
, w_class = 0xffffffff80a19fe0, w_list = { stqe_next = 0xffffffff80accce0}, w_typelist = {stqe_next = 0xffffffff80accce0}, w_children = 0x0, w_file = 0xffffffff877d1fa0
, w_line = 307, w_level = 0, w_refcount = 2, w_Giant_squawked = 0 '\0', w_other_squawked = 0 '\0', w_same_squawked = 0 '\0', w_displayed = 0 '\0'} (kgdb) p *w->w_class $6 = {lc_name = 0xffffffff808564e0 "sleep mutex", lc_flags = 9, lc_ddb_show = 0xffffffff80492e6b , lc_lock = 0xffffffff804938be , lc_unlock = 0xffffffff804933fc } This happens after modevent exists. What puzzles me here is w_refcount of 2, while w_name is out of bounds. Locks I've created I properly destroyed (at least I think I did :)). Cheers, Nikola From csjp at freebsd.org Wed Feb 4 10:27:18 2009 From: csjp at freebsd.org (Christian Peron) Date: Wed Feb 4 10:27:32 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <4989AC31.6000904@gmx.de> References: <49874CA8.5090605@gmx.de> <20090203231155.GA69101@jnz.sqrt.ca> <4989AC31.6000904@gmx.de> Message-ID: <20090204182715.GA75911@jnz.sqrt.ca> On Wed, Feb 04, 2009 at 03:54:41PM +0100, Christoph Mallon wrote: [..] > > Yes, function arguments are considered being read. The problem is > different here: mtod() should be a macro, but the macro declaration was > missing (*cough* hacked build process *cough*). So the parser tried to > parse this as function call. Then it hit the "void *", which confused it > - it got a type while parsing an expression. I improved the error > correction, resolved a few other problems, too, and generated a new list: > > http://tron.homeunix.org/unread_variables.log > (The list has a date at the top, if it is missing, you see the old list > in your browser cache) > > The false positives, which you mentioned, are gone now - thanks for > reporting this. The list now contains about 1.000 entries and about 60 > concern variables named 'error'. Also.. one other thing I noticed: void bpf_buffer_append_mbuf(struct bpf_d *d, caddr_t buf, u_int offset, void *src, u_int len) { const struct mbuf *m; u_char *dst; u_int count; m = (struct mbuf *)src; dst = (u_char *)buf + offset; while (len > 0) { if (m == NULL) panic("bpf_mcopy"); count = min(m->m_len, len); bcopy(mtod(m, void *), dst, count); m = m->m_next; dst += count; len -= count; } } dst += count In this expression, both dst and count are read since this is the same thing as: dst = dst + count; From christoph.mallon at gmx.de Wed Feb 4 10:36:37 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Wed Feb 4 10:36:48 2009 Subject: write-only variables in src/sys/ - possible bugs In-Reply-To: <20090204182715.GA75911@jnz.sqrt.ca> References: <49874CA8.5090605@gmx.de> <20090203231155.GA69101@jnz.sqrt.ca> <4989AC31.6000904@gmx.de> <20090204182715.GA75911@jnz.sqrt.ca> Message-ID: <4989E030.20609@gmx.de> Christian Peron schrieb: > On Wed, Feb 04, 2009 at 03:54:41PM +0100, Christoph Mallon wrote: > [..] >> Yes, function arguments are considered being read. The problem is >> different here: mtod() should be a macro, but the macro declaration was >> missing (*cough* hacked build process *cough*). So the parser tried to >> parse this as function call. Then it hit the "void *", which confused it >> - it got a type while parsing an expression. I improved the error >> correction, resolved a few other problems, too, and generated a new list: >> >> http://tron.homeunix.org/unread_variables.log >> (The list has a date at the top, if it is missing, you see the old list >> in your browser cache) >> >> The false positives, which you mentioned, are gone now - thanks for >> reporting this. The list now contains about 1.000 entries and about 60 >> concern variables named 'error'. > > Also.. one other thing I noticed: > > void > bpf_buffer_append_mbuf(struct bpf_d *d, caddr_t buf, u_int offset, void *src, > u_int len) > { > const struct mbuf *m; > u_char *dst; > u_int count; > > m = (struct mbuf *)src; > dst = (u_char *)buf + offset; > while (len > 0) { > if (m == NULL) > panic("bpf_mcopy"); > count = min(m->m_len, len); > bcopy(mtod(m, void *), dst, count); > m = m->m_next; > dst += count; > len -= count; > } > } > > dst += count > > In this expression, both dst and count are read since this is the > same thing as: > > dst = dst + count; No, the analysis *explicitly* marks "x" in neither "x += 1" nor "x = x + 1" as read. The value of the variable in these expressions is only read to calculate its own new value. Therefore it will complain about int x = 23; x++; x += 1; x = x + 1; return 0; This is not a bug, it is a feature. (: The problem here solely was the insufficient error recovery in the bcopy() line, which caused the only "real" user of "dst" to disappear. I corrected this problem and as you can see the false positives are no longer on the list. From avg at icyb.net.ua Wed Feb 4 11:12:01 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Feb 4 11:12:08 2009 Subject: kobj methods (DEVMETHOD) that have differing signatures (in src/sys) Message-ID: <4989E87B.5010000@icyb.net.ua> This based on the (much) earlier proposal described here: http://lists.freebsd.org/pipermail/freebsd-arch/2008-April/007982.html The patch was applied to the recent current sources and LINT kernels for all architectures that have LINT/NOTES (i.e. arm excluded) were built. Here's a link to the list of files and line numbers where KOBJ methods are set with functions that have differing signatures: http://www.icyb.net.ua/~avg/kobj_method_sigs.txt List of the most common issues can be found at the first link. Hope this is useful. -- Andriy Gapon From avg at icyb.net.ua Wed Feb 4 11:19:11 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Feb 4 11:19:18 2009 Subject: NO_WERROR vs kernel builds Message-ID: <4989EA2A.6050601@icyb.net.ua> It seems that kernel builds ignore NO_WERROR. Is this on purpose or by accident? I think that this happens because of the following lines in sys/conf/kern.pre.mk: .if ${CC} != "icc" CFLAGS+= -fno-common -finline-limit=${INLINE_LIMIT} CFLAGS+= --param inline-unit-growth=100 CFLAGS+= --param large-function-growth=1000 .if ${MACHINE_ARCH} == "amd64" || ${MACHINE} == "i386" || \ ${MACHINE_ARCH} == "ia64" || ${MACHINE_ARCH} == "powerpc" || \ ${MACHINE_ARCH} == "sparc64" WERROR?= -Werror .endif .endif I had to specify WERROR= on make's command line to catch a certain kind of warnings in bulk instead of one by one. This was not obvious. -- Andriy Gapon From imp at bsdimp.com Wed Feb 4 12:12:31 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Wed Feb 4 12:12:44 2009 Subject: kobj methods (DEVMETHOD) that have differing signatures (in src/sys) In-Reply-To: <4989E87B.5010000@icyb.net.ua> References: <4989E87B.5010000@icyb.net.ua> Message-ID: <20090204.131104.2064955776.imp@bsdimp.com> In message: <4989E87B.5010000@icyb.net.ua> Andriy Gapon writes: : : This based on the (much) earlier proposal described here: : http://lists.freebsd.org/pipermail/freebsd-arch/2008-April/007982.html : : The patch was applied to the recent current sources and LINT kernels for : all architectures that have LINT/NOTES (i.e. arm excluded) were built. : : Here's a link to the list of files and line numbers where KOBJ methods : are set with functions that have differing signatures: : http://www.icyb.net.ua/~avg/kobj_method_sigs.txt : : List of the most common issues can be found at the first link. : : Hope this is useful. This is very helpful. I'll work through the low-hanging fruit. If others want to work as well, please ping me. Warner From pluknet at gmail.com Thu Feb 5 03:01:42 2009 From: pluknet at gmail.com (pluknet) Date: Thu Feb 5 03:01:49 2009 Subject: linking .a lib with another library Message-ID: Hi. The problem is: I have a program which is statically linked to my static lib during a build. The lib is in turn uses kvm*() calls, hence it should be linked with libkvm. During the build of my binaries which are linked to my libucron.a I get the following: /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x6e): In function `check_pidfile': : undefined reference to `kvm_open' /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x8b): In function `check_pidfile': : undefined reference to `kvm_getprocs' /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0xbe): In function `check_pidfile': : undefined reference to `kvm_getprocs' /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x106): In function `check_pidfile': : undefined reference to `kvm_close' *** Error code 1 Ok, if I add SHLIB_MAJOR directive in lib/Makefile (or remove any kvm* references) then all is ok. But the lib is used to be static. So the question is can I somehow link two .a libraries? e.g. libkvm.a + libucron.a + my binaries. Thanks. -- wbr, pluknet From cornek at striata.com Thu Feb 5 04:14:03 2009 From: cornek at striata.com (Corne Kotze) Date: Thu Feb 5 04:14:14 2009 Subject: SSH Problem In-Reply-To: References: <1229934159.8928.20.camel@jackal> <1229937727.8928.24.camel@jackal> Message-ID: <1233835135.10262.31.camel@jackal> Hi all, Just an update on the ssh with keys issue I had. To refresh, I run a sftp server which chroot users to their assigned folders. http://www.bsdguides.org/guides/freebsd/security/sftp_chroot_users.php That disabled the keys authentication I had running on my server. The solution that worked for me: Verify the connecting server SSH version [local-host]$ssh -V OpenSSH_5.0p1, OpenSSL 0.9.8g 19 Oct 2007 Generate key-pair on the connecting server without a password [local-host]$ ssh-keygen ?t dsa Generating public/private dsa key pair. Enter file in which to save the key (/home/user/.ssh/id_dsa): Created directory '/home/user/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/user/.ssh/id_dsa. Your public key has been saved in /home/user/.ssh/id_dsa.pub. The key fingerprint is: 3b:2a:d2:ac:8c:71:81:7e:b7:31:21:11:b8:e8:31:ad user@localhost The public key and private key are typically stored in .ssh folder under your home directory. In this example, it is under /home/user/.sshd. You should not share the private key with anybody. Convert openSSH public key to SSH2 public key On the connecting server that is running openSSH, convert the openSSH public key to SSH2 public key using ssh-keygen as shown below. [local-host]$ ssh-keygen -e -f ~/.ssh/id_dsa.pub > ~/.ssh/id_dsa_ssh2.pub Install the public-key on the remote-host that is running SSH2 Create a new public key file on remote-host and copy paste the converted SSH2 key from the connecting server. [remote-host]$ mkdir .ssh2 [remote-host]$ vi ~/.ssh2/server.lan_ssh2_key.pub ?- BEGIN SSH2 PUBLIC KEY ?- Comment: ?2048-bit RSA, converted from OpenSSH by jsmith@local-host? DDDDB3NzaC1yc2EAAAABDmbrdomPh9rWfjZ1+7Q369zsBEa7wS1RxzWRQ0Bmr9FSplI 3ADBEBC/6cbdf/v0r6Cp5y5kusP07AOzo2F7MBDSZBtS/MbYJiIxvocoaxG2bQyz3yYjU YcpzGMD182bnA8kRxmGg+R5pVXM34lx3iSSgd8r3RzZKnDpEvEInnI7pQvUBoEbYCXPUeZ LQvQAkz6+Pb6SsNp-dop/qgv9qyfbyMz1iKUZGadG146GtanL5QtRwyAeD187gMzzrGzMFP LWjdzWpGILdZ5gq7wwRpbcXFUskVrS2ZjDe676XlTN1k5QSZmSYUuttDdrjB5SFiMpsre8 a7cQuMS178i9eDBEC== ?- END SSH2 PUBLIC KEY ?- [remote-host]$ chmod 700 ~/.ssh2 [remote-host]$ chmod go-rwx ~/.ssh2/* Add the above public key file name to the authorization file on the remote-host as shown below. [remote-host]$ vi ~/.ssh2/authorization Key server.lan_ssh2_key.pub Verify the Login from the connecting server to remote-host using the SSH2 key authentication. [local-host]$ ssh -l user remote-host The authenticity of host ?local-host? can?t be established. DSA key fingerprint is a5:f6:2e:e6:a9:b2:7b:0e:e7:ae:cb:6c:7b:f5:6d:06. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ?local-host? (DSA) to the list of known hosts. Last login: Sat Jun 21 2008 23:13:00 -0700 from 192.168.1.102 No mail. [remote-host]$ Hope this will help someone. On Mon, 2008-12-22 at 13:22 +0300, Eygene Ryabinkin wrote: > Corne, > > Mon, Dec 22, 2008 at 11:22:07AM +0200, Corne Kotze wrote: > > Thank for the reply. > > Sorry for the ignorance, but I should have added this as well. > > > > I am running apart from other things, a secure ftp server on this box as > > well that chroot the users to their home directories. > > > > I got the setup information from the following link: > > http://www.bsdguides.org/guides/freebsd/security/sftp_chroot_users.php > > Ahm, SSH.com's realization of SSH suite. Forgot about this, sorry. > I had never used it, so can't say how to make it work with public key > authentication. But read on ;)) > > However, OpenSSH had gained the chroot ability in February 2008, > http://undeadly.org/cgi?action=article&sid=20080220110039 > > But if you're running 6.x, you won't be able to use it -- it was > imported only to 7.x and -CURRENT, > SVN rev 182634 on 2008-09-01 20:03:13Z by des > > Though, no hope is lost -- security/openssh-portable is at 5.0p1, and > chroot support is there. But it is prone to the X11 MITM attack (at > least on HP/UX, don't currently know is FreeBSD is affected), > http://www.openssh.com/txt/release-5.1 > Your mileage may vary, if, for example, you're not using X11 forwarding, > then you might be fine with this. > > > Setting the "rc.conf" file to: > > sshd_enable="YES" > > sshd2_enable="NO" > > > > Then my sftp setup does not work properly, unless I am missing something > > that I can set in the "/etc/ssh/sshd_config" file. > > Ooookey, if you still prefer SSH.com's software, you may find the following > article very enlightening, > http://www.ssh.com/support/documentation/online/ssh/adminguide/32/Public-Key_Authentication-2.html > > At least for me it looks very sane and verbose. Corne Kotze Systems Administrator Striata messaging innovation E: corne.kotze@za.striata.com T: +27 11 530 9600 F: +27 11 447 9122 This email and all contents are subject to the following disclaimer: http://www.striata.com/_disclaimer/ From avg at icyb.net.ua Thu Feb 5 04:21:32 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Thu Feb 5 04:21:45 2009 Subject: kobj methods (DEVMETHOD) that have differing signatures (in src/sys) In-Reply-To: <20090204.131104.2064955776.imp@bsdimp.com> References: <4989E87B.5010000@icyb.net.ua> <20090204.131104.2064955776.imp@bsdimp.com> Message-ID: <498AD9BF.20503@icyb.net.ua> on 04/02/2009 22:11 M. Warner Losh said the following: > In message: <4989E87B.5010000@icyb.net.ua> > Andriy Gapon writes: > : > : This based on the (much) earlier proposal described here: > : http://lists.freebsd.org/pipermail/freebsd-arch/2008-April/007982.html > : > : The patch was applied to the recent current sources and LINT kernels for > : all architectures that have LINT/NOTES (i.e. arm excluded) were built. > : > : Here's a link to the list of files and line numbers where KOBJ methods > : are set with functions that have differing signatures: > : http://www.icyb.net.ua/~avg/kobj_method_sigs.txt > : > : List of the most common issues can be found at the first link. > : > : Hope this is useful. > > This is very helpful. I'll work through the low-hanging fruit. If > others want to work as well, please ping me. Warner, thanks a lot for your attention! I've updated the list to include actual source lines in addition to file names and line numbers - I think this should be useful since current is a moving target. http://www.icyb.net.ua/~avg/kobj_method_sigs.txt -- Andriy Gapon From pluknet at gmail.com Thu Feb 5 04:36:36 2009 From: pluknet at gmail.com (pluknet) Date: Thu Feb 5 04:36:42 2009 Subject: linking .a lib with another library In-Reply-To: References: Message-ID: 2009/2/5 pluknet : > Hi. > > The problem is: > > I have a program which is statically linked to my static lib during a build. > The lib is in turn uses kvm*() calls, hence it should be linked with libkvm. > > During the build of my binaries which are linked to my libucron.a I > get the following: > > /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x6e): > In function `check_pidfile': > : undefined reference to `kvm_open' > /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x8b): > In function `check_pidfile': > : undefined reference to `kvm_getprocs' > /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0xbe): > In function `check_pidfile': > : undefined reference to `kvm_getprocs' > /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x106): > In function `check_pidfile': > : undefined reference to `kvm_close' > *** Error code 1 > > Ok, if I add SHLIB_MAJOR directive in lib/Makefile (or remove any kvm* > references) then all is ok. > But the lib is used to be static. So the question is can I somehow > link two .a libraries? > e.g. libkvm.a + libucron.a + my binaries. > Ok, I finally got it: gcc crond/crond.o lib/libucron.a /usr/lib/libkvm.a -L/usr/local/lib/mysql -lmysqlclient or in BNF (bmake normal form): LDADD+= -lmysqlclient -lucron ${LIBKVM} $ ldd crond/crond crond/crond: libmysqlclient.so.14 => /usr/local/lib/mysql/libmysqlclient.so.14 (0x28080000) libc.so.6 => /lib/libc.so.6 (0x280d5000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x281ba000) libm.so.4 => /lib/libm.so.4 (0x281d2000) libz.so.3 => /lib/libz.so.3 (0x281e8000) The only drawback is that I had to add that to crond/Makefile, not to lib/Makefile. crond knows nothing about kvm calls and now it has to (and lib does vice versa). -- wbr, pluknet From jhb at freebsd.org Thu Feb 5 08:51:41 2009 From: jhb at freebsd.org (John Baldwin) Date: Thu Feb 5 08:51:48 2009 Subject: blockable sleep lock (sleep mutex) 16 In-Reply-To: References: <02026848-7F83-405C-B4F3-EDD8B47DA294@gmail.com> <498736C2.3040207@elischer.org> Message-ID: <200902050819.22726.jhb@freebsd.org> On Wednesday 04 February 2009 11:05:02 am Nikola Kne?evi? wrote: > On 2 Feb 2009, at 19:09 , Julian Elischer wrote: > > >>> It says "non-sleepable locks", yet it classifies click_instance > >>> as sleep mutex. I think witness code should emit messages which > >>> are more clear. > >> It is confusing, but you can't do an M_WAITOK malloc while holding > >> a mutex. Basically, sleeping actually means calling "*sleep() > >> (such as mtx_sleep()) or cv_*wait*()". Blocking on a mutex is not > >> sleeping, it's "blocking". Some locks (such as sx(9)) do "sleep" > >> when you contest them. In the scheduler, sleeping and blocking are > >> actually quite different (blocking uses turnstiles that handle > >> priority inversions via priority propagation, sleeping uses sleep > >> queues which do not do any of that). The underyling idea is that > >> mutexes should be held for "short" periods of time, and that any > >> sleeps are potentially unbounded. Holding a mutex while sleeping > >> could result in a mutex being held for a long time. > > > > > > the locking overview page > > man 9 locking > > tries to explain this.. > > I've been pestering John to proofread it and make suggestiosn for a > > while now. > > > Thanks John and Julian. I agree, man pages should be more clear :) > > I've switched from using mtx to sx locks, since they offer sleeping > while hold. > > Unfortunately, I've ran into something really weird now, when I unload > the module: > ---8<--- > #0 doadump () at pcpu.h:195 > #1 0xffffffff8049ef98 in boot (howto=260) at /usr/src/sys/kern/ > kern_shutdown.c:418 > #2 0xffffffff8049f429 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xffffffff8075cd26 in trap_fatal (frame=0xc, eva=Variable "eva" is > not available. > ) at /usr/src/sys/amd64/amd64/trap.c:764 > #4 0xffffffff8075da62 in trap (frame=0xffffffff87699940) at /usr/src/ > sys/amd64/amd64/trap.c:290 > #5 0xffffffff80743bfe in calltrap () at /usr/src/sys/amd64/amd64/ > exception.S:209 > #6 0xffffffff8052a411 in strcmp (s1=0xffffffff80824a0c "sigacts", > s2=0xffffffff877cd3a9
) > at /usr/src/sys/libkern/strcmp.c:45 > #7 0xffffffff804d7c61 in enroll (description=0xffffffff80824a0c > "sigacts", lock_class=0xffffffff80a19fe0) > at /usr/src/sys/kern/subr_witness.c:1439 > #8 0xffffffff804d7fb1 in witness_init (lock=0xffffff00016f4ca8) at / > usr/src/sys/kern/subr_witness.c:618 > #9 0xffffffff8049fd31 in sigacts_alloc () at /usr/src/sys/kern/ > kern_sig.c:3280 > #10 0xffffffff80481121 in fork1 (td=0xffffff0001384a50, flags=20, > pages=Variable "pages" is not available. > ) at /usr/src/sys/kern/kern_fork.c:453 > #11 0xffffffff80481450 in fork (td=0xffffff0001384a50, uap=Variable > "uap" is not available. > ) at /usr/src/sys/kern/kern_fork.c:106 > #12 0xffffffff8075d260 in syscall (frame=0xffffffff87699c80) at /usr/ > src/sys/amd64/amd64/trap.c:907 > #13 0xffffffff80743e0b in Xfast_syscall () at /usr/src/sys/amd64/amd64/ > exception.S:330 > #14 0x0000000800ca0a6c in ?? () > --->8--- > > and in fra 7: > (kgdb) p *w > $5 = {w_name = 0xffffffff877cd3a9
bounds>, w_class = 0xffffffff80a19fe0, w_list = { > stqe_next = 0xffffffff80accce0}, w_typelist = {stqe_next = > 0xffffffff80accce0}, w_children = 0x0, > w_file = 0xffffffff877d1fa0
bounds>, w_line = 307, w_level = 0, w_refcount = 2, > w_Giant_squawked = 0 '\0', w_other_squawked = 0 '\0', > w_same_squawked = 0 '\0', w_displayed = 0 '\0'} > (kgdb) p *w->w_class > $6 = {lc_name = 0xffffffff808564e0 "sleep mutex", lc_flags = 9, > lc_ddb_show = 0xffffffff80492e6b , > lc_lock = 0xffffffff804938be , lc_unlock = > 0xffffffff804933fc } > > This happens after modevent exists. > > What puzzles me here is w_refcount of 2, while w_name is out of > bounds. Locks I've created I properly destroyed (at least I think I > did :)). You are probably missing some sx_destroy()'s. You need to destroy each lock you create with sx_init(). -- John Baldwin From julian at elischer.org Thu Feb 5 09:24:25 2009 From: julian at elischer.org (Julian Elischer) Date: Thu Feb 5 09:24:50 2009 Subject: linking .a lib with another library In-Reply-To: References: Message-ID: <498B1DCB.801@elischer.org> pluknet wrote: > 2009/2/5 pluknet : >> Hi. >> >> The problem is: >> >> I have a program which is statically linked to my static lib during a build. >> The lib is in turn uses kvm*() calls, hence it should be linked with libkvm. in static linking order is important.. you need to put libkvm after the library that has the refrences. (At least that is my first reaction) >> >> During the build of my binaries which are linked to my libucron.a I >> get the following: >> >> /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x6e): >> In function `check_pidfile': >> : undefined reference to `kvm_open' >> /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x8b): >> In function `check_pidfile': >> : undefined reference to `kvm_getprocs' >> /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0xbe): >> In function `check_pidfile': >> : undefined reference to `kvm_getprocs' >> /usr/home/pluknet/svn/ucron/ksucron/crond/../lib/libucron.a(misc.o)(.text+0x106): >> In function `check_pidfile': >> : undefined reference to `kvm_close' >> *** Error code 1 >> >> Ok, if I add SHLIB_MAJOR directive in lib/Makefile (or remove any kvm* >> references) then all is ok. >> But the lib is used to be static. So the question is can I somehow >> link two .a libraries? >> e.g. libkvm.a + libucron.a + my binaries. >> > > Ok, I finally got it: > gcc crond/crond.o lib/libucron.a /usr/lib/libkvm.a > -L/usr/local/lib/mysql -lmysqlclient > > or in BNF (bmake normal form): > LDADD+= -lmysqlclient -lucron ${LIBKVM} > > $ ldd crond/crond > crond/crond: > libmysqlclient.so.14 => > /usr/local/lib/mysql/libmysqlclient.so.14 (0x28080000) > libc.so.6 => /lib/libc.so.6 (0x280d5000) > libcrypt.so.3 => /lib/libcrypt.so.3 (0x281ba000) > libm.so.4 => /lib/libm.so.4 (0x281d2000) > libz.so.3 => /lib/libz.so.3 (0x281e8000) > > The only drawback is that I had to add that to crond/Makefile, not to > lib/Makefile. > crond knows nothing about kvm calls and now it has to (and lib does vice versa). > From olli at lurza.secnetix.de Thu Feb 5 14:18:40 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Thu Feb 5 14:18:53 2009 Subject: CFT: Graphics support for /boot/loader Message-ID: <200902052218.n15MIaEa026891@lurza.secnetix.de> Hello fellow hackers, Some of you might remember that I'm working on graphics support for our /boot/loader. Unfortunately, progress has been rather slow because of non-FreeBSD-related activity. Anyway, I have now prepared a tarball containing a loader binary for public testing. If you are eager to give it a try, please feel free to do so. It should work with any FreeBSD version on i386 and amd64 platforms. I have posted detailed instructions on the FreeBSD wiki: http://wiki.freebsd.org/OliverFromme/BootLoaderTest Any kind of feedback is welcome. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From thompsa at FreeBSD.org Thu Feb 5 14:37:03 2009 From: thompsa at FreeBSD.org (Andrew Thompson) Date: Thu Feb 5 14:37:18 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <20090205223657.GC88414@citylink.fud.org.nz> On Thu, Feb 05, 2009 at 11:18:36PM +0100, Oliver Fromme wrote: > Hello fellow hackers, > > Some of you might remember that I'm working on graphics > support for our /boot/loader. Unfortunately, progress has > been rather slow because of non-FreeBSD-related activity. > > Anyway, I have now prepared a tarball containing a loader > binary for public testing. If you are eager to give it a > try, please feel free to do so. It should work with any > FreeBSD version on i386 and amd64 platforms. > > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. Works well here, tried various combinations of the options. This is very cool. Andrew From bomberboy at gmail.com Thu Feb 5 15:03:10 2009 From: bomberboy at gmail.com (Bruno Van Den Bossche) Date: Thu Feb 5 15:18:13 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <6e77fe4e0902051432n25e68edes341495f994c64bca@mail.gmail.com> On Thu, Feb 5, 2009 at 11:18 PM, Oliver Fromme wrote: [graphical bootloader] > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. I've just tested it on my laptop (Fujitsu T4220) with current and it works very well, Can select/unselect any options and everything seems to work as it is supposed. And I love the look :) Regards, Bruno From max at love2party.net Thu Feb 5 15:21:34 2009 From: max at love2party.net (Max Laier) Date: Thu Feb 5 15:21:43 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <200902060021.30883.max@love2party.net> On Thursday 05 February 2009 23:18:36 Oliver Fromme wrote: > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. quick test in qemu - works well. Very cool! -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From gonzo at bluezbox.com Thu Feb 5 15:25:18 2009 From: gonzo at bluezbox.com (Oleksandr Tymoshenko) Date: Thu Feb 5 15:25:26 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <498B754B.1050901@bluezbox.com> Oliver Fromme wrote: > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. Works fine on Thinkpad T400 (CURRENT/i386). From william at futurecis.com Thu Feb 5 15:45:02 2009 From: william at futurecis.com (william@futurecis.com) Date: Thu Feb 5 15:45:09 2009 Subject: CFT: Graphics support for /boot/loader Message-ID: <1396974090-1233875762-cardhu_decombobulator_blackberry.rim.net-991069870-@bxe176.bisx.prod.on.blackberry> Works beautifully! Loving it. Thanks for the work. ------Original Message------ From: Oliver Fromme Sender: owner-freebsd-hackers@freebsd.org To: freebsd-hackers@FreeBSD.ORG To: freebsd-current@FreeBSD.ORG Sent: Feb 5, 2009 17:18 Subject: CFT: Graphics support for /boot/loader Hello fellow hackers, Some of you might remember that I'm working on graphics support for our /boot/loader. Unfortunately, progress has been rather slow because of non-FreeBSD-related activity. Anyway, I have now prepared a tarball containing a loader binary for public testing. If you are eager to give it a try, please feel free to do so. It should work with any FreeBSD version on i386 and amd64 platforms. I have posted detailed instructions on the FreeBSD wiki: http://wiki.freebsd.org/OliverFromme/BootLoaderTest Any kind of feedback is welcome. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" Sent from my BlackBerry? wireless device From julian at elischer.org Thu Feb 5 16:44:05 2009 From: julian at elischer.org (Julian Elischer) Date: Thu Feb 5 16:44:12 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902060021.30883.max@love2party.net> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> <200902060021.30883.max@love2party.net> Message-ID: <498B811D.50405@elischer.org> Max Laier wrote: > On Thursday 05 February 2009 23:18:36 Oliver Fromme wrote: >> I have posted detailed instructions on the FreeBSD wiki: >> >> http://wiki.freebsd.org/OliverFromme/BootLoaderTest >> >> Any kind of feedback is welcome. > > quick test in qemu - works well. Very cool! > can you send a screenshot for those of us who can't test it now? From william at futurecis.com Thu Feb 5 17:23:54 2009 From: william at futurecis.com (william@futurecis.com) Date: Thu Feb 5 17:24:08 2009 Subject: CFT: Graphics support for /boot/loader Message-ID: <1332767982-1233881690-cardhu_decombobulator_blackberry.rim.net-720693194-@bxe176.bisx.prod.on.blackberry> There is a screenshot on the wiki site ------Original Message------ From: Julian Elischer Sender: owner-freebsd-hackers@freebsd.org To: Max Laier Cc: freebsd-hackers@freebsd.org Cc: freebsd-current@freebsd.org Cc: Oliver Fromme Sent: Feb 5, 2009 19:15 Subject: Re: CFT: Graphics support for /boot/loader Max Laier wrote: > On Thursday 05 February 2009 23:18:36 Oliver Fromme wrote: >> I have posted detailed instructions on the FreeBSD wiki: >> >> http://wiki.freebsd.org/OliverFromme/BootLoaderTest >> >> Any kind of feedback is welcome. > > quick test in qemu - works well. Very cool! > can you send a screenshot for those of us who can't test it now? _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" Sent from my BlackBerry? wireless device From chflags at gmail.com Thu Feb 5 18:10:22 2009 From: chflags at gmail.com (Kevin Foo) Date: Thu Feb 5 18:10:59 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <25cb30902051743r13950bfk50c0919cf7a60835@mail.gmail.com> Very nice! Thanks for the good work. -- Regards Kevin Foo On Fri, Feb 6, 2009 at 6:18 AM, Oliver Fromme wrote: > Hello fellow hackers, > > Some of you might remember that I'm working on graphics > support for our /boot/loader. Unfortunately, progress has > been rather slow because of non-FreeBSD-related activity. > > Anyway, I have now prepared a tarball containing a loader > binary for public testing. If you are eager to give it a > try, please feel free to do so. It should work with any > FreeBSD version on i386 and amd64 platforms. > > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. > > Best regards > Oliver > > -- > Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. > Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: > secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- > chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart > > FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd > > "If Java had true garbage collection, most programs > would delete themselves upon execution." > -- Robert Sewell > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From scottl at samsco.org Thu Feb 5 17:01:05 2009 From: scottl at samsco.org (Scott Long) Date: Thu Feb 5 18:16:20 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <498B811D.50405@elischer.org> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> <200902060021.30883.max@love2party.net> <498B811D.50405@elischer.org> Message-ID: <498B88CC.9030903@samsco.org> Julian Elischer wrote: > Max Laier wrote: >> On Thursday 05 February 2009 23:18:36 Oliver Fromme wrote: >>> I have posted detailed instructions on the FreeBSD wiki: >>> >>> http://wiki.freebsd.org/OliverFromme/BootLoaderTest >>> >>> Any kind of feedback is welcome. >> >> quick test in qemu - works well. Very cool! >> > > can you send a screenshot for those of us who can't test it now? http://wiki.freebsd.org/OliverFromme/BootLoader From biancalana at gmail.com Thu Feb 5 18:28:34 2009 From: biancalana at gmail.com (Alexandre Biancalana) Date: Thu Feb 5 18:28:40 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902060021.30883.max@love2party.net> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> <200902060021.30883.max@love2party.net> Message-ID: <8e10486b0902051758t2044238co70f89f1beb52afd1@mail.gmail.com> On 2/5/09, Max Laier wrote: > On Thursday 05 February 2009 23:18:36 Oliver Fromme wrote: > > I have posted detailed instructions on the FreeBSD wiki: > > > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > > > Any kind of feedback is welcome. > > > quick test in qemu - works well. Very cool tested in Parallels, works perfect and looks great. Congratulations! From scottl at samsco.org Thu Feb 5 18:52:08 2009 From: scottl at samsco.org (Scott Long) Date: Thu Feb 5 19:20:06 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <498BA5D5.4080100@samsco.org> Oliver Fromme wrote: > Hello fellow hackers, > > Some of you might remember that I'm working on graphics > support for our /boot/loader. Unfortunately, progress has > been rather slow because of non-FreeBSD-related activity. > > Anyway, I have now prepared a tarball containing a loader > binary for public testing. If you are eager to give it a > try, please feel free to do so. It should work with any > FreeBSD version on i386 and amd64 platforms. > > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. > I think that this is really neat, you've done an impressive job with it good job. However, I do take issue with your criticism of the ASCII logo; I actually spent a decent amount of time designing the block text logo =-) I wish that there hadn't been moronic politics over the beastie logo, as that does look a lot better, even if it is text. And text is still required for serial consoles. Scott From gnemmi at gmail.com Thu Feb 5 19:24:13 2009 From: gnemmi at gmail.com (Gonzalo Nemmi) Date: Thu Feb 5 19:24:19 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <200902060101.58024.gnemmi@gmail.com> On Thursday 05 February 2009 8:18:36 pm Oliver Fromme wrote: > Hello fellow hackers, > > Some of you might remember that I'm working on graphics > support for our /boot/loader. Unfortunately, progress has > been rather slow because of non-FreeBSD-related activity. > > Anyway, I have now prepared a tarball containing a loader > binary for public testing. If you are eager to give it a > try, please feel free to do so. It should work with any > FreeBSD version on i386 and amd64 platforms. > > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. > > Best regards > Oliver No at problem in here .. Just added graphics_enable="YES" beastie_theme="/boot/themes/default/theme.conf" to my loader.conf after decompressing your file as explained in your page and that was it .. it works like a charm ... [gonzalo@inferna ~]% uname -a FreeBSD inferna.inferna.com.ar 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Fri Dec 12 12:03:51 ARST 2008 root@inferna.inferna.com.ar:/usr/obj/usr/src/sys/INFERNA i386 [gonzalo@inferna ~]% Thanks a lot for your hard work :D Regards -- Blessings Gonzalo Nemmi From julian at elischer.org Thu Feb 5 20:28:54 2009 From: julian at elischer.org (Julian Elischer) Date: Thu Feb 5 20:29:01 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <498BBC85.3070408@elischer.org> Oliver Fromme wrote: > Hello fellow hackers, > > Some of you might remember that I'm working on graphics > support for our /boot/loader. Unfortunately, progress has > been rather slow because of non-FreeBSD-related activity. > > Anyway, I have now prepared a tarball containing a loader > binary for public testing. If you are eager to give it a > try, please feel free to do so. It should work with any > FreeBSD version on i386 and amd64 platforms. > > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. > > Best regards > Oliver > very nice... BTW most of these things seem to have drop out of graphics mode.. do you have something like that? (or maybe should go to loader prompt...?) If I had a machine that it didn't work on I think I'd try hitting esc but I don't think I'd think of F7. From olli at lurza.secnetix.de Thu Feb 5 23:11:38 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Thu Feb 5 23:11:47 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <498BA5D5.4080100@samsco.org> Message-ID: <200902060711.n167BZ9a048166@lurza.secnetix.de> Scott Long wrote: > Oliver Fromme wrote: > > [...] > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > I think that this is really neat, you've done an impressive job > with it good job. However, I do take issue with your criticism > of the ASCII logo; I actually spent a decent amount of time > designing the block text logo =-) I'm sorry, it wasn't my intention to disrespect your work. What I was trying to say is that ASCII graphics look old- fashioned in general. I wasn't actually picking on your rendition of the text logo in particular. Yeah, I noticed your smiley, but I agree that my wording on the wiki page was misleading, so I changed it. > I wish that there hadn't been > moronic politics over the beastie logo, as that does look a lot > better, even if it is text. And text is still required for > serial consoles. Absolutely. Text is also required for machines that aren't supported by the graphics code, or machines that don't have any graphics hardware at all. Don't worry, I'm not going to rip your ASCII logo out. :-) Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Being really good at C++ is like being really good at using rocks to sharpen sticks." -- Thant Tessman From olli at lurza.secnetix.de Thu Feb 5 23:21:09 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Thu Feb 5 23:21:21 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: Message-ID: <200902060721.n167L2px048711@lurza.secnetix.de> Julian Elischer wrote: > BTW most of these things seem to have drop out of > graphics mode.. > do you have something like that? > (or maybe should go to loader prompt...?) Good question. The screen layout isn't final, of course, and I'm open to suggestions. (Also, there will be a short descriptive text for the countdown and how to pause it.) I think it might make sense to provide an additional action using the key that leaves graphics mode and displays the old text menu instead. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "C++ is over-complicated nonsense. And Bjorn Shoestrap's book a danger to public health. I tried reading it once, I was in recovery for months." -- Cliff Sarginson From danny at cs.huji.ac.il Fri Feb 6 00:51:14 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri Feb 6 00:51:27 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <8e10486b0902051758t2044238co70f89f1beb52afd1@mail.gmail.com> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> <200902060021.30883.max@love2party.net> <8e10486b0902051758t2044238co70f89f1beb52afd1@mail.gmail.com> Message-ID: > On Thursday 05 February 2009 23:18:36 Oliver Fromme wrote: > I have posted detailed instructions on the FreeBSD wiki: [...] just tried it via pxe: panic: free: guard1 @ 0x7f3a4aec from /usr/src/lib/libstand/close.c:79 what changes are needed in pxeboot? cheers, danny From olli at lurza.secnetix.de Fri Feb 6 01:25:50 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri Feb 6 01:25:57 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: Message-ID: <200902060925.n169PObG053288@lurza.secnetix.de> Danny Braniss wrote: > just tried it via pxe: > > panic: free: guard1 @ 0x7f3a4aec from /usr/src/lib/libstand/close.c:79 > > what changes are needed in pxeboot? The panic message means that the heap memory was corruped. It could be caused by a buffer overflow or similar. I'll try to look into it. When does that message appear? Could you provide a screen shot? Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd We're sysadmins. To us, data is a protocol-overhead. From ale at FreeBSD.org Fri Feb 6 02:33:24 2009 From: ale at FreeBSD.org (Alex Dupre) Date: Fri Feb 6 02:33:30 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <498C0BAE.5020803@FreeBSD.org> Oliver Fromme ha scritto: > Some of you might remember that I'm working on graphics > support for our /boot/loader. Just a side question: are you going to improve also the splash(4) support? Graphical loader is great, but unfortunately on amd64 the boot splash screen is unusable. In any case, good job. -- Alex Dupre From skip at menantico.com Fri Feb 6 03:22:37 2009 From: skip at menantico.com (Skip Ford) Date: Fri Feb 6 03:22:50 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902060721.n167L2px048711@lurza.secnetix.de> References: <200902060721.n167L2px048711@lurza.secnetix.de> Message-ID: <20090206102106.GA881@menantico.com> Oliver Fromme wrote: > Julian Elischer wrote: > > BTW most of these things seem to have drop out of > > graphics mode.. > > do you have something like that? > > (or maybe should go to loader prompt...?) > > Good question. The screen layout isn't final, of course, > and I'm open to suggestions. (Also, there will be a short > descriptive text for the countdown and how to pause it.) > > I think it might make sense to provide an additional action > using the key that leaves graphics mode and displays > the old text menu instead. What about F8? That's what I'd guess a majority of people would instinctively reach for to get to a boot menu. -- Skip From skip at menantico.com Fri Feb 6 03:28:24 2009 From: skip at menantico.com (Skip Ford) Date: Fri Feb 6 03:28:31 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <20090206102106.GA881@menantico.com> References: <200902060721.n167L2px048711@lurza.secnetix.de> <20090206102106.GA881@menantico.com> Message-ID: <20090206112926.GB881@menantico.com> Skip Ford wrote: > Oliver Fromme wrote: > > Julian Elischer wrote: > > > BTW most of these things seem to have drop out of > > > graphics mode.. > > > do you have something like that? > > > (or maybe should go to loader prompt...?) > > > > Good question. The screen layout isn't final, of course, > > and I'm open to suggestions. (Also, there will be a short > > descriptive text for the countdown and how to pause it.) > > > > I think it might make sense to provide an additional action > > using the key that leaves graphics mode and displays > > the old text menu instead. > > What about F8? That's what I'd guess a majority of people would > instinctively reach for to get to a boot menu. Ok, scratch that idea. Windows uses F8 to stop an automatic boot, not really to drop to a text menu from a graphics menu. We can press any key to stop an automatic boot already, so F8 could be used to halt an automatic boot and jump straight to a boot menu, but that has nothing to do with this graphical boot menu. It could even jump to the graphical boot menu, not necessarily text. Windows just happens to not have a graphical menu. -- Skip From olli at lurza.secnetix.de Fri Feb 6 03:39:17 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri Feb 6 03:39:24 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <498C0BAE.5020803@FreeBSD.org> Message-ID: <200902061139.n16BdB6b058473@lurza.secnetix.de> Alex Dupre wrote: > Oliver Fromme ha scritto: > > Some of you might remember that I'm working on graphics > > support for our /boot/loader. > > Just a side question: are you going to improve also the splash(4) > support? Graphical loader is great, but unfortunately on amd64 the boot > splash screen is unusable. The problem is related to the fact that a 64bit kernel cannot use VESA BIOS functions. You should be able to use standard VGA modes though, which don't require VESA support. Anyway, there have been several ideas floating around to fix or work-around the VESA problem for amd64. One of them involves letting the loader prepare graphics mode (doing all the VESA stuff) and hand all the necessary information to the kernel, so the kernel only has to perform framebuffer access, but no VESA BIOS calls. It is my plan to try to look into that, but I would like to continue with the current work on the boot loader first. As soon as the graphics support in the loader is stable and "finished", I can start thinking about how to interface that graphics support with the kernel's syscons driver (which is a very sensitive piece of code). Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "File names are infinite in length, where infinity is set to 255 characters." -- Peter Collinson, "The Unix File System" From avg at icyb.net.ua Fri Feb 6 04:22:43 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 04:22:49 2009 Subject: (openbsd) sensors framework Message-ID: <498C2B8B.4000101@icyb.net.ua> Is there anybody who uses sensors framework and maintains it in their source tree? I mean the one that was ported from OpenBSD by Constantine Murenin, the one was added to our source tree and removed shortly after. I am interested in a patch that would apply to stable/7 or trunk. Thank you in advance. -- Andriy Gapon From rbgarga at gmail.com Fri Feb 6 04:02:56 2009 From: rbgarga at gmail.com (Renato Botelho) Date: Fri Feb 6 04:27:17 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <747dc8f30902060330l33fb0bbo489dc4fe3009747a@mail.gmail.com> On Thu, Feb 5, 2009 at 8:18 PM, Oliver Fromme wrote: > Hello fellow hackers, > > Some of you might remember that I'm working on graphics > support for our /boot/loader. Unfortunately, progress has > been rather slow because of non-FreeBSD-related activity. > > Anyway, I have now prepared a tarball containing a loader > binary for public testing. If you are eager to give it a > try, please feel free to do so. It should work with any > FreeBSD version on i386 and amd64 platforms. > > I have posted detailed instructions on the FreeBSD wiki: > > http://wiki.freebsd.org/OliverFromme/BootLoaderTest > > Any kind of feedback is welcome. Hello Oliver It worked here, on a 8.0-current i386 r188003, the only small thing is it show a red border when show the menu. There is a dmidecode output attached, just to give you some information about the bios. Thanks and congrats for the nice job -- Renato Botelho -------------- next part -------------- A non-text attachment was scrubbed... Name: dmidecode.out Type: application/octet-stream Size: 11534 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090206/02e984c2/dmidecode.obj From avg at icyb.net.ua Fri Feb 6 04:35:49 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 04:36:09 2009 Subject: Super I/O driver Message-ID: <498C2EA0.6080604@icyb.net.ua> Do we have any drivers for special functions provided by ISA/LPC Super I/Os. I mean many of them have watchdog capabilities, GPIO, etc. I do not meant fdc, lpt, sio/uart and hwm which are 9or can be) handled as the devices of their own. I have some quite hairy code for Winbond W83977EF that provides watchdog and led interfaces (the latter is for power led). I'd like to understand what is the best/canonical way to access superio's ports, enter configuration mode, etc. Also, it would be interesting to see how various configurations/wirings of the same superio could be handled. I looked at some Linux drivers but their coding ways are quite different and they typically handle only one wiring type (probably the one that was found on author's motherboard). I also would be interested in a general discussion on this topic (superios). BTW, do we have some interface to userland or GPIO? Something similar to led(4), but more generic (and supporting 'I' as well 'O'). -- Andriy Gapon From avg at icyb.net.ua Fri Feb 6 05:31:13 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 05:31:19 2009 Subject: i386: rtc, nvram and the upper bank Message-ID: <498C3B9E.4090100@icyb.net.ua> It seems that on at least "real Intel" i386 platform since 440/PIIX4 chipset times there 256 bytes of RTC NVRAM (minus actual timer/control registers). Access to upper 128 bytes can enabled/disabled via chipset configuration. For earlier chipsets it can be done via PCI config (e.g. register 0xcb of Host-ISA bridge on PIIX4) or via memory mapped "root complex" (e.g. for ICH9). Ports 0x72 and 0x73 are used for access to the upper NVRAM when it is enabled similarly to 0x70/0x71. When the upper NVRAM is disabled then 0x72/0x73 access the lower NVRAM. Multi-piece suggestion: 1. fix signatures of rtcin/writertc functions - uint8_t for register number and value (instead of ints), also rtcin() definition in sys/i386/isa/clock.c seems to be "old-style". 2. either add rtcin2/writertc2 for access to upper NVRAM, or make rtcin/writertc interpret reg numbers > 0x7f as access to upper NVRAM; the latter proposal has a shortcoming on not being to explicitly control 0x80 bit ("NMI something") anymore, but I really doubt that that bit is any useful or that it is actually used. 3. add a check/quirk somewhere[?] to enable access to upper NVRAM, because sometimes BIOS disables it or chipset has it disabled by default. 4. update sys/dev/nvram to be able to access 256 bytes Also, I understand that nvram device has an interface compatible with ts Linux "brother", but I think it would better to address NVRAM bytes by their true address and simply "mask out" control registers. I.e. always return 0x00 or 0xff on reads and ignore writes. It is not really userland's business to mess with those registers. And I am not sure about checksum calculation/verification. I think it is not the kernel's business. If a userland program modifies NVRAM content it must already understand its structure and meaning, so it should update any checksums on its own. Also, something to look at, long-term. Why not use rtc device/framework for i386 and amd64. Why not re-use mc146818. Why not extend rtc interface with NVRAM access methods. I already have some code for 1-4 (used on a running system). -- Andriy Gapon From olli at lurza.secnetix.de Fri Feb 6 05:35:26 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri Feb 6 05:35:34 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <747dc8f30902060330l33fb0bbo489dc4fe3009747a@mail.gmail.com> Message-ID: <200902061335.n16DZNA9063755@lurza.secnetix.de> Renato Botelho wrote: > It worked here, on a 8.0-current i386 r188003, the only small > thing is it show a red border when show the menu. Do you mean a red line at the top right corner? That problem has already been reported and fixed in my local source tree. > There is a dmidecode output attached, just to give you some > information about the bios. Thanks! Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "If you aim the gun at your foot and pull the trigger, it's UNIX's job to ensure reliable delivery of the bullet to where you aimed the gun (in this case, Mr. Foot)." -- Terry Lambert, FreeBSD-hackers mailing list. From c47g at gmx.at Fri Feb 6 05:40:14 2009 From: c47g at gmx.at (Christian Gusenbauer) Date: Fri Feb 6 05:40:23 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902060925.n169PObG053288@lurza.secnetix.de> References: <200902060925.n169PObG053288@lurza.secnetix.de> Message-ID: <200902061413.53508.c47g@gmx.at> Hi Oliver! On Friday 06 February 2009, Oliver Fromme wrote: > Danny Braniss wrote: > > just tried it via pxe: > > > > panic: free: guard1 @ 0x7f3a4aec from /usr/src/lib/libstand/close.c:79 > > > > what changes are needed in pxeboot? > > The panic message means that the heap memory was corruped. > It could be caused by a buffer overflow or similar. > I'll try to look into it. I got this some years ago when I played with FreeBSD 6.1. It has something to do with reading/parsing the loader.conf file. Inserting some dummy lines (comments etc.) into loader.conf solves it (at least that's a workaround). As I've never seen it again since 6.1 I thought it has already been fixed :-(. Christian. > > When does that message appear? Could you provide a screen > shot? > > Best regards > Oliver From olli at lurza.secnetix.de Fri Feb 6 06:13:54 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri Feb 6 06:14:07 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902061413.53508.c47g@gmx.at> Message-ID: <200902061413.n16EDgii065856@lurza.secnetix.de> Christian Gusenbauer wrote: > Oliver Fromme wrote: > > Danny Braniss wrote: > > > just tried it via pxe: > > > > > > panic: free: guard1 @ 0x7f3a4aec from /usr/src/lib/libstand/close.c:79 > > > > > > what changes are needed in pxeboot? > > > > The panic message means that the heap memory was corruped. > > It could be caused by a buffer overflow or similar. > > I'll try to look into it. > > I got this some years ago when I played with FreeBSD 6.1. It has something to > do with reading/parsing the loader.conf file. Inserting some dummy lines > (comments etc.) into loader.conf solves it (at least that's a workaround). As > I've never seen it again since 6.1 I thought it has already been fixed :-(. I think that's unrelated. That guard panic just means that the program has written beyond the memory that was allocated. Unfortunately it is difficult to find the piece of code responsible for that behaviour (especially when I can't reproduce the problem myself because I don't have a PXE- capable machine). It could be almost anywhere. In fact, the bug doesn't even have to be in the C code: FORTH supports (and even encourages) pointer arithmetic, too. This is real fun ... Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Unix gives you just enough rope to hang yourself -- and then a couple of more feet, just to be sure." -- Eric Allman From olli at lurza.secnetix.de Fri Feb 6 07:08:14 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri Feb 6 07:08:21 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <747dc8f30902060655r143b5ac9ud6f060d1d8351ae1@mail.gmail.com> Message-ID: <200902061508.n16F8AVX068443@lurza.secnetix.de> Renato Botelho wrote: > Oliver Fromme wrote: > > Renato Botelho wrote: > > > It worked here, on a 8.0-current i386 r188003, the only small > > > thing is it show a red border when show the menu. > > > > Do you mean a red line at the top right corner? > > That problem has already been reported and fixed > > in my local source tree. > > Not exactly, it's a big red border on 4 sides of screen, something > like there is a red background bigger than image and image is in > front of it. I see. Is that an old CRT monitor (not a TFT display)? I think I have an idea what might be causing it. On CRT monitors, the border beyond the pixel area is set to the color of the first palette entry by default. The background PCX image happens to have red as the first palette entry. So you see a red border. In my tests I didn't notice the problem because I only tested with TFT displays and qemu. These don't have a visible border. The fix should be to set the border color to the same value as the background color (in this case that would be black). Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd It's trivial to make fun of Microsoft products, but it takes a real man to make them work, and a God to make them do anything useful. From rbgarga at gmail.com Fri Feb 6 06:55:17 2009 From: rbgarga at gmail.com (Renato Botelho) Date: Fri Feb 6 07:09:31 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902061335.n16DZNA9063755@lurza.secnetix.de> References: <747dc8f30902060330l33fb0bbo489dc4fe3009747a@mail.gmail.com> <200902061335.n16DZNA9063755@lurza.secnetix.de> Message-ID: <747dc8f30902060655r143b5ac9ud6f060d1d8351ae1@mail.gmail.com> On Fri, Feb 6, 2009 at 11:35 AM, Oliver Fromme wrote: > Renato Botelho wrote: > > It worked here, on a 8.0-current i386 r188003, the only small > > thing is it show a red border when show the menu. > > Do you mean a red line at the top right corner? > That problem has already been reported and fixed > in my local source tree. Not exactly, it's a big red border on 4 sides of screen, something like there is a red background bigger than image and image is in front of it. I don't have a camera here at the moment to give you a picture but i can do it on monday. -- Renato Botelho From danny at cs.huji.ac.il Fri Feb 6 07:10:36 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri Feb 6 07:10:44 2009 Subject: i386: rtc, nvram and the upper bank In-Reply-To: <498C3B9E.4090100@icyb.net.ua> References: <498C3B9E.4090100@icyb.net.ua> Message-ID: take a look in: http://www.cs.huji.ac.il/~danny/ftp/freebsd/gpio/ it's a primitive hack/attempt :-) danny From avg at icyb.net.ua Fri Feb 6 07:21:38 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 07:21:46 2009 Subject: i386: rtc, nvram and the upper bank In-Reply-To: References: <498C3B9E.4090100@icyb.net.ua> Message-ID: <498C5567.6070803@icyb.net.ua> on 06/02/2009 17:10 Danny Braniss said the following: > take a look in: > http://www.cs.huji.ac.il/~danny/ftp/freebsd/gpio/ > it's a primitive hack/attempt :-) Thanks a lot! P.S. I guess you intended to reply to my other post ("SuperIO blah blah") :-) -- Andriy Gapon From danny at cs.huji.ac.il Fri Feb 6 07:46:41 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri Feb 6 07:46:48 2009 Subject: i386: rtc, nvram and the upper bank In-Reply-To: Your message of Fri, 06 Feb 2009 17:21:11 +0200 . Message-ID: > on 06/02/2009 17:10 Danny Braniss said the following: > > take a look in: > > http://www.cs.huji.ac.il/~danny/ftp/freebsd/gpio/ > > it's a primitive hack/attempt :-) > > Thanks a lot! > > P.S. I guess you intended to reply to my other post ("SuperIO blah > blah") :-) yeah, typical case of brain to fingers packet loss :-) danny > > > -- > Andriy Gapon From pluknet at gmail.com Fri Feb 6 09:17:21 2009 From: pluknet at gmail.com (pluknet) Date: Fri Feb 6 09:17:28 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <498BA5D5.4080100@samsco.org> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> <498BA5D5.4080100@samsco.org> Message-ID: Hi, Scott! 2009/2/6 Scott Long : > Oliver Fromme wrote: >> >> Hello fellow hackers, >> >> Some of you might remember that I'm working on graphics >> support for our /boot/loader. Unfortunately, progress has >> been rather slow because of non-FreeBSD-related activity. >> >> Anyway, I have now prepared a tarball containing a loader >> binary for public testing. If you are eager to give it a >> try, please feel free to do so. It should work with any >> FreeBSD version on i386 and amd64 platforms. >> >> I have posted detailed instructions on the FreeBSD wiki: >> >> http://wiki.freebsd.org/OliverFromme/BootLoaderTest >> >> Any kind of feedback is welcome. >> > > I think that this is really neat, you've done an impressive job > with it good job. However, I do take issue with your criticism > of the ASCII logo; I actually spent a decent amount of time > designing the block text logo =-) I wish that there hadn't been > moronic politics over the beastie logo, as that does look a lot > better, even if it is text. And text is still required for > serial consoles. Hey, then what's about that? ;) --- /boot/beastie.4th 2008-08-19 23:59:01.000000000 +0600 +++ /boot/beastie.4th 2008-12-24 18:14:48.000000000 +0500 @@ -109,6 +109,27 @@ at-xy ." |____/|_____/|_____/" ; +: share-logo ( x y -- ) +2dup at-xy ." `````" 1+ +2dup at-xy ." `-/oo/-```.:oshddmmddhso:.```:+oo/." 1+ +2dup at-xy ." .--/shdmddNMMMMMMMMNNNdyhmddNNmh+--" 1+ +2dup at-xy ." `:--/smMMMMMMMMMMMMMMNy+mMMNmho/-::" 1+ +2dup at-xy ." `/+dMMNMMMNNNNNNNNNNNmddNmho/:-/+" 1+ +2dup at-xy ." /mNNNMMMNmmmddddhhhhhhsso/:-:+y" 1+ +2dup at-xy ." :dmdmNMNmdhyso++///:::////::-++sh" 1+ +2dup at-xy ." `hdhdNMmhs+////:::::---------:://sy" 1+ +2dup at-xy ." -dsoyhhso/:::::::::--------.--.--+m" 1+ +2dup at-xy ." :d+/+++++/:::::::------------....oN" 1+ +2dup at-xy ." -m/://////:::::-------------...-/dm" 1+ +2dup at-xy ." ys-:://////:::------------..--omN" 1+ +2dup at-xy ." .d+-::///////:--------------:ohNd" 1+ +2dup at-xy ." .ho-:::://////:::::::::///++shy" 1+ +2dup at-xy ." `+s:-:::::////////+++++o+oyh+" 1+ +2dup at-xy ." .+o/:::::///////++++osyy+" 1+ +2dup at-xy ." `-//////////++ossys/-" 1+ + at-xy ." `..--::::::-." +; + : print-logo ( x y -- ) s" loader_logo" getenv dup -1 = if @@ -131,6 +152,11 @@ beastie-logo exit then + 2dup s" share" compare-insensitive 0= if + 2drop + share-logo + exit + then 2dup s" none" compare-insensitive 0= if 2drop \ no logo (spied out from http://www.opennet.ru/opennews/art.shtml?num=20136) -- wbr, pluknet From avg at icyb.net.ua Fri Feb 6 09:39:13 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 09:39:25 2009 Subject: NO_WERROR vs kernel builds In-Reply-To: <4989EA2A.6050601@icyb.net.ua> References: <4989EA2A.6050601@icyb.net.ua> Message-ID: <498C75BD.5040205@icyb.net.ua> on 04/02/2009 21:19 Andriy Gapon said the following: > It seems that kernel builds ignore NO_WERROR. > Is this on purpose or by accident? > > I think that this happens because of the following lines in > sys/conf/kern.pre.mk: > > .if ${CC} != "icc" > CFLAGS+= -fno-common -finline-limit=${INLINE_LIMIT} > CFLAGS+= --param inline-unit-growth=100 > CFLAGS+= --param large-function-growth=1000 > .if ${MACHINE_ARCH} == "amd64" || ${MACHINE} == "i386" || \ > ${MACHINE_ARCH} == "ia64" || ${MACHINE_ARCH} == "powerpc" || \ > ${MACHINE_ARCH} == "sparc64" > WERROR?= -Werror > .endif > .endif > > I had to specify WERROR= on make's command line to catch a certain kind > of warnings in bulk instead of one by one. This was not obvious. > Can anybody please explain or comment (or rub my nose into it)? -- Andriy Gapon From olli at lurza.secnetix.de Fri Feb 6 10:40:37 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri Feb 6 10:40:51 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: Message-ID: <200902061840.n16IeYGn077754@lurza.secnetix.de> pluknet wrote: > Scott Long wrote: > > I think that this is really neat, you've done an impressive job > > with it good job. However, I do take issue with your criticism > > of the ASCII logo; I actually spent a decent amount of time > > designing the block text logo =-) I wish that there hadn't been > > moronic politics over the beastie logo, as that does look a lot > > better, even if it is text. And text is still required for > > serial consoles. > > Hey, then what's about that? ;) > [...] I have to admit that I like Scott's text logo *much* better. Trying to render the "horned ball" logo with ASCII letters looks butt-ugly, IMHO. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd $ dd if=/dev/urandom of=test.pl count=1 $ file test.pl test.pl: perl script text executable From dima_bsd at inbox.lv Fri Feb 6 11:00:48 2009 From: dima_bsd at inbox.lv (Dmitriy Demidov) Date: Fri Feb 6 12:36:03 2009 Subject: CFT: Graphics support for /boot/loader Message-ID: <200902062000.43820.dima_bsd@inbox.lv> It works for me. Systems: FreeBSD 7.1-PRERELEASE running under VMWare server FreeBSD 7-STABLE running at my home PC (Gigabyte GA-8PE800, AWARD BIOS) Thanks for you work! :) From voidpointer at bsd.com.br Fri Feb 6 15:57:47 2009 From: voidpointer at bsd.com.br (Diego Rocha) Date: Fri Feb 6 15:57:59 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: it's work very nice to me FreeBSD blackbird 7.0-RELEASE-p9 FreeBSD 7.0-RELEASE-p9 #1: Wed Jan 28 22:56:31 BRST 2009 void@blackbird:/usr/obj/usr/src/sys/VPKERNEL i386 From paul at fletchermoorland.co.uk Fri Feb 6 16:05:57 2009 From: paul at fletchermoorland.co.uk (Paul Wootton) Date: Fri Feb 6 16:06:04 2009 Subject: ZFS and Graphics support for /boot/loader In-Reply-To: <200902052218.n15MIaEa026891@lurza.secnetix.de> References: <200902052218.n15MIaEa026891@lurza.secnetix.de> Message-ID: <1CDDD463937A44A09B6F4D9FD4969293@apollo> Hi Oliver, This doesn?t work for me. I am booting off a ZFS mirror with GPT partitions (built from current on an amd64). Is there any change of a version of gloader but with ZFS support? Cheers Paul -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Oliver Fromme Sent: 05 February 2009 22:19 To: freebsd-hackers@freebsd.org; freebsd-current@freebsd.org Subject: CFT: Graphics support for /boot/loader Hello fellow hackers, Some of you might remember that I'm working on graphics support for our /boot/loader. Unfortunately, progress has been rather slow because of non-FreeBSD-related activity. Anyway, I have now prepared a tarball containing a loader binary for public testing. If you are eager to give it a try, please feel free to do so. It should work with any FreeBSD version on i386 and amd64 platforms. I have posted detailed instructions on the FreeBSD wiki: http://wiki.freebsd.org/OliverFromme/BootLoaderTest Any kind of feedback is welcome. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From sam at freebsd.org Fri Feb 6 18:35:49 2009 From: sam at freebsd.org (Sam Leffler) Date: Fri Feb 6 18:35:56 2009 Subject: Super I/O driver [generic gpio driver] In-Reply-To: <498C2EA0.6080604@icyb.net.ua> References: <498C2EA0.6080604@icyb.net.ua> Message-ID: <498CF37F.2060704@freebsd.org> Andriy Gapon wrote: > Do we have any drivers for special functions provided by ISA/LPC Super > I/Os. I mean many of them have watchdog capabilities, GPIO, etc. I do > not meant fdc, lpt, sio/uart and hwm which are 9or can be) handled as > the devices of their own. > > I have some quite hairy code for Winbond W83977EF that provides watchdog > and led interfaces (the latter is for power led). > I'd like to understand what is the best/canonical way to access > superio's ports, enter configuration mode, etc. > Also, it would be interesting to see how various configurations/wirings > of the same superio could be handled. > > I looked at some Linux drivers but their coding ways are quite different > and they typically handle only one wiring type (probably the one that > was found on author's motherboard). > > I also would be interested in a general discussion on this topic (superios). > > BTW, do we have some interface to userland or GPIO? Something similar to > led(4), but more generic (and supporting 'I' as well 'O'). > There's been some discussion amongst embedded folks but nothing yet. I noticed openbsd commit something recently but don't know if we can reuse anything they've done. Sam From omerfsen at gmail.com Sat Feb 7 00:00:40 2009 From: omerfsen at gmail.com (Omer Faruk Sen) Date: Sat Feb 7 00:00:52 2009 Subject: FreeBSD 6.3/7.1 and Linux disk performance test Message-ID: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> Hi, I have installed a new server to test performance results. BIOS and RAID BIOS is the latest in this server ( Raid controller is a Intel SRCSASBB8I ) # dmesg |grep -i mfi mfi0: port 0x2000-0x20ff mem 0xb8b00000-0xb8b3ffff,0xb8b40000-0xb8b7ffff irq 16 at device 0.0 on pci10 mfi0: Megaraid SAS driver Ver 3.00 mfi0: 1870 (287314652s/0x0020/info) - Shutdown command received from host mfi0: 1871 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0060/1000/1013/8086) mfi0: 1872 (boot + 3s/0x0020/info) - Firmware version 1.20.72-0562 mfi0: 1873 (boot + 3s/0x0020/info) - Board Revision mfi0: 1874 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) mfi0: 1875 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=5000c5000bcb90b1,0000000000000000 mfi0: 1876 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) mfi0: 1877 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) Info: enclPd=ffff, scsiType=0, portMap=01, sasAddr=5000c5000bcb962d,0000000000000000 mfi0: 1878 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) mfi0: 1879 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) Info: enclPd=ffff, scsiType=0, portMap=02, sasAddr=5000c5000bcb8f8d,0000000000000000 mfi0: [ITHREAD] mfi0: 1880 (287314711s/0x0020/info) - Time established as 02/07/09 9:38:31; (52 seconds since power on) mfid0: on mfi0 mfid0: 556928MB (1140588544 sectors) RAID volume '' I have installed RHEL 5.3 x64 on this server and here is simple dd performance test: Last login: Fri Feb 6 12:15:32 2009 from 10.0.0.51 [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 1.8473 seconds, 443 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 2.05412 seconds, 399 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 1.94096 seconds, 422 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 2.1092 seconds, 388 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 1.84414 seconds, 444 MB/s [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes (819 MB) copied, 2.09829 seconds, 390 MB/s And after that I have tested with FreeBSD 6.3 and 7.1: 6.3: # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.534685 secs (231760388 bytes/sec) # cd /var/tmp/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.051668 secs (268443342 bytes/sec) # cd / # cd /var/tmp/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.001266 secs (272951481 bytes/sec) # cd /home/ cd: can't cd to /home/ # cd /usr/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.678156 secs (222720290 bytes/sec) # cd /boot # dd if=/dev/zero of=bigfile bs=8192 count=100000 cd /usr/100000+0 records in 100000+0 records out 819200000 bytes transferred in 2.985471 secs (274395563 bytes/sec) # local # pwd /usr/local # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.501986 secs (233924406 bytes/sec) 7.1: Filesystem Size Used Avail Capacity Mounted on /dev/mfid0s1a 9.7G 2.6G 6.3G 29% / devfs 1.0K 1.0K 0B 100% /dev /dev/mfid0s1d 478G 4.0K 440G 0% /opt # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.441476 secs (238037393 bytes/sec) # cd / # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.132512 secs (261515371 bytes/sec) # cd /usr/local/ # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.296514 secs (248504951 bytes/sec) # cd /usr # dd if=/dev/zero of=bigfile bs=8192 count=100000 100000+0 records in 100000+0 records out 819200000 bytes transferred in 3.069655 secs (266870386 bytes/sec) as you can see there is a big difference in just simple dd test. Is there additional steps that I can follow to increase performance? By the way If I use write-thru cache on this raid card FreeBSD 6.3 and FreeBSD 7.1 only gives 13MB/s~ which is very very bad. Linux gives a slight decrease on dd test with write-thru cache but freebsd goes from 230-270 MB/s to only 13 MB/s Regards. From rwatson at FreeBSD.org Sat Feb 7 04:26:17 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Sat Feb 7 04:26:31 2009 Subject: FreeBSD 6.3/7.1 and Linux disk performance test In-Reply-To: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> References: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> Message-ID: On Sat, 7 Feb 2009, Omer Faruk Sen wrote: > I have installed a new server to test performance results. BIOS and RAID > BIOS is the latest in this server ( Raid controller is a Intel SRCSASBB8I ) Hi Omer-- Comparing I/O and file system performance is a bit fraught with peril, especially given that the performance of disks varies a great deal based on where on the disk you're writing, etc. Ellard and Seltzer have a nice NFS benchmarking paper that includes a "Benchmarking traps" section of note in this regard: http://www.eecs.harvard.edu/~margo/papers/freenix03/ The source of the performance difference you're seeing could come from a number of places: - I/O to different parts of the disk due to partition layouts can perform quite differently -- try out diskinfo -t to get an idea of the kind of variance you can see. For example, I get this on my local 3ware array: Transfer rates: outside: 102400 kbytes in 4.056540 sec = 25243 kbytes/sec middle: 102400 kbytes in 2.531003 sec = 40458 kbytes/sec inside: 102400 kbytes in 3.947725 sec = 25939 kbytes/sec That's a massive performance difference and presumably entirely a result of disk layout. The best way to address this is to make sure you're using the same part of the disk for all tests -- so if you're installing different OS's on different partitions, use a single shared partition for the test. - File system can behave quite differently, especially if they vary in size and layout strategies. Do the dd(1) directly to a disk partition, making sure to use the right device on Linux (I believe you want the character device in order to bypass the buffer cache and compare apples to apples). So if you go ahead and do dd(1) directly to the same partition using unbuffered I/O and no file system, it should become more obvious as to whether we're looking at performance loss due to a device driver difference, a file system difference, or perhaps just a disk layout difference. Robert N M Watson Computer Laboratory University of Cambridge > > > > # dmesg |grep -i mfi > mfi0: port 0x2000-0x20ff mem > 0xb8b00000-0xb8b3ffff,0xb8b40000-0xb8b7ffff irq 16 at device 0.0 on > pci10 > mfi0: Megaraid SAS driver Ver 3.00 > mfi0: 1870 (287314652s/0x0020/info) - Shutdown command received from host > mfi0: 1871 (boot + 3s/0x0020/info) - Firmware initialization started > (PCI ID 0060/1000/1013/8086) > mfi0: 1872 (boot + 3s/0x0020/info) - Firmware version 1.20.72-0562 > mfi0: 1873 (boot + 3s/0x0020/info) - Board Revision > mfi0: 1874 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) > mfi0: 1875 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) Info: > enclPd=ffff, scsiType=0, portMap=00, > sasAddr=5000c5000bcb90b1,0000000000000000 > mfi0: 1876 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) > mfi0: 1877 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) Info: > enclPd=ffff, scsiType=0, portMap=01, > sasAddr=5000c5000bcb962d,0000000000000000 > mfi0: 1878 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) > mfi0: 1879 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) Info: > enclPd=ffff, scsiType=0, portMap=02, > sasAddr=5000c5000bcb8f8d,0000000000000000 > mfi0: [ITHREAD] > mfi0: 1880 (287314711s/0x0020/info) - Time established as 02/07/09 > 9:38:31; (52 seconds since power on) > mfid0: on mfi0 > mfid0: 556928MB (1140588544 sectors) RAID volume '' > > > I have installed RHEL 5.3 x64 on this server and here is simple dd > performance test: > > Last login: Fri Feb 6 12:15:32 2009 from 10.0.0.51 > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 1.8473 seconds, 443 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 2.05412 seconds, 399 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 1.94096 seconds, 422 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 2.1092 seconds, 388 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 1.84414 seconds, 444 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 2.09829 seconds, 390 MB/s > > And after that I have tested with FreeBSD 6.3 and 7.1: > > 6.3: > > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.534685 secs (231760388 bytes/sec) > # cd /var/tmp/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.051668 secs (268443342 bytes/sec) > # cd / > # cd /var/tmp/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.001266 secs (272951481 bytes/sec) > # cd /home/ > cd: can't cd to /home/ > # cd /usr/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.678156 secs (222720290 bytes/sec) > # cd /boot > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > cd /usr/100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 2.985471 secs (274395563 bytes/sec) > # local > # pwd > /usr/local > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.501986 secs (233924406 bytes/sec) > > 7.1: > > Filesystem Size Used Avail Capacity Mounted on > /dev/mfid0s1a 9.7G 2.6G 6.3G 29% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/mfid0s1d 478G 4.0K 440G 0% /opt > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.441476 secs (238037393 bytes/sec) > # cd / > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.132512 secs (261515371 bytes/sec) > # cd /usr/local/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.296514 secs (248504951 bytes/sec) > > # cd /usr > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.069655 secs (266870386 bytes/sec) > > as you can see there is a big difference in just simple dd test. Is > there additional steps that I can follow to increase performance? By > the way If I use write-thru cache on this raid card FreeBSD 6.3 and > FreeBSD 7.1 only gives 13MB/s~ which is very very bad. Linux gives a > slight decrease on dd test with write-thru cache but freebsd goes from > 230-270 MB/s to only 13 MB/s > > > Regards. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From pieter at degoeje.nl Sat Feb 7 05:37:30 2009 From: pieter at degoeje.nl (Pieter de Goeje) Date: Sat Feb 7 05:37:38 2009 Subject: FreeBSD 6.3/7.1 and Linux disk performance test In-Reply-To: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> References: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> Message-ID: <200902071409.27458.pieter@degoeje.nl> On Saturday 07 February 2009 09:00:37 Omer Faruk Sen wrote: > Hi, > > I have installed a new server to test performance results. BIOS and > RAID BIOS is the latest in this server ( Raid controller is a Intel > SRCSASBB8I ) > > > > # dmesg |grep -i mfi > mfi0: port 0x2000-0x20ff mem > 0xb8b00000-0xb8b3ffff,0xb8b40000-0xb8b7ffff irq 16 at device 0.0 on > pci10 > mfi0: Megaraid SAS driver Ver 3.00 > mfi0: 1870 (287314652s/0x0020/info) - Shutdown command received from host > mfi0: 1871 (boot + 3s/0x0020/info) - Firmware initialization started > (PCI ID 0060/1000/1013/8086) > mfi0: 1872 (boot + 3s/0x0020/info) - Firmware version 1.20.72-0562 > mfi0: 1873 (boot + 3s/0x0020/info) - Board Revision > mfi0: 1874 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) > mfi0: 1875 (boot + 15s/0x0002/info) - Inserted: PD 08(e0xff/s8) Info: > enclPd=ffff, scsiType=0, portMap=00, > sasAddr=5000c5000bcb90b1,0000000000000000 > mfi0: 1876 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) > mfi0: 1877 (boot + 15s/0x0002/info) - Inserted: PD 09(e0xff/s9) Info: > enclPd=ffff, scsiType=0, portMap=01, > sasAddr=5000c5000bcb962d,0000000000000000 > mfi0: 1878 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) > mfi0: 1879 (boot + 15s/0x0002/info) - Inserted: PD 0a(e0xff/s10) Info: > enclPd=ffff, scsiType=0, portMap=02, > sasAddr=5000c5000bcb8f8d,0000000000000000 > mfi0: [ITHREAD] > mfi0: 1880 (287314711s/0x0020/info) - Time established as 02/07/09 > 9:38:31; (52 seconds since power on) > mfid0: on mfi0 > mfid0: 556928MB (1140588544 sectors) RAID volume '' > > > I have installed RHEL 5.3 x64 on this server and here is simple dd > performance test: > > Last login: Fri Feb 6 12:15:32 2009 from 10.0.0.51 > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 1.8473 seconds, 443 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 2.05412 seconds, 399 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 1.94096 seconds, 422 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 2.1092 seconds, 388 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 1.84414 seconds, 444 MB/s > [root@localhost ~]# dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes (819 MB) copied, 2.09829 seconds, 390 MB/s > > And after that I have tested with FreeBSD 6.3 and 7.1: > > 6.3: > > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.534685 secs (231760388 bytes/sec) > # cd /var/tmp/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.051668 secs (268443342 bytes/sec) > # cd / > # cd /var/tmp/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.001266 secs (272951481 bytes/sec) > # cd /home/ > cd: can't cd to /home/ > # cd /usr/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.678156 secs (222720290 bytes/sec) > # cd /boot > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > cd /usr/100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 2.985471 secs (274395563 bytes/sec) > # local > # pwd > /usr/local > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.501986 secs (233924406 bytes/sec) > > 7.1: > > Filesystem Size Used Avail Capacity Mounted on > /dev/mfid0s1a 9.7G 2.6G 6.3G 29% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/mfid0s1d 478G 4.0K 440G 0% /opt > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.441476 secs (238037393 bytes/sec) > # cd / > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.132512 secs (261515371 bytes/sec) > # cd /usr/local/ > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.296514 secs (248504951 bytes/sec) > > # cd /usr > # dd if=/dev/zero of=bigfile bs=8192 count=100000 > 100000+0 records in > 100000+0 records out > 819200000 bytes transferred in 3.069655 secs (266870386 bytes/sec) > > as you can see there is a big difference in just simple dd test. Is > there additional steps that I can follow to increase performance? By > the way If I use write-thru cache on this raid card FreeBSD 6.3 and > FreeBSD 7.1 only gives 13MB/s~ which is very very bad. Linux gives a > slight decrease on dd test with write-thru cache but freebsd goes from > 230-270 MB/s to only 13 MB/s I noticed lately that the blocksize and fragment size can make a huge difference on filesystem performance when dealing with big files. Performance went up from 130MB/s to 200MB/s (2 disk raid0), after I added the following parameters to newfs(8): -b 65536 -f 8192. This increases the block and fragment size by a factor of 4 over the the defaults. Also, you use dd with a small blocksize. Try 64k or 128k instead. - Pieter de Goeje From aryeh.friedman at gmail.com Sat Feb 7 11:51:30 2009 From: aryeh.friedman at gmail.com (Aryeh M. Friedman) Date: Sat Feb 7 11:51:38 2009 Subject: setting up net/cvsup-mirror correctly Message-ID: <498DE640.4090904@gmail.com> I have installed net/cvsup-mirror with all the defaults but when I attempt to actually run cvsupd I get the following: prod# cvsupd & [1] 99334 prod# cvsup -g -h localhost /usr/share/examples/cvsup/cvs-supfile Connected to localhost 2009.02.07 14:48:28 EST [99334]: +0 aryeh@localhost (prod.istudentunion.com) [SNAP_16_1h/17.0] 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "src-all" 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "ports-all" 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "doc-all" 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "www" 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "projects-all" 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "cvsroot-all" Server message: Unknown collection "src-all" Server message: Unknown collection "ports-all" Server message: Unknown collection "doc-all" Server message: Unknown collection "www" Server message: Unknown collection "projects-all" Server message: Unknown collection "cvsroot-all" Skipping collection src-all/cvs Skipping collection ports-all/cvs Skipping collection doc-all/cvs Skipping collection www/cvs Skipping collection projects-all/cvs Skipping collection cvsroot-all/cvs Finished successfully 2009.02.07 14:48:28 EST [99334]: -0 [0Kin+0Kout] Finished successfully 2009.02.07 14:48:28 EST [99334]: Going down [1] + Done cvsupd prod# After reading through some examples on the web of how to set cvsupd up I can not see any difference between my setup and other peoples except most of the examples on the web use something else then /usr/local/etc/cvsup as the base dir. Any ideas on what I need to do to get it working? BTW once I get it going since I have a 20 MB/s upload on my fios I am going to make this an official cvsupXXX.us.freebsd.org server (Only one in the NYC metro area I know of) From aryeh.friedman at gmail.com Sat Feb 7 11:53:48 2009 From: aryeh.friedman at gmail.com (Aryeh M. Friedman) Date: Sat Feb 7 11:53:56 2009 Subject: setting up net/cvsup-mirror correctly In-Reply-To: <498DE640.4090904@gmail.com> References: <498DE640.4090904@gmail.com> Message-ID: <498DE6CA.4080605@gmail.com> oops forgot to mention I have been running a local repo (as per the instructions in development(7)) for almost a year now and updating my /usr/src and /usr/ports from works without a hitch Aryeh M. Friedman wrote: > I have installed net/cvsup-mirror with all the defaults but when I > attempt to actually run cvsupd I get the following: > > prod# cvsupd & > [1] 99334 > prod# cvsup -g -h localhost /usr/share/examples/cvsup/cvs-supfile > Connected to localhost > 2009.02.07 14:48:28 EST [99334]: +0 aryeh@localhost > (prod.istudentunion.com) [SNAP_16_1h/17.0] > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "src-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "ports-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "doc-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "www" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "projects-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "cvsroot-all" > Server message: Unknown collection "src-all" > Server message: Unknown collection "ports-all" > Server message: Unknown collection "doc-all" > Server message: Unknown collection "www" > Server message: Unknown collection "projects-all" > Server message: Unknown collection "cvsroot-all" > Skipping collection src-all/cvs > Skipping collection ports-all/cvs > Skipping collection doc-all/cvs > Skipping collection www/cvs > Skipping collection projects-all/cvs > Skipping collection cvsroot-all/cvs > Finished successfully > 2009.02.07 14:48:28 EST [99334]: -0 [0Kin+0Kout] Finished successfully > 2009.02.07 14:48:28 EST [99334]: Going down > [1] + Done cvsupd > prod# > > After reading through some examples on the web of how to set cvsupd up > I can not see any difference between my setup and other peoples except > most of the examples on the web use something else then > /usr/local/etc/cvsup as the base dir. Any ideas on what I need to do > to get it working? BTW once I get it going since I have a 20 MB/s > upload on my fios I am going to make this an official > cvsupXXX.us.freebsd.org server (Only one in the NYC metro area I know of) > > > From aryeh.friedman at gmail.com Sat Feb 7 16:52:57 2009 From: aryeh.friedman at gmail.com (Aryeh M. Friedman) Date: Sat Feb 7 16:53:04 2009 Subject: setting up net/cvsup-mirror correctly In-Reply-To: <498DE640.4090904@gmail.com> References: <498DE640.4090904@gmail.com> Message-ID: <498E2CE1.6020606@gmail.com> Aryeh M. Friedman wrote: > I have installed net/cvsup-mirror with all the defaults but when I > attempt to actually run cvsupd I get the following: > > prod# cvsupd & > [1] 99334 > prod# cvsup -g -h localhost /usr/share/examples/cvsup/cvs-supfile > Connected to localhost > 2009.02.07 14:48:28 EST [99334]: +0 aryeh@localhost > (prod.istudentunion.com) [SNAP_16_1h/17.0] > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "src-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "ports-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "doc-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "www" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "projects-all" > 2009.02.07 14:48:28 EST [99334]: =0 Unknown collection "cvsroot-all" > Server message: Unknown collection "src-all" > Server message: Unknown collection "ports-all" > Server message: Unknown collection "doc-all" > Server message: Unknown collection "www" > Server message: Unknown collection "projects-all" > Server message: Unknown collection "cvsroot-all" > Skipping collection src-all/cvs > Skipping collection ports-all/cvs > Skipping collection doc-all/cvs > Skipping collection www/cvs > Skipping collection projects-all/cvs > Skipping collection cvsroot-all/cvs > Finished successfully > 2009.02.07 14:48:28 EST [99334]: -0 [0Kin+0Kout] Finished successfully > 2009.02.07 14:48:28 EST [99334]: Going down > [1] + Done cvsupd > prod# > > After reading through some examples on the web of how to set cvsupd up > I can not see any difference between my setup and other peoples except > most of the examples on the web use something else then > /usr/local/etc/cvsup as the base dir. Any ideas on what I need to do > to get it working? BTW once I get it going since I have a 20 MB/s > upload on my fios I am going to make this an official > cvsupXXX.us.freebsd.org server (Only one in the NYC metro area I know of) > > > Problem fixed... I am going to file a bug report on it though because cvsup-master.freebsd.org is an inapproriate default upstream for most sites (you need a password) From danny at cs.huji.ac.il Sun Feb 8 01:21:09 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Sun Feb 8 01:21:16 2009 Subject: impossible packet length ... In-Reply-To: Your message of Fri, 06 Feb 2009 08:32:27 +0200 . Message-ID: I'm reposting this to hackers, and there is some more info. > Hi, > on 2 different servers, running 7.1-stable + zfs, I get this > error rather frequently: > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (543383918) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1936028704) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1869363744) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1667787057) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (976040755) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1953459488) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1348825156) from > nfs server sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (0) from nfs server > sunfire:/dist > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1647208041) from > nfs server sunfire:/dist > > in this case the server is running Freebsd-7.0-stable, but I also get it when > the server is a > netapp. > > is there a connection? > > thanks, > danny going through the logs, after it happened again, I got a glimps of this: Feb 6 18:00:13 warhol-00.cs.huji.ac.il kernel: bce0: discard frame w/o leading ethernet header (len 0 pkt len 0) Feb 6 18:00:19 klee-05.cs.huji.ac.il kernel: nfs: server warhol-00 not responding, timed out ... Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: More than a single value for /defaults in hesiod.local Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: Unknown $ sequence in "rhost:=${RHOST};type:=nfsl;fs:=${FS};rfs:=$huldig#^ZM-^KoM- abase" Feb 6 19:00:00 warhol-00.cs.huji.ac.il kernel: impossible packet length (2068989523) from nfs server sunfire:/dist which seems to point fingers at bce... danny From kris at FreeBSD.org Sun Feb 8 01:26:58 2009 From: kris at FreeBSD.org (Kris Kennaway) Date: Sun Feb 8 01:27:10 2009 Subject: FreeBSD 6.3/7.1 and Linux disk performance test In-Reply-To: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> References: <75a268720902070000r7dbe758aq60454092fcab0198@mail.gmail.com> Message-ID: <498EA55F.5020904@FreeBSD.org> Omer Faruk Sen wrote: > as you can see there is a big difference in just simple dd test. Is > there additional steps that I can follow to increase performance? Use a benchmark that matches your actual workload, and then see how things look. I would be surprised if your target workload was "dd" :-) Kris From danny at cs.huji.ac.il Sun Feb 8 01:31:47 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Sun Feb 8 01:32:01 2009 Subject: impossible packet length ... In-Reply-To: <20090208091656.GA31876@test71.vk2pj.dyndns.org> References: <20090208091656.GA31876@test71.vk2pj.dyndns.org> Message-ID: > > --jI8keyz6grp/JLjh > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > Content-Transfer-Encoding: quoted-printable > > On 2009-Feb-08 10:45:13 +0200, Danny Braniss wrote: > >Feb 6 18:00:13 warhol-00.cs.huji.ac.il kernel: bce0: discard frame w/o=20 > >leading ethernet header (len 0 pkt len 0) > =2E.. > >Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: Unknown $ sequence in=20 > >"rhost:=3D${RHOST};type:=3Dnfsl;fs:=3D${FS};rfs:=3D$huldig#^ZM-^KoM- a= > base" > >Feb 6 19:00:00 warhol-00.cs.huji.ac.il kernel: impossible packet length= > =20 > >(2068989523) from nfs server sunfire:/dist > > > >which seems to point fingers at bce... > > It does rather suggest that bce is not behaving. What happens if you > turn off checksum off-loading? This should make the kernel drop the > corrupt packets instead of trying to process them. If practical, you > could also try (temporarily) plugging in a different NIC. > I have, and now it's a matter of waiting... Q: with rxcsum on, and a bad checksum packet is received, is it dropped by the NIC? if not, then it somewhat explains the behaviour changing the nic is tough, but if needed will be done. danny > Peter Jeremy From kostikbel at gmail.com Sun Feb 8 03:23:05 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Sun Feb 8 03:23:18 2009 Subject: impossible packet length ... In-Reply-To: References: Message-ID: <20090208112259.GW9427@deviant.kiev.zoral.com.ua> On Sun, Feb 08, 2009 at 10:45:13AM +0200, Danny Braniss wrote: > I'm reposting this to hackers, and there is some more info. > > > Hi, > > on 2 different servers, running 7.1-stable + zfs, I get this > > error rather frequently: > > > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (543383918) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1936028704) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1869363744) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1667787057) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (976040755) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1953459488) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1348825156) from > > nfs server sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (0) from nfs server > > sunfire:/dist > > Feb 5 17:01:03 warhol-00 kernel: impossible packet length (1647208041) from > > nfs server sunfire:/dist > > > > in this case the server is running Freebsd-7.0-stable, but I also get it when > > the server is a > > netapp. > > > > is there a connection? > > > > thanks, > > danny > > going through the logs, after it happened again, I got a glimps of this: > > Feb 6 18:00:13 warhol-00.cs.huji.ac.il kernel: bce0: discard frame w/o > leading ethernet header (len 0 pkt len 0) > Feb 6 18:00:19 klee-05.cs.huji.ac.il kernel: nfs: server warhol-00 not > responding, timed out > ... > Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: More than a single value for > /defaults in hesiod.local > Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: Unknown $ sequence in > "rhost:=${RHOST};type:=nfsl;fs:=${FS};rfs:=$huldig#^ZM-^KoM- abase" > Feb 6 19:00:00 warhol-00.cs.huji.ac.il kernel: impossible packet length > (2068989523) from nfs server sunfire:/dist > > which seems to point fingers at bce... bce(4) is broken in stable, your best option is to revert to the driver in releng 7.1. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090208/933fa35a/attachment.pgp From kostikbel at gmail.com Sun Feb 8 04:11:47 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Sun Feb 8 04:11:55 2009 Subject: Possible VFS KPI and KBI breakage on stable/7 Message-ID: <20090208121142.GX9427@deviant.kiev.zoral.com.ua> There are three sets of changes that would benefit stable/7. Namely, there are 1. Improvements for the UFS unmount or rw->ro remount, that perform suspension during the operation. The changes depend on the the suspension mechanism path, that introduced the suspension owner, and added new VFS OP into the mount method table. This might also fix the hangs with gjournal or gjournal together with snapshots experienced by some users. Since the only real consumer of the suspension is UFS, I believe that MFC would have quite low impact, if any. Corresponding revision is 183073. 2. The openat(2) and similar syscalls. The new ZFS requires openat() functionality. We have to change struct nameidata to merge NDINIT_ATVP(). All modules using namei() need to be recompiled. 3. The Marcus' work on vn_fullpath() support for synthetic filesystems introduces new VOP, vop_vptocnp. This would allow procstat(1) to work on devfs and pseudofs vnodes. As I understand, this would also improve Gnome experience on FreeBSD. All fs modules need to be recompiled. There was one very magisterial voice that objected against KBI breakage on stable branch in principle. In my opinion, the benefits of the bug fixes and functionality improvements with the proposed merges are much greater then inconvenience of the need to recompile out-of-tree fs modules. Changes were discussed with re@ to some extent. In case there is vocal objection against the merge, I would abstain from doing this. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090208/76ae61b3/attachment.pgp From olli at lurza.secnetix.de Sun Feb 8 04:17:48 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Sun Feb 8 04:18:02 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902081001.04075.matt@chronos.org.uk> Message-ID: <200902081217.n18CHYlx014125@lurza.secnetix.de> Matt Dawson wrote: > On Saturday 07 February 2009 18:59:43 Oliver Fromme wrote: > > In fact I have prepared a theme with beastie; here's > > a screen shot (preliminary): > > > > http://www.secnetix.de/olli/FreeBSD/vloader/screenshot5.png > > Perfect. Clean, logical, concise, the three words I associate above all with > FreeBSD. I would change all machines' loaders to this in a heartbeat, although > others may have different ideas, which is where the space hopper comes in, I > suppose. Nothing against the horned ball and your variant of the new graphics > looks very neat and clean, but I'm used to having Beastie around. For some > reason, this strikes the correct balance for me between nice graphics and too > much "bling." My personal tastes only, naturally. > > That screenshot looks very professional. Well done, Oliver. Any chance > of rolling another tarball with that theme for we traditionalists? Please? > Good news: I don't have to roll another tarball for that, because this is quite easy to configure. Please download the existing tarball and follow the instructions that I posted. Please verify that it works. Then download the background image that contains beastie: http://www.secnetix.de/olli/FreeBSD/vloader/background/beastie.pcx Create a directory /boot/themes/beastie and save the image as beastie.pcx in that directory. Then create a text file themes.conf in the same directory, containing these lines: theme_background="/boot/themes/beastie/beastie.pcx" theme_font="/boot/themes/default/stdfont" theme_fgcolor="0 0 0" # black theme_bgcolor="255 255 255" # white theme_litcolor="255 64 32" # bright red theme_dimcolor="64 64 128" # dark bluish grey theme_options_xy="17 170" theme_actions_xy="17 281" Finally, change the beastie_theme line in /boot/loader.conf like this: beastie_theme="/boot/themes/beastie/theme.conf" Done. Of course, you can also use your own image if you want to create one. Just make sure it's 640 x 480 pixels at 4 bit depth (16 colors maximum). I recommend to use ppmtopcx (from ports/graphics/netpbm), because I have used this extensively for testing with my PCX decoder. Also note that there should be appropriate palette entries for the text (e.g. black and white). The code will try to use the closest possible palette entry if there is no exact match for the given RGB values. There's ONE IMPORTANT THING I have to say: I cannot take credit for any of the artwork. I am not an artist. All of the graphics images where taken from the FreeBSD website. I only scaled and arranged it a little bit, adapted the palette etc. The only thing I created myself is the proportional font; the "source code" is here: http://www.secnetix.de/olli/FreeBSD/vloader/stdfont.txt Modifying the font or creating a completely new font is also very easy. There's a tool that "compiles" the text source code (like the one above) to binary format. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Software gets slower faster than hardware gets faster." -- Niklaus Wirth From olli at lurza.secnetix.de Sun Feb 8 04:39:56 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Sun Feb 8 04:40:02 2009 Subject: ZFS and Graphics support for /boot/loader In-Reply-To: Message-ID: <200902081239.n18CdnOk015298@lurza.secnetix.de> Paul Wootton wrote: > Hi Oliver, > > This doesn?t work for me. > > I am booting off a ZFS mirror with GPT partitions (built from current on an > amd64). I'm sorry, I've compiled this loader binary with the default options, which does not include ZFS support. > Is there any change of a version of gloader but with ZFS support? I'll put it on my to-do list. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Software gets slower faster than hardware gets faster." -- Niklaus Wirth From olli at lurza.secnetix.de Sun Feb 8 04:52:54 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Sun Feb 8 04:53:06 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <20090208073927.77e2c829@bhuda.mired.org> Message-ID: <200902081252.n18CqhZf016155@lurza.secnetix.de> Mike Meyer wrote: > I'm curious - is there a reason that the numbers from the old screen > have turned into function keys on this one? No. That screen shot is an old one. In the current code, the number keys are used as usual, no function keys. In fact, it is not possible to use function keys from the FORTH code without resorting to dirty hacks. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Share your knowledge. It is a way to achieve immortality." -- The Dalai Lama From matt at chronos.org.uk Sun Feb 8 04:39:58 2009 From: matt at chronos.org.uk (Matt Dawson) Date: Sun Feb 8 04:56:39 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902081217.n18CHYlx014125@lurza.secnetix.de> References: <200902081217.n18CHYlx014125@lurza.secnetix.de> Message-ID: <200902081239.54250.matt@chronos.org.uk> On Sunday 08 February 2009 12:17:34 Oliver Fromme wrote: > http://www.secnetix.de/olli/FreeBSD/vloader/background/beastie.pcx > > Create a directory /boot/themes/beastie and save the image > as beastie.pcx in that directory. ?Then create a text file > themes.conf in the same directory, containing these lines: > > theme_background="/boot/themes/beastie/beastie.pcx" > theme_font="/boot/themes/default/stdfont" > theme_fgcolor="0 0 0" ? ? ? ? ? # black > theme_bgcolor="255 255 255" ? ? # white > theme_litcolor="255 64 32" ? ? ?# bright red > theme_dimcolor="64 64 128" ? ? ?# dark bluish grey > theme_options_xy="17 170" > theme_actions_xy="17 281" > > Finally, change the beastie_theme line in /boot/loader.conf > like this: > > beastie_theme="/boot/themes/beastie/theme.conf" > > Done. ...and it is. Thanks, Oliver! -- Matt Dawson MTD15-RIPE matt@chronos.org.uk From mwm-keyword-freebsdhackers2.e313df at mired.org Sun Feb 8 05:06:50 2009 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sun Feb 8 05:07:17 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902081217.n18CHYlx014125@lurza.secnetix.de> References: <200902081001.04075.matt@chronos.org.uk> <200902081217.n18CHYlx014125@lurza.secnetix.de> Message-ID: <20090208073927.77e2c829@bhuda.mired.org> I'm curious - is there a reason that the numbers from the old screen have turned into function keys on this one? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From peter at vk2pj.dyndns.org Sun Feb 8 05:39:25 2009 From: peter at vk2pj.dyndns.org (Peter Jeremy) Date: Sun Feb 8 07:51:47 2009 Subject: impossible packet length ... In-Reply-To: References: Message-ID: <20090208091656.GA31876@test71.vk2pj.dyndns.org> On 2009-Feb-08 10:45:13 +0200, Danny Braniss wrote: >Feb 6 18:00:13 warhol-00.cs.huji.ac.il kernel: bce0: discard frame w/o >leading ethernet header (len 0 pkt len 0) ... >Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: Unknown $ sequence in >"rhost:=${RHOST};type:=nfsl;fs:=${FS};rfs:=$huldig#^ZM-^KoM- abase" >Feb 6 19:00:00 warhol-00.cs.huji.ac.il kernel: impossible packet length >(2068989523) from nfs server sunfire:/dist > >which seems to point fingers at bce... It does rather suggest that bce is not behaving. What happens if you turn off checksum off-loading? This should make the kernel drop the corrupt packets instead of trying to process them. If practical, you could also try (temporarily) plugging in a different NIC. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090208/3c54a215/attachment.pgp From anderson at freebsd.org Sun Feb 8 20:09:34 2009 From: anderson at freebsd.org (Eric Anderson) Date: Sun Feb 8 20:09:41 2009 Subject: impossible packet length ... In-Reply-To: References: <20090208091656.GA31876@test71.vk2pj.dyndns.org> Message-ID: <205598DD-B746-4236-9140-855811BAE21C@freebsd.org> On Feb 8, 2009, at 3:31 AM, Danny Braniss wrote: >> >> --jI8keyz6grp/JLjh >> Content-Type: text/plain; charset=us-ascii >> Content-Disposition: inline >> Content-Transfer-Encoding: quoted-printable >> >> On 2009-Feb-08 10:45:13 +0200, Danny Braniss >> wrote: >>> Feb 6 18:00:13 warhol-00.cs.huji.ac.il kernel: bce0: discard >>> frame w/o=20 >>> leading ethernet header (len 0 pkt len 0) >> =2E.. >>> Feb 6 19:00:00 warhol-00.cs.huji.ac.il amd[715]: Unknown $ >>> sequence in=20 >>> "rhost:=3D${RHOST};type:=3Dnfsl;fs:=3D${FS};rfs:=3D$huldig#^ZM- >>> ^KoM- a= >> base" >>> Feb 6 19:00:00 warhol-00.cs.huji.ac.il kernel: impossible packet >>> length= >> =20 >>> (2068989523) from nfs server sunfire:/dist >>> >>> which seems to point fingers at bce... >> >> It does rather suggest that bce is not behaving. What happens if you >> turn off checksum off-loading? This should make the kernel drop the >> corrupt packets instead of trying to process them. If practical, you >> could also try (temporarily) plugging in a different NIC. >> > I have, and now it's a matter of waiting... > Q: with rxcsum on, and a bad checksum packet is received, is it > dropped by the NIC? if not, then it somewhat explains the behaviour > > changing the nic is tough, but if needed will be done. > danny > >> Peter Jeremy We were hitting this quite a bit (also bce), and updated to a recent 7- branch and it seems to be behaving better for now. Running 12 days so far (which is better than what we had been seeing). Eric From davidcollins001 at gmail.com Mon Feb 9 14:09:17 2009 From: davidcollins001 at gmail.com (David Collins) Date: Mon Feb 9 14:09:24 2009 Subject: broken ports In-Reply-To: <20090209213249.GA4401@melon.esperance-linux.co.uk> References: <20090207075521.GA93084@melon.esperance-linux.co.uk> <20090208203553.GA99661@melon.esperance-linux.co.uk> <1b30fd140902081424p1d75e304y9f4ef9f9b472328c@mail.gmail.com> <20090209055634.GA2116@melon.esperance-linux.co.uk> <1b30fd140902082330m21b4aecyf99cb83a080e972a@mail.gmail.com> <20090209083338.GA2666@melon.esperance-linux.co.uk> <1b30fd140902090108m292ef929tdce1ce2c946ead6b@mail.gmail.com> <20090209093505.GA2835@melon.esperance-linux.co.uk> <1b30fd140902091152x17ed90e5sf915b361927067e@mail.gmail.com> <20090209213249.GA4401@melon.esperance-linux.co.uk> Message-ID: <1b30fd140902091348j56068545hb90905eb87050acc@mail.gmail.com> Thank you for your input. I feel like this problem is too far over my head to be able to give adequate enough debugging. I am not against any alternative methods of resolving this (even removing and reinstalling ports) rather than fixing, I just don't want to have to jump into sysinstall or reinstall because i believe that is a soluion. David On 09/02/2009, Frank Shute wrote: > On Mon, Feb 09, 2009 at 07:52:58PM +0000, David Collins wrote: >> >> > Then you should be able to build rtorrent. >> > >> > Then what I'd do is deinstall gcc42: >> > >> > # pkg_deinstall -f gcc-4.2.5_20080702 >> > >> > Comment out the CC line in /etc/make.conf & run ldconfig: >> > >> > # /etc/rc.d/ldconfig start >> > >> > and hopefully your system is then back to normal. >> >> I tried all this and rtorrent would still not install, with the same >> error at configure. I was however able to pkg_deinstall the gcc it >> installed. That didn't solve any problems though. >> >> I feel that it might be easier to focus on the following compiling >> problem that still exists: >> >> viper:~$ gcc -o hello hello.c >> /usr/bin/ld: cannot find -lgcc_s > > What this is telling you is the linker, ld(1), can't find the gcc_s > library. ldconfig(8) tells ld where to look. > > What does: > > $ ldconfig -r | grep gcc_s > > tell you now that you've removed that compiler? You should get > something like: > > $ ldconfig -r | grep gcc_s > 30:-lgcc_s.1 => /lib/libgcc_s.so.1 > > i.e. It's looking in the system libs. > > If it still tells you that the lib is under /usr/local/lib, then > there's your problem & you have to regenerate a fresh hints file, I > think. > > The problem is that I've never (IIRC) installed a compiler from > ports & I don't know how it screws with the compiler toolchain & hence > how to put it right. Hence, I've cc'd this to hackers@ in the hope that > someone who is more familiar with the toolchain can give advice. > > > Regards, > > -- > > Frank > > > Contact info: http://www.shute.org.uk/misc/contact.html > > From frank at shute.org.uk Mon Feb 9 13:48:11 2009 From: frank at shute.org.uk (Frank Shute) Date: Mon Feb 9 14:44:49 2009 Subject: broken ports In-Reply-To: <1b30fd140902091152x17ed90e5sf915b361927067e@mail.gmail.com> References: <20090207075521.GA93084@melon.esperance-linux.co.uk> <1b30fd140902080816v7dfb8bb6g6f3c9a3d162f48c0@mail.gmail.com> <20090208203553.GA99661@melon.esperance-linux.co.uk> <1b30fd140902081424p1d75e304y9f4ef9f9b472328c@mail.gmail.com> <20090209055634.GA2116@melon.esperance-linux.co.uk> <1b30fd140902082330m21b4aecyf99cb83a080e972a@mail.gmail.com> <20090209083338.GA2666@melon.esperance-linux.co.uk> <1b30fd140902090108m292ef929tdce1ce2c946ead6b@mail.gmail.com> <20090209093505.GA2835@melon.esperance-linux.co.uk> <1b30fd140902091152x17ed90e5sf915b361927067e@mail.gmail.com> Message-ID: <20090209213249.GA4401@melon.esperance-linux.co.uk> On Mon, Feb 09, 2009 at 07:52:58PM +0000, David Collins wrote: > > > Then you should be able to build rtorrent. > > > > Then what I'd do is deinstall gcc42: > > > > # pkg_deinstall -f gcc-4.2.5_20080702 > > > > Comment out the CC line in /etc/make.conf & run ldconfig: > > > > # /etc/rc.d/ldconfig start > > > > and hopefully your system is then back to normal. > > I tried all this and rtorrent would still not install, with the same > error at configure. I was however able to pkg_deinstall the gcc it > installed. That didn't solve any problems though. > > I feel that it might be easier to focus on the following compiling > problem that still exists: > > viper:~$ gcc -o hello hello.c > /usr/bin/ld: cannot find -lgcc_s What this is telling you is the linker, ld(1), can't find the gcc_s library. ldconfig(8) tells ld where to look. What does: $ ldconfig -r | grep gcc_s tell you now that you've removed that compiler? You should get something like: $ ldconfig -r | grep gcc_s 30:-lgcc_s.1 => /lib/libgcc_s.so.1 i.e. It's looking in the system libs. If it still tells you that the lib is under /usr/local/lib, then there's your problem & you have to regenerate a fresh hints file, I think. The problem is that I've never (IIRC) installed a compiler from ports & I don't know how it screws with the compiler toolchain & hence how to put it right. Hence, I've cc'd this to hackers@ in the hope that someone who is more familiar with the toolchain can give advice. Regards, -- Frank Contact info: http://www.shute.org.uk/misc/contact.html From yuri at rawbw.com Mon Feb 9 14:59:24 2009 From: yuri at rawbw.com (Yuri) Date: Mon Feb 9 14:59:41 2009 Subject: How to troubleshoot why ath0 can't connect to a passwordless wireless network? Message-ID: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> I have a several wireless networks without password that my linux box easily connects to. On FreeBSD 'ifconfig ath0 up scan' command shows it. 'ifconfig ath0 ssid up' brings interface to 'associated' state. But dhclient fails to set it up. I have another device on the same system: ral0. It sometimes connects to these networks ok, sometimes has the same problem. What can I do to understand what may be a problem with ath0 in my case? I tried to use tcpdump. It shows outbound DHCP packets and nothing is inbound. I asked similar question here before, somebody asked me to downgrade atheros driver to one particular lower version. But this didn't help. Relevant dmesg lines are: ath_hal: 0.9.30.13 (AR5210, AR5211, AR5212, AR5216, RF5111, RF5112, RF2413, RF5413, RF2133) ath0: mem 0xcffe0000-0xcffeffff irq 16 at device 5.0 on pci0 ath0: mac 7.8 phy 4.5 radio 5.6 I use 71-PRERELEASE Yuri From avg at icyb.net.ua Tue Feb 10 03:42:06 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue Feb 10 03:42:13 2009 Subject: superio and "shared" io port Message-ID: <4991680A.5010905@icyb.net.ua> Here I am with Super I/O question again. There is a certain Super I/O that is configured to use the same port for entering/exiting configuration mode as the one that is used by fdc in the same Super I/O. This is port 0x3f0. I guess the hardware works by examining a value written to this port and forwarding non-magic accesses to the fdc. I wonder what is the best way to handle this double-use of an io port with newbus isa code. -- Andriy Gapon From scf at FreeBSD.org Tue Feb 10 06:42:11 2009 From: scf at FreeBSD.org (Sean C. Farley) Date: Tue Feb 10 06:42:18 2009 Subject: How to troubleshoot why ath0 can't connect to a passwordless wireless network? In-Reply-To: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> References: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> Message-ID: On Mon, 9 Feb 2009, Yuri wrote: > I have a several wireless networks without password that my linux box > easily connects to. What version of wpa_supplicant does the Linux box run? > On FreeBSD 'ifconfig ath0 up scan' command shows it. 'ifconfig ath0 > ssid up' brings interface to 'associated' state. But > dhclient fails to set it up. > > I have another device on the same system: ral0. It sometimes connects > to these networks ok, sometimes has the same problem. > > What can I do to understand what may be a problem with ath0 in my > case? > > I tried to use tcpdump. It shows outbound DHCP packets and nothing is > inbound. > > I asked similar question here before, somebody asked me to downgrade > atheros driver to one particular lower version. But this didn't help. > > Relevant dmesg lines are: > ath_hal: 0.9.30.13 (AR5210, AR5211, AR5212, AR5216, RF5111, RF5112, RF2413, > RF5413, RF2133) > ath0: mem 0xcffe0000-0xcffeffff irq 16 at device 5.0 on pci0 > ath0: mac 7.8 phy 4.5 radio 5.6 I had a lot of trouble with ath(4) and my work's wireless network (Aruba Networks based). After patching wpa_supplicant from 0.5.10 to 0.5.11, everything has worked well. Before, it would associate, yet DHCP would not work. Sam Leffler (sam@) has since committed it to CURRENT. I still have the original patch to RELENG_7[1] that the CURRENT patch was based upon. It is a bit old, yet I think it may still apply at least for the most part. Sean 1. http://people.freebsd.org/~scf/wpa_supplicant-0.5.11-RELENG_7.patch -- scf@FreeBSD.org From sam at freebsd.org Tue Feb 10 08:39:28 2009 From: sam at freebsd.org (Sam Leffler) Date: Tue Feb 10 08:39:36 2009 Subject: How to troubleshoot why ath0 can't connect to a passwordless wireless network? In-Reply-To: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> References: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> Message-ID: <4991ADB5.9050803@freebsd.org> Yuri wrote: > I have a several wireless networks without password that my linux box > easily connects to. > > On FreeBSD 'ifconfig ath0 up scan' command shows it. 'ifconfig ath0 > ssid up' brings interface to 'associated' state. But > dhclient fails to set it up. > > I have another device on the same system: ral0. It sometimes connects > to these networks ok, sometimes has the same problem. > > What can I do to understand what may be a problem with ath0 in my case? > > I tried to use tcpdump. It shows outbound DHCP packets and nothing is > inbound. > > I asked similar question here before, somebody asked me to downgrade > atheros driver to one particular lower version. But this didn't help. > > Relevant dmesg lines are: > ath_hal: 0.9.30.13 (AR5210, AR5211, AR5212, AR5216, RF5111, RF5112, > RF2413, RF5413, RF2133) > ath0: mem 0xcffe0000-0xcffeffff irq 16 at device 5.0 on > pci0 > ath0: mac 7.8 phy 4.5 radio 5.6 > > I use 71-PRERELEASE You seem to say your network is open (no security). If not you need to show your network configuration. wlanstats shows statistics collected by the 802.11 layer. athstats show stats collected by the ath driver. Both are important tools for diagnosing problems. tcpdump can be used to tap traffic at 3 layers: 802.3, 802.11, and driver. It can be used to identify where packets are lost in the hierarchy (if at all). Assuming packets are going out but not coming back you can sniff from a 3rd station to look for traffic in the air but not received. Given how little info you posted it's virtually impossible to advise you what is wrong. When in doubt c&p real output; describing a problem often causes useful info to be left out. BTW hal version 0.9.30.13 was a test build; RELENG_7 has 0.9.20.3 and HEAD has source code for a much newer version. Sam From avg at icyb.net.ua Tue Feb 10 11:21:22 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue Feb 10 11:21:29 2009 Subject: a little bit of c++ in kernel [module] Message-ID: <4991D3AF.1070001@icyb.net.ua> I've detected so far several postings on the lists where people ask about such a small nits of having c++ kernel code that some people already have some code in [subset of] c++ working kernel. I wonder if there is any howto about that or maybe somebody is willing to share his experience. What interests me the most: 1. what subset of c++ is used 2. what compilation options/tricks are used 3. how much of run-time support had to be added to kernel (and if that's code is available) 4. what else had to be changed in kernel code 5. so, how is it? :-) I think that I am not the only with such interest. P.S. no, I am not interested in re-writing the whole kernel in c++ or any discussion on that topic, I am interested in a small controlled injection. -- Andriy Gapon From yuri at rawbw.com Tue Feb 10 11:25:56 2009 From: yuri at rawbw.com (Yuri) Date: Tue Feb 10 11:26:02 2009 Subject: How to troubleshoot why ath0 can't connect to a passwordless wireless network? In-Reply-To: <4991ADB5.9050803@freebsd.org> References: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> <4991ADB5.9050803@freebsd.org> Message-ID: <20090210112554.15896t24mpahwggs@webmail.rawbw.com> Quoting "Sam Leffler" : > You seem to say your network is open (no security). If not you need > to show your network configuration. That's right, network is open (no security). > tcpdump can be used to tap traffic at 3 layers: 802.3, 802.11, and > driver. It can be used to identify where packets are lost in the > hierarchy (if at all). 'tcpdump -vv -i ath0' shows nothing when I run 'dhclient ath0' or 'dhcping -s 255.255.255.255'. dhclient always fails, dhcping says "no answer". But tcpdump shows traffic between other hosts on the same network. > Assuming packets are going out but not coming back you can sniff > from a 3rd station to look for traffic in the air but not received. No, now packets don't seem to go out at all. I said before that packets went out when interface in fact was in 'monitor' mode. > Given how little info you posted it's virtually impossible to advise > you what is wrong. When in doubt c&p real output; describing a > problem often causes useful info to be left out. There is no "real output" from tcpdump, this is a problem. How to find out for example why 'sendto' call from dhcping doesn't result in any packets on ath0 interface which is 'UP' and 'associated'? I used this same system to connect to other open WiFi networks without the problem. So it seems to be network-specific. And it fails only for certain networks. Yuri From bsd.quest at googlemail.com Tue Feb 10 12:21:51 2009 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Tue Feb 10 12:22:01 2009 Subject: taskqueue (9) Message-ID: <671bb5fc0902101221n2dbb1cf0hbc43164ffa6f6124@mail.gmail.com> Hello, the structure task(9) contain field ta_priority. Which role plays this priority if the task will wake up for run. Or it is used only for order of task in waitqueue while pending ? Thanks Alexej From abohra at gmail.com Tue Feb 10 13:04:47 2009 From: abohra at gmail.com (Aniruddha Bohra) Date: Tue Feb 10 13:10:13 2009 Subject: a little bit of c++ in kernel [module] Message-ID: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> You can see Click: http://read.cs.ucla.edu/click/ It does not run on FreeBSD >4. I have an old diff which builds on the work by Marko Zec and Bruce Simpson, that allows me to load the click module. http://www.cs.rutgers.edu/~bohra/click-1.5.0.diff Hope this helps Aniruddha From ale at FreeBSD.org Wed Feb 11 03:02:38 2009 From: ale at FreeBSD.org (Alex Dupre) Date: Wed Feb 11 03:02:45 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902061139.n16BdB6b058473@lurza.secnetix.de> References: <200902061139.n16BdB6b058473@lurza.secnetix.de> Message-ID: <4992B049.30903@FreeBSD.org> Oliver Fromme ha scritto: > The problem is related to the fact that a 64bit kernel > cannot use VESA BIOS functions. You should be able to > use standard VGA modes though, which don't require VESA > support. Actually I cannot see any splash screen on amd64, at least on the machines I tried (most with ATI ES1000 cards), with a 320x200x8 bitmap. No problems with i386 (on the same machines). -- Alex Dupre From olli at lurza.secnetix.de Wed Feb 11 04:01:07 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Wed Feb 11 04:01:21 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <4992B049.30903@FreeBSD.org> Message-ID: <200902111200.n1BC0wni006842@lurza.secnetix.de> Alex Dupre wrote: > Oliver Fromme ha scritto: > > The problem is related to the fact that a 64bit kernel > > cannot use VESA BIOS functions. You should be able to > > use standard VGA modes though, which don't require VESA > > support. > > Actually I cannot see any splash screen on amd64, at least on the > machines I tried (most with ATI ES1000 cards), with a 320x200x8 bitmap. > No problems with i386 (on the same machines). Yeah, my fingers were faster than my brain. The current syscons code cannot switch modes (no matter if VESA or standard VGA) if there is no BIOS support. It probably makes sense to let the boot loader set up graphics mode (including VESA support), so it is already active when the kernel comes up. Then the kernel will only have to deal with the frame buffer, not with the BIOS. That will work on both i386 and amd64 platforms. The only drawback is that the mode cannot be changed by the kernel once it is running, i.e. you have to stay in that mode till reboot. That solution requires support by the loader and by syscons. It is my plan to look into that, as soon as the basic graphics support in the loader is finished. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "[...] one observation we can make here is that Python makes an excellent pseudocoding language, with the wonderful attribute that it can actually be executed." -- Bruce Eckel From avg at icyb.net.ua Wed Feb 11 04:53:59 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Feb 11 04:54:07 2009 Subject: generalizing fd allocation code to id allocation Message-ID: <4992CA63.8080601@icyb.net.ua> Guys, anybody has ideas (or code) for generalizing code in sys/kern/kern_descrip.c for managing fd-s ("small" integer numbers)? I mean divorcing that code from filedesc and making it re-usable wherever we need some (constrained) id allocation. By constrained I mean that either ids are allocated from a sufficiently small limited pool or it is desirable to keep ids at small values. I needed this in a small driver that I am slowly writing and here's some bits from it: #define NDSLOTTYPE uint64_t #define NDSLOTSIZE sizeof(NDSLOTTYPE) #define NDENTRIES (NDSLOTSIZE * __CHAR_BIT) #define NDSLOT(x) ((x) / NDENTRIES) #define NDBIT(x) ((NDSLOTTYPE)1 << ((x) % NDENTRIES)) #define NDSLOTS(x) (((x) + NDENTRIES - 1) / NDENTRIES) ... /* XXX ugly */ NDSLOTTYPE host_addr_map[NDSLOTS(sizeof(uint8_t) * __CHAR_BIT)]; ... static uint8_t alloc_host_addr(struct heci_softc *sc) { static const int maxoff = sizeof(sc->host_addr_map) / sizeof(sc->host_addr_map[0]); NDSLOTTYPE *map = sc->host_addr_map; int off; for (off = 0; off < maxoff; ++off) { if (map[off] != ~0UL) { uint8_t addr = off * NDENTRIES + ffsl(~map[off]) - 1; map[NDSLOT(addr)] |= NDBIT(addr); return (addr); } } /* XXX what to return if all addresses are in use? */ /* use the fact that zero is a reserved address */ return 0; } static void release_host_addr(struct heci_softc *sc, uint8_t addr) { NDSLOTTYPE *map = sc->host_addr_map; if (!(map[NDSLOT(addr)] & NDBIT(addr))) /* XXX make KASSERT? */ device_printf(sc->dev, "release for unused host addr 0x%02x\n", addr); map[NDSLOT(addr)] &= ~NDBIT(addr); } Essentially this is almost a copy/paste. -- Andriy Gapon From avg at icyb.net.ua Wed Feb 11 05:09:50 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Feb 11 05:09:56 2009 Subject: generalizing fd allocation code to id allocation In-Reply-To: <4992CA63.8080601@icyb.net.ua> References: <4992CA63.8080601@icyb.net.ua> Message-ID: <4992CE1A.6060106@icyb.net.ua> My nose has just been rubbed into alloc_unr(9) :) Thanks, Roman! on 11/02/2009 14:53 Andriy Gapon said the following: > Guys, > > anybody has ideas (or code) for generalizing code in > sys/kern/kern_descrip.c for managing fd-s ("small" integer numbers)? > > I mean divorcing that code from filedesc and making it re-usable > wherever we need some (constrained) id allocation. > By constrained I mean that either ids are allocated from a sufficiently > small limited pool or it is desirable to keep ids at small values. > > I needed this in a small driver that I am slowly writing and here's some > bits from it: > > #define NDSLOTTYPE uint64_t > #define NDSLOTSIZE sizeof(NDSLOTTYPE) > #define NDENTRIES (NDSLOTSIZE * __CHAR_BIT) > #define NDSLOT(x) ((x) / NDENTRIES) > #define NDBIT(x) ((NDSLOTTYPE)1 << ((x) % NDENTRIES)) > #define NDSLOTS(x) (((x) + NDENTRIES - 1) / NDENTRIES) > ... > /* XXX ugly */ > NDSLOTTYPE host_addr_map[NDSLOTS(sizeof(uint8_t) * __CHAR_BIT)]; > ... > static uint8_t alloc_host_addr(struct heci_softc *sc) > { > static const int maxoff = sizeof(sc->host_addr_map) / > sizeof(sc->host_addr_map[0]); > NDSLOTTYPE *map = sc->host_addr_map; > int off; > > for (off = 0; off < maxoff; ++off) { > if (map[off] != ~0UL) { > uint8_t addr = off * NDENTRIES + ffsl(~map[off]) > - 1; > map[NDSLOT(addr)] |= NDBIT(addr); > return (addr); > } > } > > /* XXX what to return if all addresses are in use? */ > /* use the fact that zero is a reserved address */ > return 0; > } > > static void > release_host_addr(struct heci_softc *sc, uint8_t addr) > { > NDSLOTTYPE *map = sc->host_addr_map; > if (!(map[NDSLOT(addr)] & NDBIT(addr))) /* XXX make KASSERT? */ > device_printf(sc->dev, "release for unused host addr > 0x%02x\n", addr); > map[NDSLOT(addr)] &= ~NDBIT(addr); > } > > > Essentially this is almost a copy/paste. > > -- Andriy Gapon From hselasky at c2i.net Wed Feb 11 05:23:20 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Wed Feb 11 05:23:27 2009 Subject: generalizing fd allocation code to id allocation In-Reply-To: <4992CE1A.6060106@icyb.net.ua> References: <4992CA63.8080601@icyb.net.ua> <4992CE1A.6060106@icyb.net.ua> Message-ID: <200902111425.45981.hselasky@c2i.net> On Wednesday 11 February 2009, Andriy Gapon wrote: > My nose has just been rubbed into alloc_unr(9) :) > Thanks, Roman! The only problem about alloc_unr() is that you cannot allocate multiple contiguous units? --HPS From rink at FreeBSD.org Wed Feb 11 05:28:35 2009 From: rink at FreeBSD.org (Rink Springer) Date: Wed Feb 11 05:28:42 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902111200.n1BC0wni006842@lurza.secnetix.de> References: <4992B049.30903@FreeBSD.org> <200902111200.n1BC0wni006842@lurza.secnetix.de> Message-ID: <20090211131331.GA78543@rink.nu> On Wed, Feb 11, 2009 at 01:00:58PM +0100, Oliver Fromme wrote: > It probably makes sense to let the boot loader set up > graphics mode (including VESA support), so it is already > active when the kernel comes up. Then the kernel will > only have to deal with the frame buffer, not with the BIOS. > That will work on both i386 and amd64 platforms. The only > drawback is that the mode cannot be changed by the kernel > once it is running, i.e. you have to stay in that mode > till reboot. FWIW, this is exactly what FreeBSD/xbox does; the boot loader is responsible for setting up the video mode, and all it does is remap the framebuffer to a more sensible location (the way to do this is just writing to a register which is the same for any Xbox, and most bootloaders set the framebuffer to 4MB, which is a bit much for 640x480x16M especially if your machine only has 64MB of memory :-) > That solution requires support by the loader and by > syscons. It is my plan to look into that, as soon as the > basic graphics support in the loader is finished. I think that is a good approach. Go for it! Regards, -- Rink P.W. Springer - http://rink.nu "Chance favours the prepared mind" - Penn From avg at icyb.net.ua Wed Feb 11 05:40:31 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Feb 11 05:40:37 2009 Subject: a little bit of c++ in kernel [module] In-Reply-To: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> References: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> Message-ID: <4992D1B0.8020708@icyb.net.ua> on 10/02/2009 22:43 Aniruddha Bohra said the following: > You can see Click: http://read.cs.ucla.edu/click/ > It does not run on FreeBSD >4. > I have an old diff which builds on the work by Marko Zec and Bruce > Simpson, that allows me to load the click module. > http://www.cs.rutgers.edu/~bohra/click-1.5.0.diff > Aniruddha, thank you very much for the feedback! I looked through the code and the patch and I see the following: 1. options -fpermissive -fno-exceptions -fno-rtti are passed to c++ compiler 2. there are new/delete implementations that use kernel malloc I think that #1 means that there are no exceptions, (non-trivial) dynamic_cast and typeid for kernel c++ code. The questions that I have left: 1. do you use any global/static objects with constructors? did you have to write any code to call on those constructors when the module is loaded? 2. did you have to write any other run-time support code or platform glue code (besides new/delete)? 3. I assume virtual inheritance can be used in kernel code? do you use it? Thank you again! -- Andriy Gapon From mehulc87 at gmail.com Wed Feb 11 05:45:12 2009 From: mehulc87 at gmail.com (Mehul Chadha) Date: Wed Feb 11 05:45:19 2009 Subject: understanding of PTD symbol Message-ID: <251d650c0902110545w7c521e66tcb74f8ecc83e17a1@mail.gmail.com> Hello all, I am having doubts in understanding the usage of the symbol PTD. Could anyone pls explain the usage of PTD in the FreeBSD kernel?? In pmap_growkernel when we execute the following instruction wht is returned by pdir_pde ? Is it the physical address of a page table for KVA addressing?? while (pdir_pde(PTD, kernel_vm_end)) When I used objdump on the kernel executable i found the address of PTD symbol to be 0xBFEFF000 , I could not understand this , as in a virtual memory with 1G/3G split the kernel addressing must be between the addresses 0xC0000000 and 0xFFFFFFFF. Regards, Mehul From avg at icyb.net.ua Wed Feb 11 06:15:11 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Feb 11 06:15:22 2009 Subject: userland driver? Message-ID: <4992DD6B.3050507@icyb.net.ua> has anybody tried anything in the area of userland drivers on FreeBSD? I mean a driver for something sufficiently simple and standalone and not driven by interrupts. E.g. some sort of a watchdog driver that simply reads/writes some io registers from time to time. Brute-force and a very bad way is to go through /dev/mem, /dev/io, /dev/pci, but I am thinking about something that would allow to plug into newbus framework from userland. -- Andriy Gapon From jhb at freebsd.org Wed Feb 11 06:50:23 2009 From: jhb at freebsd.org (John Baldwin) Date: Wed Feb 11 06:50:33 2009 Subject: taskqueue (9) In-Reply-To: <671bb5fc0902101221n2dbb1cf0hbc43164ffa6f6124@mail.gmail.com> References: <671bb5fc0902101221n2dbb1cf0hbc43164ffa6f6124@mail.gmail.com> Message-ID: <200902110925.49633.jhb@freebsd.org> On Tuesday 10 February 2009 3:21:49 pm Alexej Sokolov wrote: > Hello, > the structure task(9) contain field ta_priority. Which role plays this > priority if the task will wake up for run. > Or it is used only for order of task in waitqueue while pending ? It is solely used to order requests within the queue. -- John Baldwin From brooks at freebsd.org Wed Feb 11 07:30:53 2009 From: brooks at freebsd.org (Brooks Davis) Date: Wed Feb 11 07:31:00 2009 Subject: generalizing fd allocation code to id allocation In-Reply-To: <200902111425.45981.hselasky@c2i.net> References: <4992CA63.8080601@icyb.net.ua> <4992CE1A.6060106@icyb.net.ua> <200902111425.45981.hselasky@c2i.net> Message-ID: <20090211150104.GA54097@lor.one-eyed-alien.net> On Wed, Feb 11, 2009 at 02:25:45PM +0100, Hans Petter Selasky wrote: > On Wednesday 11 February 2009, Andriy Gapon wrote: > > My nose has just been rubbed into alloc_unr(9) :) > > Thanks, Roman! > > The only problem about alloc_unr() is that you cannot allocate multiple > contiguous units? You might want rman for that. The thing I would like to see alloc_urn() grow is that ability to request a particular value so we could use it in the ifnet cloning code. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090211/64e4ae38/attachment.pgp From das at FreeBSD.ORG Wed Feb 11 07:43:33 2009 From: das at FreeBSD.ORG (David Schultz) Date: Wed Feb 11 07:43:39 2009 Subject: generalizing fd allocation code to id allocation In-Reply-To: <200902111425.45981.hselasky@c2i.net> References: <4992CA63.8080601@icyb.net.ua> <4992CE1A.6060106@icyb.net.ua> <200902111425.45981.hselasky@c2i.net> Message-ID: <20090211152243.GA11468@zim.MIT.EDU> On Wed, Feb 11, 2009, Hans Petter Selasky wrote: > On Wednesday 11 February 2009, Andriy Gapon wrote: > > My nose has just been rubbed into alloc_unr(9) :) > > Thanks, Roman! > > The only problem about alloc_unr() is that you cannot allocate multiple > contiguous units? Both Solaris' vmem(9) and NetBSD's extent(9) can do this... might be worth looking into. From abohra at gmail.com Wed Feb 11 08:40:07 2009 From: abohra at gmail.com (Aniruddha Bohra) Date: Wed Feb 11 08:45:11 2009 Subject: a little bit of c++ in kernel [module] In-Reply-To: <4992D1B0.8020708@icyb.net.ua> References: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> <4992D1B0.8020708@icyb.net.ua> Message-ID: <43a2dc1a0902110840t1e4c8856lb5b33cc153ea4acb@mail.gmail.com> On Wed, Feb 11, 2009 at 8:25 AM, Andriy Gapon wrote: > on 10/02/2009 22:43 Aniruddha Bohra said the following: >> You can see Click: http://read.cs.ucla.edu/click/ >> It does not run on FreeBSD >4. >> I have an old diff which builds on the work by Marko Zec and Bruce >> Simpson, that allows me to load the click module. >> http://www.cs.rutgers.edu/~bohra/click-1.5.0.diff > > 1. options -fpermissive -fno-exceptions -fno-rtti are passed to c++ compiler > 2. there are new/delete implementations that use kernel malloc > > I think that #1 means that there are no exceptions, (non-trivial) > dynamic_cast and typeid for kernel c++ code. Correct. > 1. do you use any global/static objects with constructors? did you have > to write any code to call on those constructors when the module is loaded? Not sure about this one. But AFAIK, there are no global static objects with constructors in Click code. There is one router object that is always initialized and it is updated by writing to the clickfs file system. The other objects are created with new. > 2. did you have to write any other run-time support code or platform > glue code (besides new/delete)? Apart from the new and delete, I think the other things were the pseudofs code to initialize the file system, the locks in include/click/sync.hh, the glue code in atomic.hh. > 3. I assume virtual inheritance can be used in kernel code? do you use it? Yes. For example, all objects inherit from "Element" and that defines virtual functions. (include/click/element.hh) Hope this helps. Aniruddha From tinguely at casselton.net Wed Feb 11 10:32:02 2009 From: tinguely at casselton.net (Mark Tinguely) Date: Wed Feb 11 10:35:55 2009 Subject: understanding of PTD symbol In-Reply-To: <251d650c0902110545w7c521e66tcb74f8ecc83e17a1@mail.gmail.com> Message-ID: <200902111815.n1BIFdYa036622@casselton.net> > Hello all, > I am having doubts in understanding the usage of the > symbol PTD. Could anyone pls explain the usage of PTD in the FreeBSD > kernel?? > > In pmap_growkernel when we execute the following instruction wht is > returned by pdir_pde ? Is it the physical address of a page table for > KVA addressing?? > > while (pdir_pde(PTD, kernel_vm_end)) > > When I used objdump on the kernel executable i found the address of > PTD symbol to be 0xBFEFF000 , I could not understand this , as in a > virtual memory with 1G/3G split the kernel addressing must be between > the addresses 0xC0000000 and 0xFFFFFFFF. > > Regards, > Mehul On the i386/amd64, the page table sits just below the kernel virtual memory. The recursive virtual address reference to the page table is inside this address. Assume i386 with 4 byte pointers and 4GB virtual address space with the 3GB user vm and 1GB kernel vm: the PTmap will start at 0xbfc00000 (which is one 4MB entry below the KVA). the PTD page directory descripters for the page table descripter will be 0xdeff000 which is 767 4K entries into the page table. Note: 768 is the 3G mark, so 767 is one less than 3/4 into the page table. PTDpde is the page table descriptor the the page table. It is PTD + 767 * 4 I found it easier to draw the various layouts and the page table itself. The i386 in 32 bit mode is 10 bits for the page directory, 10 bits for the page table and 12 bits for the physical page. The ARM processor is funner with 12/8/12 bits. --Mark Tinguely. From christoph.mallon at gmx.de Wed Feb 11 13:51:09 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Wed Feb 11 13:51:37 2009 Subject: a little bit of c++ in kernel [module] In-Reply-To: <43a2dc1a0902110840t1e4c8856lb5b33cc153ea4acb@mail.gmail.com> References: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> <4992D1B0.8020708@icyb.net.ua> <43a2dc1a0902110840t1e4c8856lb5b33cc153ea4acb@mail.gmail.com> Message-ID: <49934846.10905@gmx.de> Aniruddha Bohra schrieb: > On Wed, Feb 11, 2009 at 8:25 AM, Andriy Gapon wrote: >> on 10/02/2009 22:43 Aniruddha Bohra said the following: >>> You can see Click: http://read.cs.ucla.edu/click/ >>> It does not run on FreeBSD >4. >>> I have an old diff which builds on the work by Marko Zec and Bruce >>> Simpson, that allows me to load the click module. >>> http://www.cs.rutgers.edu/~bohra/click-1.5.0.diff >> 1. options -fpermissive -fno-exceptions -fno-rtti are passed to c++ compiler >> 2. there are new/delete implementations that use kernel malloc >> >> I think that #1 means that there are no exceptions, (non-trivial) >> dynamic_cast and typeid for kernel c++ code. > > Correct. That's a pity. Lack of exceptions negates some major benefits of C++. >> 1. do you use any global/static objects with constructors? did you have >> to write any code to call on those constructors when the module is loaded? > > Not sure about this one. But AFAIK, there are no global static objects > with constructors in Click code. > There is one router object that is always initialized and it is > updated by writing to the clickfs file system. > The other objects are created with new. > >> 2. did you have to write any other run-time support code or platform >> glue code (besides new/delete)? > > Apart from the new and delete, I think the other things were the > pseudofs code to initialize the file system, > the locks in include/click/sync.hh, the glue code in atomic.hh. > > >> 3. I assume virtual inheritance can be used in kernel code? do you use it? Virtual inheritence needs no support from the "outside", so it is available. > Yes. For example, all objects inherit from "Element" and that defines > virtual functions. (include/click/element.hh) Virtual inheritance is something completely different than virtual methods. From bruce at cran.org.uk Wed Feb 11 14:22:21 2009 From: bruce at cran.org.uk (Bruce Cran) Date: Wed Feb 11 14:22:27 2009 Subject: a little bit of c++ in kernel [module] In-Reply-To: <49934846.10905@gmx.de> References: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> <4992D1B0.8020708@icyb.net.ua> <43a2dc1a0902110840t1e4c8856lb5b33cc153ea4acb@mail.gmail.com> <49934846.10905@gmx.de> Message-ID: <20090211222213.7509f5e3@gluon> On Wed, 11 Feb 2009 22:51:02 +0100 Christoph Mallon wrote: > Aniruddha Bohra schrieb: > > On Wed, Feb 11, 2009 at 8:25 AM, Andriy Gapon > > wrote: > >> on 10/02/2009 22:43 Aniruddha Bohra said the following: > >>> You can see Click: http://read.cs.ucla.edu/click/ > >>> It does not run on FreeBSD >4. > >>> I have an old diff which builds on the work by Marko Zec and Bruce > >>> Simpson, that allows me to load the click module. > >>> http://www.cs.rutgers.edu/~bohra/click-1.5.0.diff > >> 1. options -fpermissive -fno-exceptions -fno-rtti are passed to > >> c++ compiler 2. there are new/delete implementations that use > >> kernel malloc > >> > >> I think that #1 means that there are no exceptions, (non-trivial) > >> dynamic_cast and typeid for kernel c++ code. > > > > Correct. > > That's a pity. Lack of exceptions negates some major benefits of C++. > > >> 1. do you use any global/static objects with constructors? did you > >> have to write any code to call on those constructors when the > >> module is loaded? > > > > Not sure about this one. But AFAIK, there are no global static > > objects with constructors in Click code. > > There is one router object that is always initialized and it is > > updated by writing to the clickfs file system. > > The other objects are created with new. > > > >> 2. did you have to write any other run-time support code or > >> platform glue code (besides new/delete)? > > > > Apart from the new and delete, I think the other things were the > > pseudofs code to initialize the file system, > > the locks in include/click/sync.hh, the glue code in atomic.hh. > > > > > >> 3. I assume virtual inheritance can be used in kernel code? do you > >> use it? > > Virtual inheritence needs no support from the "outside", so it is > available. > > > Yes. For example, all objects inherit from "Element" and that > > defines virtual functions. (include/click/element.hh) > > Virtual inheritance is something completely different than virtual > methods. Microsoft has an overview of using C++ in kernel drivers at http://www.microsoft.com/whdc/driver/kernel/KMcode.mspx . It sounds like the situation on FreeBSD may be somewhat similar. -- Bruce Cran From das at FreeBSD.ORG Wed Feb 11 14:49:37 2009 From: das at FreeBSD.ORG (David Schultz) Date: Wed Feb 11 14:49:44 2009 Subject: a little bit of c++ in kernel [module] In-Reply-To: <20090211222213.7509f5e3@gluon> References: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> <4992D1B0.8020708@icyb.net.ua> <43a2dc1a0902110840t1e4c8856lb5b33cc153ea4acb@mail.gmail.com> <49934846.10905@gmx.de> <20090211222213.7509f5e3@gluon> Message-ID: <20090211225019.GA13784@zim.MIT.EDU> On Wed, Feb 11, 2009, Bruce Cran wrote: > On Wed, 11 Feb 2009 22:51:02 +0100 > Christoph Mallon wrote: > > > Aniruddha Bohra schrieb: > > > On Wed, Feb 11, 2009 at 8:25 AM, Andriy Gapon > > > wrote: > > >> on 10/02/2009 22:43 Aniruddha Bohra said the following: > > >>> You can see Click: http://read.cs.ucla.edu/click/ > > >>> It does not run on FreeBSD >4. > > >>> I have an old diff which builds on the work by Marko Zec and Bruce > > >>> Simpson, that allows me to load the click module. > > >>> http://www.cs.rutgers.edu/~bohra/click-1.5.0.diff > > >> 1. options -fpermissive -fno-exceptions -fno-rtti are passed to > > >> c++ compiler 2. there are new/delete implementations that use > > >> kernel malloc > > >> > > >> I think that #1 means that there are no exceptions, (non-trivial) > > >> dynamic_cast and typeid for kernel c++ code. > > > > > > Correct. > > > > That's a pity. Lack of exceptions negates some major benefits of C++. > > > > >> 1. do you use any global/static objects with constructors? did you > > >> have to write any code to call on those constructors when the > > >> module is loaded? > > > > > > Not sure about this one. But AFAIK, there are no global static > > > objects with constructors in Click code. > > > There is one router object that is always initialized and it is > > > updated by writing to the clickfs file system. > > > The other objects are created with new. > > > > > >> 2. did you have to write any other run-time support code or > > >> platform glue code (besides new/delete)? > > > > > > Apart from the new and delete, I think the other things were the > > > pseudofs code to initialize the file system, > > > the locks in include/click/sync.hh, the glue code in atomic.hh. > > > > > > > > >> 3. I assume virtual inheritance can be used in kernel code? do you > > >> use it? > > > > Virtual inheritence needs no support from the "outside", so it is > > available. > > > > > Yes. For example, all objects inherit from "Element" and that > > > defines virtual functions. (include/click/element.hh) > > > > Virtual inheritance is something completely different than virtual > > methods. > > Microsoft has an overview of using C++ in kernel drivers at > http://www.microsoft.com/whdc/driver/kernel/KMcode.mspx . It sounds > like the situation on FreeBSD may be somewhat similar. FreeBSD doesn't support loading kernel code and data into pageable memory like Windows does, so a lot of the paging-related problems they're talking about aren't issues for us. From avg at icyb.net.ua Thu Feb 12 03:37:12 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Thu Feb 12 03:37:18 2009 Subject: a little bit of c++ in kernel [module] In-Reply-To: <49934846.10905@gmx.de> References: <43a2dc1a0902101243w5ce35609x35dbe440c39d80a8@mail.gmail.com> <4992D1B0.8020708@icyb.net.ua> <43a2dc1a0902110840t1e4c8856lb5b33cc153ea4acb@mail.gmail.com> <49934846.10905@gmx.de> Message-ID: <499409DF.4070404@icyb.net.ua> on 11/02/2009 23:51 Christoph Mallon said the following: > Aniruddha Bohra schrieb: >> On Wed, Feb 11, 2009 at 8:25 AM, Andriy Gapon wrote: [snip] >>> 3. I assume virtual inheritance can be used in kernel code? do you >>> use it? > > Virtual inheritence needs no support from the "outside", so it is > available. > >> Yes. For example, all objects inherit from "Element" and that defines >> virtual functions. (include/click/element.hh) > > Virtual inheritance is something completely different than virtual methods. The foul is on me, I should have known to write in proper terms. I meant virtual methods and their overriding, but used the wrong term. Thank you. -- Andriy Gapon From rwatson at FreeBSD.org Thu Feb 12 04:09:04 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Feb 12 04:09:10 2009 Subject: userland driver? In-Reply-To: <4992DD6B.3050507@icyb.net.ua> References: <4992DD6B.3050507@icyb.net.ua> Message-ID: On Wed, 11 Feb 2009, Andriy Gapon wrote: > has anybody tried anything in the area of userland drivers on FreeBSD? I > mean a driver for something sufficiently simple and standalone and not > driven by interrupts. E.g. some sort of a watchdog driver that simply > reads/writes some io registers from time to time. > > Brute-force and a very bad way is to go through /dev/mem, /dev/io, /dev/pci, > but I am thinking about something that would allow to plug into newbus > framework from userland. I recently had a related question from a colleague about implementing a synthetic USB device in userspace so that software part development can occur concurrently with hardware part development. Robert N M Watson Computer Laboratory University of Cambridge From onemda at gmail.com Thu Feb 12 04:17:07 2009 From: onemda at gmail.com (Paul B. Mahol) Date: Thu Feb 12 04:17:14 2009 Subject: How to troubleshoot why ath0 can't connect to a passwordless wireless network? In-Reply-To: <20090210112554.15896t24mpahwggs@webmail.rawbw.com> References: <20090209143642.168421bp9jjvs7i8@webmail.rawbw.com> <4991ADB5.9050803@freebsd.org> <20090210112554.15896t24mpahwggs@webmail.rawbw.com> Message-ID: <3a142e750902120417mee92d86m5a3a871c9145fa54@mail.gmail.com> On 2/10/09, Yuri wrote: > Quoting "Sam Leffler" : > >> You seem to say your network is open (no security). If not you need >> to show your network configuration. > > That's right, network is open (no security). > >> tcpdump can be used to tap traffic at 3 layers: 802.3, 802.11, and >> driver. It can be used to identify where packets are lost in the >> hierarchy (if at all). > > 'tcpdump -vv -i ath0' shows nothing when I run 'dhclient ath0' or > 'dhcping -s 255.255.255.255'. dhclient always fails, dhcping > says "no answer". > But tcpdump shows traffic between other hosts on the same network. > >> Assuming packets are going out but not coming back you can sniff >> from a 3rd station to look for traffic in the air but not received. > > No, now packets don't seem to go out at all. I said before that packets > went out when interface in fact was in 'monitor' mode. Really? Try wlandebug(8) to see what 802.11 code is doing vs ath driver. >> Given how little info you posted it's virtually impossible to advise >> you what is wrong. When in doubt c&p real output; describing a >> problem often causes useful info to be left out. > > There is no "real output" from tcpdump, this is a problem. > > How to find out for example why 'sendto' call from dhcping doesn't > result in any packets on ath0 interface which is 'UP' and 'associated'? > > I used this same system to connect to other open WiFi networks without > the problem. So it seems to be network-specific. And it fails only for > certain networks. > > Yuri > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Paul From brampton+freebsd-hackers at gmail.com Thu Feb 12 06:08:25 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Thu Feb 12 06:08:32 2009 Subject: pahole - Finding holes in kernel structs Message-ID: I found this useful tool called pahole[1]. It basically finds holes within structs, so for example on my 64bit machine this struct: struct test { int foo; const char *bar; int blah; } Would have a hole between foo and bar of 4 bytes because both for and bar have been aligned on a 8 byte boundary, and the struct would also have 4 bytes of padding on the end. However, if I simply moved blah between foo and bar then the struct has shrunk by 8 bytes, which could be a good thing. This could also help keep structs within single cache lines, and just generally keep memory usage to a minimum when the struct is used many times (for example in an array). So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that many of the struct had holes, and some of which could be rearranged to fill the gap. I've made the list available here[2]. So my questions are two fold: 1) Is it worth my time trying to rearrange structs? If so do you think many of my patches would be accepted? 2) Is there a way to find out the most heavily used structs? There are ~3600 structs, and ~2000 holes, it might be a waste of my time fixing the structs which are only used once. thanks Andrew [1] http://lwn.net/Articles/206805/ [2] http://bramp.net/projects/kernel.pahole.bz2 (~260kB) From kostikbel at gmail.com Thu Feb 12 06:13:19 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Thu Feb 12 06:13:26 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: References: Message-ID: <20090212141313.GD2723@deviant.kiev.zoral.com.ua> On Thu, Feb 12, 2009 at 02:08:22PM +0000, Andrew Brampton wrote: > I found this useful tool called pahole[1]. It basically finds holes > within structs, so for example on my 64bit machine this struct: > > struct test { > int foo; > const char *bar; > int blah; > } > > Would have a hole between foo and bar of 4 bytes because both for and > bar have been aligned on a 8 byte boundary, and the struct would also > have 4 bytes of padding on the end. However, if I simply moved blah > between foo and bar then the struct has shrunk by 8 bytes, which could > be a good thing. This could also help keep structs within single cache > lines, and just generally keep memory usage to a minimum when the > struct is used many times (for example in an array). > > So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that Did you ported it to FreeBSD, or run on the Linux host ? > many of the struct had holes, and some of which could be rearranged to > fill the gap. I've made the list available here[2]. So my questions > are two fold: > > 1) Is it worth my time trying to rearrange structs? If so do you think > many of my patches would be accepted? > > 2) Is there a way to find out the most heavily used structs? There are > ~3600 structs, and ~2000 holes, it might be a waste of my time fixing > the structs which are only used once. > > thanks > Andrew > > [1] http://lwn.net/Articles/206805/ > [2] http://bramp.net/projects/kernel.pahole.bz2 (~260kB) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090212/cd45de8e/attachment.pgp From brampton+freebsd-hackers at gmail.com Thu Feb 12 06:45:24 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Thu Feb 12 06:45:31 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <20090212141313.GD2723@deviant.kiev.zoral.com.ua> References: <20090212141313.GD2723@deviant.kiev.zoral.com.ua> Message-ID: 2009/2/12 Kostik Belousov : > On Thu, Feb 12, 2009 at 02:08:22PM +0000, Andrew Brampton wrote: >> >> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that > Did you ported it to FreeBSD, or run on the Linux host ? > Sorry no, I just ran it from a Linux host, but to my surprised it seemed to work fine. Andrew From redcrash at gmail.com Thu Feb 12 06:52:29 2009 From: redcrash at gmail.com (Harald Servat) Date: Thu Feb 12 06:52:36 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <20090212141313.GD2723@deviant.kiev.zoral.com.ua> References: <20090212141313.GD2723@deviant.kiev.zoral.com.ua> Message-ID: On Thu, Feb 12, 2009 at 3:13 PM, Kostik Belousov wrote: > On Thu, Feb 12, 2009 at 02:08:22PM +0000, Andrew Brampton wrote: > > I found this useful tool called pahole[1]. It basically finds holes > > within structs, so for example on my 64bit machine this struct: > > > > struct test { > > int foo; > > const char *bar; > > int blah; > > } > > > > Would have a hole between foo and bar of 4 bytes because both for and > > bar have been aligned on a 8 byte boundary, and the struct would also > > have 4 bytes of padding on the end. However, if I simply moved blah > > between foo and bar then the struct has shrunk by 8 bytes, which could > > be a good thing. This could also help keep structs within single cache > > lines, and just generally keep memory usage to a minimum when the > > struct is used many times (for example in an array). > > > > So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that > Did you ported it to FreeBSD, or run on the Linux host ? > > > many of the struct had holes, and some of which could be rearranged to > > fill the gap. I've made the list available here[2]. So my questions > > are two fold: > > > > 1) Is it worth my time trying to rearrange structs? If so do you think > > many of my patches would be accepted? > > > > 2) Is there a way to find out the most heavily used structs? There are > > ~3600 structs, and ~2000 holes, it might be a waste of my time fixing > > the structs which are only used once. > > > > thanks > > Andrew > Interesting utility Andrew! Remember that size of some types depend on the memory ABI (32 or 64 bits), so this influences on the result of this utility. > > > [1] http://lwn.net/Articles/206805/ > > [2] http://bramp.net/projects/kernel.pahole.bz2 (~260kB) > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > From jhs at berklix.org Thu Feb 12 08:01:16 2009 From: jhs at berklix.org (Julian Stacey) Date: Thu Feb 12 08:01:25 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: Your message "Thu, 12 Feb 2009 14:08:22 GMT." Message-ID: <200902121549.n1CFnLdt002361@fire.js.berklix.net> > 1) Is it worth my time trying to rearrange structs? I wondered whether as a sensitivity test, some version of gcc (or its competitor ?) might have capability to automatically re-order variables ? but found nothing in man gcc "Optimization Options". Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From max at love2party.net Thu Feb 12 08:17:53 2009 From: max at love2party.net (Max Laier) Date: Thu Feb 12 08:18:00 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: References: Message-ID: <200902121717.47841.max@love2party.net> On Thursday 12 February 2009 15:08:22 Andrew Brampton wrote: > So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that > many of the struct had holes, and some of which could be rearranged to > fill the gap. Interesting tool ... > I've made the list available here[2]. So my questions > are two fold: > > 1) Is it worth my time trying to rearrange structs? If so do you think > many of my patches would be accepted? > > 2) Is there a way to find out the most heavily used structs? There are > ~3600 structs, and ~2000 holes, it might be a waste of my time fixing > the structs which are only used once. That's the tricky part. Rearranging the structs itself is not that difficult, but identifying which should be rearranged and if, how ... that's the problem. The fact that gaps might be different for 64 vs. 32 bit architectures has already been mentioned. In addition one needs to keep in mind that changing a struct layout is a ABI change. So if we do identify structs that we want to change we should do them all at once to keep the different versions down to a minimum. So to answer your first question, submitting 101 patches to rearrange 101 structs is certainly a wasted effort. However, if you take a good look at the 2000 holes, identify an interesting subset and submit a patch to fix that subset ... that would be a worthwhile effort ... IMHO. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From ed at 80386.nl Thu Feb 12 08:33:46 2009 From: ed at 80386.nl (Ed Schouten) Date: Thu Feb 12 08:33:54 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121717.47841.max@love2party.net> References: <200902121717.47841.max@love2party.net> Message-ID: <20090212163344.GR79178@hoeg.nl> * Max Laier wrote: > So to answer your first question, submitting 101 patches to rearrange 101 > structs is certainly a wasted effort. However, if you take a good look at the > 2000 holes, identify an interesting subset and submit a patch to fix that > subset ... that would be a worthwhile effort ... IMHO. I guess it's also a wasted effort to reduce struct tty from 8xx to 7xx bytes, because it still allocates 1024 bytes of memory using malloc(9). I guess we should mainly focus on structures that are allocated using uma(9) or are slightly bigger than 2^n. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090212/6826b83d/attachment.pgp From sam at freebsd.org Thu Feb 12 08:42:21 2009 From: sam at freebsd.org (Sam Leffler) Date: Thu Feb 12 08:42:28 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121717.47841.max@love2party.net> References: <200902121717.47841.max@love2party.net> Message-ID: <4994516B.8060703@freebsd.org> Max Laier wrote: > On Thursday 12 February 2009 15:08:22 Andrew Brampton wrote: > >> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that >> many of the struct had holes, and some of which could be rearranged to >> fill the gap. >> > > Interesting tool ... > Someone should be able to do the same thing with coverity but it's obviously less effort to use something that exists. If I recall this and related tools like sparse use dwarf symbols which we don't generate by default. But with dtrace support I think we now can in fact generate the symbols easily so maybe someone can look at porting the tools... > >> I've made the list available here[2]. So my questions >> are two fold: >> >> 1) Is it worth my time trying to rearrange structs? If so do you think >> many of my patches would be accepted? >> >> 2) Is there a way to find out the most heavily used structs? There are >> ~3600 structs, and ~2000 holes, it might be a waste of my time fixing >> the structs which are only used once. >> > > That's the tricky part. Rearranging the structs itself is not that difficult, > but identifying which should be rearranged and if, how ... that's the problem. > The fact that gaps might be different for 64 vs. 32 bit architectures has > already been mentioned. > > In addition one needs to keep in mind that changing a struct layout is a ABI > change. So if we do identify structs that we want to change we should do them > all at once to keep the different versions down to a minimum. > > So to answer your first question, submitting 101 patches to rearrange 101 > structs is certainly a wasted effort. However, if you take a good look at the > 2000 holes, identify an interesting subset and submit a patch to fix that > subset ... that would be a worthwhile effort ... IMHO. > > The other thing to keep in mind is that structure layout can have a noticeable effect on cache locality. Arbitrarily rearranging structure members can generate many more cache misses so one should sanity check changes w/ something like hwpmc. However as noted because layout may be platform-dependent even if something shows no change on x86 it may be a loss on another architecture and finding that performance drop may be really hard. Sam From jille at quis.cx Thu Feb 12 08:50:10 2009 From: jille at quis.cx (Jille Timmermans) Date: Thu Feb 12 08:50:18 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121549.n1CFnLdt002361@fire.js.berklix.net> References: <200902121549.n1CFnLdt002361@fire.js.berklix.net> Message-ID: <49944F8F.5080104@quis.cx> Julian Stacey schreef: >> 1) Is it worth my time trying to rearrange structs? > > I wondered whether as a sensitivity test, some version of gcc (or > its competitor ?) might have capability to automatically re-order > variables ? but found nothing in man gcc "Optimization Options". There is a __packed attribute, I don't know what it exactly does and whether it is an improvement. -- Jille > > Cheers, > Julian From max at love2party.net Thu Feb 12 08:54:27 2009 From: max at love2party.net (Max Laier) Date: Thu Feb 12 08:54:41 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <4994516B.8060703@freebsd.org> References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> Message-ID: <200902121754.24536.max@love2party.net> On Thursday 12 February 2009 17:42:19 Sam Leffler wrote: > Max Laier wrote: > > On Thursday 12 February 2009 15:08:22 Andrew Brampton wrote: > >> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that > >> many of the struct had holes, and some of which could be rearranged to > >> fill the gap. > > > > Interesting tool ... > > Someone should be able to do the same thing with coverity but it's > obviously less effort to use something that exists. If I recall this > and related tools like sparse use dwarf symbols which we don't generate > by default. But with dtrace support I think we now can in fact generate > the symbols easily so maybe someone can look at porting the tools... > > >> I've made the list available here[2]. So my questions > >> are two fold: > >> > >> 1) Is it worth my time trying to rearrange structs? If so do you think > >> many of my patches would be accepted? > >> > >> 2) Is there a way to find out the most heavily used structs? There are > >> ~3600 structs, and ~2000 holes, it might be a waste of my time fixing > >> the structs which are only used once. > > > > That's the tricky part. Rearranging the structs itself is not that > > difficult, but identifying which should be rearranged and if, how ... > > that's the problem. The fact that gaps might be different for 64 vs. 32 > > bit architectures has already been mentioned. > > > > In addition one needs to keep in mind that changing a struct layout is a > > ABI change. So if we do identify structs that we want to change we > > should do them all at once to keep the different versions down to a > > minimum. > > > > So to answer your first question, submitting 101 patches to rearrange 101 > > structs is certainly a wasted effort. However, if you take a good look > > at the 2000 holes, identify an interesting subset and submit a patch to > > fix that subset ... that would be a worthwhile effort ... IMHO. > > The other thing to keep in mind is that structure layout can have a > noticeable effect on cache locality. Arbitrarily rearranging structure > members can generate many more cache misses so one should sanity check > changes w/ something like hwpmc. However as noted because layout may be > platform-dependent even if something shows no change on x86 it may be a > loss on another architecture and finding that performance drop may be > really hard. Let's not be too "glass half empty" about it, though. The same is true in the opposite direction. If we can identify and eliminate an unnecessary hole in an important structure we might gain that same performance just by reshuffling a few lines. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From bsd.quest at googlemail.com Thu Feb 12 08:57:40 2009 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Thu Feb 12 08:57:47 2009 Subject: strange output in /var/log/messages Message-ID: <671bb5fc0902120857h3b789447q64a8c396728bdbd6@mail.gmail.com> Hello, I try now to debug a kernel module and I make some output with printf(9). But the text appears in /var/log/messages in very strange form: Feb 12 17:54:34 myhost kernel: b Feb 12 17:54:34 myhost kernel: eg Feb 12 17:54:34 myhost kernel: in Feb 12 17:54:34 myhost kernel: . Feb 12 17:54:34 myhost kernel: De Feb 12 17:54:34 myhost kernel: vice Feb 12 17:54:34 myhost kernel: U Feb 12 17:54:34 myhost kernel: ni Feb 12 17:54:34 myhost kernel: t: Could anyone explain the reason of this kind of output. And how can I correct it? Alexej From brooks at freebsd.org Thu Feb 12 09:02:35 2009 From: brooks at freebsd.org (Brooks Davis) Date: Thu Feb 12 09:02:44 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121754.24536.max@love2party.net> References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> <200902121754.24536.max@love2party.net> Message-ID: <20090212170119.GA7951@lor.one-eyed-alien.net> On Thu, Feb 12, 2009 at 05:54:24PM +0100, Max Laier wrote: > On Thursday 12 February 2009 17:42:19 Sam Leffler wrote: > > Max Laier wrote: > > > On Thursday 12 February 2009 15:08:22 Andrew Brampton wrote: > > >> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that > > >> many of the struct had holes, and some of which could be rearranged to > > >> fill the gap. > > > > > > Interesting tool ... > > > > Someone should be able to do the same thing with coverity but it's > > obviously less effort to use something that exists. If I recall this > > and related tools like sparse use dwarf symbols which we don't generate > > by default. But with dtrace support I think we now can in fact generate > > the symbols easily so maybe someone can look at porting the tools... > > > > >> I've made the list available here[2]. So my questions > > >> are two fold: > > >> > > >> 1) Is it worth my time trying to rearrange structs? If so do you think > > >> many of my patches would be accepted? > > >> > > >> 2) Is there a way to find out the most heavily used structs? There are > > >> ~3600 structs, and ~2000 holes, it might be a waste of my time fixing > > >> the structs which are only used once. > > > > > > That's the tricky part. Rearranging the structs itself is not that > > > difficult, but identifying which should be rearranged and if, how ... > > > that's the problem. The fact that gaps might be different for 64 vs. 32 > > > bit architectures has already been mentioned. > > > > > > In addition one needs to keep in mind that changing a struct layout is a > > > ABI change. So if we do identify structs that we want to change we > > > should do them all at once to keep the different versions down to a > > > minimum. > > > > > > So to answer your first question, submitting 101 patches to rearrange 101 > > > structs is certainly a wasted effort. However, if you take a good look > > > at the 2000 holes, identify an interesting subset and submit a patch to > > > fix that subset ... that would be a worthwhile effort ... IMHO. > > > > The other thing to keep in mind is that structure layout can have a > > noticeable effect on cache locality. Arbitrarily rearranging structure > > members can generate many more cache misses so one should sanity check > > changes w/ something like hwpmc. However as noted because layout may be > > platform-dependent even if something shows no change on x86 it may be a > > loss on another architecture and finding that performance drop may be > > really hard. > > Let's not be too "glass half empty" about it, though. The same is true in the > opposite direction. If we can identify and eliminate an unnecessary hole in > an important structure we might gain that same performance just by reshuffling > a few lines. Remember however, that any structure that is exposed to userland can't just be changed. If it's part a syscall interface then the old layout must be supported. If that's going to be done, it's worth showing an actual, measurable improvement before writing the compatibility code. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090212/0d77610b/attachment.pgp From jhs at berklix.org Thu Feb 12 09:06:54 2009 From: jhs at berklix.org (Julian Stacey) Date: Thu Feb 12 09:07:01 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: Your message "Thu, 12 Feb 2009 17:34:23 +0100." <49944F8F.5080104@quis.cx> Message-ID: <200902121707.n1CH75XW003391@fire.js.berklix.net> Hi, Reference: > From: Jille Timmermans > Date: Thu, 12 Feb 2009 17:34:23 +0100 > Message-id: <49944F8F.5080104@quis.cx> Jille Timmermans wrote: > Julian Stacey schreef: > >> 1) Is it worth my time trying to rearrange structs? > > > > I wondered whether as a sensitivity test, some version of gcc (or > > its competitor ?) might have capability to automatically re-order > > variables ? but found nothing in man gcc "Optimization Options". > There is a __packed attribute, I don't know what it exactly does and > whether it is an improvement. Ah: gcc -fpack-struct Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From rwatson at FreeBSD.org Thu Feb 12 09:51:05 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Feb 12 09:51:12 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121717.47841.max@love2party.net> References: <200902121717.47841.max@love2party.net> Message-ID: On Thu, 12 Feb 2009, Max Laier wrote: > That's the tricky part. Rearranging the structs itself is not that > difficult, but identifying which should be rearranged and if, how ... that's > the problem. The fact that gaps might be different for 64 vs. 32 bit > architectures has already been mentioned. > > In addition one needs to keep in mind that changing a struct layout is a ABI > change. So if we do identify structs that we want to change we should do > them all at once to keep the different versions down to a minimum. Leaving aside the potential memory savings, etc, another potential use for a tool like this is actually in ABI monitoring and maintenance. I.e., run a tool that generates a description of the structure ABI of the kernel and user interfaces of the system, then run it intermittently to detect and report changes, perhaps correlated with symbol version information, etc. Right now we discover ABI changes in three ways: a developer changing the ABI realizes it has changed and (probably) deals with it, a developer reviews changes later and finds it, or a user finds it the hardware way due to application crashes, data loss, etc. Mechanizing tracking of API signatures, structure layout, etc, would help make all of this a lot more deterministic. Robert N M Watson Computer Laboratory University of Cambridge From rwatson at FreeBSD.org Thu Feb 12 09:52:10 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Feb 12 09:52:16 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <20090212163344.GR79178@hoeg.nl> References: <200902121717.47841.max@love2party.net> <20090212163344.GR79178@hoeg.nl> Message-ID: On Thu, 12 Feb 2009, Ed Schouten wrote: > * Max Laier wrote: >> So to answer your first question, submitting 101 patches to rearrange 101 >> structs is certainly a wasted effort. However, if you take a good look at the >> 2000 holes, identify an interesting subset and submit a patch to fix that >> subset ... that would be a worthwhile effort ... IMHO. > > I guess it's also a wasted effort to reduce struct tty from 8xx to 7xx > bytes, because it still allocates 1024 bytes of memory using malloc(9). I > guess we should mainly focus on structures that are allocated using uma(9) > or are slightly bigger than 2^n. There are two things we can do: we can allocate using a specific UMA zone, but we could also retune our malloc buckets to waste less space while avoiding commiting memory to specific types where it's unhelpful (when UMA is used, we populate caches of pre-initialized instances, so we don't want to use UMA for things allocated, say, six times). Robert N M Watson Computer Laboratory University of Cambridge From stefan.lambrev at moneybookers.com Thu Feb 12 09:52:23 2009 From: stefan.lambrev at moneybookers.com (Stefan Lambrev) Date: Thu Feb 12 09:52:32 2009 Subject: strange output in /var/log/messages In-Reply-To: <671bb5fc0902120857h3b789447q64a8c396728bdbd6@mail.gmail.com> References: <671bb5fc0902120857h3b789447q64a8c396728bdbd6@mail.gmail.com> Message-ID: Hi, On Feb 12, 2009, at 6:57 PM, Alexej Sokolov wrote: > Hello, > I try now to debug a kernel module and I make some output with > printf(9). > But the text appears in /var/log/messages in very strange form: > > Feb 12 17:54:34 myhost kernel: b > Feb 12 17:54:34 myhost kernel: eg > Feb 12 17:54:34 myhost kernel: in > Feb 12 17:54:34 myhost kernel: . > Feb 12 17:54:34 myhost kernel: De > Feb 12 17:54:34 myhost kernel: vice > Feb 12 17:54:34 myhost kernel: U > Feb 12 17:54:34 myhost kernel: ni > Feb 12 17:54:34 myhost kernel: t: > > Could anyone explain the reason of this kind of output. And how can I > correct it? But those kernel messages are displayed properly if you type dmesg? If yes I think you can blame syslogd. > > > Alexej > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org > " -- Best Wishes, Stefan Lambrev ICQ# 24134177 From sam at freebsd.org Thu Feb 12 09:53:19 2009 From: sam at freebsd.org (Sam Leffler) Date: Thu Feb 12 09:53:26 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121754.24536.max@love2party.net> References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> <200902121754.24536.max@love2party.net> Message-ID: <4994620E.5000804@freebsd.org> Max Laier wrote: > On Thursday 12 February 2009 17:42:19 Sam Leffler wrote: > >> Max Laier wrote: >> >>> On Thursday 12 February 2009 15:08:22 Andrew Brampton wrote: >>> >>>> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that >>>> many of the struct had holes, and some of which could be rearranged to >>>> fill the gap. >>>> >>> Interesting tool ... >>> >> Someone should be able to do the same thing with coverity but it's >> obviously less effort to use something that exists. If I recall this >> and related tools like sparse use dwarf symbols which we don't generate >> by default. But with dtrace support I think we now can in fact generate >> the symbols easily so maybe someone can look at porting the tools... >> >> >>>> I've made the list available here[2]. So my questions >>>> are two fold: >>>> >>>> 1) Is it worth my time trying to rearrange structs? If so do you think >>>> many of my patches would be accepted? >>>> >>>> 2) Is there a way to find out the most heavily used structs? There are >>>> ~3600 structs, and ~2000 holes, it might be a waste of my time fixing >>>> the structs which are only used once. >>>> >>> That's the tricky part. Rearranging the structs itself is not that >>> difficult, but identifying which should be rearranged and if, how ... >>> that's the problem. The fact that gaps might be different for 64 vs. 32 >>> bit architectures has already been mentioned. >>> >>> In addition one needs to keep in mind that changing a struct layout is a >>> ABI change. So if we do identify structs that we want to change we >>> should do them all at once to keep the different versions down to a >>> minimum. >>> >>> So to answer your first question, submitting 101 patches to rearrange 101 >>> structs is certainly a wasted effort. However, if you take a good look >>> at the 2000 holes, identify an interesting subset and submit a patch to >>> fix that subset ... that would be a worthwhile effort ... IMHO. >>> >> The other thing to keep in mind is that structure layout can have a >> noticeable effect on cache locality. Arbitrarily rearranging structure >> members can generate many more cache misses so one should sanity check >> changes w/ something like hwpmc. However as noted because layout may be >> platform-dependent even if something shows no change on x86 it may be a >> loss on another architecture and finding that performance drop may be >> really hard. >> > > Let's not be too "glass half empty" about it, though. The same is true in the > opposite direction. If we can identify and eliminate an unnecessary hole in > an important structure we might gain that same performance just by reshuffling > a few lines. > > Certainly plugging holes can also be beneficial but just cautioning that changes of this sort need to be checked if made to critical data structures. OTOH there aren't that many that matter in practice. Sam From rwatson at FreeBSD.org Thu Feb 12 09:55:11 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Feb 12 09:55:18 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <200902121754.24536.max@love2party.net> References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> <200902121754.24536.max@love2party.net> Message-ID: On Thu, 12 Feb 2009, Max Laier wrote: >>> So to answer your first question, submitting 101 patches to rearrange 101 >>> structs is certainly a wasted effort. However, if you take a good look at >>> the 2000 holes, identify an interesting subset and submit a patch to fix >>> that subset ... that would be a worthwhile effort ... IMHO. >> >> The other thing to keep in mind is that structure layout can have a >> noticeable effect on cache locality. Arbitrarily rearranging structure >> members can generate many more cache misses so one should sanity check >> changes w/ something like hwpmc. However as noted because layout may be >> platform-dependent even if something shows no change on x86 it may be a >> loss on another architecture and finding that performance drop may be >> really hard. > > Let's not be too "glass half empty" about it, though. The same is true in > the opposite direction. If we can identify and eliminate an unnecessary > hole in an important structure we might gain that same performance just by > reshuffling a few lines. Well, I think we want to inform this through actual measurement. Right now, tools like hwpmc track cache misses by point in executable code, but what would be nice is if we could post-process to generate cache miss information by data structure field... Robert N M Watson Computer Laboratory University of Cambridge From bsd.quest at googlemail.com Thu Feb 12 10:03:15 2009 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Thu Feb 12 10:03:24 2009 Subject: strange output in /var/log/messages In-Reply-To: References: <671bb5fc0902120857h3b789447q64a8c396728bdbd6@mail.gmail.com> Message-ID: <671bb5fc0902121003i4f9e606drc70e35bb0a91d0f8@mail.gmail.com> 2009/2/12 Stefan Lambrev > Hi, > On Feb 12, 2009, at 6:57 PM, Alexej Sokolov wrote: > > Hello, > I try now to debug a kernel module and I make some output with printf(9). > But the text appears in /var/log/messages in very strange form: > > Feb 12 17:54:34 myhost kernel: b > Feb 12 17:54:34 myhost kernel: eg > Feb 12 17:54:34 myhost kernel: in > Feb 12 17:54:34 myhost kernel: . > Feb 12 17:54:34 myhost kernel: De > Feb 12 17:54:34 myhost kernel: vice > Feb 12 17:54:34 myhost kernel: U > Feb 12 17:54:34 myhost kernel: ni > Feb 12 17:54:34 myhost kernel: t: > > Could anyone explain the reason of this kind of output. And how can I > correct it? > > > But those kernel messages are displayed properly if you type dmesg? > If yes I think you can blame syslogd. > Yes , dmesg makes correct output :( > > > > Alexej > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > -- > Best Wishes, > Stefan Lambrev > ICQ# 24134177 > > > > > > From bsd.quest at googlemail.com Thu Feb 12 12:31:51 2009 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Thu Feb 12 12:31:58 2009 Subject: bus_setup_intr (9) Message-ID: <671bb5fc0902121231v45317860ne3ff399c688261b1@mail.gmail.com> hello, from man: int bus_setup_intr(device_t dev, struct resource *r, int flags, driver_filter_t filter, driver_intr_t ithread, void *arg, void **cookiep); The function filter returns value of type driver_filter_t (int). This function will run if interrupt happen. Question: Which function will get this returned "int value" of filter function. Or How/where can I catch it ? Alexej From bakul at bitblocks.com Thu Feb 12 13:15:09 2009 From: bakul at bitblocks.com (Bakul Shah) Date: Thu Feb 12 13:15:16 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: Your message of "Thu, 12 Feb 2009 09:53:18 PST." <4994620E.5000804@freebsd.org> References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> <200902121754.24536.max@love2party.net> <4994620E.5000804@freebsd.org> Message-ID: <20090212211508.7CF795B21@mail.bitblocks.com> > >>>> So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that > >>>> many of the struct had holes, and some of which could be rearranged to > >>>> fill the gap. ... > Certainly plugging holes can also be beneficial but just cautioning that > changes of this sort need to be checked if made to critical data > structures. OTOH there aren't that many that matter in practice. But why do it? Are the benefits worth churning any ABIs? From xcllnt at mac.com Thu Feb 12 13:22:10 2009 From: xcllnt at mac.com (Marcel Moolenaar) Date: Thu Feb 12 13:22:17 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: <49944F8F.5080104@quis.cx> References: <200902121549.n1CFnLdt002361@fire.js.berklix.net> <49944F8F.5080104@quis.cx> Message-ID: On Feb 12, 2009, at 8:34 AM, Jille Timmermans wrote: > Julian Stacey schreef: >>> 1) Is it worth my time trying to rearrange structs? >> I wondered whether as a sensitivity test, some version of gcc (or >> its competitor ?) might have capability to automatically re-order >> variables ? but found nothing in man gcc "Optimization Options". > There is a __packed attribute, I don't know what it exactly does and > whether it is an improvement. > __packed is always a gross pessimization. The side-effect of packing a structure is that the alignment of the structure drops to 1. That means that any field will be read 1 byte at a time and reconstructed by logical operations. For best results, __packed should be used with __aligned(X), in case __packed is needed of course to address the side- effect. Of course multi-byte fields that are unaligned in the structure as the result of packing, will still be read in parts. In other words: don't use __packed when you don't have to. -- Marcel Moolenaar xcllnt@mac.com From p.pisati at oltrelinux.com Thu Feb 12 13:24:37 2009 From: p.pisati at oltrelinux.com (Paolo Pisati) Date: Thu Feb 12 13:24:44 2009 Subject: bus_setup_intr (9) In-Reply-To: <671bb5fc0902121231v45317860ne3ff399c688261b1@mail.gmail.com> References: <671bb5fc0902121231v45317860ne3ff399c688261b1@mail.gmail.com> Message-ID: <499490D7.5020306@oltrelinux.com> Alexej Sokolov wrote: > from man: > int > bus_setup_intr(device_t dev, struct resource *r, int flags, > driver_filter_t filter, driver_intr_t ithread, void *arg, > void **cookiep); > > The function filter returns value of type driver_filter_t (int). This > function will run if interrupt happen. > > Question: Which function will get this returned "int value" of filter > function. Or How/where can I catch it ? > the interrupt handling framework: see sys/kern/kern_intr.c::intr_event_handle(). bye, P. From kmacy at freebsd.org Thu Feb 12 13:40:15 2009 From: kmacy at freebsd.org (Kip Macy) Date: Thu Feb 12 13:40:30 2009 Subject: bus_setup_intr (9) In-Reply-To: <671bb5fc0902121231v45317860ne3ff399c688261b1@mail.gmail.com> References: <671bb5fc0902121231v45317860ne3ff399c688261b1@mail.gmail.com> Message-ID: <3c1674c90902121307k7cbe71d0of4940f3876f7a01c@mail.gmail.com> in HEAD, see intr_event_handle in sys/kern/kern_intr.c Cheers, Kip On Thu, Feb 12, 2009 at 12:31 PM, Alexej Sokolov wrote: > hello, > > from man: > int > bus_setup_intr(device_t dev, struct resource *r, int flags, > driver_filter_t filter, driver_intr_t ithread, void *arg, > void **cookiep); > > The function filter returns value of type driver_filter_t (int). This > function will run if interrupt happen. > > Question: Which function will get this returned "int value" of filter > function. Or How/where can I catch it ? > > Alexej > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From neldredge at math.ucsd.edu Thu Feb 12 14:05:44 2009 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu Feb 12 14:05:50 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: References: <200902121549.n1CFnLdt002361@fire.js.berklix.net> <49944F8F.5080104@quis.cx> Message-ID: On Thu, 12 Feb 2009, Marcel Moolenaar wrote: > > On Feb 12, 2009, at 8:34 AM, Jille Timmermans wrote: > >> Julian Stacey schreef: >>>> 1) Is it worth my time trying to rearrange structs? >>> I wondered whether as a sensitivity test, some version of gcc (or >>> its competitor ?) might have capability to automatically re-order >>> variables ? but found nothing in man gcc "Optimization Options". >> There is a __packed attribute, I don't know what it exactly does and >> whether it is an improvement. >> > > __packed is always a gross pessimization. The side-effect of > packing a structure is that the alignment of the structure > drops to 1. That means that any field will be read 1 byte at > a time and reconstructed by logical operations. The other alternative is to read/write that member by unaligned operations, on platforms that support it. This also typically comes with a performance penalty, of course. Usually it means the hardware reads the two words that overlap the member and pieces it back together. But on such a platform the software does not need to handle it specially; it executes the same instruction, but it takes more time. The only reason to use this would be (1) if you needed to have your structure occupy as little memory as possible; for instance, if your structure had two elements, one 'int' and one 'char', and you had 1 billion of them, using __packed__ would save you 3 gigabytes. Or (2) if you need to conform to an externally defined data structure that already does this. Most places in the kernel, I don't think either of these would be true. -- Nate Eldredge neldredge@math.ucsd.edu From gad at FreeBSD.org Thu Feb 12 18:41:29 2009 From: gad at FreeBSD.org (Garance A Drosehn) Date: Thu Feb 12 18:41:39 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: References: Message-ID: At 2:08 PM +0000 2/12/09, Andrew Brampton wrote: >I found this useful tool called pahole[1]. It basically finds holes >within structs, so for example on my 64bit machine this struct: > >struct test { > int foo; > const char *bar; > int blah; >} > >Would have a hole between foo and bar of 4 bytes because both for and >bar have been aligned on a 8 byte boundary, and the struct would also >have 4 bytes of padding on the end. However, if I simply moved blah >between foo and bar then the struct has shrunk by 8 bytes, which could >be a good thing. This could also help keep structs within single cache >lines, and just generally keep memory usage to a minimum when the >struct is used many times (for example in an array). > >So I ran the tool pahole over a 7.1 FreeBSD Kernel, and found that >many of the struct had holes, and some of which could be rearranged to >fill the gap. I've made the list available here[2]. So my questions >are two fold: > >1) Is it worth my time trying to rearrange structs? If so do you >think many of my patches would be accepted? In most cases, re-arranging things is probably not worth it. That's just my opinion, though. >2) Is there a way to find out the most heavily used structs? There >are ~3600 structs, and ~2000 holes, it might be a waste of my time >fixing the structs which are only used once. I used a similar tool (which I wrote myself) when making some changes to 'struct kinfo_proc' some time ago. The first issue which comes up is that a hole on one hardware platform may not be a hole on other platforms. I think it is perhaps useful to shuffle around fields in a struct if you're doing some *other* change at the same time. A tool like this pahole program seems like a good idea at those times. And I do think it can be useful to document the holes which exist, especially if you take the time to figure out what is happening on different platforms which have different alignment requirements. -- Garance Alistair Drosehn = drosehn@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA From ttw+bsd at cobbled.net Fri Feb 13 04:21:09 2009 From: ttw+bsd at cobbled.net (ttw+bsd@cobbled.net) Date: Fri Feb 13 04:58:49 2009 Subject: removal of NGROUPS_MAX dependancy from base Message-ID: <20090213115426.GA15211@holyman.cobbled.net> attached is the first in a series of patches that is intended to remove the current limitation on group membership. this patch should remove the dependancy on the definition of NGROUPS_MAX as a static constant and implement it as a writable sysconf variable of the same. it should also make the necessary changes to the codebase to support those. i need some guidance as to what i should re-define NGROUPS_MAX to be (so that code that depends on it can continue to operate, i'm thinking just make it 16 but perhaps it would be worth extending the default while we're at it to something like 64??). i also need feedback on any braindamage in the current changes. the next step will be to extend the kernel groups and map them back to the user structs / calls. finally i'll extend the user groups and implement those calls. nb: not tested the code (it builds) ... was intending to test it on my XEN box but only just realised that Xen on amd64 isn't working. :-( happy for any questions that may help guide the process. -------------- next part -------------- Index: contrib/openpam/lib/openpam_borrow_cred.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/contrib/openpam/lib/openpam_borrow_cred.c,v retrieving revision 1.1.1.9 diff -b -u -r1.1.1.9 openpam_borrow_cred.c --- contrib/openpam/lib/openpam_borrow_cred.c 21 Dec 2007 11:49:29 -0000 1.1.1.9 +++ contrib/openpam/lib/openpam_borrow_cred.c 4 Feb 2009 16:38:46 -0000 @@ -60,6 +60,7 @@ struct pam_saved_cred *scred; const void *scredp; int r; + int ngroups ; ENTERI(pwd->pw_uid); r = pam_get_data(pamh, PAM_SAVED_CRED, &scredp); @@ -73,26 +74,55 @@ (int)geteuid()); RETURNC(PAM_PERM_DENIED); } - scred = calloc(1, sizeof *scred); - if (scred == NULL) - RETURNC(PAM_BUF_ERR); - scred->euid = geteuid(); - scred->egid = getegid(); - r = getgroups(NGROUPS_MAX, scred->groups); - if (r < 0) { - FREE(scred); - RETURNC(PAM_SYSTEM_ERR); - } - scred->ngroups = r; +/* get the maximum number of system groups */ +#if _POSIX_VERSION > 199212 + ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + ngroups = NGROUPS_MAX ; +#else + ngroups = _NGROUPS_COMPAT ; +#endif +/* initally allocate enough memory for max_groups */ + scred = malloc( sizeof(struct pam_saved_cred) + + ngroups*sizeof(gid_t) ) ; + if( scred == NULL ) + RETURNC( PAM_BUF_ERR ) ; +/* set the save values */ + scred->euid = geteuid() ; + scred->egid = getegid() ; +/* save groups into our (probably) oversized memory allocation */ + r = getgroups( ngroups, scred->groups ) ; + if( r < 0 ) { + FREE( scred ) ; /* call PAM's free macro */ + RETURNC( PAM_SYSTEM_ERR ) ; + } ; + scred->ngroups = r ; + ngroups = r < ngroups ? r : ngroups ; /* choose the smallest */ + /* ... number of groups to allocate */ + ngroups = ngroups < _NGROUPS_COMPAT ? ngroups : _NGROUPS_COMPAT ; + /* but keep it within expected minimum value */ + /* XXX: we don't really want this but until we get + * educated on the implications this is probably safe + * and certainaly compatible */ +/* realloc, releasing unneeded memory */ + scred = realloc( (void*)scred, + sizeof(struct pam_saved_cred)+ngroups*sizeof(gid_t) ) ; + /* nb: we ignore failure and try to store the larger + * ... structure as initially requested. catching the + * ... error in 'pam_set_data' if neccessary. */ +/* save the credentials to PAM user data area */ r = pam_set_data(pamh, PAM_SAVED_CRED, scred, &openpam_free_data); if (r != PAM_SUCCESS) { FREE(scred); RETURNC(r); } +/* set the new credentials */ if (geteuid() == pwd->pw_uid) RETURNC(PAM_SUCCESS); if (initgroups(pwd->pw_name, pwd->pw_gid) < 0 || - setegid(pwd->pw_gid) < 0 || seteuid(pwd->pw_uid) < 0) { + setegid(pwd->pw_gid) < 0 || seteuid(pwd->pw_uid) < 0) + { + /* if any of the set calls failed, then restore and fail */ openpam_restore_cred(pamh); RETURNC(PAM_SYSTEM_ERR); } Index: contrib/openpam/lib/openpam_impl.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/contrib/openpam/lib/openpam_impl.h,v retrieving revision 1.1.1.17 diff -b -u -r1.1.1.17 openpam_impl.h --- contrib/openpam/lib/openpam_impl.h 21 Dec 2007 11:49:29 -0000 1.1.1.17 +++ contrib/openpam/lib/openpam_impl.h 5 Feb 2009 15:41:19 -0000 @@ -110,13 +110,17 @@ int env_size; }; -#ifdef NGROUPS_MAX +#if _POSIX_VERSION > 199212 #define PAM_SAVED_CRED "pam_saved_cred" struct pam_saved_cred { uid_t euid; gid_t egid; - gid_t groups[NGROUPS_MAX]; int ngroups; + gid_t groups[]; + /* keep this last so that we can simply + .. over-allocate the amount of space + .. nb: don't use sizeof' unless you adjust + .. for the number of groups */ }; #endif Index: include/rpc/auth_unix.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/include/rpc/auth_unix.h,v retrieving revision 1.11 diff -b -u -r1.11 auth_unix.h --- include/rpc/auth_unix.h 23 Mar 2002 17:24:55 -0000 1.11 +++ include/rpc/auth_unix.h 14 Jan 2009 11:15:21 -0000 @@ -52,7 +52,7 @@ #define MAX_MACHINE_NAME 255 /* gids compose part of a credential; there may not be more than 16 of them */ -#define NGRPS 16 +#define AUTH_UNIX_NGROUPS 16 /* * Unix style credentials. Index: lib/libc/rpc/auth_unix.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/auth_unix.c,v retrieving revision 1.18 diff -b -u -r1.18 auth_unix.c --- lib/libc/rpc/auth_unix.c 14 Jun 2007 20:07:35 -0000 1.18 +++ lib/libc/rpc/auth_unix.c 4 Feb 2009 15:31:57 -0000 @@ -182,27 +182,48 @@ * Returns an auth handle with parameters determined by doing lots of * syscalls. */ -AUTH * +AUTH* authunix_create_default() { - int len; char machname[MAXHOSTNAMELEN + 1]; + AUTH* auth_unix ; uid_t uid; gid_t gid; - gid_t gids[NGROUPS_MAX]; - - if (gethostname(machname, sizeof machname) == -1) - abort(); - machname[sizeof(machname) - 1] = 0; + gid_t *gids ; + uint ngroups ; + uint max_ngroups ; + +/* get hostname or fail */ + if( gethostname(machname,sizeof(machname)) == -1 ) + abort() ; + machname[sizeof(machname)-1] = 0 ; /* add a null terminator */ +/* set uid/gid from current effective values */ uid = geteuid(); gid = getegid(); - if ((len = getgroups(NGROUPS_MAX, gids)) < 0) - abort(); - if (len > NGRPS) - len = NGRPS; - /* XXX: interface problem; those should all have been unsigned */ - return (authunix_create(machname, (int)uid, (int)gid, len, - (int *)gids)); +/* set the group set */ +#if _POSIX_VERSION > 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = 16 ; +#endif + gids = (gid_t*)calloc( max_ngroups, sizeof(gid_t) ) ; + if( gids == NULL ) + abort () ; + if( (ngroups=getgroups(max_ngroups,gids)) < 0 ) { + free( gids ) ; + abort() ; + } +/* clip the groups to a transmissable size */ + if( ngroups > AUTH_UNIX_NGROUPS ) + ngroups = AUTH_UNIX_NGROUPS ; +/* XXX: interface problem; those should all have been unsigned */ + auth_unix = authunix_create( machname, + (int)uid, (int)gid, (int)ngroups, + (int*)gids ) ; + free( (void*)gids ) ; + return( auth_unix ) ; } /* Index: lib/libc/rpc/authunix_prot.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/authunix_prot.c,v retrieving revision 1.10 diff -b -u -r1.10 authunix_prot.c --- lib/libc/rpc/authunix_prot.c 20 Nov 2007 01:51:20 -0000 1.10 +++ lib/libc/rpc/authunix_prot.c 4 Feb 2009 16:03:29 -0000 @@ -67,13 +67,14 @@ paup_gids = &p->aup_gids; - if (xdr_u_long(xdrs, &(p->aup_time)) - && xdr_string(xdrs, &(p->aup_machname), MAX_MACHINE_NAME) - && xdr_int(xdrs, &(p->aup_uid)) - && xdr_int(xdrs, &(p->aup_gid)) - && xdr_array(xdrs, (char **) paup_gids, - &(p->aup_len), NGRPS, sizeof(int), (xdrproc_t)xdr_int) ) { - return (TRUE); + if( xdr_u_long(xdrs,&(p->aup_time)) && + xdr_string(xdrs,&(p->aup_machname),MAX_MACHINE_NAME) && + xdr_int(xdrs,&(p->aup_uid)) && + xdr_int(xdrs,&(p->aup_gid)) && + xdr_array(xdrs,(char**)paup_gids,&(p->aup_len), + AUTH_UNIX_NGROUPS,sizeof(int),(xdrproc_t)xdr_int) ) + { + return( TRUE ) ; } - return (FALSE); + return( FALSE ) ; } Index: lib/libc/rpc/netname.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/netname.c,v retrieving revision 1.8 diff -b -u -r1.8 netname.c --- lib/libc/rpc/netname.c 16 Oct 2004 06:11:35 -0000 1.8 +++ lib/libc/rpc/netname.c 14 Jan 2009 01:29:47 -0000 @@ -61,6 +61,7 @@ #ifndef MAXHOSTNAMELEN #define MAXHOSTNAMELEN 256 #endif + #ifndef NGROUPS #define NGROUPS 16 #endif Index: lib/libc/rpc/netnamer.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/netnamer.c,v retrieving revision 1.12 diff -b -u -r1.12 netnamer.c --- lib/libc/rpc/netnamer.c 10 Mar 2005 00:58:21 -0000 1.12 +++ lib/libc/rpc/netnamer.c 3 Feb 2009 17:55:48 -0000 @@ -69,7 +69,6 @@ #ifndef NGROUPS #define NGROUPS 16 #endif - /* * Convert network-name into unix credential */ @@ -104,7 +103,7 @@ return (0); } *gidp = (gid_t) atol(p); - for (gidlen = 0; gidlen < NGROUPS; gidlen++) { + for (gidlen = 0; gidlen < _NGROUPS_RPC_MAX; gidlen++) { p = strsep(&res, "\n,"); if (p == NULL) break; @@ -157,7 +156,7 @@ static int _getgroups(uname, groups) char *uname; - gid_t groups[NGROUPS]; + gid_t groups[_NGROUPS_RPC_MAX]; { gid_t ngroups = 0; struct group *grp; @@ -169,10 +168,11 @@ while ((grp = getgrent())) { for (i = 0; grp->gr_mem[i]; i++) if (!strcmp(grp->gr_mem[i], uname)) { - if (ngroups == NGROUPS) { + if( ngroups == _NGROUPS_RPC_MAX ) { #ifdef DEBUG - fprintf(stderr, - "initgroups: %s is in too many groups\n", uname); + fprintf( stderr, + "initgroups: %s is in too many groups\n", + uname ) ; #endif goto toomany; } Index: lib/libc/rpc/svc_auth_des.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/svc_auth_des.c,v retrieving revision 1.9 diff -b -u -r1.9 svc_auth_des.c --- lib/libc/rpc/svc_auth_des.c 22 Mar 2002 23:18:37 -0000 1.9 +++ lib/libc/rpc/svc_auth_des.c 3 Feb 2009 17:51:01 -0000 @@ -452,7 +452,7 @@ short uid; /* cached uid */ short gid; /* cached gid */ short grouplen; /* length of cached groups */ - short groups[NGROUPS]; /* cached groups */ + short groups[_NGROUPS_RPC_MAX]; /* cached groups */ }; /* Index: lib/libc/rpc/svc_auth_unix.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/svc_auth_unix.c,v retrieving revision 1.11 diff -b -u -r1.11 svc_auth_unix.c --- lib/libc/rpc/svc_auth_unix.c 16 Oct 2004 06:11:35 -0000 1.11 +++ lib/libc/rpc/svc_auth_unix.c 4 Feb 2009 16:04:10 -0000 @@ -68,7 +68,7 @@ struct area { struct authunix_parms area_aup; char area_machname[MAX_MACHINE_NAME+1]; - int area_gids[NGRPS]; + int area_gids[AUTH_UNIX_NGROUPS] ; } *area; u_int auth_len; size_t str_len, gid_len; @@ -98,7 +98,7 @@ aup->aup_uid = (int)IXDR_GET_INT32(buf); aup->aup_gid = (int)IXDR_GET_INT32(buf); gid_len = (size_t)IXDR_GET_U_INT32(buf); - if (gid_len > NGRPS) { + if( gid_len > AUTH_UNIX_NGROUPS ) { stat = AUTH_BADCRED; goto done; } Index: lib/librpcsec_gss/svc_rpcsec_gss.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/librpcsec_gss/svc_rpcsec_gss.c,v retrieving revision 1.4 diff -b -u -r1.4 svc_rpcsec_gss.c --- lib/librpcsec_gss/svc_rpcsec_gss.c 3 Nov 2008 10:38:00 -0000 1.4 +++ lib/librpcsec_gss/svc_rpcsec_gss.c 5 Feb 2009 16:09:37 -0000 @@ -127,7 +127,7 @@ rpc_gss_ucred_t cl_ucred; /* unix-style credentials */ bool_t cl_done_callback; /* TRUE after call */ void *cl_cookie; /* user cookie from callback */ - gid_t cl_gid_storage[NGRPS]; + gid_t cl_gid_storage[AUTH_UNIX_NGROUPS]; gss_OID cl_mech; /* mechanism */ gss_qop_t cl_qop; /* quality of protection */ u_int cl_seq; /* current sequence number */ @@ -578,7 +578,7 @@ getpwuid_r(uid, &pwd, buf, sizeof(buf), &pw); if (pw) { - int len = NGRPS; + int len = AUTH_UNIX_NGROUPS; uc->uid = pw->pw_uid; uc->gid = pw->pw_gid; uc->gidlist = client->cl_gid_storage; Index: sys/compat/svr4/svr4_misc.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/compat/svr4/svr4_misc.c,v retrieving revision 1.101 diff -b -u -r1.101 svr4_misc.c --- sys/compat/svr4/svr4_misc.c 21 Apr 2008 21:24:08 -0000 1.101 +++ sys/compat/svr4/svr4_misc.c 14 Jan 2009 11:58:47 -0000 @@ -710,7 +710,12 @@ *retval = 0; break; case SVR4_CONFIG_NGROUPS: - *retval = NGROUPS_MAX; + *retval = _NGROUPS_COMPAT; + /* XXX: this should pull the value + * from sysctl but i cannot find + * the definitions for the similar + * varaibles here (i.e. 'maxproc') + */ break; case SVR4_CONFIG_CHILD_MAX: *retval = maxproc; Index: sys/fs/portalfs/portal.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/fs/portalfs/portal.h,v retrieving revision 1.10 diff -b -u -r1.10 portal.h --- sys/fs/portalfs/portal.h 6 Jan 2005 18:10:40 -0000 1.10 +++ sys/fs/portalfs/portal.h 16 Jan 2009 23:44:50 -0000 @@ -43,7 +43,7 @@ int pcr_flag; /* File open mode */ uid_t pcr_uid; /* From ucred */ short pcr_ngroups; /* From ucred */ - gid_t pcr_groups[NGROUPS]; /* From ucred */ + gid_t pcr_groups[_NGROUPS_COMPAT]; /* From ucred */ }; #ifdef _KERNEL Index: sys/i386/ibcs2/ibcs2_misc.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/i386/ibcs2/ibcs2_misc.c,v retrieving revision 1.70 diff -b -u -r1.70 ibcs2_misc.c --- sys/i386/ibcs2/ibcs2_misc.c 13 Jan 2008 14:44:07 -0000 1.70 +++ sys/i386/ibcs2/ibcs2_misc.c 14 Jan 2009 12:24:56 -0000 @@ -659,14 +659,14 @@ struct thread *td; struct ibcs2_getgroups_args *uap; { - ibcs2_gid_t iset[NGROUPS_MAX]; - gid_t gp[NGROUPS_MAX]; + ibcs2_gid_t iset[_NGROUPS_COMPAT]; + gid_t gp[_NGROUPS_COMPAT]; u_int i, ngrp; int error; if (uap->gidsetsize < 0) return (EINVAL); - ngrp = MIN(uap->gidsetsize, NGROUPS_MAX); + ngrp = MIN(uap->gidsetsize, _NGROUPS_COMPAT); error = kern_getgroups(td, &ngrp, gp); if (error) return (error); @@ -685,11 +685,11 @@ struct thread *td; struct ibcs2_setgroups_args *uap; { - ibcs2_gid_t iset[NGROUPS_MAX]; - gid_t gp[NGROUPS_MAX]; + ibcs2_gid_t iset[_NGROUPS_COMPAT]; + gid_t gp[_NGROUPS_COMPAT]; int error, i; - if (uap->gidsetsize < 0 || uap->gidsetsize > NGROUPS_MAX) + if (uap->gidsetsize < 0 || uap->gidsetsize > _NGROUPS_COMPAT) return (EINVAL); if (uap->gidsetsize && uap->gidset) { error = copyin(uap->gidset, iset, sizeof(ibcs2_gid_t) * @@ -789,8 +789,13 @@ return 0; case IBCS2_SC_NGROUPS_MAX: - mib[1] = KERN_NGROUPS; - break; + /* XXX: IBCS2 compat with group limits not known to + * me, so i'll just return a compatibile/safe limit + * for now */ + PROC_LOCK(p) ; + td->td_retval[0] = _NGROUPS_COMPAT ; + PROC_UNLOCK(p) ; + return( 0 ) ; case IBCS2_SC_OPEN_MAX: PROC_LOCK(p); Index: sys/kern/kern_mib.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/kern/kern_mib.c,v retrieving revision 1.93 diff -b -u -r1.93 kern_mib.c --- sys/kern/kern_mib.c 28 Jan 2009 19:58:05 -0000 1.93 +++ sys/kern/kern_mib.c 4 Feb 2009 13:15:06 -0000 @@ -124,8 +124,8 @@ SYSCTL_INT(_kern, KERN_POSIX1, posix1version, CTLFLAG_RD, 0, _POSIX_VERSION, "Version of POSIX attempting to comply to"); -SYSCTL_INT(_kern, KERN_NGROUPS, ngroups, CTLFLAG_RD, - 0, NGROUPS_MAX, "Maximum number of groups a user can belong to"); +SYSCTL_INT(_kern, KERN_NGROUPS, ngroups, CTLFLAG_RW, + 0, _NGROUPS_COMPAT, "Maximum number of groups allocated to a user"); SYSCTL_INT(_kern, KERN_JOB_CONTROL, job_control, CTLFLAG_RD, 0, 1, "Whether job control is available"); Index: sys/sys/param.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/sys/param.h,v retrieving revision 1.382 diff -b -u -r1.382 param.h --- sys/sys/param.h 28 Jan 2009 17:57:16 -0000 1.382 +++ sys/sys/param.h 4 Feb 2009 14:11:55 -0000 @@ -57,7 +57,7 @@ * is created, otherwise 1. */ #undef __FreeBSD_version -#define __FreeBSD_version 800062 /* Master, propagated to newvers */ +#define __FreeBSD_version 800060 /* Master, propagated to newvers */ #ifndef LOCORE #include @@ -77,7 +77,8 @@ #define MAXLOGNAME 17 /* max login name length (incl. NUL) */ #define MAXUPRC CHILD_MAX /* max simultaneous processes */ #define NCARGS ARG_MAX /* max bytes for an exec function */ -#define NGROUPS NGROUPS_MAX /* max number groups */ +#define NGROUPS _NGROUPS_COMPAT + /* depreciated check sysctl/sysconf for NGROUPS_MAX value instead */ #define NOFILE OPEN_MAX /* max open files per process */ #define NOGROUP 65535 /* marker for empty group set member */ #define MAXHOSTNAMELEN 256 /* max hostname size */ Index: sys/sys/syslimits.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/sys/syslimits.h,v retrieving revision 1.23 diff -b -u -r1.23 syslimits.h --- sys/sys/syslimits.h 29 May 2007 15:14:46 -0000 1.23 +++ sys/sys/syslimits.h 3 Feb 2009 18:02:22 -0000 @@ -54,7 +54,6 @@ #define MAX_CANON 255 /* max bytes in term canon input line */ #define MAX_INPUT 255 /* max bytes in terminal input */ #define NAME_MAX 255 /* max bytes in a file name */ -#define NGROUPS_MAX 16 /* max supplemental group id's */ #ifndef OPEN_MAX #define OPEN_MAX 64 /* max open files per process */ #endif @@ -66,9 +65,35 @@ * We leave the following values undefined to force applications to either * assume conservative values or call sysconf() to get the current value. * - * HOST_NAME_MAX + * HOST_NAME_MAX NGROUPS_MAX * * (We should do this for most of the values currently defined here, * but many programs are not prepared to deal with this yet.) */ +/* + * here are some reference values in respect of the obsoleted + * NGROUPS_MAX value. + * nb: some apps appear to check NGROUPS_MAX as meaning that + * ... system has user groups (i.e. to #ifdef chunks of code). + * ... this is easy to change but maybe historically defined? + */ +#define _NGROUPS_RPC_MAX 16 /* reference only */ + /* nb: this is the old system max, so named + * ... because it's limit appears to + * ... have been derived from a limitation + * ... in RPC (and thereby NFS), where it's + * ... the max number of groups we can exchange */ +#define _NGROUPS_COMPAT _NGROUPS_RPC_MAX /* reference only */ + /* nb: although this is defined as equal to the rpc + * ... limit, i have defined it distintly so that + * ... we may distinguish (whilst updating) usage + * ... that is correctly explicit (i.e. should be 16) + * ... and usage that is only 16 because of an expected + * ... convention. hopefully we may remove these and + * ... define additional _NGROUPS_*_MAX for those defined + * ... uses. */ +#define _NGROUPS_SYS_MAX 65536 /* reference only */ + /* nb: the idea's to have this extensible + * ... indefinately, this is what linux have and + * ... should more than cover immediate needs */ #endif Index: usr.bin/catman/catman.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.bin/catman/catman.c,v retrieving revision 1.14 diff -b -u -r1.14 catman.c --- usr.bin/catman/catman.c 5 Dec 2005 14:22:12 -0000 1.14 +++ usr.bin/catman/catman.c 8 Feb 2009 22:51:44 -0000 @@ -93,8 +93,9 @@ enum Ziptype {NONE, BZIP, GZIP}; static uid_t uid; -static gid_t gids[NGROUPS_MAX]; +static gid_t *gids; static int ngids; +static int max_ngroups ; static int starting_dir; static char tmp_file[MAXPATHLEN]; struct stat test_st; @@ -789,7 +790,15 @@ /* NOTREACHED */ } } - ngids = getgroups(NGROUPS_MAX, gids); +/* allocate memory for group ids */ +#if _POSIX_VERSION > 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + ngids = getgroups( max_ngroups, gids ) ; if ((starting_dir = open(".", 0)) < 0) { err(1, "."); } Index: usr.bin/newgrp/newgrp.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.bin/newgrp/newgrp.c,v retrieving revision 1.2 diff -b -u -r1.2 newgrp.c --- usr.bin/newgrp/newgrp.c 30 Oct 2003 15:14:34 -0000 1.2 +++ usr.bin/newgrp/newgrp.c 9 Feb 2009 22:05:53 -0000 @@ -146,9 +146,10 @@ static void addgroup(const char *grpname) { - gid_t grps[NGROUPS_MAX]; + gid_t *grps; long lgid; - int dbmember, i, ngrps; + int dbmember, i, ngrps, max_ngroups ; + /* XXX: should 'max_ngroups' be a static const variable? */ gid_t egid; struct group *grp; char *ep, *pass; @@ -185,9 +186,21 @@ } } - if ((ngrps = getgroups(NGROUPS_MAX, (gid_t *)grps)) < 0) { +#if _POSIX_VERSION >= 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + grps = (gid_t*)calloc( max_ngroups, sizeof(gid_t) ) ; + if( grps == NULL ) { + warn( "group set memory allocation" ) ; + return ; + } + if( (ngrps=getgroups(max_ngroups,(gid_t*)grps)) < 0 ) { warn("getgroups"); - return; + goto error_free ; } /* Remove requested gid from supp. list if it exists. */ @@ -201,7 +214,7 @@ if (setgroups(ngrps, (const gid_t *)grps) < 0) { PRIV_END; warn("setgroups"); - return; + goto error_free ; } PRIV_END; } @@ -210,14 +223,14 @@ if (setgid(grp->gr_gid)) { PRIV_END; warn("setgid"); - return; + goto error_free ; } PRIV_END; grps[0] = grp->gr_gid; /* Add old effective gid to supp. list if it does not exist. */ if (egid != grp->gr_gid && !inarray(egid, grps, ngrps)) { - if (ngrps == NGROUPS_MAX) + if( ngrps == max_ngroups ) warnx("too many groups"); else { grps[ngrps++] = egid; @@ -225,12 +238,15 @@ if (setgroups(ngrps, (const gid_t *)grps)) { PRIV_END; warn("setgroups"); - return; + goto error_free ; } PRIV_END; } } +error_free: + free( grps ) ; + return ; } static int Index: usr.sbin/chown/chown.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/chown/chown.c,v retrieving revision 1.29 diff -b -u -r1.29 chown.c --- usr.sbin/chown/chown.c 7 Aug 2004 04:19:37 -0000 1.29 +++ usr.sbin/chown/chown.c 8 Feb 2009 16:22:31 -0000 @@ -269,7 +269,8 @@ { static uid_t euid = -1; static int ngroups = -1; - gid_t groups[NGROUPS_MAX]; + static int max_groups ; + gid_t *groups; /* Check for chown without being root. */ if (errno != EPERM || (uid != (uid_t)-1 && @@ -279,16 +280,31 @@ } /* Check group membership; kernel just returns EPERM. */ +#if _POSIX_VERSION >= 199212 + max_groups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_groups = NGROUPS_MAX ; +#else + max_groups = _NGROUPS_COMPAT ; +#endif + groups = (gid_t*)calloc( max_groups, sizeof(gid_t) ) ; + if( groups == NULL ) { + warnx( "failed to allocate memory for group set" ) ; + goto exit_cleanup ; + } if (gid != (gid_t)-1 && ngroups == -1 && euid == (uid_t)-1 && (euid = geteuid()) != 0) { - ngroups = getgroups(NGROUPS_MAX, groups); + ngroups = getgroups( max_groups, groups ) ; while (--ngroups >= 0 && gid != groups[ngroups]); if (ngroups < 0) { warnx("you are not a member of group %s", gname); - return; + goto exit_cleanup ; } } warn("%s", file); +exit_cleanup: + free( groups ) ; + return ; } void Index: usr.sbin/chroot/chroot.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/chroot/chroot.c,v retrieving revision 1.11 diff -b -u -r1.11 chroot.c --- usr.sbin/chroot/chroot.c 7 Aug 2004 04:19:37 -0000 1.11 +++ usr.sbin/chroot/chroot.c 5 Feb 2009 23:29:48 -0000 @@ -59,6 +59,7 @@ char *user; /* user to switch to before running program */ char *group; /* group to switch to ... */ char *grouplist; /* group list to switch to ... */ +int max_ngroups; /* max number of groups allowable */ int main(argc, argv) @@ -69,12 +70,25 @@ struct passwd *pw; char *endp, *p; const char *shell; - gid_t gid, gidlist[NGROUPS_MAX]; + gid_t gid, *gidlist ; uid_t uid; - int ch, gids; + int ch, gids ; +/* set some defaults */ gid = 0; uid = 0; + user = NULL ; + group = NULL ; + grouplist = NULL ; +#if _POSIX_VERSION >= 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + +/* process command line options */ while ((ch = getopt(argc, argv, "G:g:u:")) != -1) { switch(ch) { case 'u': @@ -103,9 +117,12 @@ if (argc < 1) usage(); +/* if a group argument was passed then process it */ if (group != NULL) { + /* if the first char's a digit then assume it's a gid ... */ if (isdigit((unsigned char)*group)) { gid = (gid_t)strtoul(group, &endp, 0); + /* ... and back out that assumption if it proves wrong */ if (*endp != '\0') goto getgroup; } else { @@ -117,8 +134,15 @@ } } - for (gids = 0; - (p = strsep(&grouplist, ",")) != NULL && gids < NGROUPS_MAX; ) { +/* process command line group list */ + if( grouplist != NULL ) { + gidlist = (gid_t*)calloc( max_ngroups, sizeof(gid_t) ) ; + if( gidlist == NULL ) + errx( 1, "inadquate memory for group list" ) ; + for( gids = 0 ; + gids < max_ngroups && + (p=strsep(&grouplist,",")) != NULL ; ) + { if (*p == '\0') continue; @@ -135,9 +159,11 @@ } gids++; } - if (p != NULL && gids == NGROUPS_MAX) + if( p != NULL && gids == max_ngroups ) errx(1, "too many supplementary groups provided"); + } +/* set user from command line option, if supplied */ if (user != NULL) { if (isdigit((unsigned char)*user)) { uid = (uid_t)strtoul(user, &endp, 0); @@ -152,9 +178,11 @@ } } +/* change root */ if (chdir(argv[0]) == -1 || chroot(".") == -1) err(1, "%s", argv[0]); +/* set credentials */ if (gids && setgroups(gids, gidlist) == -1) err(1, "setgroups"); if (group && setgid(gid) == -1) @@ -162,11 +190,14 @@ if (user && setuid(uid) == -1) err(1, "setuid"); +/* exec the remaining arguments as the chroot'd command ... */ if (argv[1]) { execvp(argv[1], &argv[1]); err(1, "%s", argv[1]); + /* NOTREACHED */ } +/* ... or execute the default system shell */ if (!(shell = getenv("SHELL"))) shell = _PATH_BSHELL; execlp(shell, shell, "-i", (char *)NULL); Index: usr.sbin/gssd/gssd.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/gssd/gssd.c,v retrieving revision 1.1 diff -b -u -r1.1 gssd.c --- usr.sbin/gssd/gssd.c 3 Nov 2008 10:38:00 -0000 1.1 +++ usr.sbin/gssd/gssd.c 5 Feb 2009 16:16:37 -0000 @@ -464,8 +464,8 @@ result->uid = uid; getpwuid_r(uid, &pwd, buf, sizeof(buf), &pw); if (pw) { - int len = NGRPS; - int groups[NGRPS]; + int len = AUTH_UNIX_NGROUPS ; + int groups[AUTH_UNIX_NGROUPS] ; result->gid = pw->pw_gid; getgrouplist(pw->pw_name, pw->pw_gid, groups, &len); Index: usr.sbin/mount_portalfs/cred.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/mount_portalfs/cred.c,v retrieving revision 1.1 diff -b -u -r1.1 cred.c --- usr.sbin/mount_portalfs/cred.c 11 Mar 2005 08:39:58 -0000 1.1 +++ usr.sbin/mount_portalfs/cred.c 16 Jan 2009 23:49:36 -0000 @@ -46,7 +46,7 @@ set_user_credentials(struct portal_cred *user, struct portal_cred *save) { save->pcr_uid = geteuid(); - if ((save->pcr_ngroups = getgroups(NGROUPS_MAX, save->pcr_groups)) < 0) + if( (save->pcr_ngroups=getgroups(_NGROUPS_COMPAT,save->pcr_groups)) < 0 ) return (-1); if (setgroups(user->pcr_ngroups, user->pcr_groups) < 0) return (-1); Index: usr.sbin/pppd/options.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/pppd/options.c,v retrieving revision 1.26 diff -b -u -r1.26 options.c --- usr.sbin/pppd/options.c 7 Nov 2007 10:53:38 -0000 1.26 +++ usr.sbin/pppd/options.c 10 Feb 2009 09:11:47 -0000 @@ -72,10 +72,6 @@ char *strdup(char *); #endif -#ifndef GIDSET_TYPE -#define GIDSET_TYPE gid_t -#endif - /* * Option variables and default values. */ @@ -779,23 +775,64 @@ int fd; { uid_t uid; - int ngroups, i; + int ngroups, max_ngroups, i; struct stat sbuf; - GIDSET_TYPE groups[NGROUPS_MAX]; + gid_t *groups; +/* get the uid */ uid = getuid(); +/* ... and return true if root */ +/* XXX: needs credential check */ if (uid == 0) return 1; + +/* if we're not root, get some info about the file */ if (fstat(fd, &sbuf) != 0) return 0; + +/* test for owner match with current process */ if (sbuf.st_uid == uid) return sbuf.st_mode & S_IRUSR; +/* ... and a group match */ if (sbuf.st_gid == getgid()) return sbuf.st_mode & S_IRGRP; - ngroups = getgroups(NGROUPS_MAX, groups); - for (i = 0; i < ngroups; ++i) - if (sbuf.st_gid == groups[i]) - return sbuf.st_mode & S_IRGRP; + +/* if we've still no luck then check the group list for permission match */ +#if _POSIX_VERSION >= 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + groups = (gid_t*) calloc( max_ngroups, sizeof(gid_t) ) ; + if( groups == NULL ) { + /* if we cannot check groups correctly then assume 'fd' is unreadable + * XXX: this may be false as the converse is more likely. + * i.e. it would be failed readable on available groups + * and granted on full list, however, we just can't be + * psychic and i'm not about to code some idiotic loop that tries + * to get 'some' memory for partial testing. probably a better + * recourse would be to simply die here but that seems severe + * for a 'readable' test. + * NB: we don't need a 'full' allocation of memory to test the + * group list, only to store it. one idea would be to do this in + * 'blocks' + */ + option_error( 1, "unable to allocate memory for group list" ) ; + return( 0 ) ; + } +/* get groups */ + ngroups = getgroups( max_ngroups, groups ) ; +/* ... and test the group permission if matching */ + for( i = 0 ; i < ngroups ; ++i ) { + if (sbuf.st_gid == groups[i]) { + free( (void*)groups) ; + return( sbuf.st_mode & S_IRGRP ) ; + } + } +/* otherwise return other permissions match */ + free( (void*)groups ) ; return sbuf.st_mode & S_IROTH; } Index: usr.sbin/rpc.lockd/kern.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/rpc.lockd/kern.c,v retrieving revision 1.21 diff -b -u -r1.21 kern.c --- usr.sbin/rpc.lockd/kern.c 17 Aug 2006 05:55:20 -0000 1.21 +++ usr.sbin/rpc.lockd/kern.c 5 Feb 2009 16:22:17 -0000 @@ -239,15 +239,15 @@ int ngroups; ngroups = xucred->cr_ngroups - 1; - if (ngroups > NGRPS) - ngroups = NGRPS; - if (cl->cl_auth != NULL) - cl->cl_auth->ah_ops->ah_destroy(cl->cl_auth); - cl->cl_auth = authunix_create(hostname, + if( ngroups > AUTH_UNIX_NGROUPS ) + ngroups = AUTH_UNIX_NGROUPS ; + if( cl->cl_auth != NULL ) + cl->cl_auth->ah_ops->ah_destroy( cl->cl_auth ) ; + cl->cl_auth = authunix_create( hostname, xucred->cr_uid, xucred->cr_groups[0], ngroups, - &xucred->cr_groups[1]); + &xucred->cr_groups[1] ) ; } From patfbsd at davenulle.org Fri Feb 13 06:39:43 2009 From: patfbsd at davenulle.org (Patrick =?ISO-8859-15?Q?Lamaizi=E8re?=) Date: Fri Feb 13 06:39:50 2009 Subject: bus_dmamem_alloc() and BUS_DMA_WAITOK Message-ID: <20090213152205.0eccafb0@baby-jane.lamaiziere.net> Hello, I would like to know if it is safe to use the BUS_DMA_WAITOK flag with bus_dmamem_alloc(), in the context of the device_attach routine? Many drivers use BUS_DMA_NOWAIT. I need a 32Ko buffer and on low memory my driver cannot attach (bus_dmamem_alloc returns ENOMEM). Will BUS_DMA_WAITOK help in this case? Thanks in advance, regards. From jhb at freebsd.org Fri Feb 13 07:40:00 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 13 07:40:07 2009 Subject: bus_dmamem_alloc() and BUS_DMA_WAITOK In-Reply-To: <20090213152205.0eccafb0@baby-jane.lamaiziere.net> References: <20090213152205.0eccafb0@baby-jane.lamaiziere.net> Message-ID: <200902130957.10997.jhb@freebsd.org> On Friday 13 February 2009 9:22:05 am Patrick Lamaizi?re wrote: > Hello, > > I would like to know if it is safe to use the BUS_DMA_WAITOK flag with > bus_dmamem_alloc(), in the context of the device_attach routine? Many > drivers use BUS_DMA_NOWAIT. Yes, it is ok to use in an attach routine. > I need a 32Ko buffer and on low memory my driver cannot attach > (bus_dmamem_alloc returns ENOMEM). Will BUS_DMA_WAITOK help in this > case? I'm not sure if it will really help, though. You might end up blocking forever (or a long time) until a 32k region is freed. -- John Baldwin From kpielorz_lst at tdx.co.uk Fri Feb 13 10:14:54 2009 From: kpielorz_lst at tdx.co.uk (Karl Pielorz) Date: Fri Feb 13 10:15:01 2009 Subject: Tyan S2895 7.1 amd64 >4Gb RAM support? Message-ID: <12320CD678FB9B76CA7A29F1@Octa64> Hi, I've a Tyan S2895 (bios 1.04), w/10Gb of ECC RAM onboard using 2 * Opteron 285's. The machine used to run WinXP x64, and Vista x64 (mostly doing video production, ray tracing etc.) I recently switched this machine to FreeBSD 7.1 amd64 - to run ZFS on it, but I've been having horrific problems with it. Basically, with more than ~3Gb of RAM usable in the system - it shows signs of chronic RAM problems (everything from sig11's through to failing to compile the kernel with 'weird' errors - as well as kernel panics, and spontaneous reboots). I've tested all the RAM (ECC is enabled on the BIOS) - it all tests fine (even if I jumble it up between different simms in different sockets etc.) By setting: hw.physmem="3G" In loader.conf - I get a stable system. I've not setup any ZFS pools or anything yet, until I get the system stable. I've also tried changing the BIOS settings for the Memory Hole, IOMMU, MTRR etc. - all to no avail (nor does a BIOS 'use safe defaults' make any difference). It boots off the onboard nVidia RAID controller (a pair of 36Gb drives configured as RAID1), this shows up as: " atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1400-0x140f at device 6.0 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] atapci1: port 0x1440-0x1447,0x1434-0x1437,0x1438-0x143f,0x1430-0x1433,0x1410-0x141f mem 0xc0002000-0xc0002fff irq 22 at device 7.0 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] atapci2: port 0x1458-0x145f,0x144c-0x144f,0x1450-0x1457,0x1448-0x144b,0x1420-0x142f mem 0xc0003000-0xc0003fff irq 23 at device 8.0 on pci0 atapci2: [ITHREAD] ata4: on atapci2 ata4: [ITHREAD] ata5: on atapci2 ata5: [ITHREAD] ... ad8: 35304MB at ata4-master SATA150 ad10: 35304MB at ata5-master SATA150 ... ar0: 35304MB status: READY ar0: disk0 READY (master) using ad10 at ata5-master ar0: disk1 READY (mirror) using ad8 at ata4-master Trying to mount root from ufs:/dev/ar0s1a " Anyone got any ideas? - At this time, I can't prove 100% whether it's the disk controller messing up and corrupting data as it's loaded into RAM, or data getting corrupt once in RAM that's causing the crashes - all I know is with ~3Gb RAM - either by physically pulling SIMMs or using the hw.physmem option - it works fine. I tried booting 8.0-CURRENT-200812-amd64-disc1.iso - to see if anything was different with this hardware in 8.0 - but unfortunately that doesn't boot past the BTX loader on this machine, regardless of how much RAM is / isn't in it :( -Kp From rdivacky at freebsd.org Fri Feb 13 10:53:46 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Fri Feb 13 10:53:52 2009 Subject: TUNABLE_INT question Message-ID: <20090213183229.GA94272@freebsd.org> hi #define TUNABLE_INT(path, var) \ static struct tunable_int __CONCAT(__tunable_int_, __LINE__) = { \ (path), \ (var), \ }; \ SYSINIT(__CONCAT(__Tunable_init_, __LINE__), \ SI_SUB_TUNABLES, SI_ORDER_MIDDLE, tunable_int_init, \ &__CONCAT(__tunable_int_, __LINE__)) this is the code for TUNABLE_INT macro, the first param to SYSINIT is an "uniquifier" which is "__Tunable_init_X" where is is a number specifying line. if I read it correctly there is no protection from collisions, am I right? is it a proper fix to concat __FILE__ there too? thnx roman -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090213/3d5c0ecc/attachment.pgp From max at love2party.net Fri Feb 13 11:08:40 2009 From: max at love2party.net (Max Laier) Date: Fri Feb 13 11:08:48 2009 Subject: Tyan S2895 7.1 amd64 >4Gb RAM support? In-Reply-To: <12320CD678FB9B76CA7A29F1@Octa64> References: <12320CD678FB9B76CA7A29F1@Octa64> Message-ID: <200902132008.38110.max@love2party.net> On Friday 13 February 2009 19:00:31 Karl Pielorz wrote: > Hi, > > I've a Tyan S2895 (bios 1.04), w/10Gb of ECC RAM onboard using 2 * Opteron > 285's. The machine used to run WinXP x64, and Vista x64 (mostly doing video > production, ray tracing etc.) > > I recently switched this machine to FreeBSD 7.1 amd64 - to run ZFS on it, > but I've been having horrific problems with it. > > Basically, with more than ~3Gb of RAM usable in the system - it shows signs > of chronic RAM problems (everything from sig11's through to failing to > compile the kernel with 'weird' errors - as well as kernel panics, and > spontaneous reboots). > > I've tested all the RAM (ECC is enabled on the BIOS) - it all tests fine > (even if I jumble it up between different simms in different sockets etc.) > > By setting: > > hw.physmem="3G" > > In loader.conf - I get a stable system. > > I've not setup any ZFS pools or anything yet, until I get the system > stable. I've also tried changing the BIOS settings for the Memory Hole, > IOMMU, MTRR etc. - all to no avail (nor does a BIOS 'use safe defaults' > make any difference). I have a S2882-D populated with 6GB (6x1G) running FreeBSD since early 7. It took some tuning of the memory timing in the BIOS to get it to work, but ever since it works like a charm. > It boots off the onboard nVidia RAID controller (a pair of 36Gb drives > configured as RAID1), this shows up as: Can you maybe try to take the nVidia RAID out of the equation? I figure the "professional" version of the chip is not that common so maybe the corruption stems from the disk controller. > " > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1400-0x140f at device 6.0 on pci0 > ata0: on atapci0 > ata0: [ITHREAD] > ata1: on atapci0 > ata1: [ITHREAD] > atapci1: port > 0x1440-0x1447,0x1434-0x1437,0x1438-0x143f,0x1430-0x1433,0x1410-0x141f mem > 0xc0002000-0xc0002fff irq 22 at device 7.0 on pci0 > atapci1: [ITHREAD] > ata2: on atapci1 > ata2: [ITHREAD] > ata3: on atapci1 > ata3: [ITHREAD] > atapci2: port > 0x1458-0x145f,0x144c-0x144f,0x1450-0x1457,0x1448-0x144b,0x1420-0x142f mem > 0xc0003000-0xc0003fff irq 23 at device 8.0 on pci0 > atapci2: [ITHREAD] > ata4: on atapci2 > ata4: [ITHREAD] > ata5: on atapci2 > ata5: [ITHREAD] > ... > ad8: 35304MB at ata4-master SATA150 > ad10: 35304MB at ata5-master SATA150 > ... > ar0: 35304MB status: READY > ar0: disk0 READY (master) using ad10 at ata5-master > ar0: disk1 READY (mirror) using ad8 at ata4-master > Trying to mount root from ufs:/dev/ar0s1a > " > > Anyone got any ideas? - At this time, I can't prove 100% whether it's the > disk controller messing up and corrupting data as it's loaded into RAM, or > data getting corrupt once in RAM that's causing the crashes - all I know is > with ~3Gb RAM - either by physically pulling SIMMs or using the hw.physmem > option - it works fine. > > I tried booting 8.0-CURRENT-200812-amd64-disc1.iso - to see if anything was > different with this hardware in 8.0 - but unfortunately that doesn't boot > past the BTX loader on this machine, regardless of how much RAM is / isn't > in it :( Any more details on how it fails would help. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From rysto32 at gmail.com Fri Feb 13 13:28:50 2009 From: rysto32 at gmail.com (Ryan Stone) Date: Fri Feb 13 13:29:03 2009 Subject: TUNABLE_INT question In-Reply-To: <20090213183229.GA94272@freebsd.org> References: <20090213183229.GA94272@freebsd.org> Message-ID: __FILE__ is a string so you can't concat that with anything to produce an identifier. In any case, the variable is static so there can't be any collision problems with other files. Ryan Stone From rdivacky at freebsd.org Fri Feb 13 14:19:05 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Fri Feb 13 14:19:12 2009 Subject: TUNABLE_INT question In-Reply-To: References: <20090213183229.GA94272@freebsd.org> Message-ID: <20090213221607.GA25161@freebsd.org> On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > __FILE__ is a string so you can't concat that with anything to produce an > identifier. In any case, the variable is static so there can't be any > collision problems with other files. I was talking about the SYSINIT parameter. thats a section in a .o file, and I am getting collisions there... From lists at yamagi.org Sat Feb 14 01:53:27 2009 From: lists at yamagi.org (Yamagi Burmeister) Date: Sat Feb 14 01:53:38 2009 Subject: Tyan S2895 7.1 amd64 >4Gb RAM support? In-Reply-To: <12320CD678FB9B76CA7A29F1@Octa64> References: <12320CD678FB9B76CA7A29F1@Octa64> Message-ID: <20090214092248.GA1281@yamagi.org> Am Fri, Feb 13, 2009 at 06:00:31PM +0000 schrieb Karl Pielorz: > > Hi, > > I've a Tyan S2895 (bios 1.04), w/10Gb of ECC RAM onboard using 2 * Opteron > 285's. The machine used to run WinXP x64, and Vista x64 (mostly doing video > production, ray tracing etc.) > > I recently switched this machine to FreeBSD 7.1 amd64 - to run ZFS on it, > but I've been having horrific problems with it. Hello, I'm using some S2895 for various purposes, as webservers, databaseservers and workstations. Most of them are populated using 2 * Opteron 265 in stepping E2. They work like a charm _after_ upgrading the BIOS to version 1.05e_beta or later[1]. But it's necessary to use /boot/loader from 7.0 since later version won't work to some bios-bugs. Ciao, Yamagi 1: http://www.tyan.de/support_download_bios.aspx?model=S.S2895 -- Homepage: www.yamagi.org Jabber: yamagi@yamagi.org GnuPG/GPG: 0xEFBCCBCB -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090214/2acf356e/attachment.pgp From zbeeble at gmail.com Sat Feb 14 11:25:41 2009 From: zbeeble at gmail.com (Zaphod Beeblebrox) Date: Sat Feb 14 11:25:47 2009 Subject: When does the pool get bigger? Message-ID: <5f67a8c40902141100w406b0a73h7cf487369e15ec8f@mail.gmail.com> I have a ZFS raid-Z array (FreeBSD-7.1p2) that I use for storing backups and media. I'm keenly awaiting the MFC of the ZFS v13 code, but I'm not in a hurry to run -CURRENT on this box. Anyways... The array was 5x 750G drives and I decided to upgrade to 5x 1.5T drives. I removed one 750G drive and inserted a 1.5T drive each time. All 5 are done resilvering now. When does the pool get bigger? The resilver of the last drive has finished, but the pool still reads [1:20:320]root@virtual:/usr/local/etc> zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT vr2 3.41T 3.16T 251G 92% ONLINE - ... which is the size with 750G drives. From ticso at cicely7.cicely.de Sat Feb 14 12:58:30 2009 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Sat Feb 14 12:58:38 2009 Subject: When does the pool get bigger? In-Reply-To: <5f67a8c40902141100w406b0a73h7cf487369e15ec8f@mail.gmail.com> References: <5f67a8c40902141100w406b0a73h7cf487369e15ec8f@mail.gmail.com> Message-ID: <20090214203919.GV84964@cicely7.cicely.de> On Sat, Feb 14, 2009 at 02:00:23PM -0500, Zaphod Beeblebrox wrote: > I have a ZFS raid-Z array (FreeBSD-7.1p2) that I use for storing backups and > media. I'm keenly awaiting the MFC of the ZFS v13 code, but I'm not in a > hurry to run -CURRENT on this box. > > Anyways... The array was 5x 750G drives and I decided to upgrade to 5x 1.5T > drives. I removed one 750G drive and inserted a 1.5T drive each time. All > 5 are done resilvering now. > > When does the pool get bigger? The resilver of the last drive has finished, > but the pool still reads > > [1:20:320]root@virtual:/usr/local/etc> zpool list > NAME SIZE USED AVAIL CAP HEALTH ALTROOT > vr2 3.41T 3.16T 251G 92% ONLINE - > > ... which is the size with 750G drives. You need to export/import the pool once. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From kpielorz_lst at tdx.co.uk Sat Feb 14 16:51:43 2009 From: kpielorz_lst at tdx.co.uk (Karl Pielorz) Date: Sat Feb 14 16:51:51 2009 Subject: Tyan S2895 7.1 amd64 >8Gb RAM support? In-Reply-To: <200902132008.38110.max@love2party.net> References: <12320CD678FB9B76CA7A29F1@Octa64> <200902132008.38110.max@love2party.net> Message-ID: <4FF0CBDE2E90DAFEDE59D777@Octa64> --On 13 February 2009 20:08 +0100 Max Laier wrote: > Can you maybe try to take the nVidia RAID out of the equation? I figure > the "professional" version of the chip is not that common so maybe the > corruption stems from the disk controller. Hi, I've tested with both Marvell (PCI-X), and Promise (PCI 32 Bit) SATA controllers now - it makes no difference. I upgraded the BIOS on the machine, and did a CMOS reset, then load factory defaults. I also then slowly upped the hw.physmem setting to see what would happen. I can now get this running at 8Gb [I've changed the email subject accordingly]. Any attempt to go over that (or remove the line from loader.conf completely) and it ends in the previous random crashes, compiler errors (e.g. warnings of internal bugs in gcc) - and occasional sig11's... e.g. >From compiling the kernel one time I got: " mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/src/sys/amd64/compile/GENERIC /usr/src/sys/modules/uslcom/../../dev/usb/uslcom.c ===> utopia (depend) @ -> /usr/src/sys /libexec/ld-elf.so.1: /lib/libc.so.7: Unsupported relocation type 88 in non-PLT relocations " I could probably live with only 8Gb in the machine, but it's going to be running some large ZFS pools (and a number of other tasks) - I'd like to have all 10Gb usable (especially if I move to 8.x eventually - and obviously, as it's physically in there, it'd be good to 'use it') Can anyone think of anything that is likely to break if you go >8Gb? [up from 4Gb since the BIOS was reflashed/reset & factory defaulted]. -Kp From ticso at cicely7.cicely.de Sun Feb 15 05:06:42 2009 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Sun Feb 15 05:06:49 2009 Subject: When does the pool get bigger? In-Reply-To: <4998034D.5020203@louko.com> References: <5f67a8c40902141100w406b0a73h7cf487369e15ec8f@mail.gmail.com> <20090214203919.GV84964@cicely7.cicely.de> <4998034D.5020203@louko.com> Message-ID: <20090215130636.GM84964@cicely7.cicely.de> On Sun, Feb 15, 2009 at 01:58:05PM +0200, Antti Louko wrote: > Bernd Walter wrote: > > On Sat, Feb 14, 2009 at 02:00:23PM -0500, Zaphod Beeblebrox wrote: > > >> When does the pool get bigger? The resilver of the last drive has finished, > >> but the pool still reads > > >> ... which is the size with 750G drives. > > > You need to export/import the pool once. > > A related issue. It is probably more of a generic ZFS code base thing, > but what do you think? > > It would be a nice idea to be able to _prevent_ ZFS pool from groving > automatically. This could be a flag in the pool label or anything. This > is not a real issue in FreeBSD at least for me because it is in any case > better to use glabel to label partitions so that the names don't change > between reboots when devices are added and removed. > > But with pool using whole disks this would be useful if one wants to > keep pool at certain size eg. to be able to temporarily use larger disks > and later move back to original-size disks. This wouldn't be an issue at all if the pool could shrink. Since it is on the TODO list I will wait. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From alo-freebsd-lists at louko.com Sun Feb 15 06:10:50 2009 From: alo-freebsd-lists at louko.com (Antti Louko) Date: Sun Feb 15 06:10:57 2009 Subject: When does the pool get bigger? In-Reply-To: <20090214203919.GV84964@cicely7.cicely.de> References: <5f67a8c40902141100w406b0a73h7cf487369e15ec8f@mail.gmail.com> <20090214203919.GV84964@cicely7.cicely.de> Message-ID: <4998034D.5020203@louko.com> Bernd Walter wrote: > On Sat, Feb 14, 2009 at 02:00:23PM -0500, Zaphod Beeblebrox wrote: >> When does the pool get bigger? The resilver of the last drive has finished, >> but the pool still reads >> ... which is the size with 750G drives. > You need to export/import the pool once. A related issue. It is probably more of a generic ZFS code base thing, but what do you think? It would be a nice idea to be able to _prevent_ ZFS pool from groving automatically. This could be a flag in the pool label or anything. This is not a real issue in FreeBSD at least for me because it is in any case better to use glabel to label partitions so that the names don't change between reboots when devices are added and removed. But with pool using whole disks this would be useful if one wants to keep pool at certain size eg. to be able to temporarily use larger disks and later move back to original-size disks. From jkoshy at FreeBSD.org Sun Feb 15 06:33:46 2009 From: jkoshy at FreeBSD.org (Joseph Koshy) Date: Sun Feb 15 06:34:20 2009 Subject: pahole - Finding holes in kernel structs In-Reply-To: References: <200902121717.47841.max@love2party.net> <4994516B.8060703@freebsd.org> <200902121754.24536.max@love2party.net> Message-ID: <86skmfztrm.wl%koshy@unixconsulting.co.in> > Well, I think we want to inform this through actual measurement. Right now, > tools like hwpmc track cache misses by point in executable code, but what > would be nice is if we could post-process to generate cache miss information > by data structure field... That is one of the tools that I've had in mind. The hard part in the process is tracking sample locations back to source artifacts with acceptable accuracy. I couldn't find existing libraries with a compatible open-source license that I could build on, and thus the motivation for some of the libraries listed in the roadmap at http://elftoolchain.sourceforge.net/. Also, hwpmc(4) would need to be augmented to use "precise" sampling on the hardware that supports it, but this is relatively straightforward. Koshy From avg at icyb.net.ua Mon Feb 16 10:26:51 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Feb 16 10:26:58 2009 Subject: booting/loading a tool, not kernel Message-ID: <4999AFE7.9070001@icyb.net.ua> I'd like to add an entry to my loader menu to load/execute memtest86 instead of a kernel. It is built as standalone ELF executable, of course. Currently I go to a loader prompt and do: > unload > load /boot/memtest86 > boot Where should I look to get this into the forth code? Suppose I want to extend this approach. What would be a good name for subdirectory to put such tools? What would be a good way to add a sub-menu for such tools? Also, my current approach means that loader still first loads a standard kernel and its modules and then replaces them with memetest86. How to get to a loader menu without pre-loading anything? -- Andriy Gapon From nick at van-laarhoven.org Tue Feb 17 02:37:35 2009 From: nick at van-laarhoven.org (Nick Hibma) Date: Tue Feb 17 02:37:42 2009 Subject: booting/loading a tool, not kernel Message-ID: <200902171125.24160.nick@van-laarhoven.org> You are aware of nextboot(8)? That you could use to specify the 'kernel' to load on next boot. Also boot.config(5) might be of use. Nick > I'd like to add an entry to my loader menu to load/execute memtest86 > instead of a kernel. It is built as standalone ELF executable, of course. > > Currently I go to a loader prompt and do: > > unload > > load /boot/memtest86 > > boot > > Where should I look to get this into the forth code? > > Suppose I want to extend this approach. > What would be a good name for subdirectory to put such tools? > What would be a good way to add a sub-menu for such tools? > > Also, my current approach means that loader still first loads a standard > kernel and its modules and then replaces them with memetest86. How to > get to a loader menu without pre-loading anything? From avg at icyb.net.ua Tue Feb 17 03:16:40 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue Feb 17 03:16:46 2009 Subject: booting/loading a tool, not kernel In-Reply-To: <200902171125.24160.nick@van-laarhoven.org> References: <200902171125.24160.nick@van-laarhoven.org> Message-ID: <499A9C47.4070602@icyb.net.ua> on 17/02/2009 12:25 Nick Hibma said the following: > You are aware of nextboot(8)? That you could use to specify the 'kernel' to > load on next boot. > > Also boot.config(5) might be of use. Thanks a lot, these are very useful. But I am still thinking about an interactive menu where I could e.g. press '9' to go to "Utilities" and then could select memtest86 or memtest2009 or cputest :-) >> I'd like to add an entry to my loader menu to load/execute memtest86 >> instead of a kernel. It is built as standalone ELF executable, of course. >> >> Currently I go to a loader prompt and do: >>> unload >>> load /boot/memtest86 >>> boot >> Where should I look to get this into the forth code? >> >> Suppose I want to extend this approach. >> What would be a good name for subdirectory to put such tools? >> What would be a good way to add a sub-menu for such tools? >> >> Also, my current approach means that loader still first loads a standard >> kernel and its modules and then replaces them with memetest86. How to >> get to a loader menu without pre-loading anything? > > -- Andriy Gapon From avg at icyb.net.ua Tue Feb 17 04:08:26 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue Feb 17 04:08:34 2009 Subject: booting/loading a tool, not kernel In-Reply-To: <200902172230.25492.doconnor@gsoft.com.au> References: <200902171125.24160.nick@van-laarhoven.org> <499A9C47.4070602@icyb.net.ua> <200902172230.25492.doconnor@gsoft.com.au> Message-ID: <499AA897.3020703@icyb.net.ua> on 17/02/2009 14:00 Daniel O'Connor said the following: > On Tuesday 17 February 2009 21:45:19 Andriy Gapon wrote: >> on 17/02/2009 12:25 Nick Hibma said the following: >>> You are aware of nextboot(8)? That you could use to specify the 'kernel' >>> to load on next boot. >>> >>> Also boot.config(5) might be of use. >> Thanks a lot, these are very useful. >> But I am still thinking about an interactive menu where I could e.g. >> press '9' to go to "Utilities" and then could select memtest86 or >> memtest2009 or cputest :-) > > That would be very handy IMO. > > What file do you use to run memtest etc? > Stephan Eisvogel shared his work on memtest86+ for FreeBSD. https://www.seitics.de/?e=47 I am now trying to make a minimalistic port out of his work. I mean - all the features that he added are very useful, but I want to get "official" memtest86+ and/or memtest86 ported with minimal changes. -- Andriy Gapon From doconnor at gsoft.com.au Tue Feb 17 04:35:04 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Tue Feb 17 04:35:12 2009 Subject: booting/loading a tool, not kernel In-Reply-To: <499A9C47.4070602@icyb.net.ua> References: <200902171125.24160.nick@van-laarhoven.org> <499A9C47.4070602@icyb.net.ua> Message-ID: <200902172230.25492.doconnor@gsoft.com.au> On Tuesday 17 February 2009 21:45:19 Andriy Gapon wrote: > on 17/02/2009 12:25 Nick Hibma said the following: > > You are aware of nextboot(8)? That you could use to specify the 'kernel' > > to load on next boot. > > > > Also boot.config(5) might be of use. > > Thanks a lot, these are very useful. > But I am still thinking about an interactive menu where I could e.g. > press '9' to go to "Utilities" and then could select memtest86 or > memtest2009 or cputest :-) That would be very handy IMO. What file do you use to run memtest etc? -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090217/455c730e/attachment.pgp From ccowart at rescomp.berkeley.edu Tue Feb 17 10:29:34 2009 From: ccowart at rescomp.berkeley.edu (Chris Cowart) Date: Tue Feb 17 10:29:41 2009 Subject: Kernel Crashes Message-ID: <20090217181347.GC94579@hal.rescomp.berkeley.edu> Hello, We have a system that has been experiencing 3-5 crashes per day for about 2 weeks now. The affected machine started as a 7.0 virtual machine cloned from our staging build. The production machine is running apache with mod_proxy and mongrel_cluster for a ruby on rails webapp. Shortly after rolling into production, the crashes began. All of the crashes have been: fault code = supervisor read, page not present But there appears to be no rhyme or reason to the current process or current syscall (to my untrained eyes, at least). We also tried building a 7.1 vm from scratch, installing all the software, and rolling it out last night. Today, it crashed too. We are fairly certain this isn't a hardware problem, because we have many other FreeBSD vms running successfully on this ESX server. Here's today's 7.1 backtrace: | root optimus crash # kgdb /boot/kernel/kernel vmcore.0 | GNU gdb 6.1.1 [FreeBSD] | Copyright 2004 Free Software Foundation, Inc. | GDB is free software, covered by the GNU General Public License, and you are | welcome to change it and/or distribute copies of it under certain conditions. | Type "show copying" to see the conditions. | There is absolutely no warranty for GDB. Type "show warranty" for details. | This GDB was configured as "i386-marcel-freebsd"... | | Unread portion of the kernel message buffer: | | | Fatal trap 12: page fault while in kernel mode | cpuid = 0; apic id = 00 | fault virtual address = 0xc526b305 | fault code = supervisor read, page not present | instruction pointer = 0x20:0xc07e95b5 | stack pointer = 0x28:0xcd08fb18 | frame pointer = 0x28:0xcd08fb44 | code segment = base 0x0, limit 0xfffff, type 0x1b | = DPL 0, pres 1, def32 1, gran 1 | processor eflags = interrupt enabled, resume, IOPL = 0 | current process = 1158 (ruby18) | trap number = 12 | panic: page fault | cpuid = 0 | Uptime: 7h47m5s | Physical memory: 243 MB | Dumping 70 MB: 55 39 23 7 | | Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done. | done. | Loaded symbols for /boot/kernel/acpi.ko | Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /boot/kernel/ipfw.ko.symbols...done. | done. | Loaded symbols for /boot/kernel/ipfw.ko | #0 doadump () at pcpu.h:196 | 196 pcpu.h: No such file or directory. | in pcpu.h | (kgdb) where | #0 doadump () at pcpu.h:196 | #1 0xc07998c7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 | #2 0xc0799b99 in panic (fmt=Variable "fmt" is not available. | ) at /usr/src/sys/kern/kern_shutdown.c:574 | #3 0xc0ad6a9c in trap_fatal (frame=0xcd08fad8, eva=3307647749) | at /usr/src/sys/i386/i386/trap.c:939 | #4 0xc0ad6d20 in trap_pfault (frame=0xcd08fad8, usermode=0, eva=3307647749) | at /usr/src/sys/i386/i386/trap.c:852 | #5 0xc0ad76dc in trap (frame=0xcd08fad8) at /usr/src/sys/i386/i386/trap.c:530 | #6 0xc0abd54b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 | #7 0xc07e95b5 in m_uiotombuf (uio=0xcd08fbe8, how=2, len=2048, align=76, | flags=2) at /usr/src/sys/kern/uipc_mbuf.c:1747 | #8 0xc07f16d5 in sosend_generic (so=0xc24d2680, addr=0x0, uio=0xcd08fbe8, | top=0x0, control=0x0, flags=0, td=0xc23a6000) | at /usr/src/sys/kern/uipc_socket.c:1219 | #9 0xc07ed2ff in sosend (so=0xc24d2680, addr=0x0, uio=0xcd08fbe8, top=0x0, | control=0x0, flags=0, td=0xc23a6000) | at /usr/src/sys/kern/uipc_socket.c:1288 | #10 0xc07f4606 in kern_sendit (td=0xc23a6000, s=5, mp=0xcd08fc64, flags=0, | control=0x0, segflg=UIO_USERSPACE) at /usr/src/sys/kern/uipc_syscalls.c:805 | #11 0xc07f7851 in sendit (td=0xc23a6000, s=5, mp=0xcd08fc64, flags=0) | at /usr/src/sys/kern/uipc_syscalls.c:742 | #12 0xc07f7968 in sendto (td=0xc23a6000, uap=0xcd08fcfc) | at /usr/src/sys/kern/uipc_syscalls.c:857 | #13 0xc0ad7075 in syscall (frame=0xcd08fd38) | at /usr/src/sys/i386/i386/trap.c:1090 | #14 0xc0abd5b0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255 | #15 0x00000033 in ?? () | Previous frame inner to this frame (corrupt stack?) And here's a previous 7.0 backtrace: | ccowart hal optimus $ cat vmcore.1.bt | GNU gdb 6.1.1 [FreeBSD] | Copyright 2004 Free Software Foundation, Inc. | GDB is free software, covered by the GNU General Public License, and you are | welcome to change it and/or distribute copies of it under certain conditions. | Type "show copying" to see the conditions. | There is absolutely no warranty for GDB. Type "show warranty" for details. | This GDB was configured as "i386-marcel-freebsd". | | Unread portion of the kernel message buffer: | | | Fatal trap 12: page fault while in kernel mode | cpuid = 0; apic id = 00 | fault virtual address = 0x640d6b5d | fault code = supervisor read, page not present | instruction pointer = 0x20:0xc07a9f7b | stack pointer = 0x28:0xcc793a10 | frame pointer = 0x28:0xcc793a20 | code segment = base 0x0, limit 0xfffff, type 0x1b | = DPL 0, pres 1, def32 1, gran 1 | processor eflags = interrupt enabled, resume, IOPL = 0 | current process = 28 (irq18: le0) | trap number = 12 | panic: page fault | cpuid = 0 | Uptime: 17h8m27s | Physical memory: 243 MB | Dumping 98 MB: 83 67 51 35 19 3 | | #0 doadump () at pcpu.h:195 | in pcpu.h | (kgdb) #0 doadump () at pcpu.h:195 | #1 0xc075cf37 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 | #2 0xc075d1f9 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:563 | #3 0xc0a7c6ac in trap_fatal (frame=0xcc7939d0, eva=1678601053) | at /usr/src/sys/i386/i386/trap.c:899 | #4 0xc0a7c930 in trap_pfault (frame=0xcc7939d0, usermode=0, eva=1678601053) | at /usr/src/sys/i386/i386/trap.c:812 | #5 0xc0a7d2dc in trap (frame=0xcc7939d0) at /usr/src/sys/i386/i386/trap.c:490 | #6 0xc0a6325b in calltrap () at /usr/src/sys/i386/i386/exception.s:139 | #7 0xc07a9f7b in m_tag_delete_chain (m=0xc49da000, t=0x0) | at /usr/src/sys/kern/uipc_mbuf2.c:355 | #8 0xc074ef05 in mb_dtor_mbuf (mem=0xc49da000, size=256, arg=) | at /usr/src/sys/kern/kern_mbuf.c:409 | #9 0xc099a1ef in uma_zfree_arg (zone=0xc1466d20, item=0xc49da000, udata=0x0) | at /usr/src/sys/vm/uma_core.c:2255 | #10 0xc07ab683 in sbdrop_internal (sb=0xc26eda24, len=122) at uma.h:305 | #11 0xc07ab77e in sbdrop_locked (sb=0xc26eda24, len=122) | at /usr/src/sys/kern/uipc_sockbuf.c:899 | #12 0xc0899c9b in tcp_do_segment (m=0xc244c800, th=0xc36ba024, so=0xc26ed948, | tp=0xc25051d0, drop_hdrlen=52, tlen=666) | at /usr/src/sys/netinet/tcp_input.c:2031 | #13 0xc089b501 in tcp_input (m=0xc244c800, off0=20) | at /usr/src/sys/netinet/tcp_input.c:845 | #14 0xc083cb59 in ip_input (m=0xc244c800) | at /usr/src/sys/netinet/ip_input.c:665 | #15 0xc07fc945 in netisr_dispatch (num=2, m=0xc244c800) | at /usr/src/sys/net/netisr.c:185 | #16 0xc07f2981 in ether_demux (ifp=0xc2227000, m=0xc244c800) | at /usr/src/sys/net/if_ethersubr.c:834 | #17 0xc07f2d73 in ether_input (ifp=0xc2227000, m=0xc244c800) | at /usr/src/sys/net/if_ethersubr.c:692 | #18 0xc05e520c in am79900_intr (arg=0xc2183000) | at /usr/src/sys/dev/le/am79900.c:340 | #19 0xc074015b in ithread_loop (arg=0xc22266f0) | at /usr/src/sys/kern/kern_intr.c:1036 | #20 0xc073cf59 in fork_exit (callout=0xc073ffb0 , | arg=0xc22266f0, frame=0xcc793d38) at /usr/src/sys/kern/kern_fork.c:781 | #21 0xc0a632d0 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:205 And one more from 7.0, just for show: | ccowart hal optimus $ cat vmcore.3.bt | GNU gdb 6.1.1 [FreeBSD] | Copyright 2004 Free Software Foundation, Inc. | GDB is free software, covered by the GNU General Public License, and you are | welcome to change it and/or distribute copies of it under certain conditions. | Type "show copying" to see the conditions. | There is absolutely no warranty for GDB. Type "show warranty" for details. | This GDB was configured as "i386-marcel-freebsd". | | Unread portion of the kernel message buffer: | | | Fatal trap 12: page fault while in kernel mode | cpuid = 0; apic id = 00 | fault virtual address = 0xf98ca8a2 | fault code = supervisor read, page not present | instruction pointer = 0x20:0xc07a82b4 | stack pointer = 0x28:0xcd057248 | frame pointer = 0x28:0xcd057274 | code segment = base 0x0, limit 0xfffff, type 0x1b | = DPL 0, pres 1, def32 1, gran 1 | processor eflags = interrupt enabled, resume, IOPL = 0 | current process = 1032 (syslog-ng) | trap number = 12 | panic: page fault | cpuid = 0 | Uptime: 1d21h31m10s | Physical memory: 243 MB | Dumping 68 MB: 52 36 20 4 | | #0 doadump () at pcpu.h:195 | in pcpu.h | (kgdb) #0 doadump () at pcpu.h:195 | #1 0xc075cf37 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 | #2 0xc075d1f9 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:563 | #3 0xc0a7c6ac in trap_fatal (frame=0xcd057208, eva=4186744994) | at /usr/src/sys/i386/i386/trap.c:899 | #4 0xc0a7c930 in trap_pfault (frame=0xcd057208, usermode=0, eva=4186744994) | at /usr/src/sys/i386/i386/trap.c:812 | #5 0xc0a7d2dc in trap (frame=0xcd057208) at /usr/src/sys/i386/i386/trap.c:490 | #6 0xc0a6325b in calltrap () at /usr/src/sys/i386/i386/exception.s:139 | #7 0xc07a82b4 in m_copym (m=0xf98ca88e, off0=1500, len=16, wait=1) | at mbuf.h:454 | #8 0xc083df55 in ip_fragment (ip=0xc2310434, m_frag=0xcd057344, mtu=1500, | if_hwassist_flags=0, sw_csum=1) at /usr/src/sys/netinet/ip_output.c:726 | #9 0xc083ec2e in ip_output (m=0xc2310400, opt=0x0, ro=0xcd057308, flags=2, | imo=0x0, inp=0x0) at /usr/src/sys/netinet/ip_output.c:565 | #10 0xc08d711d in ipsec_process_done (m=0xc2310400, isr=0xc2435080) | at /usr/src/sys/netipsec/ipsec_output.c:177 | #11 0xc08e49a5 in ah_output_cb (crp=0xc2497528) | at /usr/src/sys/netipsec/xform_ah.c:1193 | #12 0xc0924330 in crypto_done (crp=0xc2497528) | at /usr/src/sys/opencrypto/crypto.c:1148 | #13 0xc092773e in swcr_process (dev=0xc21f5a00, crp=0xc2497528, hint=0) | at /usr/src/sys/opencrypto/cryptosoft.c:975 | #14 0xc0925376 in crypto_invoke (cap=0xc21f5a00, crp=0xc2497528, hint=0) | at cryptodev_if.h:53 | #15 0xc0925dbc in crypto_dispatch (crp=0xc2497528) | at /usr/src/sys/opencrypto/crypto.c:798 | #16 0xc08e504f in ah_output (m=0xc2310400, isr=0xc2435080, mp=0x0, skip=20, | protoff=9) at /usr/src/sys/netipsec/xform_ah.c:1102 | #17 0xc08d743b in ipsec4_process_packet (m=0xc24d4a00, isr=0xc2435080, | flags=0, tunalready=0) at /usr/src/sys/netipsec/ipsec_output.c:486 | #18 0xc08d70ae in ipsec_process_done (m=0xc24d4a00, isr=0xc2435100) | at /usr/src/sys/netipsec/ipsec_output.c:160 | #19 0xc08e629c in esp_output_cb (crp=0xc24975a0) | at /usr/src/sys/netipsec/xform_esp.c:965 | #20 0xc0924330 in crypto_done (crp=0xc24975a0) | at /usr/src/sys/opencrypto/crypto.c:1148 | #21 0xc092773e in swcr_process (dev=0xc21f5a00, crp=0xc24975a0, hint=0) | at /usr/src/sys/opencrypto/cryptosoft.c:975 | #22 0xc0925376 in crypto_invoke (cap=0xc21f5a00, crp=0xc24975a0, hint=0) | at cryptodev_if.h:53 | #23 0xc0925dbc in crypto_dispatch (crp=0xc24975a0) | at /usr/src/sys/opencrypto/crypto.c:798 | #24 0xc08e6906 in esp_output (m=0xc24d4a00, isr=0xc2435100, mp=0x0, skip=20, | protoff=9) at /usr/src/sys/netipsec/xform_esp.c:875 | #25 0xc08d743b in ipsec4_process_packet (m=0xc24d4a00, isr=0xc2435100, | flags=0, tunalready=0) at /usr/src/sys/netipsec/ipsec_output.c:486 | #26 0xc083cee3 in ip_ipsec_output (m=0xcd057af8, inp=0xc23d0000, | flags=0xcd057b04, error=0xcd057ad8, ro=0xcd057b00, iproute=0xcd057abc, | dst=0xcd057ad4, ia=0xcd057ad0, ifp=0xcd057ae0) | at /usr/src/sys/netinet/ip_ipsec.c:331 | #27 0xc083e9af in ip_output (m=0xc24d4a00, opt=0x0, ro=0xcd057abc, flags=0, | imo=0x0, inp=0xc23d0000) at /usr/src/sys/netinet/ip_output.c:418 | #28 0xc08a85b3 in udp_send (so=0xc24a1000, flags=0, m=0xc24d4a00, addr=0x0, | control=0x0, td=0xc2337420) at /usr/src/sys/netinet/udp_usrreq.c:972 | #29 0xc07af576 in sosend_dgram (so=0xc24a1000, addr=0x0, uio=0xcd057c60, | top=0xc24d4a00, control=0x0, flags=) | at /usr/src/sys/kern/uipc_socket.c:1053 | #30 0xc07ac41f in sosend (so=0xc24a1000, addr=0x0, uio=0xcd057c60, top=0x0, | control=0x0, flags=0, td=0xc2337420) | at /usr/src/sys/kern/uipc_socket.c:1286 | #31 0xc0796a1b in soo_write (fp=0xc2354e58, uio=0xcd057c60, | active_cred=0xc20f8200, flags=0, td=0xc2337420) | at /usr/src/sys/kern/sys_socket.c:103 | #32 0xc07900c7 in dofilewrite (td=0xc2337420, fd=8, fp=0xc2354e58, | auio=0xcd057c60, offset=-1, flags=0) at file.h:254 | #33 0xc07903a8 in kern_writev (td=0xc2337420, fd=8, auio=0xcd057c60) | at /usr/src/sys/kern/sys_generic.c:401 | #34 0xc079041f in write (td=0xc2337420, uap=0xcd057cfc) | at /usr/src/sys/kern/sys_generic.c:317 | #35 0xc0a7cc85 in syscall (frame=0xcd057d38) | at /usr/src/sys/i386/i386/trap.c:1035 | #36 0xc0a632c0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:196 | #37 0x00000033 in ?? () Our custom kernel config is: | include GENERIC | ident RCBSD_REL7 | options IPSEC | options IPSEC_FILTERTUNNEL | device crypto | options QUOTA I'm pretty sure the kernel isn't supposed to crash, so any tips on fixing whatever it is that's broken here? -- Chris Cowart Network Technical Lead Network & Infrastructure Services, RSSP-IT UC Berkeley -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 834 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090217/2126dd10/attachment.pgp From jhb at freebsd.org Tue Feb 17 14:23:23 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Feb 17 14:23:35 2009 Subject: TUNABLE_INT question In-Reply-To: <20090213221607.GA25161@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <20090213221607.GA25161@freebsd.org> Message-ID: <200902170931.12983.jhb@freebsd.org> On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > __FILE__ is a string so you can't concat that with anything to produce an > > identifier. In any case, the variable is static so there can't be any > > collision problems with other files. > > I was talking about the SYSINIT parameter. thats a section in a .o > file, and I am getting collisions there... Hmm, are you doing something like this: #define FOO(string) \ TUNABLE_INT(string ## ".bar", &bar); \ TUNABLE_INT(string ## ".foo", &foo); \ FOO(baz) That would collide as both of the TUNABLE_INT() invocations would have the same __LINE__ (the line number of the 'FOO(baz)'). -- John Baldwin From jhb at freebsd.org Tue Feb 17 14:23:23 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Feb 17 14:23:36 2009 Subject: TUNABLE_INT question In-Reply-To: <20090213221607.GA25161@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <20090213221607.GA25161@freebsd.org> Message-ID: <200902170931.12983.jhb@freebsd.org> On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > __FILE__ is a string so you can't concat that with anything to produce an > > identifier. In any case, the variable is static so there can't be any > > collision problems with other files. > > I was talking about the SYSINIT parameter. thats a section in a .o > file, and I am getting collisions there... Hmm, are you doing something like this: #define FOO(string) \ TUNABLE_INT(string ## ".bar", &bar); \ TUNABLE_INT(string ## ".foo", &foo); \ FOO(baz) That would collide as both of the TUNABLE_INT() invocations would have the same __LINE__ (the line number of the 'FOO(baz)'). -- John Baldwin From rdivacky at freebsd.org Tue Feb 17 14:24:44 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Tue Feb 17 14:24:52 2009 Subject: TUNABLE_INT question In-Reply-To: <200902170931.12983.jhb@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <20090213221607.GA25161@freebsd.org> <200902170931.12983.jhb@freebsd.org> Message-ID: <20090217222142.GA94925@freebsd.org> On Tue, Feb 17, 2009 at 09:31:12AM -0500, John Baldwin wrote: > On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > > __FILE__ is a string so you can't concat that with anything to produce an > > > identifier. In any case, the variable is static so there can't be any > > > collision problems with other files. > > > > I was talking about the SYSINIT parameter. thats a section in a .o > > file, and I am getting collisions there... > > Hmm, are you doing something like this: > > #define FOO(string) \ > TUNABLE_INT(string ## ".bar", &bar); \ > TUNABLE_INT(string ## ".foo", &foo); \ > > FOO(baz) > > That would collide as both of the TUNABLE_INT() invocations would have the > same __LINE__ (the line number of the 'FOO(baz)'). no.. it was just two tunables in two files that happened to end up in the same line. fixed now From rdivacky at freebsd.org Tue Feb 17 14:40:40 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Tue Feb 17 14:40:46 2009 Subject: TUNABLE_INT question In-Reply-To: <200902170931.12983.jhb@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <20090213221607.GA25161@freebsd.org> <200902170931.12983.jhb@freebsd.org> Message-ID: <20090217222142.GA94925@freebsd.org> On Tue, Feb 17, 2009 at 09:31:12AM -0500, John Baldwin wrote: > On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > > __FILE__ is a string so you can't concat that with anything to produce an > > > identifier. In any case, the variable is static so there can't be any > > > collision problems with other files. > > > > I was talking about the SYSINIT parameter. thats a section in a .o > > file, and I am getting collisions there... > > Hmm, are you doing something like this: > > #define FOO(string) \ > TUNABLE_INT(string ## ".bar", &bar); \ > TUNABLE_INT(string ## ".foo", &foo); \ > > FOO(baz) > > That would collide as both of the TUNABLE_INT() invocations would have the > same __LINE__ (the line number of the 'FOO(baz)'). no.. it was just two tunables in two files that happened to end up in the same line. fixed now From jhb at freebsd.org Tue Feb 17 15:09:44 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Feb 17 15:09:56 2009 Subject: TUNABLE_INT question In-Reply-To: <20090217222142.GA94925@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <200902170931.12983.jhb@freebsd.org> <20090217222142.GA94925@freebsd.org> Message-ID: <200902171751.13962.jhb@freebsd.org> On Tuesday 17 February 2009 5:21:42 pm Roman Divacky wrote: > On Tue, Feb 17, 2009 at 09:31:12AM -0500, John Baldwin wrote: > > On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > > > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > > > __FILE__ is a string so you can't concat that with anything to produce an > > > > identifier. In any case, the variable is static so there can't be any > > > > collision problems with other files. > > > > > > I was talking about the SYSINIT parameter. thats a section in a .o > > > file, and I am getting collisions there... > > > > Hmm, are you doing something like this: > > > > #define FOO(string) \ > > TUNABLE_INT(string ## ".bar", &bar); \ > > TUNABLE_INT(string ## ".foo", &foo); \ > > > > FOO(baz) > > > > That would collide as both of the TUNABLE_INT() invocations would have the > > same __LINE__ (the line number of the 'FOO(baz)'). > > no.. it was just two tunables in two files that happened to end up in the same > line. fixed now Hmmm, odd. Those should be static and not matter. -- John Baldwin From jhb at freebsd.org Tue Feb 17 15:09:44 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Feb 17 15:09:57 2009 Subject: TUNABLE_INT question In-Reply-To: <20090217222142.GA94925@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <200902170931.12983.jhb@freebsd.org> <20090217222142.GA94925@freebsd.org> Message-ID: <200902171751.13962.jhb@freebsd.org> On Tuesday 17 February 2009 5:21:42 pm Roman Divacky wrote: > On Tue, Feb 17, 2009 at 09:31:12AM -0500, John Baldwin wrote: > > On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > > > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > > > __FILE__ is a string so you can't concat that with anything to produce an > > > > identifier. In any case, the variable is static so there can't be any > > > > collision problems with other files. > > > > > > I was talking about the SYSINIT parameter. thats a section in a .o > > > file, and I am getting collisions there... > > > > Hmm, are you doing something like this: > > > > #define FOO(string) \ > > TUNABLE_INT(string ## ".bar", &bar); \ > > TUNABLE_INT(string ## ".foo", &foo); \ > > > > FOO(baz) > > > > That would collide as both of the TUNABLE_INT() invocations would have the > > same __LINE__ (the line number of the 'FOO(baz)'). > > no.. it was just two tunables in two files that happened to end up in the same > line. fixed now Hmmm, odd. Those should be static and not matter. -- John Baldwin From doconnor at gsoft.com.au Tue Feb 17 17:18:16 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Tue Feb 17 17:18:23 2009 Subject: booting/loading a tool, not kernel In-Reply-To: <499AA897.3020703@icyb.net.ua> References: <200902171125.24160.nick@van-laarhoven.org> <200902172230.25492.doconnor@gsoft.com.au> <499AA897.3020703@icyb.net.ua> Message-ID: <200902181148.10206.doconnor@gsoft.com.au> On Tuesday 17 February 2009 22:37:51 Andriy Gapon wrote: > on 17/02/2009 14:00 Daniel O'Connor said the following: > > On Tuesday 17 February 2009 21:45:19 Andriy Gapon wrote: > >> on 17/02/2009 12:25 Nick Hibma said the following: > >>> You are aware of nextboot(8)? That you could use to specify the > >>> 'kernel' to load on next boot. > >>> > >>> Also boot.config(5) might be of use. > >> > >> Thanks a lot, these are very useful. > >> But I am still thinking about an interactive menu where I could e.g. > >> press '9' to go to "Utilities" and then could select memtest86 or > >> memtest2009 or cputest :-) > > > > That would be very handy IMO. > > > > What file do you use to run memtest etc? > > Stephan Eisvogel shared his work on memtest86+ for FreeBSD. > https://www.seitics.de/?e=47 Thanks. > I am now trying to make a minimalistic port out of his work. > I mean - all the features that he added are very useful, but I want to > get "official" memtest86+ and/or memtest86 ported with minimal changes. Yes, it would be very useful IMO. If you have the loader look for extra stuff in some directory under /boot then other things could be added as ports without requiring changes to the loader each time. Although, just a fixed test for memtest would be fantastic :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090218/cef4a900/attachment.pgp From rdivacky at freebsd.org Wed Feb 18 00:02:59 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Wed Feb 18 00:03:12 2009 Subject: TUNABLE_INT question In-Reply-To: <200902171751.13962.jhb@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <200902170931.12983.jhb@freebsd.org> <20090217222142.GA94925@freebsd.org> <200902171751.13962.jhb@freebsd.org> Message-ID: <20090218075955.GA66426@freebsd.org> On Tue, Feb 17, 2009 at 05:51:13PM -0500, John Baldwin wrote: > On Tuesday 17 February 2009 5:21:42 pm Roman Divacky wrote: > > On Tue, Feb 17, 2009 at 09:31:12AM -0500, John Baldwin wrote: > > > On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > > > > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > > > > __FILE__ is a string so you can't concat that with anything to produce > an > > > > > identifier. In any case, the variable is static so there can't be any > > > > > collision problems with other files. > > > > > > > > I was talking about the SYSINIT parameter. thats a section in a .o > > > > file, and I am getting collisions there... > > > > > > Hmm, are you doing something like this: > > > > > > #define FOO(string) \ > > > TUNABLE_INT(string ## ".bar", &bar); \ > > > TUNABLE_INT(string ## ".foo", &foo); \ > > > > > > FOO(baz) > > > > > > That would collide as both of the TUNABLE_INT() invocations would have the > > > same __LINE__ (the line number of the 'FOO(baz)'). > > > > no.. it was just two tunables in two files that happened to end up in the > same > > line. fixed now > > Hmmm, odd. Those should be static and not matter. the symbol itself is static, but the section ELF section created might collide. anyway, this has been fixed and is not a problem anymore for me From rdivacky at freebsd.org Wed Feb 18 00:02:59 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Wed Feb 18 00:03:13 2009 Subject: TUNABLE_INT question In-Reply-To: <200902171751.13962.jhb@freebsd.org> References: <20090213183229.GA94272@freebsd.org> <200902170931.12983.jhb@freebsd.org> <20090217222142.GA94925@freebsd.org> <200902171751.13962.jhb@freebsd.org> Message-ID: <20090218075955.GA66426@freebsd.org> On Tue, Feb 17, 2009 at 05:51:13PM -0500, John Baldwin wrote: > On Tuesday 17 February 2009 5:21:42 pm Roman Divacky wrote: > > On Tue, Feb 17, 2009 at 09:31:12AM -0500, John Baldwin wrote: > > > On Friday 13 February 2009 5:16:07 pm Roman Divacky wrote: > > > > On Fri, Feb 13, 2009 at 03:55:44PM -0500, Ryan Stone wrote: > > > > > __FILE__ is a string so you can't concat that with anything to produce > an > > > > > identifier. In any case, the variable is static so there can't be any > > > > > collision problems with other files. > > > > > > > > I was talking about the SYSINIT parameter. thats a section in a .o > > > > file, and I am getting collisions there... > > > > > > Hmm, are you doing something like this: > > > > > > #define FOO(string) \ > > > TUNABLE_INT(string ## ".bar", &bar); \ > > > TUNABLE_INT(string ## ".foo", &foo); \ > > > > > > FOO(baz) > > > > > > That would collide as both of the TUNABLE_INT() invocations would have the > > > same __LINE__ (the line number of the 'FOO(baz)'). > > > > no.. it was just two tunables in two files that happened to end up in the > same > > line. fixed now > > Hmmm, odd. Those should be static and not matter. the symbol itself is static, but the section ELF section created might collide. anyway, this has been fixed and is not a problem anymore for me From xorquewasp at googlemail.com Wed Feb 18 00:34:28 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Wed Feb 18 00:34:35 2009 Subject: Simulating bad network conditions Message-ID: <20090218083424.GA62198@logik.internal.network> Hello. I want to verify that a program I'm developing works correctly in poor network conditions (high latency, random packet loss, etc). I seem to remember reading something once about somebody simulating these conditions with PF but can't remember where. Anybody got any ideas about how I could go about doing this? Ideally it won't involve starting up a hundred large BitTorrent downloads and randomly ripping out network cables... thanks, xw From maxim at macomnet.ru Wed Feb 18 00:42:02 2009 From: maxim at macomnet.ru (Maxim Konovalov) Date: Wed Feb 18 00:42:09 2009 Subject: Simulating bad network conditions In-Reply-To: <20090218083424.GA62198@logik.internal.network> References: <20090218083424.GA62198@logik.internal.network> Message-ID: <20090218114128.B73352@mp2.macomnet.net> On Wed, 18 Feb 2009, 08:34-0000, xorquewasp@googlemail.com wrote: > Hello. > > I want to verify that a program I'm developing works correctly > in poor network conditions (high latency, random packet loss, etc). > > I seem to remember reading something once about somebody simulating > these conditions with PF but can't remember where. > > Anybody got any ideas about how I could go about doing this? > ipfw(8) prob + dummynet(8). -- Maxim Konovalov From xorquewasp at googlemail.com Wed Feb 18 00:57:19 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Wed Feb 18 00:57:26 2009 Subject: Simulating bad network conditions In-Reply-To: <20090218114128.B73352@mp2.macomnet.net> References: <20090218083424.GA62198@logik.internal.network> <20090218114128.B73352@mp2.macomnet.net> Message-ID: <20090218085715.GA9821@logik.internal.network> On 2009-02-18 11:42:00, Maxim Konovalov wrote: > > ipfw(8) prob + dummynet(8). > Hi. Thanks for the quick response. Is there, by any chance, an equivalent for PF? I see there's 'ALTQ' but it looks to be poorly supported (unless I misunderstand). I have quite a complicated setup here with PF forwarding and jails and I'm not sure how well ipfw will play along. thanks, xw From kpielorz_lst at tdx.co.uk Wed Feb 18 01:01:52 2009 From: kpielorz_lst at tdx.co.uk (Karl Pielorz) Date: Wed Feb 18 01:01:59 2009 Subject: Tyan S2895 7.1 amd64 >8Gb RAM support? - resolved In-Reply-To: <4FF0CBDE2E90DAFEDE59D777@Octa64> References: <12320CD678FB9B76CA7A29F1@Octa64> <200902132008.38110.max@love2party.net> <4FF0CBDE2E90DAFEDE59D777@Octa64> Message-ID: <626FC588D3A7707FC1186924@Works64.dmpriest.net.uk> Hi, I finally resolved this... (posted to the list for completeness / incase someone else hits this issue). Brief Solution: Reduce dram timing in the bios from DDR400 to DDR333. Gory details: Having taken quite a trip through mptables, bioses, pulling / pushing DIMM's etc. - all the DIMM's in the machine are DDR400 rated - all do test OK. Also remember this machine was used for years under WinXP64/Vista (including using things like a 6Gb RAM drive for video production) - if it had been corrupting data as much as under FreeBSD it would have been noticed :( Anyway, it transpires earlier Opterons were either incapable of running DDR400, or had a 'caveat' that when running lots of memory modules they recommended dropping the timing from DDR400 downwards. This was 'apparently' fixed in later Opterons (my system has 2 * E Stepping 285's). To counter this the BIOS would detect which CPU was in use, and drop the maximum memory speed accordingly vs. installed DIMM slots. This restriction was apparently lifted for Opteron 285's (at least for the stepping I'm using) as their onboard memory controller should be capable of driving all memory banks, at DDR400 speeds - the BIOS reflects this by defaulting to DDR400 timing for my system. Obviously, this works on this system under Windows memory usage patterns / loading / stress - but not for FreeBSD's memory usage / loading. With the RAM backed down to DDR333 - the system once again runs flawlessly. RAM tests didn't catch this - as I could only effectively test 4Gb of RAM 'at a time' - I also doubt that the memory usage patterns software such as memtest86 use would 'load' the memory system enough to cause the timing problem. Max Laier did say he'd had to make memory timing adjustments on his S2895 system, but I presumed that with DDR400 rated memory, and the BIOS using the SPD details on the chips (all branded RAM) that it wouldn't be that (and the fact windows gave years of service with the current settings). -Kp [Who now knows far more than I ever wanted to about mptables, APIC, ASL and hopefully a whole bunch of other stuff I can forget now ;)] From tim at clewlow.org Wed Feb 18 02:24:37 2009 From: tim at clewlow.org (Tim Clewlow) Date: Wed Feb 18 02:26:05 2009 Subject: Simulating bad network conditions In-Reply-To: <20090218085715.GA9821@logik.internal.network> References: <20090218083424.GA62198@logik.internal.network> <20090218114128.B73352@mp2.macomnet.net> <20090218085715.GA9821@logik.internal.network> Message-ID: <2294403bfa2d98673c5d5748496a6d64.squirrel@192.168.1.100> Here is a link to a mailing list post about a patch to give dummynet support in pf. It is _fairly_ recent and so may still be a little buggy, but the poster seems quite confident that it works as intended. http://www.mail-archive.com/freebsd-pf@freebsd.org/msg03857.html Cheers, Tim. > On 2009-02-18 11:42:00, Maxim Konovalov wrote: >> >> ipfw(8) prob + dummynet(8). >> > > Hi. Thanks for the quick response. > > Is there, by any chance, an equivalent for PF? I see there's 'ALTQ' > but it looks to be poorly supported (unless I misunderstand). I have > quite a complicated setup here with PF forwarding and jails and I'm > not sure how well ipfw will play along. > > thanks, > xw From bruce at cran.org.uk Wed Feb 18 02:39:26 2009 From: bruce at cran.org.uk (Bruce Cran) Date: Wed Feb 18 02:39:33 2009 Subject: Simulating bad network conditions In-Reply-To: <20090218085715.GA9821@logik.internal.network> References: <20090218083424.GA62198@logik.internal.network> <20090218114128.B73352@mp2.macomnet.net> <20090218085715.GA9821@logik.internal.network> Message-ID: <20090218103903.5aee2d7d@tau.draftnet> On Wed, 18 Feb 2009 08:57:15 +0000 xorquewasp@googlemail.com wrote: > On 2009-02-18 11:42:00, Maxim Konovalov wrote: > > > > ipfw(8) prob + dummynet(8). > > > > Hi. Thanks for the quick response. > > Is there, by any chance, an equivalent for PF? I see there's 'ALTQ' > but it looks to be poorly supported (unless I misunderstand). I have > quite a complicated setup here with PF forwarding and jails and I'm > not sure how well ipfw will play along. ALTQ can drop 20% of packets using something like: block in proto icmp probability 20% It seems ALTQ can't delay packets though, so you'd need to use dummynet for that. -- Bruce Cran From julian at elischer.org Wed Feb 18 10:12:29 2009 From: julian at elischer.org (Julian Elischer) Date: Wed Feb 18 10:12:36 2009 Subject: Simulating bad network conditions In-Reply-To: <20090218103903.5aee2d7d@tau.draftnet> References: <20090218083424.GA62198@logik.internal.network> <20090218114128.B73352@mp2.macomnet.net> <20090218085715.GA9821@logik.internal.network> <20090218103903.5aee2d7d@tau.draftnet> Message-ID: <499C48F8.1080604@elischer.org> Bruce Cran wrote: > On Wed, 18 Feb 2009 08:57:15 +0000 > xorquewasp@googlemail.com wrote: > >> On 2009-02-18 11:42:00, Maxim Konovalov wrote: >>> ipfw(8) prob + dummynet(8). >>> >> Hi. Thanks for the quick response. >> >> Is there, by any chance, an equivalent for PF? I see there's 'ALTQ' >> but it looks to be poorly supported (unless I misunderstand). I have >> quite a complicated setup here with PF forwarding and jails and I'm >> not sure how well ipfw will play along. > > ALTQ can drop 20% of packets using something like: > > block in proto icmp probability 20% > > It seems ALTQ can't delay packets though, so you'd need to use dummynet > for that. > vimage it the tool for you it can REALLY simulate a lot of network setups. From laladelausanne at gmail.com Wed Feb 18 11:18:25 2009 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Wed Feb 18 11:18:31 2009 Subject: Obtaining l2 misses and cpu utilization in a module Message-ID: Hi, I would like to obtain fore mentioned data from within my module. I need these performance metrics to see how certain code executes, and make decisions during the runtime. pmc(3) seems complete, but it also seems to be intended for use in the userland. How to use pmc from modules? Is there any other way? Also, module is amd64 only, so I don't care about portability. Cheers, Nikola From lgj at usenix.org Wed Feb 18 09:37:35 2009 From: lgj at usenix.org (Lionel Garth Jones) Date: Wed Feb 18 11:31:19 2009 Subject: HotPar '09: Attendance Now Open to Everyone Message-ID: <499C3F8A.5070607@usenix.org> The First USENIX Workshop on Hot Topics in Parallelism will take place March 30-31 in Berkeley, CA. http://www.usenix.org/hotpar09/proga HotPar '09 will bring together researchers and practitioners doing innovative work in the area of parallel computing. HotPar recognizes the broad impact of multicore computing and seeks relevant contributions from all fields, including application design, languages and compilers, systems, and architecture. Attendance to the workshop is now open to the USENIX community. To ensure a productive environment, it will be limited to 75 participants. Register today to ensure your place. This year's outstanding program includes a diverse range of hot research areas including models and paradigms, operating systems and middleware, transactional memory, and more. Please join us at the Claremont Resort & Spa for this new workshop covering the most innovative research in parallel computing. We look forward to seeing you there. Sincerely, Alexandra Fedorova, Simon Fraser University Jim Larus, Microsoft HotPar '09 Program Co-Chairs hotpar09chairs@usenix.org ------------------------------------------------------------------------ First USENIX Workshop on Hot Topics in Parallelism (HotPar '09) March 30-31, 2009 Berkeley, CA, USA http://www.usenix.org/hotpar09/proga ------------------------------------------------------------------------ From fbsd.hackers at rachie.is-a-geek.net Wed Feb 18 11:39:48 2009 From: fbsd.hackers at rachie.is-a-geek.net (Mel) Date: Wed Feb 18 11:39:55 2009 Subject: Simulating bad network conditions In-Reply-To: <20090218103903.5aee2d7d@tau.draftnet> References: <20090218083424.GA62198@logik.internal.network> <20090218085715.GA9821@logik.internal.network> <20090218103903.5aee2d7d@tau.draftnet> Message-ID: <200902180948.04532.fbsd.hackers@rachie.is-a-geek.net> On Wednesday 18 February 2009 01:39:03 Bruce Cran wrote: > It seems ALTQ can't delay packets though, so you'd need to use dummynet > for that. You can work around it using hfsc's realtime: hfsc(realtime (0Kb 300 128Kb)) 1. 0Kbit of bandwidth guaranteed for the first 300ms 2. After 300ms 128Kbit of bandwidth is provided 3. When using this method, the first value must be 0. Though, this only works when under pressure. If enough bandwidth is available, the delay won't happen. More info here: http://www.probsd.net/pf/index.php/Hednod%27s_HFSC_explained -- Mel From sean.bruno at dsl-only.net Wed Feb 18 13:28:45 2009 From: sean.bruno at dsl-only.net (Sean Bruno) Date: Wed Feb 18 13:28:52 2009 Subject: Module Dependencies Message-ID: <1234990923.20261.26.camel@localhost.localdomain> I'm messing around with dev/firewire/sbp_targ at home this week and wanted to know if there was a way to warn or notify when trying to run config against a kernel configuration file that has "device sbp_targ" but doesn't have "device targbh". Is there such a mechanism? Sean From david at catwhisker.org Wed Feb 18 13:56:20 2009 From: david at catwhisker.org (David Wolfskill) Date: Wed Feb 18 13:56:36 2009 Subject: Obtaining l2 misses and cpu utilization in a module In-Reply-To: References: Message-ID: <20090218215618.GV81076@albert.catwhisker.org> On Wed, Feb 18, 2009 at 08:18:20PM +0100, Nikola Kne??evi?? wrote: > Hi, > > I would like to obtain fore mentioned data from within my module. I > need these performance metrics to see how certain code executes, and > make decisions during the runtime. pmc(3) seems complete, but it also > seems to be intended for use in the userland. > > How to use pmc from modules? Is there any other way? Also, module is > amd64 only, so I don't care about portability. I'm not sure what you mean by "module" ini this context. But CPU utilization my be found in the kern.cp_time sysctl value; per-core utilization may be found in the kern.cp_times sysctl value. Note that these are arrays of counters; a given set of values will be foir "user", "nice", "system", "interrupt", and "idle" CPU time (in that order). The differences between respective counters at each end of a time interval of interest will be informative. Peace, david -- David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090218/2d5a3db0/attachment.pgp From pluknet at gmail.com Wed Feb 18 14:35:27 2009 From: pluknet at gmail.com (pluknet) Date: Wed Feb 18 14:35:35 2009 Subject: OT: svn commit: r188764 - head/sys/kern Message-ID: Hi. Am I right that td_estcpu is only updated under SCHED_4BSD ? (well, user's ki_estcpu is not used in top anyway. ps shows all zeroes under SCHED_ULE (as it should then though). I'm just interested.) -- wbr, pluknet From joseph.koshy at gmail.com Wed Feb 18 19:07:49 2009 From: joseph.koshy at gmail.com (Joseph Koshy) Date: Wed Feb 18 19:07:57 2009 Subject: Obtaining l2 misses and cpu utilization in a module In-Reply-To: References: Message-ID: <84dead720902181846o7c1ca09fx93fa156d262cc610@mail.gmail.com> > I would like to obtain fore mentioned data from within my module. I need > these performance metrics to see how certain code executes, and make > decisions during the runtime. pmc(3) seems complete, but it also seems to be > intended for use in the userland. > > How to use pmc from modules? Is there any other way? Also, module is amd64 > only, so I don't care about portability. At this point of time there isn't an in-kernel API for using hwpmc(4) (but it is on the task list, see: http://code.google.com/p/pmctools/issues/detail?id=17). For now, you could configure hwpmc(4) to not touch specific PMC registers using the OP_PMCADMIN request (i.e., use pmccontrol -d). After this you can use those registers from your kernel module without interference from hwpmc. Koshy From laladelausanne at gmail.com Thu Feb 19 05:48:38 2009 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Thu Feb 19 05:48:45 2009 Subject: Obtaining l2 misses and cpu utilization in a module In-Reply-To: <84dead720902190315y3dc25bfaj315409988bb5c460@mail.gmail.com> References: <84dead720902181846o7c1ca09fx93fa156d262cc610@mail.gmail.com> <402671A8-53C8-4F24-980B-85519146761E@gmail.com> <84dead720902190315y3dc25bfaj315409988bb5c460@mail.gmail.com> Message-ID: <26699242-E776-4920-A8AE-011BEA4DA18E@gmail.com> On 19 Feb 2009, at 12:15 , Joseph Koshy wrote: >> % pmccontrol -L >> pmccontrol: Initialization of the pmc(3) library failed: No such >> file or >> directory. > > Did you load hwpmc(4) into the kernel? > How can I check that? kldload pmc reports that pmc interface exists in kernel. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0xffffffff804ae160, 0xffffffff809c4c60) error 78 WARNING: WITNESS option enabled, expect reduced performance. cd3 at umass-sim3 bus 3 target 0 lun 0 cd3: Removable CD-ROM SCSI-3 device cd3: 40.000MB/s transfers cd3: Attempt to query device size failed: NOT READY, Medium not present cd0 at umass-sim0 bus 0 target 0 lun 0 cd0: Removable CD-ROM SCSI-3 device cd0: 40.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present cd1 at umass-sim1 bus 1 target 0 lun 0 cd1: Removable CD-ROM SCSI-3 device cd1: 40.000MB/s transfers cd1: Attempt to query device size failed: NOT READY, Medium not present cd2 at umass-sim2 bus 2 target 0 lun 0 cd2: Removable CD-ROM SCSI-3 device cd2: 40.000MB/s transfers cd2: Attempt to query device size failed: NOT READY, Medium not present Trying to mount root from ufs:/dev/ad4s1a ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based forwarding disabled, default to deny, logging disabled interface pmc.33554432 already present in the KLD 'kernel'! kldload: /boot/kernel/hwpmc.ko: Unsupported file type > Model 10676H should be supported in -current. If you are running 7.X > you can find a patch by Fabien at wiki.freebsd.org/PmcTools. I've applied only . Should I apply the other one? My system is: FreeBSD nslrack54.epfl.ch 7.1-RELEASE-p2 FreeBSD 7.1-RELEASE-p2 #12 Cheers, Nikola From mail at sebastianstumpf.de Thu Feb 19 13:17:30 2009 From: mail at sebastianstumpf.de (Sebastian Stumpf) Date: Thu Feb 19 13:17:37 2009 Subject: gbde and missing lock files Message-ID: <20090219205519.GA5161@sebastianstumpf.de> Hi. I've got a small GEOM_BDE setup at my computer. I used gbde to encrypt ad4s1e and stored its lock file under /etc/gbde/ad4s1e.lock. Accidently I deleted this file via a cronjob. I know that the lock file only stores the lock sector position on the disk. I've still got my password. I've read a bit through src/sbin/geom, but I don't think that I'm skilled enough to write a hack that will try every sector with my password and call fsck to check for a valid ufs filesystem. I'm watching this list for quite some time now and I know that there are some people out there how could help me with this little brute force problem in a few minutes by putting together a small hack/patch. And yes: Of course there was a backup of the lock file. But guess where... I'd be very grateful if somebody could help me mounting this filesystem one more time. :-) Cheers, Sebastian -- VI VI VI - The editor of the beast. - perlhacker.org -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090219/52f79fa5/attachment.pgp From mail at sebastianstumpf.de Thu Feb 19 13:34:09 2009 From: mail at sebastianstumpf.de (Sebastian Stumpf) Date: Thu Feb 19 13:34:16 2009 Subject: gbde and missing lock file Message-ID: <20090219210041.GA5299@sebastianstumpf.de> Hi. I've got a small GEOM_BDE setup at my computer. I used gbde to encrypt ad4s1e and stored its lock file under /etc/gbde/ad4s1e.lock. Accidently I deleted this file via a cronjob. I know that the lock file only stores the lock sector position on the disk. I've still got my password. I've read a bit through src/sbin/geom, but I don't think that I'm skilled enough to write a hack that will try every sector with my password and call fsck to check for a valid ufs filesystem. I'm watching this list for quite some time now and I know that there are some people out there how could help me with this little brute force problem in a few minutes by putting together a small hack/patch. And yes: Of course there was a backup of the lock file. But guess where... I'd be very grateful if somebody could help me mounting this filesystem one more time. :-) Cheers, Sebastian -- VI VI VI - The editor of the beast. - perlhacker.org -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090219/057c5959/attachment.pgp From fernando.apesteguia at gmail.com Fri Feb 20 08:45:32 2009 From: fernando.apesteguia at gmail.com (=?ISO-8859-1?Q?Fernando_Apestegu=EDa?=) Date: Fri Feb 20 08:45:39 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) Message-ID: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> Hi all, I copy here the mail I sent to freebsd-questions cause I didn't get any answers: Yesterday I updated to 7.1-p3 on AMD64 arch. Since then, the NIC is not detected anymore. ifconfig doesn't show it and I can't connect to the Internet. There were well-known issues with this NIC model before, (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) but the weird thing is that it seemed to be fine with 7.1-RELEASE and newer till this -p3. It doesn't recognize the card in 4/5 boot sequences (really annoying). Anybody with this problem? Thanks in advance. From wblock at wonkity.com Fri Feb 20 10:40:35 2009 From: wblock at wonkity.com (Warren Block) Date: Fri Feb 20 10:43:28 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) In-Reply-To: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> References: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> Message-ID: On Fri, 20 Feb 2009, Fernando Apestegu?a wrote: > Yesterday I updated to 7.1-p3 on AMD64 arch. > > Since then, the NIC is not detected anymore. ifconfig doesn't show it > and I can't connect to the Internet. > > There were well-known issues with this NIC model before, > (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) > but the weird thing is that it seemed to be fine with > 7.1-RELEASE and newer till this -p3. Please identify the NIC more precisely with pciconf -lv | grep -B2 Ethernet > It doesn't recognize the card in 4/5 boot sequences (really annoying). > > Anybody with this problem? If you can do 'ifconfig re0' and then the NIC is active, see this: http://www.freebsd.org/cgi/query-pr.cgi?pr=130586&cat= -Warren Block * Rapid City, South Dakota USA From fernando.apesteguia at gmail.com Fri Feb 20 11:05:38 2009 From: fernando.apesteguia at gmail.com (=?ISO-8859-1?Q?Fernando_Apestegu=EDa?=) Date: Fri Feb 20 11:08:35 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) In-Reply-To: References: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> Message-ID: <1bd550a00902201105v3a893450x21e3001185d39283@mail.gmail.com> On Fri, Feb 20, 2009 at 7:15 PM, Warren Block wrote: > On Fri, 20 Feb 2009, Fernando Apestegu?a wrote: > >> Yesterday I updated to 7.1-p3 on AMD64 arch. >> >> Since then, the NIC is not detected anymore. ifconfig doesn't show it >> and I can't connect to the Internet. >> >> There were well-known issues with this NIC model before, >> >> (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) >> but the weird thing is that it seemed to be fine with >> 7.1-RELEASE and newer till this -p3. > > Please identify the NIC more precisely with > > pciconf -lv | grep -B2 Ethernet re0@pci0:2:0:0: class=0x020000 card=0x2a6f103c chip=0x816810ec rev=0x02 hdr=0x00 vendor = 'Realtek Semiconductor' device = 'RTL8168/8111 PCI-E Gigabit Ethernet NIC' > >> It doesn't recognize the card in 4/5 boot sequences (really annoying). >> >> Anybody with this problem? > > If you can do 'ifconfig re0' and then the NIC is active, see this: No, I can't. The re0 interface is not present. After executing the command it is still missing. Thanks for the help. Should you need more information or me doing some tests, just tell me. > > http://www.freebsd.org/cgi/query-pr.cgi?pr=130586&cat= > > -Warren Block * Rapid City, South Dakota USA > From pyunyh at gmail.com Fri Feb 20 18:17:23 2009 From: pyunyh at gmail.com (Pyun YongHyeon) Date: Fri Feb 20 18:17:29 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) In-Reply-To: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> References: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> Message-ID: <20090221015518.GA44336@michelle.cdnetworks.co.kr> On Fri, Feb 20, 2009 at 05:25:26PM +0100, Fernando Apestegu?a wrote: > Hi all, > > I copy here the mail I sent to freebsd-questions cause I didn't get any answers: > > > Yesterday I updated to 7.1-p3 on AMD64 arch. > > Since then, the NIC is not detected anymore. ifconfig doesn't show it > and I can't connect to the Internet. > > There were well-known issues with this NIC model before, > (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) > but the weird thing is that it seemed to be fine with > 7.1-RELEASE and newer till this -p3. > > It doesn't recognize the card in 4/5 boot sequences (really annoying). > > Anybody with this problem? I'm not sure you're suffering from MAC power saving issue of RealTek PCIe controller. Sometimes re(4) used to fail to wakeup the controller which in turn resulted in 'no driver' for the controller. If this is the case you can see "MII without any phy!" message in dmesg output. r188358(cvs if_re.c 1.95.2.40) should fix the issue so please try latest 7-stable or copy if_re.c, if_rlreg.h and if_rl.c from HEAD/ 7-stable to your 7.1-RELEASE box and rebuild kernel. If you still see the same issue please let me know. Btw, stable@ is more appropriate list for this type of issues. From ttw+bsd at cobbled.net Sat Feb 21 10:09:47 2009 From: ttw+bsd at cobbled.net (ttw+bsd@cobbled.net) Date: Sat Feb 21 10:09:55 2009 Subject: removal of NGROUPS_MAX dependancy from base Message-ID: <20090213115426.GA15211@holyman.cobbled.net> attached is the first in a series of patches that is intended to remove the current limitation on group membership. this patch should remove the dependancy on the definition of NGROUPS_MAX as a static constant and implement it as a writable sysconf variable of the same. it should also make the necessary changes to the codebase to support those. i need some guidance as to what i should re-define NGROUPS_MAX to be (so that code that depends on it can continue to operate, i'm thinking just make it 16 but perhaps it would be worth extending the default while we're at it to something like 64??). i also need feedback on any braindamage in the current changes. the next step will be to extend the kernel groups and map them back to the user structs / calls. finally i'll extend the user groups and implement those calls. nb: not tested the code (it builds) ... was intending to test it on my XEN box but only just realised that Xen on amd64 isn't working. :-( happy for any questions that may help guide the process. -------------- next part -------------- Index: contrib/openpam/lib/openpam_borrow_cred.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/contrib/openpam/lib/openpam_borrow_cred.c,v retrieving revision 1.1.1.9 diff -b -u -r1.1.1.9 openpam_borrow_cred.c --- contrib/openpam/lib/openpam_borrow_cred.c 21 Dec 2007 11:49:29 -0000 1.1.1.9 +++ contrib/openpam/lib/openpam_borrow_cred.c 4 Feb 2009 16:38:46 -0000 @@ -60,6 +60,7 @@ struct pam_saved_cred *scred; const void *scredp; int r; + int ngroups ; ENTERI(pwd->pw_uid); r = pam_get_data(pamh, PAM_SAVED_CRED, &scredp); @@ -73,26 +74,55 @@ (int)geteuid()); RETURNC(PAM_PERM_DENIED); } - scred = calloc(1, sizeof *scred); - if (scred == NULL) - RETURNC(PAM_BUF_ERR); - scred->euid = geteuid(); - scred->egid = getegid(); - r = getgroups(NGROUPS_MAX, scred->groups); - if (r < 0) { - FREE(scred); - RETURNC(PAM_SYSTEM_ERR); - } - scred->ngroups = r; +/* get the maximum number of system groups */ +#if _POSIX_VERSION > 199212 + ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + ngroups = NGROUPS_MAX ; +#else + ngroups = _NGROUPS_COMPAT ; +#endif +/* initally allocate enough memory for max_groups */ + scred = malloc( sizeof(struct pam_saved_cred) + + ngroups*sizeof(gid_t) ) ; + if( scred == NULL ) + RETURNC( PAM_BUF_ERR ) ; +/* set the save values */ + scred->euid = geteuid() ; + scred->egid = getegid() ; +/* save groups into our (probably) oversized memory allocation */ + r = getgroups( ngroups, scred->groups ) ; + if( r < 0 ) { + FREE( scred ) ; /* call PAM's free macro */ + RETURNC( PAM_SYSTEM_ERR ) ; + } ; + scred->ngroups = r ; + ngroups = r < ngroups ? r : ngroups ; /* choose the smallest */ + /* ... number of groups to allocate */ + ngroups = ngroups < _NGROUPS_COMPAT ? ngroups : _NGROUPS_COMPAT ; + /* but keep it within expected minimum value */ + /* XXX: we don't really want this but until we get + * educated on the implications this is probably safe + * and certainaly compatible */ +/* realloc, releasing unneeded memory */ + scred = realloc( (void*)scred, + sizeof(struct pam_saved_cred)+ngroups*sizeof(gid_t) ) ; + /* nb: we ignore failure and try to store the larger + * ... structure as initially requested. catching the + * ... error in 'pam_set_data' if neccessary. */ +/* save the credentials to PAM user data area */ r = pam_set_data(pamh, PAM_SAVED_CRED, scred, &openpam_free_data); if (r != PAM_SUCCESS) { FREE(scred); RETURNC(r); } +/* set the new credentials */ if (geteuid() == pwd->pw_uid) RETURNC(PAM_SUCCESS); if (initgroups(pwd->pw_name, pwd->pw_gid) < 0 || - setegid(pwd->pw_gid) < 0 || seteuid(pwd->pw_uid) < 0) { + setegid(pwd->pw_gid) < 0 || seteuid(pwd->pw_uid) < 0) + { + /* if any of the set calls failed, then restore and fail */ openpam_restore_cred(pamh); RETURNC(PAM_SYSTEM_ERR); } Index: contrib/openpam/lib/openpam_impl.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/contrib/openpam/lib/openpam_impl.h,v retrieving revision 1.1.1.17 diff -b -u -r1.1.1.17 openpam_impl.h --- contrib/openpam/lib/openpam_impl.h 21 Dec 2007 11:49:29 -0000 1.1.1.17 +++ contrib/openpam/lib/openpam_impl.h 5 Feb 2009 15:41:19 -0000 @@ -110,13 +110,17 @@ int env_size; }; -#ifdef NGROUPS_MAX +#if _POSIX_VERSION > 199212 #define PAM_SAVED_CRED "pam_saved_cred" struct pam_saved_cred { uid_t euid; gid_t egid; - gid_t groups[NGROUPS_MAX]; int ngroups; + gid_t groups[]; + /* keep this last so that we can simply + .. over-allocate the amount of space + .. nb: don't use sizeof' unless you adjust + .. for the number of groups */ }; #endif Index: include/rpc/auth_unix.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/include/rpc/auth_unix.h,v retrieving revision 1.11 diff -b -u -r1.11 auth_unix.h --- include/rpc/auth_unix.h 23 Mar 2002 17:24:55 -0000 1.11 +++ include/rpc/auth_unix.h 14 Jan 2009 11:15:21 -0000 @@ -52,7 +52,7 @@ #define MAX_MACHINE_NAME 255 /* gids compose part of a credential; there may not be more than 16 of them */ -#define NGRPS 16 +#define AUTH_UNIX_NGROUPS 16 /* * Unix style credentials. Index: lib/libc/rpc/auth_unix.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/auth_unix.c,v retrieving revision 1.18 diff -b -u -r1.18 auth_unix.c --- lib/libc/rpc/auth_unix.c 14 Jun 2007 20:07:35 -0000 1.18 +++ lib/libc/rpc/auth_unix.c 4 Feb 2009 15:31:57 -0000 @@ -182,27 +182,48 @@ * Returns an auth handle with parameters determined by doing lots of * syscalls. */ -AUTH * +AUTH* authunix_create_default() { - int len; char machname[MAXHOSTNAMELEN + 1]; + AUTH* auth_unix ; uid_t uid; gid_t gid; - gid_t gids[NGROUPS_MAX]; - - if (gethostname(machname, sizeof machname) == -1) - abort(); - machname[sizeof(machname) - 1] = 0; + gid_t *gids ; + uint ngroups ; + uint max_ngroups ; + +/* get hostname or fail */ + if( gethostname(machname,sizeof(machname)) == -1 ) + abort() ; + machname[sizeof(machname)-1] = 0 ; /* add a null terminator */ +/* set uid/gid from current effective values */ uid = geteuid(); gid = getegid(); - if ((len = getgroups(NGROUPS_MAX, gids)) < 0) - abort(); - if (len > NGRPS) - len = NGRPS; - /* XXX: interface problem; those should all have been unsigned */ - return (authunix_create(machname, (int)uid, (int)gid, len, - (int *)gids)); +/* set the group set */ +#if _POSIX_VERSION > 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = 16 ; +#endif + gids = (gid_t*)calloc( max_ngroups, sizeof(gid_t) ) ; + if( gids == NULL ) + abort () ; + if( (ngroups=getgroups(max_ngroups,gids)) < 0 ) { + free( gids ) ; + abort() ; + } +/* clip the groups to a transmissable size */ + if( ngroups > AUTH_UNIX_NGROUPS ) + ngroups = AUTH_UNIX_NGROUPS ; +/* XXX: interface problem; those should all have been unsigned */ + auth_unix = authunix_create( machname, + (int)uid, (int)gid, (int)ngroups, + (int*)gids ) ; + free( (void*)gids ) ; + return( auth_unix ) ; } /* Index: lib/libc/rpc/authunix_prot.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/authunix_prot.c,v retrieving revision 1.10 diff -b -u -r1.10 authunix_prot.c --- lib/libc/rpc/authunix_prot.c 20 Nov 2007 01:51:20 -0000 1.10 +++ lib/libc/rpc/authunix_prot.c 4 Feb 2009 16:03:29 -0000 @@ -67,13 +67,14 @@ paup_gids = &p->aup_gids; - if (xdr_u_long(xdrs, &(p->aup_time)) - && xdr_string(xdrs, &(p->aup_machname), MAX_MACHINE_NAME) - && xdr_int(xdrs, &(p->aup_uid)) - && xdr_int(xdrs, &(p->aup_gid)) - && xdr_array(xdrs, (char **) paup_gids, - &(p->aup_len), NGRPS, sizeof(int), (xdrproc_t)xdr_int) ) { - return (TRUE); + if( xdr_u_long(xdrs,&(p->aup_time)) && + xdr_string(xdrs,&(p->aup_machname),MAX_MACHINE_NAME) && + xdr_int(xdrs,&(p->aup_uid)) && + xdr_int(xdrs,&(p->aup_gid)) && + xdr_array(xdrs,(char**)paup_gids,&(p->aup_len), + AUTH_UNIX_NGROUPS,sizeof(int),(xdrproc_t)xdr_int) ) + { + return( TRUE ) ; } - return (FALSE); + return( FALSE ) ; } Index: lib/libc/rpc/netname.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/netname.c,v retrieving revision 1.8 diff -b -u -r1.8 netname.c --- lib/libc/rpc/netname.c 16 Oct 2004 06:11:35 -0000 1.8 +++ lib/libc/rpc/netname.c 14 Jan 2009 01:29:47 -0000 @@ -61,6 +61,7 @@ #ifndef MAXHOSTNAMELEN #define MAXHOSTNAMELEN 256 #endif + #ifndef NGROUPS #define NGROUPS 16 #endif Index: lib/libc/rpc/netnamer.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/netnamer.c,v retrieving revision 1.12 diff -b -u -r1.12 netnamer.c --- lib/libc/rpc/netnamer.c 10 Mar 2005 00:58:21 -0000 1.12 +++ lib/libc/rpc/netnamer.c 3 Feb 2009 17:55:48 -0000 @@ -69,7 +69,6 @@ #ifndef NGROUPS #define NGROUPS 16 #endif - /* * Convert network-name into unix credential */ @@ -104,7 +103,7 @@ return (0); } *gidp = (gid_t) atol(p); - for (gidlen = 0; gidlen < NGROUPS; gidlen++) { + for (gidlen = 0; gidlen < _NGROUPS_RPC_MAX; gidlen++) { p = strsep(&res, "\n,"); if (p == NULL) break; @@ -157,7 +156,7 @@ static int _getgroups(uname, groups) char *uname; - gid_t groups[NGROUPS]; + gid_t groups[_NGROUPS_RPC_MAX]; { gid_t ngroups = 0; struct group *grp; @@ -169,10 +168,11 @@ while ((grp = getgrent())) { for (i = 0; grp->gr_mem[i]; i++) if (!strcmp(grp->gr_mem[i], uname)) { - if (ngroups == NGROUPS) { + if( ngroups == _NGROUPS_RPC_MAX ) { #ifdef DEBUG - fprintf(stderr, - "initgroups: %s is in too many groups\n", uname); + fprintf( stderr, + "initgroups: %s is in too many groups\n", + uname ) ; #endif goto toomany; } Index: lib/libc/rpc/svc_auth_des.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/svc_auth_des.c,v retrieving revision 1.9 diff -b -u -r1.9 svc_auth_des.c --- lib/libc/rpc/svc_auth_des.c 22 Mar 2002 23:18:37 -0000 1.9 +++ lib/libc/rpc/svc_auth_des.c 3 Feb 2009 17:51:01 -0000 @@ -452,7 +452,7 @@ short uid; /* cached uid */ short gid; /* cached gid */ short grouplen; /* length of cached groups */ - short groups[NGROUPS]; /* cached groups */ + short groups[_NGROUPS_RPC_MAX]; /* cached groups */ }; /* Index: lib/libc/rpc/svc_auth_unix.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/libc/rpc/svc_auth_unix.c,v retrieving revision 1.11 diff -b -u -r1.11 svc_auth_unix.c --- lib/libc/rpc/svc_auth_unix.c 16 Oct 2004 06:11:35 -0000 1.11 +++ lib/libc/rpc/svc_auth_unix.c 4 Feb 2009 16:04:10 -0000 @@ -68,7 +68,7 @@ struct area { struct authunix_parms area_aup; char area_machname[MAX_MACHINE_NAME+1]; - int area_gids[NGRPS]; + int area_gids[AUTH_UNIX_NGROUPS] ; } *area; u_int auth_len; size_t str_len, gid_len; @@ -98,7 +98,7 @@ aup->aup_uid = (int)IXDR_GET_INT32(buf); aup->aup_gid = (int)IXDR_GET_INT32(buf); gid_len = (size_t)IXDR_GET_U_INT32(buf); - if (gid_len > NGRPS) { + if( gid_len > AUTH_UNIX_NGROUPS ) { stat = AUTH_BADCRED; goto done; } Index: lib/librpcsec_gss/svc_rpcsec_gss.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/lib/librpcsec_gss/svc_rpcsec_gss.c,v retrieving revision 1.4 diff -b -u -r1.4 svc_rpcsec_gss.c --- lib/librpcsec_gss/svc_rpcsec_gss.c 3 Nov 2008 10:38:00 -0000 1.4 +++ lib/librpcsec_gss/svc_rpcsec_gss.c 5 Feb 2009 16:09:37 -0000 @@ -127,7 +127,7 @@ rpc_gss_ucred_t cl_ucred; /* unix-style credentials */ bool_t cl_done_callback; /* TRUE after call */ void *cl_cookie; /* user cookie from callback */ - gid_t cl_gid_storage[NGRPS]; + gid_t cl_gid_storage[AUTH_UNIX_NGROUPS]; gss_OID cl_mech; /* mechanism */ gss_qop_t cl_qop; /* quality of protection */ u_int cl_seq; /* current sequence number */ @@ -578,7 +578,7 @@ getpwuid_r(uid, &pwd, buf, sizeof(buf), &pw); if (pw) { - int len = NGRPS; + int len = AUTH_UNIX_NGROUPS; uc->uid = pw->pw_uid; uc->gid = pw->pw_gid; uc->gidlist = client->cl_gid_storage; Index: sys/compat/svr4/svr4_misc.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/compat/svr4/svr4_misc.c,v retrieving revision 1.101 diff -b -u -r1.101 svr4_misc.c --- sys/compat/svr4/svr4_misc.c 21 Apr 2008 21:24:08 -0000 1.101 +++ sys/compat/svr4/svr4_misc.c 14 Jan 2009 11:58:47 -0000 @@ -710,7 +710,12 @@ *retval = 0; break; case SVR4_CONFIG_NGROUPS: - *retval = NGROUPS_MAX; + *retval = _NGROUPS_COMPAT; + /* XXX: this should pull the value + * from sysctl but i cannot find + * the definitions for the similar + * varaibles here (i.e. 'maxproc') + */ break; case SVR4_CONFIG_CHILD_MAX: *retval = maxproc; Index: sys/fs/portalfs/portal.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/fs/portalfs/portal.h,v retrieving revision 1.10 diff -b -u -r1.10 portal.h --- sys/fs/portalfs/portal.h 6 Jan 2005 18:10:40 -0000 1.10 +++ sys/fs/portalfs/portal.h 16 Jan 2009 23:44:50 -0000 @@ -43,7 +43,7 @@ int pcr_flag; /* File open mode */ uid_t pcr_uid; /* From ucred */ short pcr_ngroups; /* From ucred */ - gid_t pcr_groups[NGROUPS]; /* From ucred */ + gid_t pcr_groups[_NGROUPS_COMPAT]; /* From ucred */ }; #ifdef _KERNEL Index: sys/i386/ibcs2/ibcs2_misc.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/i386/ibcs2/ibcs2_misc.c,v retrieving revision 1.70 diff -b -u -r1.70 ibcs2_misc.c --- sys/i386/ibcs2/ibcs2_misc.c 13 Jan 2008 14:44:07 -0000 1.70 +++ sys/i386/ibcs2/ibcs2_misc.c 14 Jan 2009 12:24:56 -0000 @@ -659,14 +659,14 @@ struct thread *td; struct ibcs2_getgroups_args *uap; { - ibcs2_gid_t iset[NGROUPS_MAX]; - gid_t gp[NGROUPS_MAX]; + ibcs2_gid_t iset[_NGROUPS_COMPAT]; + gid_t gp[_NGROUPS_COMPAT]; u_int i, ngrp; int error; if (uap->gidsetsize < 0) return (EINVAL); - ngrp = MIN(uap->gidsetsize, NGROUPS_MAX); + ngrp = MIN(uap->gidsetsize, _NGROUPS_COMPAT); error = kern_getgroups(td, &ngrp, gp); if (error) return (error); @@ -685,11 +685,11 @@ struct thread *td; struct ibcs2_setgroups_args *uap; { - ibcs2_gid_t iset[NGROUPS_MAX]; - gid_t gp[NGROUPS_MAX]; + ibcs2_gid_t iset[_NGROUPS_COMPAT]; + gid_t gp[_NGROUPS_COMPAT]; int error, i; - if (uap->gidsetsize < 0 || uap->gidsetsize > NGROUPS_MAX) + if (uap->gidsetsize < 0 || uap->gidsetsize > _NGROUPS_COMPAT) return (EINVAL); if (uap->gidsetsize && uap->gidset) { error = copyin(uap->gidset, iset, sizeof(ibcs2_gid_t) * @@ -789,8 +789,13 @@ return 0; case IBCS2_SC_NGROUPS_MAX: - mib[1] = KERN_NGROUPS; - break; + /* XXX: IBCS2 compat with group limits not known to + * me, so i'll just return a compatibile/safe limit + * for now */ + PROC_LOCK(p) ; + td->td_retval[0] = _NGROUPS_COMPAT ; + PROC_UNLOCK(p) ; + return( 0 ) ; case IBCS2_SC_OPEN_MAX: PROC_LOCK(p); Index: sys/kern/kern_mib.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/kern/kern_mib.c,v retrieving revision 1.93 diff -b -u -r1.93 kern_mib.c --- sys/kern/kern_mib.c 28 Jan 2009 19:58:05 -0000 1.93 +++ sys/kern/kern_mib.c 4 Feb 2009 13:15:06 -0000 @@ -124,8 +124,8 @@ SYSCTL_INT(_kern, KERN_POSIX1, posix1version, CTLFLAG_RD, 0, _POSIX_VERSION, "Version of POSIX attempting to comply to"); -SYSCTL_INT(_kern, KERN_NGROUPS, ngroups, CTLFLAG_RD, - 0, NGROUPS_MAX, "Maximum number of groups a user can belong to"); +SYSCTL_INT(_kern, KERN_NGROUPS, ngroups, CTLFLAG_RW, + 0, _NGROUPS_COMPAT, "Maximum number of groups allocated to a user"); SYSCTL_INT(_kern, KERN_JOB_CONTROL, job_control, CTLFLAG_RD, 0, 1, "Whether job control is available"); Index: sys/sys/param.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/sys/param.h,v retrieving revision 1.382 diff -b -u -r1.382 param.h --- sys/sys/param.h 28 Jan 2009 17:57:16 -0000 1.382 +++ sys/sys/param.h 4 Feb 2009 14:11:55 -0000 @@ -57,7 +57,7 @@ * is created, otherwise 1. */ #undef __FreeBSD_version -#define __FreeBSD_version 800062 /* Master, propagated to newvers */ +#define __FreeBSD_version 800060 /* Master, propagated to newvers */ #ifndef LOCORE #include @@ -77,7 +77,8 @@ #define MAXLOGNAME 17 /* max login name length (incl. NUL) */ #define MAXUPRC CHILD_MAX /* max simultaneous processes */ #define NCARGS ARG_MAX /* max bytes for an exec function */ -#define NGROUPS NGROUPS_MAX /* max number groups */ +#define NGROUPS _NGROUPS_COMPAT + /* depreciated check sysctl/sysconf for NGROUPS_MAX value instead */ #define NOFILE OPEN_MAX /* max open files per process */ #define NOGROUP 65535 /* marker for empty group set member */ #define MAXHOSTNAMELEN 256 /* max hostname size */ Index: sys/sys/syslimits.h =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/sys/sys/syslimits.h,v retrieving revision 1.23 diff -b -u -r1.23 syslimits.h --- sys/sys/syslimits.h 29 May 2007 15:14:46 -0000 1.23 +++ sys/sys/syslimits.h 3 Feb 2009 18:02:22 -0000 @@ -54,7 +54,6 @@ #define MAX_CANON 255 /* max bytes in term canon input line */ #define MAX_INPUT 255 /* max bytes in terminal input */ #define NAME_MAX 255 /* max bytes in a file name */ -#define NGROUPS_MAX 16 /* max supplemental group id's */ #ifndef OPEN_MAX #define OPEN_MAX 64 /* max open files per process */ #endif @@ -66,9 +65,35 @@ * We leave the following values undefined to force applications to either * assume conservative values or call sysconf() to get the current value. * - * HOST_NAME_MAX + * HOST_NAME_MAX NGROUPS_MAX * * (We should do this for most of the values currently defined here, * but many programs are not prepared to deal with this yet.) */ +/* + * here are some reference values in respect of the obsoleted + * NGROUPS_MAX value. + * nb: some apps appear to check NGROUPS_MAX as meaning that + * ... system has user groups (i.e. to #ifdef chunks of code). + * ... this is easy to change but maybe historically defined? + */ +#define _NGROUPS_RPC_MAX 16 /* reference only */ + /* nb: this is the old system max, so named + * ... because it's limit appears to + * ... have been derived from a limitation + * ... in RPC (and thereby NFS), where it's + * ... the max number of groups we can exchange */ +#define _NGROUPS_COMPAT _NGROUPS_RPC_MAX /* reference only */ + /* nb: although this is defined as equal to the rpc + * ... limit, i have defined it distintly so that + * ... we may distinguish (whilst updating) usage + * ... that is correctly explicit (i.e. should be 16) + * ... and usage that is only 16 because of an expected + * ... convention. hopefully we may remove these and + * ... define additional _NGROUPS_*_MAX for those defined + * ... uses. */ +#define _NGROUPS_SYS_MAX 65536 /* reference only */ + /* nb: the idea's to have this extensible + * ... indefinately, this is what linux have and + * ... should more than cover immediate needs */ #endif Index: usr.bin/catman/catman.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.bin/catman/catman.c,v retrieving revision 1.14 diff -b -u -r1.14 catman.c --- usr.bin/catman/catman.c 5 Dec 2005 14:22:12 -0000 1.14 +++ usr.bin/catman/catman.c 8 Feb 2009 22:51:44 -0000 @@ -93,8 +93,9 @@ enum Ziptype {NONE, BZIP, GZIP}; static uid_t uid; -static gid_t gids[NGROUPS_MAX]; +static gid_t *gids; static int ngids; +static int max_ngroups ; static int starting_dir; static char tmp_file[MAXPATHLEN]; struct stat test_st; @@ -789,7 +790,15 @@ /* NOTREACHED */ } } - ngids = getgroups(NGROUPS_MAX, gids); +/* allocate memory for group ids */ +#if _POSIX_VERSION > 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + ngids = getgroups( max_ngroups, gids ) ; if ((starting_dir = open(".", 0)) < 0) { err(1, "."); } Index: usr.bin/newgrp/newgrp.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.bin/newgrp/newgrp.c,v retrieving revision 1.2 diff -b -u -r1.2 newgrp.c --- usr.bin/newgrp/newgrp.c 30 Oct 2003 15:14:34 -0000 1.2 +++ usr.bin/newgrp/newgrp.c 9 Feb 2009 22:05:53 -0000 @@ -146,9 +146,10 @@ static void addgroup(const char *grpname) { - gid_t grps[NGROUPS_MAX]; + gid_t *grps; long lgid; - int dbmember, i, ngrps; + int dbmember, i, ngrps, max_ngroups ; + /* XXX: should 'max_ngroups' be a static const variable? */ gid_t egid; struct group *grp; char *ep, *pass; @@ -185,9 +186,21 @@ } } - if ((ngrps = getgroups(NGROUPS_MAX, (gid_t *)grps)) < 0) { +#if _POSIX_VERSION >= 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + grps = (gid_t*)calloc( max_ngroups, sizeof(gid_t) ) ; + if( grps == NULL ) { + warn( "group set memory allocation" ) ; + return ; + } + if( (ngrps=getgroups(max_ngroups,(gid_t*)grps)) < 0 ) { warn("getgroups"); - return; + goto error_free ; } /* Remove requested gid from supp. list if it exists. */ @@ -201,7 +214,7 @@ if (setgroups(ngrps, (const gid_t *)grps) < 0) { PRIV_END; warn("setgroups"); - return; + goto error_free ; } PRIV_END; } @@ -210,14 +223,14 @@ if (setgid(grp->gr_gid)) { PRIV_END; warn("setgid"); - return; + goto error_free ; } PRIV_END; grps[0] = grp->gr_gid; /* Add old effective gid to supp. list if it does not exist. */ if (egid != grp->gr_gid && !inarray(egid, grps, ngrps)) { - if (ngrps == NGROUPS_MAX) + if( ngrps == max_ngroups ) warnx("too many groups"); else { grps[ngrps++] = egid; @@ -225,12 +238,15 @@ if (setgroups(ngrps, (const gid_t *)grps)) { PRIV_END; warn("setgroups"); - return; + goto error_free ; } PRIV_END; } } +error_free: + free( grps ) ; + return ; } static int Index: usr.sbin/chown/chown.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/chown/chown.c,v retrieving revision 1.29 diff -b -u -r1.29 chown.c --- usr.sbin/chown/chown.c 7 Aug 2004 04:19:37 -0000 1.29 +++ usr.sbin/chown/chown.c 8 Feb 2009 16:22:31 -0000 @@ -269,7 +269,8 @@ { static uid_t euid = -1; static int ngroups = -1; - gid_t groups[NGROUPS_MAX]; + static int max_groups ; + gid_t *groups; /* Check for chown without being root. */ if (errno != EPERM || (uid != (uid_t)-1 && @@ -279,16 +280,31 @@ } /* Check group membership; kernel just returns EPERM. */ +#if _POSIX_VERSION >= 199212 + max_groups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_groups = NGROUPS_MAX ; +#else + max_groups = _NGROUPS_COMPAT ; +#endif + groups = (gid_t*)calloc( max_groups, sizeof(gid_t) ) ; + if( groups == NULL ) { + warnx( "failed to allocate memory for group set" ) ; + goto exit_cleanup ; + } if (gid != (gid_t)-1 && ngroups == -1 && euid == (uid_t)-1 && (euid = geteuid()) != 0) { - ngroups = getgroups(NGROUPS_MAX, groups); + ngroups = getgroups( max_groups, groups ) ; while (--ngroups >= 0 && gid != groups[ngroups]); if (ngroups < 0) { warnx("you are not a member of group %s", gname); - return; + goto exit_cleanup ; } } warn("%s", file); +exit_cleanup: + free( groups ) ; + return ; } void Index: usr.sbin/chroot/chroot.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/chroot/chroot.c,v retrieving revision 1.11 diff -b -u -r1.11 chroot.c --- usr.sbin/chroot/chroot.c 7 Aug 2004 04:19:37 -0000 1.11 +++ usr.sbin/chroot/chroot.c 5 Feb 2009 23:29:48 -0000 @@ -59,6 +59,7 @@ char *user; /* user to switch to before running program */ char *group; /* group to switch to ... */ char *grouplist; /* group list to switch to ... */ +int max_ngroups; /* max number of groups allowable */ int main(argc, argv) @@ -69,12 +70,25 @@ struct passwd *pw; char *endp, *p; const char *shell; - gid_t gid, gidlist[NGROUPS_MAX]; + gid_t gid, *gidlist ; uid_t uid; - int ch, gids; + int ch, gids ; +/* set some defaults */ gid = 0; uid = 0; + user = NULL ; + group = NULL ; + grouplist = NULL ; +#if _POSIX_VERSION >= 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + +/* process command line options */ while ((ch = getopt(argc, argv, "G:g:u:")) != -1) { switch(ch) { case 'u': @@ -103,9 +117,12 @@ if (argc < 1) usage(); +/* if a group argument was passed then process it */ if (group != NULL) { + /* if the first char's a digit then assume it's a gid ... */ if (isdigit((unsigned char)*group)) { gid = (gid_t)strtoul(group, &endp, 0); + /* ... and back out that assumption if it proves wrong */ if (*endp != '\0') goto getgroup; } else { @@ -117,8 +134,15 @@ } } - for (gids = 0; - (p = strsep(&grouplist, ",")) != NULL && gids < NGROUPS_MAX; ) { +/* process command line group list */ + if( grouplist != NULL ) { + gidlist = (gid_t*)calloc( max_ngroups, sizeof(gid_t) ) ; + if( gidlist == NULL ) + errx( 1, "inadquate memory for group list" ) ; + for( gids = 0 ; + gids < max_ngroups && + (p=strsep(&grouplist,",")) != NULL ; ) + { if (*p == '\0') continue; @@ -135,9 +159,11 @@ } gids++; } - if (p != NULL && gids == NGROUPS_MAX) + if( p != NULL && gids == max_ngroups ) errx(1, "too many supplementary groups provided"); + } +/* set user from command line option, if supplied */ if (user != NULL) { if (isdigit((unsigned char)*user)) { uid = (uid_t)strtoul(user, &endp, 0); @@ -152,9 +178,11 @@ } } +/* change root */ if (chdir(argv[0]) == -1 || chroot(".") == -1) err(1, "%s", argv[0]); +/* set credentials */ if (gids && setgroups(gids, gidlist) == -1) err(1, "setgroups"); if (group && setgid(gid) == -1) @@ -162,11 +190,14 @@ if (user && setuid(uid) == -1) err(1, "setuid"); +/* exec the remaining arguments as the chroot'd command ... */ if (argv[1]) { execvp(argv[1], &argv[1]); err(1, "%s", argv[1]); + /* NOTREACHED */ } +/* ... or execute the default system shell */ if (!(shell = getenv("SHELL"))) shell = _PATH_BSHELL; execlp(shell, shell, "-i", (char *)NULL); Index: usr.sbin/gssd/gssd.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/gssd/gssd.c,v retrieving revision 1.1 diff -b -u -r1.1 gssd.c --- usr.sbin/gssd/gssd.c 3 Nov 2008 10:38:00 -0000 1.1 +++ usr.sbin/gssd/gssd.c 5 Feb 2009 16:16:37 -0000 @@ -464,8 +464,8 @@ result->uid = uid; getpwuid_r(uid, &pwd, buf, sizeof(buf), &pw); if (pw) { - int len = NGRPS; - int groups[NGRPS]; + int len = AUTH_UNIX_NGROUPS ; + int groups[AUTH_UNIX_NGROUPS] ; result->gid = pw->pw_gid; getgrouplist(pw->pw_name, pw->pw_gid, groups, &len); Index: usr.sbin/mount_portalfs/cred.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/mount_portalfs/cred.c,v retrieving revision 1.1 diff -b -u -r1.1 cred.c --- usr.sbin/mount_portalfs/cred.c 11 Mar 2005 08:39:58 -0000 1.1 +++ usr.sbin/mount_portalfs/cred.c 16 Jan 2009 23:49:36 -0000 @@ -46,7 +46,7 @@ set_user_credentials(struct portal_cred *user, struct portal_cred *save) { save->pcr_uid = geteuid(); - if ((save->pcr_ngroups = getgroups(NGROUPS_MAX, save->pcr_groups)) < 0) + if( (save->pcr_ngroups=getgroups(_NGROUPS_COMPAT,save->pcr_groups)) < 0 ) return (-1); if (setgroups(user->pcr_ngroups, user->pcr_groups) < 0) return (-1); Index: usr.sbin/pppd/options.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/pppd/options.c,v retrieving revision 1.26 diff -b -u -r1.26 options.c --- usr.sbin/pppd/options.c 7 Nov 2007 10:53:38 -0000 1.26 +++ usr.sbin/pppd/options.c 10 Feb 2009 09:11:47 -0000 @@ -72,10 +72,6 @@ char *strdup(char *); #endif -#ifndef GIDSET_TYPE -#define GIDSET_TYPE gid_t -#endif - /* * Option variables and default values. */ @@ -779,23 +775,64 @@ int fd; { uid_t uid; - int ngroups, i; + int ngroups, max_ngroups, i; struct stat sbuf; - GIDSET_TYPE groups[NGROUPS_MAX]; + gid_t *groups; +/* get the uid */ uid = getuid(); +/* ... and return true if root */ +/* XXX: needs credential check */ if (uid == 0) return 1; + +/* if we're not root, get some info about the file */ if (fstat(fd, &sbuf) != 0) return 0; + +/* test for owner match with current process */ if (sbuf.st_uid == uid) return sbuf.st_mode & S_IRUSR; +/* ... and a group match */ if (sbuf.st_gid == getgid()) return sbuf.st_mode & S_IRGRP; - ngroups = getgroups(NGROUPS_MAX, groups); - for (i = 0; i < ngroups; ++i) - if (sbuf.st_gid == groups[i]) - return sbuf.st_mode & S_IRGRP; + +/* if we've still no luck then check the group list for permission match */ +#if _POSIX_VERSION >= 199212 + max_ngroups = sysconf( _SC_NGROUPS_MAX ) ; +#elif defined(NGROUPS_MAX) + max_ngroups = NGROUPS_MAX ; +#else + max_ngroups = _NGROUPS_COMPAT ; +#endif + groups = (gid_t*) calloc( max_ngroups, sizeof(gid_t) ) ; + if( groups == NULL ) { + /* if we cannot check groups correctly then assume 'fd' is unreadable + * XXX: this may be false as the converse is more likely. + * i.e. it would be failed readable on available groups + * and granted on full list, however, we just can't be + * psychic and i'm not about to code some idiotic loop that tries + * to get 'some' memory for partial testing. probably a better + * recourse would be to simply die here but that seems severe + * for a 'readable' test. + * NB: we don't need a 'full' allocation of memory to test the + * group list, only to store it. one idea would be to do this in + * 'blocks' + */ + option_error( 1, "unable to allocate memory for group list" ) ; + return( 0 ) ; + } +/* get groups */ + ngroups = getgroups( max_ngroups, groups ) ; +/* ... and test the group permission if matching */ + for( i = 0 ; i < ngroups ; ++i ) { + if (sbuf.st_gid == groups[i]) { + free( (void*)groups) ; + return( sbuf.st_mode & S_IRGRP ) ; + } + } +/* otherwise return other permissions match */ + free( (void*)groups ) ; return sbuf.st_mode & S_IROTH; } Index: usr.sbin/rpc.lockd/kern.c =================================================================== RCS file: /home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/usr.sbin/rpc.lockd/kern.c,v retrieving revision 1.21 diff -b -u -r1.21 kern.c --- usr.sbin/rpc.lockd/kern.c 17 Aug 2006 05:55:20 -0000 1.21 +++ usr.sbin/rpc.lockd/kern.c 5 Feb 2009 16:22:17 -0000 @@ -239,15 +239,15 @@ int ngroups; ngroups = xucred->cr_ngroups - 1; - if (ngroups > NGRPS) - ngroups = NGRPS; - if (cl->cl_auth != NULL) - cl->cl_auth->ah_ops->ah_destroy(cl->cl_auth); - cl->cl_auth = authunix_create(hostname, + if( ngroups > AUTH_UNIX_NGROUPS ) + ngroups = AUTH_UNIX_NGROUPS ; + if( cl->cl_auth != NULL ) + cl->cl_auth->ah_ops->ah_destroy( cl->cl_auth ) ; + cl->cl_auth = authunix_create( hostname, xucred->cr_uid, xucred->cr_groups[0], ngroups, - &xucred->cr_groups[1]); + &xucred->cr_groups[1] ) ; } From barkinkilic at gmail.com Sat Feb 21 10:50:37 2009 From: barkinkilic at gmail.com (=?ISO-8859-9?B?QmFya/1uIEtJTEnH?=) Date: Sat Feb 21 10:50:44 2009 Subject: ethernet card error Message-ID: <9f8b20dc0902211021i687eefc9iff1e05b763712c03@mail.gmail.com> Hi all After a normal reboot i sometimes get errors at my log file about my ethernet card like: Feb 19 03:51:32 ortam kernel: dc0: dc_setcfg: failed to force tx to idle state Feb 19 03:52:11 ortam kernel: dc0: watchdog timeout and i loose my network connection, and it get fixed after a reboot. What is this error and how could i fix this without rebooting everytime it occurs.. uname -a : FreeBSD ortam.barkin.info 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 thanks -- barkink From gerryw at compvia.com Sat Feb 21 16:49:47 2009 From: gerryw at compvia.com (Gerry Weaver) Date: Sat Feb 21 16:49:53 2009 Subject: variable args in kernel module? Message-ID: <8AC3EBEC-71C1-4E29-9E20-680AF49626BF@compvia.com> Hello All, How does one make use of variable arguments in a kernel module? It seems doable ala printf etc. I want to front end sprintf to calculate buffer size. I've been looking through the source tree, but no luck as of yet. Thanks, Gerry From julian at elischer.org Sat Feb 21 23:00:49 2009 From: julian at elischer.org (Julian Elischer) Date: Sat Feb 21 23:00:57 2009 Subject: removal of NGROUPS_MAX dependancy from base In-Reply-To: <20090213115426.GA15211@holyman.cobbled.net> References: <20090213115426.GA15211@holyman.cobbled.net> Message-ID: <49A0F57E.2030506@elischer.org> ttw+bsd@cobbled.net wrote: > attached is the first in a series of patches that is intended to > remove the current limitation on group membership. > > this patch should remove the dependancy on the definition of > NGROUPS_MAX as a static constant and implement it as a writable > sysconf variable of the same. it should also make the necessary > changes to the codebase to support those. > > i need some guidance as to what i should re-define NGROUPS_MAX to be > (so that code that depends on it can continue to operate, i'm thinking > just make it 16 but perhaps it would be worth extending the default > while we're at it to something like 64??). i also need feedback on > any braindamage in the current changes. > > the next step will be to extend the kernel groups and map them back > to the user structs / calls. finally i'll extend the user groups > and implement those calls. > > nb: not tested the code (it builds) ... was intending to test it on > my XEN box but only just realised that Xen on amd64 isn't working. > :-( > > happy for any questions that may help guide the process. What do you do about NFS? I seem to remember that NFWS had a maximum number of groups supported.. > > > ------------------------------------------------------------------------ > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From gerryw at compvia.com Sat Feb 21 23:22:32 2009 From: gerryw at compvia.com (Gerry Weaver) Date: Sat Feb 21 23:22:39 2009 Subject: variable args in kernel module? In-Reply-To: <8AC3EBEC-71C1-4E29-9E20-680AF49626BF@compvia.com> References: <8AC3EBEC-71C1-4E29-9E20-680AF49626BF@compvia.com> Message-ID: <0C6BEA08-70DC-451D-AA87-885748574100@compvia.com> Hello All, Solved. I found machine/stdarg.h. Thanks, Gerry On Feb 21, 2009, at 6:34 PM, Gerry Weaver wrote: > Hello All, > > How does one make use of variable arguments in a kernel module? It > seems doable ala printf etc. I want to front end sprintf to > calculate buffer size. I've been looking through the source tree, > but no luck as of yet. > > Thanks, > Gerry > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org > " > From ttw+bsd at cobbled.net Sun Feb 22 03:41:56 2009 From: ttw+bsd at cobbled.net (ttw+bsd@cobbled.net) Date: Sun Feb 22 03:42:03 2009 Subject: removal of NGROUPS_MAX dependancy from base In-Reply-To: <49A0F57E.2030506@elischer.org> References: <20090213115426.GA15211@holyman.cobbled.net> <49A0F57E.2030506@elischer.org> Message-ID: <20090222110719.GA16634@holyman.cobbled.net> On 21.02-22:49, Julian Elischer wrote: [ ... ] > >this patch should remove the dependancy on the definition of > >NGROUPS_MAX as a static constant and implement it as a writable > >sysconf variable of the same. it should also make the necessary > >changes to the codebase to support those. [ ... ] > What do you do about NFS? > I seem to remember that NFWS had a maximum number of groups supported.. NFS will be supported by mapping 16 groups into the auth_unix structure dynamically. my intention is to try and make this transparent by allocating moving the 'most used' groups into that mapping as user processes check them, however, this is very conceptual at the moment and needs more thought as well as validation from others with more experience. From fernando.apesteguia at gmail.com Sun Feb 22 09:15:57 2009 From: fernando.apesteguia at gmail.com (=?ISO-8859-1?Q?Fernando_Apestegu=EDa?=) Date: Sun Feb 22 09:16:03 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) In-Reply-To: <20090221015518.GA44336@michelle.cdnetworks.co.kr> References: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> <20090221015518.GA44336@michelle.cdnetworks.co.kr> Message-ID: <1bd550a00902220915p276b8b61m4b757373a443348a@mail.gmail.com> On Sat, Feb 21, 2009 at 2:55 AM, Pyun YongHyeon wrote: > On Fri, Feb 20, 2009 at 05:25:26PM +0100, Fernando Apestegu?a wrote: >> Hi all, >> >> I copy here the mail I sent to freebsd-questions cause I didn't get any answers: >> >> >> Yesterday I updated to 7.1-p3 on AMD64 arch. >> >> Since then, the NIC is not detected anymore. ifconfig doesn't show it >> and I can't connect to the Internet. >> >> There were well-known issues with this NIC model before, >> (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) >> but the weird thing is that it seemed to be fine with >> 7.1-RELEASE and newer till this -p3. >> >> It doesn't recognize the card in 4/5 boot sequences (really annoying). >> >> Anybody with this problem? > > I'm not sure you're suffering from MAC power saving issue of > RealTek PCIe controller. Sometimes re(4) used to fail to wakeup the > controller which in turn resulted in 'no driver' for the > controller. If this is the case you can see "MII without any phy!" > message in dmesg output. You are right, I see that message. > r188358(cvs if_re.c 1.95.2.40) should fix the issue so please try > latest 7-stable or copy if_re.c, if_rlreg.h and if_rl.c from HEAD/ > 7-stable to your 7.1-RELEASE box and rebuild kernel. > If you still see the same issue please let me know. I copied the files and restarted my computer 5 times and the NIC was present in all of them. I will watch this issue and I will let you know if I notice more problems. Thanks so much for your help. > > Btw, stable@ is more appropriate list for this type of issues. Even if I'm not using -STABLE? Cheers. > > From pyunyh at gmail.com Sun Feb 22 16:19:48 2009 From: pyunyh at gmail.com (Pyun YongHyeon) Date: Sun Feb 22 16:19:55 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) In-Reply-To: <1bd550a00902220915p276b8b61m4b757373a443348a@mail.gmail.com> References: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> <20090221015518.GA44336@michelle.cdnetworks.co.kr> <1bd550a00902220915p276b8b61m4b757373a443348a@mail.gmail.com> Message-ID: <20090223002440.GA48134@michelle.cdnetworks.co.kr> On Sun, Feb 22, 2009 at 06:15:55PM +0100, Fernando Apestegu?a wrote: > On Sat, Feb 21, 2009 at 2:55 AM, Pyun YongHyeon wrote: > > On Fri, Feb 20, 2009 at 05:25:26PM +0100, Fernando Apestegu?a wrote: > >> Hi all, > >> > >> I copy here the mail I sent to freebsd-questions cause I didn't get any answers: > >> > >> > >> Yesterday I updated to 7.1-p3 on AMD64 arch. > >> > >> Since then, the NIC is not detected anymore. ifconfig doesn't show it > >> and I can't connect to the Internet. > >> > >> There were well-known issues with this NIC model before, > >> (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) > >> but the weird thing is that it seemed to be fine with > >> 7.1-RELEASE and newer till this -p3. > >> > >> It doesn't recognize the card in 4/5 boot sequences (really annoying). > >> > >> Anybody with this problem? > > > > I'm not sure you're suffering from MAC power saving issue of > > RealTek PCIe controller. Sometimes re(4) used to fail to wakeup the > > controller which in turn resulted in 'no driver' for the > > controller. If this is the case you can see "MII without any phy!" > > message in dmesg output. > > You are right, I see that message. > > > r188358(cvs if_re.c 1.95.2.40) should fix the issue so please try > > latest 7-stable or copy if_re.c, if_rlreg.h and if_rl.c from HEAD/ > > 7-stable to your 7.1-RELEASE box and rebuild kernel. > > If you still see the same issue please let me know. > > I copied the files and restarted my computer 5 times and the NIC was > present in all of them. Glad to hear that. > I will watch this issue and I will let you know if I notice more problems. > > Thanks so much for your help. > No problem. > > > > Btw, stable@ is more appropriate list for this type of issues. > > Even if I'm not using -STABLE? > Yes, you're not using CURRENT. From fernando.apesteguia at gmail.com Sun Feb 22 23:23:34 2009 From: fernando.apesteguia at gmail.com (=?ISO-8859-1?Q?Fernando_Apestegu=EDa?=) Date: Sun Feb 22 23:23:41 2009 Subject: FBSD 7.0-p3 NIC driver problem (Realtek) In-Reply-To: <20090223002440.GA48134@michelle.cdnetworks.co.kr> References: <1bd550a00902200825w3f225d0at99cfd1c6bb63ea0@mail.gmail.com> <20090221015518.GA44336@michelle.cdnetworks.co.kr> <1bd550a00902220915p276b8b61m4b757373a443348a@mail.gmail.com> <20090223002440.GA48134@michelle.cdnetworks.co.kr> Message-ID: <1bd550a00902222323o1580a434mc883b94f26aad74f@mail.gmail.com> On 2/23/09, Pyun YongHyeon wrote: > On Sun, Feb 22, 2009 at 06:15:55PM +0100, Fernando Apestegu?a wrote: > > On Sat, Feb 21, 2009 at 2:55 AM, Pyun YongHyeon wrote: > > > On Fri, Feb 20, 2009 at 05:25:26PM +0100, Fernando Apestegu?a wrote: > > >> Hi all, > > >> > > >> I copy here the mail I sent to freebsd-questions cause I didn't get any answers: > > >> > > >> > > >> Yesterday I updated to 7.1-p3 on AMD64 arch. > > >> > > >> Since then, the NIC is not detected anymore. ifconfig doesn't show it > > >> and I can't connect to the Internet. > > >> > > >> There were well-known issues with this NIC model before, > > >> (http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2008-11/msg00299.html) > > >> but the weird thing is that it seemed to be fine with > > >> 7.1-RELEASE and newer till this -p3. > > >> > > >> It doesn't recognize the card in 4/5 boot sequences (really annoying). > > >> > > >> Anybody with this problem? > > > > > > I'm not sure you're suffering from MAC power saving issue of > > > RealTek PCIe controller. Sometimes re(4) used to fail to wakeup the > > > controller which in turn resulted in 'no driver' for the > > > controller. If this is the case you can see "MII without any phy!" > > > message in dmesg output. > > > > You are right, I see that message. > > > > > r188358(cvs if_re.c 1.95.2.40) should fix the issue so please try > > > latest 7-stable or copy if_re.c, if_rlreg.h and if_rl.c from HEAD/ > > > 7-stable to your 7.1-RELEASE box and rebuild kernel. > > > If you still see the same issue please let me know. > > > > I copied the files and restarted my computer 5 times and the NIC was > > present in all of them. > > Glad to hear that. > > > I will watch this issue and I will let you know if I notice more problems. > > > > Thanks so much for your help. > > > > No problem. > > > > > > > Btw, stable@ is more appropriate list for this type of issues. > > > > Even if I'm not using -STABLE? > > > > Yes, you're not using CURRENT. OK, I'll keep that in mind. > From ttw+bsd at cobbled.net Mon Feb 23 00:11:06 2009 From: ttw+bsd at cobbled.net (n0g0013) Date: Mon Feb 23 00:11:32 2009 Subject: removal of NGROUPS_MAX dependancy from base In-Reply-To: <20090222222831.GA70072@lor.one-eyed-alien.net> References: <20090213115426.GA15211@holyman.cobbled.net> <49A0F57E.2030506@elischer.org> <20090222110719.GA16634@holyman.cobbled.net> <20090222222831.GA70072@lor.one-eyed-alien.net> Message-ID: <20090223081059.GA11713@holyman.cobbled.net> On 22.02-16:28, Brooks Davis wrote: > On Sun, Feb 22, 2009 at 11:07:19AM +0000, ttw+bsd@cobbled.net wrote: > > On 21.02-22:49, Julian Elischer wrote: > > [ ... ] > > > >this patch should remove the dependancy on the definition of > > > >NGROUPS_MAX as a static constant and implement it as a writable > > > >sysconf variable of the same. it should also make the necessary > > > >changes to the codebase to support those. > > [ ... ] > > > What do you do about NFS? > > > I seem to remember that NFWS had a maximum number of groups supported.. > > > > NFS will be supported by mapping 16 groups into the auth_unix structure > > dynamically. my intention is to try and make this transparent by > > allocating moving the 'most used' groups into that mapping as user > > processes check them, however, this is very conceptual at the moment > > and needs more thought as well as validation from others with more > > experience. > > I think this behavior will probably need to be configurable by the > administrator because some sites are probably using groups to supply > negative permissions. It's quite reasionable to argue that's a bad > idea, but we need to take some care since people do occationally use > that "feature". agree. i'm hoping to make the rpc group allocations dynamic and thus, mostly transparent, but would suggest the only consistent way administrators to set permissions (when NFS is required) is to restrict NGROUP_MAX to 16 or less. i intend this to be the default, changed by sysctl/sysconf. my current primary concern is with software that defines it static arrays with a length of NGROUPS_MAX and then forgets to sanitize 'ngroups' count against that maximum but no idea how to catch those except too say that is 'broken'. From lavalamp at spiritual-machines.org Mon Feb 23 08:05:14 2009 From: lavalamp at spiritual-machines.org (Brian A. Seklecki) Date: Mon Feb 23 08:16:17 2009 Subject: shmmax tops out at 2G? Message-ID: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> > On Wed, 2006-Dec-13 10:50:21 -0500, Bill Moran wrote: > >In response to Bill Moran : > >> sysctl kern.ipc.shmmax=2200000000 > >> kern.ipc.shmmax: 2100000000 -> -2094967296 Someone was nice enough to file a PR related to this: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/130274 We'd be happy to sponsor development in -current to address this limitation. ~BAS From csjp at freebsd.org Mon Feb 23 11:08:33 2009 From: csjp at freebsd.org (Christian Peron) Date: Mon Feb 23 11:08:40 2009 Subject: shmmax tops out at 2G? In-Reply-To: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> Message-ID: <20090223190828.GA34866@jnz.sqrt.ca> This issue has come up a number of times. I was looking into fixing this but I just have not had the time. The basic issue is our shmid_ds structure: struct shmid_ds { struct ipc_perm shm_perm; /* operation permission structure */ int shm_segsz; /* size of segment in bytes */ pid_t shm_lpid; /* process ID of last shared memory op */ pid_t shm_cpid; /* process ID of creator */ short shm_nattch; /* number of current attaches */ time_t shm_atime; /* time of last shmat() */ time_t shm_dtime; /* time of last shmdt() */ time_t shm_ctime; /* time of last change by shmctl() */ void *shm_internal; /* sysv stupidity */ }; Basically the shm_segsz member needs to be switched from 32 bits (int) to 64 bits. The problem is that this breaks the ABI and older versions of postgresql will not work. The solution is to add additional syscalls. However, everytime this issue comes up, the question on whether we should fix struct ipc_perm at the same time is asked. The answer imho is that we should, however this is more complex since semaphores, messaages and shared memory segments all use it. The fixes are straight forward, however making sure we maintain reverse compatability is where things become complicated, especially since there are multiple layers of reverse compat we need to look after. On Mon, Feb 23, 2009 at 10:50:07AM -0500, Brian A. Seklecki wrote: > > On Wed, 2006-Dec-13 10:50:21 -0500, Bill Moran wrote: > > >In response to Bill Moran : > > >> sysctl kern.ipc.shmmax=2200000000 > > >> kern.ipc.shmmax: 2100000000 -> -2094967296 > > Someone was nice enough to file a PR related to this: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/130274 > > We'd be happy to sponsor development in -current to address this > limitation. ~BAS > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From yanefbsd at gmail.com Mon Feb 23 11:52:45 2009 From: yanefbsd at gmail.com (Garrett Cooper) Date: Mon Feb 23 11:52:52 2009 Subject: shmmax tops out at 2G? In-Reply-To: <20090223190828.GA34866@jnz.sqrt.ca> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <20090223190828.GA34866@jnz.sqrt.ca> Message-ID: <78AACD88-3F94-4B39-9122-9C4199DFFDBA@gmail.com> On Feb 23, 2009, at 11:08 AM, Christian Peron wrote: > This issue has come up a number of times. I was looking into fixing > this but I > just have not had the time. The basic issue is our shmid_ds > structure: > > struct shmid_ds { > struct ipc_perm shm_perm; /* operation permission > structure */ > int shm_segsz; /* size of segment in bytes */ > pid_t shm_lpid; /* process ID of last shared > memory op */ > pid_t shm_cpid; /* process ID of creator */ > short shm_nattch; /* number of current attaches */ > time_t shm_atime; /* time of last shmat() */ > time_t shm_dtime; /* time of last shmdt() */ > time_t shm_ctime; /* time of last change by > shmctl() */ > void *shm_internal; /* sysv stupidity */ > }; > > > Basically the shm_segsz member needs to be switched from 32 bits > (int) to > 64 bits. The problem is that this breaks the ABI and older versions > of > postgresql will not work. The solution is to add additional syscalls. > > However, everytime this issue comes up, the question on whether we > should > fix struct ipc_perm at the same time is asked. The answer imho is > that > we should, however this is more complex since semaphores, messaages > and > shared memory segments all use it. > > The fixes are straight forward, however making sure we maintain > reverse > compatability is where things become complicated, especially since > there > are multiple layers of reverse compat we need to look after. > > > On Mon, Feb 23, 2009 at 10:50:07AM -0500, Brian A. Seklecki wrote: >>> On Wed, 2006-Dec-13 10:50:21 -0500, Bill Moran wrote: >>>> In response to Bill Moran : >>>>> sysctl kern.ipc.shmmax=2200000000 >>>>> kern.ipc.shmmax: 2100000000 -> -2094967296 >> >> Someone was nice enough to file a PR related to this: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/130274 >> >> We'd be happy to sponsor development in -current to address this >> limitation. ~BAS Why isn't the field an unsigned int / size_t? I don't see much value in having the size be signed... -Garrett From csjp at freebsd.org Mon Feb 23 12:12:19 2009 From: csjp at freebsd.org (Christian Peron) Date: Mon Feb 23 12:12:25 2009 Subject: shmmax tops out at 2G? In-Reply-To: <78AACD88-3F94-4B39-9122-9C4199DFFDBA@gmail.com> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <20090223190828.GA34866@jnz.sqrt.ca> <78AACD88-3F94-4B39-9122-9C4199DFFDBA@gmail.com> Message-ID: <20090223201217.GA35374@jnz.sqrt.ca> On Mon, Feb 23, 2009 at 11:58:09AM -0800, Garrett Cooper wrote: [..] > > Why isn't the field an unsigned int / size_t? I don't see much value > in having the size be signed... No idea :) This code long predates me. From wmoran at potentialtech.com Mon Feb 23 12:32:26 2009 From: wmoran at potentialtech.com (Bill Moran) Date: Mon Feb 23 12:42:21 2009 Subject: shmmax tops out at 2G? In-Reply-To: <20090223201217.GA35374@jnz.sqrt.ca> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <20090223190828.GA34866@jnz.sqrt.ca> <78AACD88-3F94-4B39-9122-9C4199DFFDBA@gmail.com> <20090223201217.GA35374@jnz.sqrt.ca> Message-ID: <20090223151611.af79586c.wmoran@potentialtech.com> In response to Christian Peron : > On Mon, Feb 23, 2009 at 11:58:09AM -0800, Garrett Cooper wrote: > [..] > > > > Why isn't the field an unsigned int / size_t? I don't see much value > > in having the size be signed... > > No idea :) This code long predates me. It's that way because the original Sun spec for the API said so. It makes little sense to change it just to unsigned. The additional 2G it would give doesn't really solve the tuning problem on a 64G system. This is simply a spec that has become outdated by modern hardware. -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/ From yanefbsd at gmail.com Tue Feb 24 00:09:43 2009 From: yanefbsd at gmail.com (Garrett Cooper) Date: Tue Feb 24 00:09:50 2009 Subject: shmmax tops out at 2G? In-Reply-To: <20090223151611.af79586c.wmoran@potentialtech.com> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <20090223190828.GA34866@jnz.sqrt.ca> <78AACD88-3F94-4B39-9122-9C4199DFFDBA@gmail.com> <20090223201217.GA35374@jnz.sqrt.ca> <20090223151611.af79586c.wmoran@potentialtech.com> Message-ID: <7d6fde3d0902240009q77e3be85n54c17f7404bc446b@mail.gmail.com> On Mon, Feb 23, 2009 at 12:16 PM, Bill Moran wrote: > In response to Christian Peron : > >> On Mon, Feb 23, 2009 at 11:58:09AM -0800, Garrett Cooper wrote: >> [..] >> > >> > ? ? Why isn't the field an unsigned int / size_t? I don't see much value >> > in having the size be signed... >> >> No idea :) This code long predates me. > > It's that way because the original Sun spec for the API said so. > > It makes little sense to change it just to unsigned. ?The additional 2G > it would give doesn't really solve the tuning problem on a 64G system. > This is simply a spec that has become outdated by modern hardware. Ah, but an unsigned integer on a 64-bit system supports that kind of precision ;). Or are you saying you're crazy enough to run PAE mode with that much RAM 0-o? Then again the bug filer's statement doesn't make sense given the data -- there must be a int32_t used somewhere that's mucking up the system. Trying to compile the test app with -Wall, this is what I see: [gcooper@optimus ~]$ gcc -Wall -o test4 test4.c test4.c: In function 'main': test4.c:11: warning: 'return' with no value, in function returning non-void test4.c:13: warning: format '%zd' expects type 'signed size_t', but argument 2 has type 'int' #include #include int main() { size_t size = 2*1024*1024*1024l - 4096; int segid; printf("Requested: %zd\n", size); segid = shmget(234, size, IPC_CREAT); if(segid == -1) { perror("Died"); return; } printf("SHM ID : %zd\n", segid); } So I'm not 100% sure that this issue isn't a coding error, or the sample app is just incorrect... When the error comes back though from the perror, it returns EINVALID, not ENOMEM or something similar to that. Not sure if it's because the value is truly invalid, or if it's just a bad return code. My 2 cents... -Garrett From neldredge at math.ucsd.edu Tue Feb 24 00:26:51 2009 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Tue Feb 24 00:26:59 2009 Subject: shmmax tops out at 2G? In-Reply-To: <7d6fde3d0902240009q77e3be85n54c17f7404bc446b@mail.gmail.com> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <20090223190828.GA34866@jnz.sqrt.ca> <78AACD88-3F94-4B39-9122-9C4199DFFDBA@gmail.com> <20090223201217.GA35374@jnz.sqrt.ca> <20090223151611.af79586c.wmoran@potentialtech.com> <7d6fde3d0902240009q77e3be85n54c17f7404bc446b@mail.gmail.com> Message-ID: On Tue, 24 Feb 2009, Garrett Cooper wrote: > On Mon, Feb 23, 2009 at 12:16 PM, Bill Moran wrote: >> In response to Christian Peron : >> >>> On Mon, Feb 23, 2009 at 11:58:09AM -0800, Garrett Cooper wrote: >>> [..] >>>> >>>> ? ? Why isn't the field an unsigned int / size_t? I don't see much value >>>> in having the size be signed... >>> >>> No idea :) This code long predates me. >> >> It's that way because the original Sun spec for the API said so. >> >> It makes little sense to change it just to unsigned. ?The additional 2G >> it would give doesn't really solve the tuning problem on a 64G system. >> This is simply a spec that has become outdated by modern hardware. > > Ah, but an unsigned integer on a 64-bit system supports that kind of > precision ;). Or are you saying you're crazy enough to run PAE mode > with that much RAM 0-o? int and unsigned on amd64 are 32-bit types. To get a 64-bit integer, you need (unsigned) long. -- Nate Eldredge neldredge@math.ucsd.edu From kostikbel at gmail.com Tue Feb 24 09:39:43 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue Feb 24 09:39:50 2009 Subject: shmmax tops out at 2G? In-Reply-To: <20090223190828.GA34866@jnz.sqrt.ca> References: <1235404207.31655.2085.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> <20090223190828.GA34866@jnz.sqrt.ca> Message-ID: <20090224161250.GP41617@deviant.kiev.zoral.com.ua> On Mon, Feb 23, 2009 at 01:08:28PM -0600, Christian Peron wrote: > This issue has come up a number of times. I was looking into fixing this but I > just have not had the time. The basic issue is our shmid_ds structure: > > struct shmid_ds { > struct ipc_perm shm_perm; /* operation permission structure */ > int shm_segsz; /* size of segment in bytes */ > pid_t shm_lpid; /* process ID of last shared memory op */ > pid_t shm_cpid; /* process ID of creator */ > short shm_nattch; /* number of current attaches */ > time_t shm_atime; /* time of last shmat() */ > time_t shm_dtime; /* time of last shmdt() */ > time_t shm_ctime; /* time of last change by shmctl() */ > void *shm_internal; /* sysv stupidity */ > }; > > > Basically the shm_segsz member needs to be switched from 32 bits (int) to > 64 bits. The problem is that this breaks the ABI and older versions of > postgresql will not work. The solution is to add additional syscalls. > > However, everytime this issue comes up, the question on whether we should > fix struct ipc_perm at the same time is asked. The answer imho is that > we should, however this is more complex since semaphores, messaages and > shared memory segments all use it. > > The fixes are straight forward, however making sure we maintain reverse > compatability is where things become complicated, especially since there > are multiple layers of reverse compat we need to look after. Yes, this is the right solution. Meantime, below is what we use ATM to get over this limitation. The struct shmid_ds is only used for IPC_STAT call in the usermode, ignoring ipcs(1). Allowing it to break for >2Gb segments, we get otherwise good workaround. The luck is that shmget takes size_t instead of int as a segment size. It might be further tweaked to only allow for >2Gb allocation after some sysctl is set, by I do not see a point. diff --git a/sys/kern/sysv_shm.c b/sys/kern/sysv_shm.c index 4e9854d..a945523 100644 --- a/sys/kern/sysv_shm.c +++ b/sys/kern/sysv_shm.c @@ -121,7 +121,8 @@ static sy_call_t *shmcalls[] = { #define SHMSEG_ALLOCATED 0x0800 #define SHMSEG_WANTED 0x1000 -static int shm_last_free, shm_nused, shm_committed, shmalloced; +static int shm_last_free, shm_nused, shmalloced; +size_t shm_committed; static struct shmid_kernel *shmsegs; struct shmmap_state { @@ -250,7 +251,7 @@ shm_deallocate_segment(shmseg) vm_object_deallocate(shmseg->u.shm_internal); shmseg->u.shm_internal = NULL; - size = round_page(shmseg->u.shm_segsz); + size = round_page(shmseg->shm_bsegsz); shm_committed -= btoc(size); shm_nused--; shmseg->u.shm_perm.mode = SHMSEG_FREE; @@ -270,7 +271,7 @@ shm_delete_mapping(struct vmspace *vm, struct shmmap_state *shmmap_s) segnum = IPCID_TO_IX(shmmap_s->shmid); shmseg = &shmsegs[segnum]; - size = round_page(shmseg->u.shm_segsz); + size = round_page(shmseg->shm_bsegsz); result = vm_map_remove(&vm->vm_map, shmmap_s->va, shmmap_s->va + size); if (result != KERN_SUCCESS) return (EINVAL); @@ -390,7 +391,7 @@ kern_shmat(td, shmid, shmaddr, shmflg) error = EMFILE; goto done2; } - size = round_page(shmseg->u.shm_segsz); + size = round_page(shmseg->shm_bsegsz); #ifdef VM_PROT_READ_IS_EXEC prot = VM_PROT_READ | VM_PROT_EXECUTE; #else @@ -422,7 +423,8 @@ kern_shmat(td, shmid, shmaddr, shmflg) vm_object_reference(shmseg->u.shm_internal); rv = vm_map_find(&p->p_vmspace->vm_map, shmseg->u.shm_internal, - 0, &attach_va, size, (flags & MAP_FIXED)?0:1, prot, prot, 0); + 0, &attach_va, size, (flags & MAP_FIXED) ? VMFS_NO_SPACE : + VMFS_ANY_SPACE, prot, prot, 0); if (rv != KERN_SUCCESS) { vm_object_deallocate(shmseg->u.shm_internal); error = ENOMEM; @@ -720,7 +722,7 @@ shmget_existing(td, uap, mode, segnum) if (error != 0) return (error); #endif - if (uap->size && uap->size > shmseg->u.shm_segsz) + if (uap->size && uap->size > shmseg->shm_bsegsz) return (EINVAL); td->td_retval[0] = IXSEQ_TO_IPCID(segnum, shmseg->u.shm_perm); return (0); @@ -732,7 +734,8 @@ shmget_allocate_segment(td, uap, mode) struct shmget_args *uap; int mode; { - int i, segnum, shmid, size; + int i, segnum, shmid; + size_t size; struct ucred *cred = td->td_ucred; struct shmid_kernel *shmseg; vm_object_t shm_object; @@ -790,6 +793,7 @@ shmget_allocate_segment(td, uap, mode) shmseg->u.shm_perm.mode = (shmseg->u.shm_perm.mode & SHMSEG_WANTED) | (mode & ACCESSPERMS) | SHMSEG_ALLOCATED; shmseg->u.shm_segsz = uap->size; + shmseg->shm_bsegsz = uap->size; shmseg->u.shm_cpid = td->td_proc->p_pid; shmseg->u.shm_lpid = shmseg->u.shm_nattch = 0; shmseg->u.shm_atime = shmseg->u.shm_dtime = 0; diff --git a/sys/sys/shm.h b/sys/sys/shm.h index 33ed7b0..c4b1369 100644 --- a/sys/sys/shm.h +++ b/sys/sys/shm.h @@ -108,6 +108,7 @@ struct shminfo { struct shmid_kernel { struct shmid_ds u; struct label *label; /* MAC label */ + size_t shm_bsegsz; }; extern struct shminfo shminfo; diff --git a/usr.bin/ipcs/ipcs.c b/usr.bin/ipcs/ipcs.c index 67364d5..1fd943a 100644 --- a/usr.bin/ipcs/ipcs.c +++ b/usr.bin/ipcs/ipcs.c @@ -452,8 +452,8 @@ print_kshmptr(int i, int option, struct shmid_kernel *kshmptr) kshmptr->u.shm_nattch); if (option & BIGGEST) - printf(" %12d", - kshmptr->u.shm_segsz); + printf(" %12zu", + kshmptr->shm_bsegsz); if (option & PID) printf(" %12d %12d", -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090224/a97239cd/attachment.pgp From spsneo at gmail.com Tue Feb 24 10:07:48 2009 From: spsneo at gmail.com (Siddharth Prakash Singh) Date: Tue Feb 24 10:07:53 2009 Subject: Google SoC 2009 Idea Message-ID: Hi all, I am a student from India. I am willing to contribute to FreeBSD as a google soc participant this year. I would like to suggest an idea. Title: Multicore Aware Process Scheduler. I have not gone through the process scheduler code of Free BSD. Hence, I am not yet aware about the current support for Multicore Architectures. Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. Please do comment and suggest on this idea. Awaiting replies ! -- Siddharth Prakash Singh http://www.spsneo.com From kitchetech at gmail.com Tue Feb 24 10:43:47 2009 From: kitchetech at gmail.com (matt donovan) Date: Tue Feb 24 10:43:54 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: Message-ID: <28283d910902241010u165c2ba3v51436c440094290e@mail.gmail.com> On Tue, Feb 24, 2009 at 12:43 PM, Siddharth Prakash Singh wrote: > Hi all, > > I am a student from India. I am willing to contribute to FreeBSD > as a google soc participant this year. I would like to suggest an > idea. > Title: Multicore Aware Process Scheduler. > I have not gone through the process scheduler code of Free BSD. > Hence, I am not yet aware about the current support for Multicore > Architectures. > > Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. > Please do comment and suggest on this idea. > > Awaiting replies ! > > -- > Siddharth Prakash Singh > http://www.spsneo.com > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" you mean ULE which FreeBSD has already but believe it's already an idea for SoC to add things to it From spsneo at gmail.com Tue Feb 24 11:07:43 2009 From: spsneo at gmail.com (Siddharth Prakash Singh) Date: Tue Feb 24 11:07:50 2009 Subject: Google SoC 2009 Idea In-Reply-To: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> Message-ID: On Wed, Feb 25, 2009 at 12:30 AM, Ray Mihm wrote: >> Title: Multicore Aware Process Scheduler. >> I have not gone through the process scheduler code of Free BSD. >> Hence, I am not yet aware about the current support for Multicore >> Architectures. > > Talk to jeff@freebsd.org, the author of ULE. What are your opinions on this project? What is the scope of this project? > >> Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. Does the current scheduler has support for "CPU affinity/binding", mechanism for distinguishing varying capability of CPUs. > > These may be there already in ULE, although I'm not sure about NUMA. > > Ray > Waiting for your response, Siddharth -- Siddharth Prakash Singh http://www.spsneo.com From brooks at freebsd.org Tue Feb 24 11:15:43 2009 From: brooks at freebsd.org (Brooks Davis) Date: Tue Feb 24 11:15:50 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> Message-ID: <20090224191426.GA2947@lor.one-eyed-alien.net> On Wed, Feb 25, 2009 at 12:37:40AM +0530, Siddharth Prakash Singh wrote: > On Wed, Feb 25, 2009 at 12:30 AM, Ray Mihm wrote: > >> Title: Multicore Aware Process Scheduler. > >> I have not gone through the process scheduler code of Free BSD. > >> Hence, I am not yet aware about the current support for Multicore > >> Architectures. > > > > Talk to jeff@freebsd.org, the author of ULE. > > What are your opinions on this project? What is the scope of this project? > > > >> Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. > > Does the current scheduler has support for "CPU affinity/binding", > mechanism for distinguishing varying capability of CPUs. I highly recommend you read Jeff's paper on an early version of ULE and look at the code. See also, cpuset(1). Successful summer of code applicants will have demonstrated the ability to follow leads on their own. -- Brooks > > > > These may be there already in ULE, although I'm not sure about NUMA. > > > > Ray > > > > Waiting for your response, > > Siddharth > > -- > Siddharth Prakash Singh > http://www.spsneo.com > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090224/647702f7/attachment.pgp From sam at freebsd.org Tue Feb 24 11:17:26 2009 From: sam at freebsd.org (Sam Leffler) Date: Tue Feb 24 11:18:11 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> Message-ID: <49A447C5.2020903@freebsd.org> Siddharth Prakash Singh wrote: > On Wed, Feb 25, 2009 at 12:30 AM, Ray Mihm wrote: > >>> Title: Multicore Aware Process Scheduler. >>> I have not gone through the process scheduler code of Free BSD. >>> Hence, I am not yet aware about the current support for Multicore >>> Architectures. >>> >> Talk to jeff@freebsd.org, the author of ULE. >> > > What are your opinions on this project? What is the scope of this project? > >>> Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. >>> > > Does the current scheduler has support for "CPU affinity/binding", > mechanism for distinguishing varying capability of CPUs. > >> These may be there already in ULE, although I'm not sure about NUMA. >> >> Ray >> >> > > Waiting for your response, > > I note you sent this same note to the netbsd mailing lists. You might want to do some more investigation before you propose a project. Sam From ray.mihm at gmail.com Tue Feb 24 11:27:45 2009 From: ray.mihm at gmail.com (Ray Mihm) Date: Tue Feb 24 11:27:52 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: Message-ID: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> > Title: Multicore Aware Process Scheduler. > I have not gone through the process scheduler code of Free BSD. > Hence, I am not yet aware about the current support for Multicore > Architectures. Talk to jeff@freebsd.org, the author of ULE. > Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. These may be there already in ULE, although I'm not sure about NUMA. Ray From jgordeev at dir.bg Tue Feb 24 12:55:50 2009 From: jgordeev at dir.bg (Jordan Gordeev) Date: Tue Feb 24 12:55:57 2009 Subject: Google SoC 2009 Idea In-Reply-To: <49A447C5.2020903@freebsd.org> References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> <49A447C5.2020903@freebsd.org> Message-ID: <49A457CA.20704@dir.bg> Sam Leffler wrote: > Siddharth Prakash Singh wrote: >> On Wed, Feb 25, 2009 at 12:30 AM, Ray Mihm wrote: >> >>>> Title: Multicore Aware Process Scheduler. >>>> I have not gone through the process scheduler code of Free BSD. >>>> Hence, I am not yet aware about the current support for Multicore >>>> Architectures. >>>> >>> Talk to jeff@freebsd.org, the author of ULE. >>> >> >> What are your opinions on this project? What is the scope of this >> project? >> >>>> Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. >>>> >> >> Does the current scheduler has support for "CPU affinity/binding", >> mechanism for distinguishing varying capability of CPUs. >> >>> These may be there already in ULE, although I'm not sure about NUMA. >>> >>> Ray >>> >>> >> >> Waiting for your response, >> >> > I note you sent this same note to the netbsd mailing lists. You might > want to do some more investigation before you propose a project. > > Sam > It was also sent to the DragonFly mailing lists. :-) From spsneo at gmail.com Tue Feb 24 17:43:07 2009 From: spsneo at gmail.com (Siddharth Prakash Singh) Date: Tue Feb 24 17:43:14 2009 Subject: Google SoC 2009 Idea In-Reply-To: <49A457CA.20704@dir.bg> References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> <49A447C5.2020903@freebsd.org> <49A457CA.20704@dir.bg> Message-ID: Yeah I sent the same proposal to all the *BSD mailing list, because I am interested in doing this project . What's wrong in proposing the same project in all the *BSD organizations? On Wed, Feb 25, 2009 at 1:55 AM, Jordan Gordeev wrote: > Sam Leffler wrote: >> >> Siddharth Prakash Singh wrote: >>> >>> On Wed, Feb 25, 2009 at 12:30 AM, Ray Mihm wrote: >>> >>>>> >>>>> Title: Multicore Aware Process Scheduler. >>>>> I have not gone through the process scheduler code of Free BSD. >>>>> Hence, I am not yet aware about the current support for Multicore >>>>> Architectures. >>>>> >>>> >>>> Talk to jeff@freebsd.org, the author of ULE. >>>> >>> >>> What are your opinions on this project? What is the scope of this >>> project? >>> >>>>> >>>>> Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. >>>>> >>> >>> Does the current scheduler has support for "CPU affinity/binding", >>> mechanism for distinguishing varying capability of CPUs. >>> >>>> >>>> These may be there already in ULE, although I'm not sure about NUMA. >>>> >>>> Ray >>>> >>>> >>> >>> Waiting for your response, >>> >>> >> >> I note you sent this same note to the netbsd mailing lists. ?You might >> want to do some more investigation before you propose a project. >> >> ? Sam >> > It was also sent to the DragonFly mailing lists. :-) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Siddharth Prakash Singh http://www.spsneo.com From chuckr at telenix.org Tue Feb 24 19:34:35 2009 From: chuckr at telenix.org (Chuck Robey) Date: Tue Feb 24 19:34:43 2009 Subject: x11 status Message-ID: <49A4B9ED.5040705@telenix.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I was wondering, I've lost track of the status of XFree86 on FreeBSD.or really, at all. It looks like all of the Xfree86 servers have been removed from ports. I was looking on the www.Xfree86.org website, and from what I see, it apparently still is generating releases. Also, I downloaded the latest cvs image from Xfree86, and it built FAR easier that xorg, far faster, far simpler to configure ... but when I look into FreeBSD-ports, the few ports which still have the Xfree86 name, they're really cheating (talking about the drivers), they seem to be really xorg drivers, just haven't had their names fixed. No servers for Xfree86 exist in ports anymore, even though it seems trivial to build. Why is that? Is XFree86 not getting any fixes? It seems that their version 4.8.0 is recent, so I'm confused. I just wanted to know if it's missing from ports for some reason, or only because someone wanted it removed? It couldn't be because of a massive prejudice for configure-based ports, is it? (I hope not). -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmkue0ACgkQz62J6PPcoOkgOQCfS+bPR3zcTvckuTtY0fxMBMTT OwsAnjVrFGOLjWhJjDbror4xgzHqjmrV =nFWV -----END PGP SIGNATURE----- From kitchetech at gmail.com Tue Feb 24 19:40:49 2009 From: kitchetech at gmail.com (matt donovan) Date: Tue Feb 24 19:40:57 2009 Subject: x11 status In-Reply-To: <49A4B9ED.5040705@telenix.org> References: <49A4B9ED.5040705@telenix.org> Message-ID: <28283d910902241940g6648dbb5yf724508485ce6c3@mail.gmail.com> On Tue, Feb 24, 2009 at 10:24 PM, Chuck Robey wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I was wondering, I've lost track of the status of XFree86 on FreeBSD.or > really, > at all. It looks like all of the Xfree86 servers have been removed from > ports. > I was looking on the www.Xfree86.org website, and from what I see, it > apparently still is generating releases. Also, I downloaded the latest cvs > image from Xfree86, and it built FAR easier that xorg, far faster, far > simpler > to configure ... but when I look into FreeBSD-ports, the few ports which > still > have the Xfree86 name, they're really cheating (talking about the drivers), > they > seem to be really xorg drivers, just haven't had their names fixed. No > servers > for Xfree86 exist in ports anymore, even though it seems trivial to build. > > Why is that? Is XFree86 not getting any fixes? It seems that their > version > 4.8.0 is recent, so I'm confused. I just wanted to know if it's missing > from > ports for some reason, or only because someone wanted it removed? It > couldn't > be because of a massive prejudice for configure-based ports, is it? (I hope > not). > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.9 (FreeBSD) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkmkue0ACgkQz62J6PPcoOkgOQCfS+bPR3zcTvckuTtY0fxMBMTT > OwsAnjVrFGOLjWhJjDbror4xgzHqjmrV > =nFWV > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" It's due to the fact that not a lot of programs will compile with xfree86. Xorg is the de-facto standard now for graphic server. Pretty much All programs that require a gui at least the ones that depend on X libs use Xorg libs. From ray.mihm at gmail.com Tue Feb 24 19:57:34 2009 From: ray.mihm at gmail.com (Ray Mihm) Date: Tue Feb 24 19:57:41 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> <49A447C5.2020903@freebsd.org> <49A457CA.20704@dir.bg> Message-ID: <1aa142960902241957r28e77868tf2b1ba400f01232@mail.gmail.com> > Yeah I sent the same proposal to all the *BSD mailing list, because I > am interested in doing this project . Have you read what others have said? do. your. homework. first. > What's wrong in proposing the same project in all the *BSD organizations? You wouldn't have asked this question if you'd done your homework. Ray From ed at 80386.nl Tue Feb 24 22:53:10 2009 From: ed at 80386.nl (Ed Schouten) Date: Tue Feb 24 22:53:17 2009 Subject: x11 status In-Reply-To: <49A4B9ED.5040705@telenix.org> References: <49A4B9ED.5040705@telenix.org> Message-ID: <20090225065308.GO19161@hoeg.nl> Hi Chuck, * Chuck Robey wrote: > I was wondering, I've lost track of the status of XFree86 on > FreeBSD.or really, at all. It looks like all of the Xfree86 servers > have been removed from ports. The XFree86 project has been dying ever since almost all the active development moved to the Xorg-project. Xorg has many new features that XFree86 doesn't have, like hardware compositing and improved device detection. > I was looking on the www.Xfree86.org website, and from what I see, it > apparently still is generating releases. Also, I downloaded the > latest cvs image from Xfree86, and it built FAR easier that xorg, far > faster, far simpler to configure ... Why should it matter how easy it is to build a piece of software? You can just run `make -C /usr/ports/x11/xorg install clean' or `pkg_add -r xorg'. > but when I look into FreeBSD-ports, the few ports which still have the > Xfree86 name, they're really cheating (talking about the drivers), > they seem to be really xorg drivers, just haven't had their names > fixed. This is a naming decision by the Xorg project. For some reason, all drivers are still called xf86-*. It's pretty hard to remove all references to XFree86 in hundreds of megabytes of source code. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090225/41a5cd2c/attachment.pgp From Alexander at Leidinger.net Wed Feb 25 02:38:04 2009 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Wed Feb 25 04:14:29 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: <1aa142960902241100u671d5f90u769ad98e08fabb43@mail.gmail.com> <49A447C5.2020903@freebsd.org> <49A457CA.20704@dir.bg> Message-ID: <20090225113751.62205vkw7ui1ax6o@webmail.leidinger.net> Quoting Siddharth Prakash Singh (from Wed, 25 Feb 2009 07:13:05 +0530): > Yeah I sent the same proposal to all the *BSD mailing list, because I > am interested in doing this project . What's wrong in proposing the > same project in all the *BSD organizations? As one of the FreeBSD mentors for some GSoC's in the past: nothing is wrong with proposing the same project to several *BSD projects, that's not unusual and happened in the past several times. What's not so nice is to propose something without looking at the existing features in this area. It's not just saying "I want to do something like this". When you submit your proposal to Google, we expect that you looked at the corresponding code and at least know most of the features. You are not supposed to know each line of code or to understand each line of code, but you should know what is there, and what you need to do until your goal is achieved. For example in one of the past GSoC's proposals told that in the XYZ subsystem A, B and C "is missing". They contained a timeframe which explained how much time the student expects until each feature is implemented. For some stuff (API compatibility) even a list of missing functions was presented. You have to understand that in the past we got between 10 and 20 students during the GSoC. For those 10-20 slots there where more than 100 proposals (more in the range of 200-300). Those proposals where filtered by Google, so we've seen only those, which where not immediately rejected by Google because of lack of content. Those proposals have to be rated by the FreeBSD committers which are willing to mentor students, and they do this based upon several checkpoints. We look at the proposal and look if it is actually possible to do what is proposed. Not only in general, also during the timeframe of the GSoC and by a student. It is also not important that all features are completed, so if we think that the student is able to e.g. handle 80% of what he proposes and if we also think that this is ok for us, then we give some points to the proposal. This means that the student has to show that he understands what he is talking about and that he has also some insight into what he has to do and some expectation how long it takes. In the end the proposals with the most points (and someone willing to mentor this project) are taken. So the better the proposal is, more likely it will be that the proposal is accepted. When you look at the FreeBSD ideas page, you see the bare minimum what information needs to be in the proposal (nobody needs to write the required skills in a proposal). When we see a proposal which is just a copy of what we have on the ideas page, it will not get that much points, as it doesn't show if the students really understands what he is proposing. Bye, Alexander. > On Wed, Feb 25, 2009 at 1:55 AM, Jordan Gordeev wrote: >> Sam Leffler wrote: >>> >>> Siddharth Prakash Singh wrote: >>>> >>>> On Wed, Feb 25, 2009 at 12:30 AM, Ray Mihm wrote: >>>> >>>>>> >>>>>> Title: Multicore Aware Process Scheduler. >>>>>> I have not gone through the process scheduler code of Free BSD. >>>>>> Hence, I am not yet aware about the current support for Multicore >>>>>> Architectures. >>>>>> >>>>> >>>>> Talk to jeff@freebsd.org, the author of ULE. >>>>> >>>> >>>> What are your opinions on this project? What is the scope of this >>>> project? >>>> >>>>>> >>>>>> Linux Kernel 2.6.* currently supports SMP, SMT, NUMA architectures. >>>>>> >>>> >>>> Does the current scheduler has support for "CPU affinity/binding", >>>> mechanism for distinguishing varying capability of CPUs. >>>> >>>>> >>>>> These may be there already in ULE, although I'm not sure about NUMA. >>>>> >>>>> Ray >>>>> >>>>> >>>> >>>> Waiting for your response, >>>> >>>> >>> >>> I note you sent this same note to the netbsd mailing lists. ?You might >>> want to do some more investigation before you propose a project. >>> >>> ? Sam >>> >> It was also sent to the DragonFly mailing lists. :-) >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> > > > > -- > Siddharth Prakash Singh > http://www.spsneo.com > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > -- There is a certain impertinence in allowing oneself to be burned for an opinion. -- Anatole France http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From linimon at lonesome.com Wed Feb 25 14:12:22 2009 From: linimon at lonesome.com (Mark Linimon) Date: Wed Feb 25 14:26:09 2009 Subject: x11 status In-Reply-To: <49A4B9ED.5040705@telenix.org> References: <49A4B9ED.5040705@telenix.org> Message-ID: <20090225221221.GA2289@lonesome.com> On Tue, Feb 24, 2009 at 10:24:29PM -0500, Chuck Robey wrote: > Why is that? Is XFree86 not getting any fixes? We didn't have anyone that wanted to support it after we switched the default to xorg. (We actually need more people willing to support xorg, as well). mcl From delphij at delphij.net Wed Feb 25 15:38:50 2009 From: delphij at delphij.net (Xin LI) Date: Wed Feb 25 15:39:17 2009 Subject: How to change an interface? Message-ID: <49A5D67C.8080102@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Just wanted to confirm that the following procedure to change an existing interface: - Remove the symbol in question from all previous FBSD_1.* namespaces with their corresponding Symbol.map files; - Add the new symbol into latest FBSD_1.* namespace, say, FBSD_1.1 for now, into corresponding Symbol.map files; - Create a new file containing the compatibility shims with prefix __ and suffix of something indicating its obsoleteness, e.g. _44bsd. For instance, for function foo(), the shim function would be called __foo_44bsd(); - At the tail of the shim file, add glues for the old symbols like this: __sym_compat(foo, __foo_44bsd, FBSD_1.0); - Double check to make sure that new .so would work with old binaries. Is that correct? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEARECAAYFAkml1nwACgkQi+vbBBjt66CgLwCgojrOaeSyuNHdHQyzxA0+UEMq PREAn0rFE8zpFez0WbccVAir8Nhf/AK0 =MBeo -----END PGP SIGNATURE----- From deischen at freebsd.org Wed Feb 25 16:21:47 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Wed Feb 25 16:21:55 2009 Subject: How to change an interface? In-Reply-To: <49A5D67C.8080102@delphij.net> References: <49A5D67C.8080102@delphij.net> Message-ID: On Wed, 25 Feb 2009, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > Just wanted to confirm that the following procedure to change an > existing interface: > > - Remove the symbol in question from all previous FBSD_1.* namespaces > with their corresponding Symbol.map files; > > - Add the new symbol into latest FBSD_1.* namespace, say, FBSD_1.1 for > now, into corresponding Symbol.map files; > > - Create a new file containing the compatibility shims with prefix __ > and suffix of something indicating its obsoleteness, e.g. _44bsd. For > instance, for function foo(), the shim function would be called > __foo_44bsd(); > > - At the tail of the shim file, add glues for the old symbols like this: > > __sym_compat(foo, __foo_44bsd, FBSD_1.0); > > - Double check to make sure that new .so would work with old binaries. > > Is that correct? Yes, I believe that is correct. I also think that the awk script that processes the Symbol.map files will also warn you if the symbol is multiply declared. -- DE From kientzle at freebsd.org Wed Feb 25 17:27:47 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Wed Feb 25 17:27:54 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: Message-ID: <49A5D6FC.1090800@freebsd.org> Siddharth Prakash Singh wrote: > > I am a student from India. I am willing to contribute to FreeBSD > as a google soc participant this year. I would like to suggest an > idea. > Title: Multicore Aware Process Scheduler. > I have not gone through the process scheduler code of Free BSD. > Hence, I am not yet aware about the current support for Multicore > Architectures. Since you posted to a lot of different lists, I think you probably don't already use FreeBSD. (If you did, why would you post to NetBSD and DragonflyBSD lists?) Scheduler work is quite complex and interacts heavily with the rest of the system; it may not be a good choice for someone who doesn't already have a lot of experience with FreeBSD. You should probably tell us a little more about yourself: What kind of student? Graduate? Undergraduate? Are you in a CS program or some other engineering program? Do you use FreeBSD? How long have you used it? What do you do with it? Have you read Kirk McKusick's book on FreeBSD internals? Have you built and installed a FreeBSD system from source code? Have you taken classes on OS internals? How much C programming have you done? What areas of FreeBSD have you had the most problems with? How would you make those areas better? Tim Kientzle From fbsd.hackers at rachie.is-a-geek.net Wed Feb 25 18:24:42 2009 From: fbsd.hackers at rachie.is-a-geek.net (Mel) Date: Wed Feb 25 18:24:49 2009 Subject: [PATCH] Support for thresholds in du(1) Message-ID: <200902251724.40212.fbsd.hackers@rachie.is-a-geek.net> Hi, attached is a small patch to add threshold support to du(1). I've been using it on 7-STABLE machines for a while, cause I got tired of the noise I get when sorting and then reformatting to human-readable. Especially since sorting isn't part of the equasion "I'd like to see all dirs exceeding a given size". I've not updated the manpage on -STABLE yet, should be the same as HEAD. Example usage: # du -xht 20m . 29M ./contrib/binutils 52M ./contrib/gcc 237M ./contrib 35M ./crypto 28M ./lib 20M ./share 55M ./sys/dev 139M ./sys 545M . I'll file a PR for it, if there's no objections to this feature / implementation, the style(9) or the usage of -t. -- Mel -------------- next part -------------- A non-text attachment was scrubbed... Name: du.HEAD.diff Type: text/x-diff Size: 2973 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090226/052d8675/du.HEAD.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: du.RELENG_7.diff Type: text/x-diff Size: 2046 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090226/052d8675/du.RELENG_7.bin From delphij at delphij.net Wed Feb 25 19:37:03 2009 From: delphij at delphij.net (Xin LI) Date: Wed Feb 25 19:37:09 2009 Subject: [PATCH] Support for thresholds in du(1) In-Reply-To: <200902251724.40212.fbsd.hackers@rachie.is-a-geek.net> References: <200902251724.40212.fbsd.hackers@rachie.is-a-geek.net> Message-ID: <49A60E4D.5040605@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mel wrote: [...] > I'll file a PR for it, if there's no objections to this feature / > implementation, the style(9) or the usage of -t. One comment: you may want to consider using expand_number(3) instead of rolling your own version Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEARECAAYFAkmmDk0ACgkQi+vbBBjt66A1pgCghpIS/bOgflo0JKNBlKZlBzDf H0IAn21ZJNk1fT6YWDzO0e6nK4dUmJd0 =FFjB -----END PGP SIGNATURE----- From fbsd.hackers at rachie.is-a-geek.net Wed Feb 25 20:03:22 2009 From: fbsd.hackers at rachie.is-a-geek.net (Mel) Date: Wed Feb 25 20:03:28 2009 Subject: [PATCH] Support for thresholds in du(1) In-Reply-To: <49A60E4D.5040605@delphij.net> References: <200902251724.40212.fbsd.hackers@rachie.is-a-geek.net> <49A60E4D.5040605@delphij.net> Message-ID: <200902251903.20430.fbsd.hackers@rachie.is-a-geek.net> On Wednesday 25 February 2009 18:36:45 Xin LI wrote: > Mel wrote: > [...] > > > I'll file a PR for it, if there's no objections to this feature / > > implementation, the style(9) or the usage of -t. > > One comment: you may want to consider using expand_number(3) instead of > rolling your own version Cool thanks, didn't know about that one and was actually considering a request for this API. Maybe strtonum/atoi/stroll and friends should .Xr this API? -- Mel From delphij at delphij.net Wed Feb 25 20:25:05 2009 From: delphij at delphij.net (Xin LI) Date: Wed Feb 25 20:25:11 2009 Subject: [PATCH] Support for thresholds in du(1) In-Reply-To: <200902251903.20430.fbsd.hackers@rachie.is-a-geek.net> References: <200902251724.40212.fbsd.hackers@rachie.is-a-geek.net> <49A60E4D.5040605@delphij.net> <200902251903.20430.fbsd.hackers@rachie.is-a-geek.net> Message-ID: <49A61995.5040500@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mel wrote: > On Wednesday 25 February 2009 18:36:45 Xin LI wrote: >> Mel wrote: >> [...] >> >>> I'll file a PR for it, if there's no objections to this feature / >>> implementation, the style(9) or the usage of -t. >> One comment: you may want to consider using expand_number(3) instead of >> rolling your own version > > Cool thanks, didn't know about that one and was actually considering a request > for this API. Maybe strtonum/atoi/stroll and friends should .Xr this API? Maybe, perhaps also humanize_numbers... We may also want NetBSD's dehumanize_number as well... - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEARECAAYFAkmmGZUACgkQi+vbBBjt66DX0ACfSge9+MA5P98MOYf4LfF+rKn8 Nq8AoLCko53l4YKQrZUQW1PJp9oUVAw4 =6ok+ -----END PGP SIGNATURE----- From spsneo at gmail.com Thu Feb 26 04:23:52 2009 From: spsneo at gmail.com (Siddharth Prakash Singh) Date: Thu Feb 26 04:24:11 2009 Subject: Google SoC 2009 Idea In-Reply-To: <49A5D6FC.1090800@freebsd.org> References: <49A5D6FC.1090800@freebsd.org> Message-ID: On Thu, Feb 26, 2009 at 5:10 AM, Tim Kientzle wrote: > Siddharth Prakash Singh wrote: >> >> I am a student from India. I am willing to contribute to FreeBSD >> as a google soc participant this year. I would like to suggest an >> idea. >> Title: Multicore Aware Process Scheduler. >> I have not gone through the process scheduler code of Free BSD. >> Hence, I am not yet aware about the current support for Multicore >> Architectures. > > Since you posted to a lot of different lists, I > think you probably don't already use FreeBSD. > (If you did, why would you post to NetBSD and > DragonflyBSD lists?) ?Scheduler work is quite complex > and interacts heavily with the rest of the system; > it may not be a good choice for someone who doesn't > already have a lot of experience with FreeBSD. > > You should probably tell us a little more about yourself: > > What kind of student? ?Graduate? ?Undergraduate? > Are you in a CS program or some other engineering program? I am junior undergraduate student at CS department of Indian Institute Of Technology, India. > > Do you use FreeBSD? ?How long have you used it? > What do you do with it? No, I haven't really used FreeBSD much. I have been using Linux since long! > > Have you read Kirk McKusick's book on FreeBSD internals? No. But I have sound knowledge of Linux and Minix Kernel. > > Have you built and installed a FreeBSD system from source code? No, But I have built and installed a Linux system from source code. > > Have you taken classes on OS internals? Yes, I have taken a course on OS internal last semester only. Ans I worked on Pintos Kernel as a part of the course. > > How much C programming have you done? I have a tremendous experience of C coding. > > What areas of FreeBSD have you had the most problems with? > How would you make those areas better? > > Tim Kientzle > I am aware that I am new to FreeBSD, but that doesn't really mean I am not qualified. -- Siddharth Prakash Singh http://www.spsneo.com From gary.jennejohn at freenet.de Thu Feb 26 04:41:38 2009 From: gary.jennejohn at freenet.de (Gary Jennejohn) Date: Thu Feb 26 04:41:45 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: <49A5D6FC.1090800@freebsd.org> Message-ID: <20090226134135.156d3b86@ernst.jennejohn.org> On Thu, 26 Feb 2009 17:53:48 +0530 Siddharth Prakash Singh wrote: > > Do you use FreeBSD? __How long have you used it? > > What do you do with it? > No, I haven't really used FreeBSD much. I have been using Linux since long! > > > > Have you read Kirk McKusick's book on FreeBSD internals? > No. But I have sound knowledge of Linux and Minix Kernel. > > Having done development on both FreeBSD and Linux I can authoritatively state that they're totally different animals. But just because you only have Linux knowledge doesn't exclude you from doing FreeBSD development. It will just take a while to get up to speed. --- Gary Jennejohn From rwatson at FreeBSD.org Thu Feb 26 08:23:21 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Feb 26 08:23:27 2009 Subject: Google SoC 2009 Idea In-Reply-To: <49A5D6FC.1090800@freebsd.org> References: <49A5D6FC.1090800@freebsd.org> Message-ID: On Wed, 25 Feb 2009, Tim Kientzle wrote: >> I have not gone through the process scheduler code of Free BSD. Hence, I am >> not yet aware about the current support for Multicore Architectures. > > Since you posted to a lot of different lists, I think you probably don't > already use FreeBSD. (If you did, why would you post to NetBSD and > DragonflyBSD lists?) Scheduler work is quite complex and interacts heavily > with the rest of the system; it may not be a good choice for someone who > doesn't already have a lot of experience with FreeBSD. All the things you say are true, but let's not be too hard on the new guy, however -- many of our GSoC students don't have previous FreeBSD kernel-hacking experience. However, it does mean that they have to pick project ideas that are well-suited to a significant warmup and investigation period on the front end of the project. I'm also not convinced that a scheduler project along these lines would be the most successful, but I wonder if a more experimental-spin proposal for looking at how to investigate poor scheduling decisions using dtrace, instrumentation and metrics to help us understand performance on NUMA systems, and exploring the impact of heuristics might go a long way. As our ULE scheduler has most of the non-NUMA features suggested in the original already, it would actually be a nice starting point for this. I understand Jeff Roberson has been doing some initial looking at NUMA, but more from a memory placement and less from a scheduling perspective? Robert N M Watson Computer Laboratory University of Cambridge > > You should probably tell us a little more about yourself: > > What kind of student? Graduate? Undergraduate? > Are you in a CS program or some other engineering program? > > Do you use FreeBSD? How long have you used it? > What do you do with it? > > Have you read Kirk McKusick's book on FreeBSD internals? > > Have you built and installed a FreeBSD system from source code? > > Have you taken classes on OS internals? > > How much C programming have you done? > > What areas of FreeBSD have you had the most problems with? > How would you make those areas better? > > Tim Kientzle > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From kientzle at freebsd.org Thu Feb 26 09:19:37 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Thu Feb 26 09:19:43 2009 Subject: Google SoC 2009 Idea In-Reply-To: References: <49A5D6FC.1090800@freebsd.org> Message-ID: <49A6CF27.3000203@freebsd.org> Robert Watson wrote: > On Wed, 25 Feb 2009, Tim Kientzle wrote: > >>> I have not gone through the process scheduler code of Free BSD. >>> Hence, I am not yet aware about the current support for Multicore >>> Architectures. >> >> Since you posted to a lot of different lists, I think you probably >> don't already use FreeBSD. (If you did, why would you post to NetBSD >> and DragonflyBSD lists?) Scheduler work is quite complex and >> interacts heavily with the rest of the system; it may not be a good >> choice for someone who doesn't already have a lot of experience with >> FreeBSD. > > All the things you say are true, but let's not be too hard on the new > guy, however -- many of our GSoC students don't have previous FreeBSD > kernel-hacking experience. However, it does mean that they have to pick > project ideas that are well-suited to a significant warmup and > investigation period on the front end of the project. I apologize to Siddharth and others if I came off overly harsh. My intention was to caution him that he should plan for a lot of work prior to GSoC if he wants to tackle something that's at the core of the OS like this. > I'm also not convinced that a scheduler project along these lines would > be the most successful, but I wonder if a more experimental-spin > proposal for looking at how to investigate poor scheduling decisions > using dtrace, instrumentation and metrics to help us understand > performance on NUMA systems, and exploring the impact of heuristics > might go a long way. That's a good idea. The thing that's always impressed me about scheduling work is how very difficult it is to test. It's easy to change the scheduler code; it's much harder to measure whether those changes have made the scheduler better or not. Some testing support would help. Ideally, something non-intrusive that could be easily run on a lot of different machines so as to collect better information about the impacts of scheduler changes: * Load balancing: How effectively are all cores being used? * CPU switching: What percentage of the time does a thread stay on the same core? * NUMA statistics: How often does a thread get scheduled on a different processor from it's allocated memory? * Priority inversion: How often is a higher-priority thread idle while a lower-priority thread is running? A student who built such a tool and then ran some tests with a variety of hardware and workloads could really do a lot to advance scheduler development. Eventually, turning such a tool into something that anyone could run and upload data to a central collection site could be a huge advance. Certainly something to think about... Tim From yanefbsd at gmail.com Thu Feb 26 09:24:24 2009 From: yanefbsd at gmail.com (Garrett Cooper) Date: Thu Feb 26 09:24:30 2009 Subject: Google SoC 2009 Idea In-Reply-To: <49A6CF27.3000203@freebsd.org> References: <49A5D6FC.1090800@freebsd.org> <49A6CF27.3000203@freebsd.org> Message-ID: <7d6fde3d0902260924r45ebb7c8i46cd6daf43a8171d@mail.gmail.com> On Thu, Feb 26, 2009 at 9:19 AM, Tim Kientzle wrote: > Robert Watson wrote: >> >> On Wed, 25 Feb 2009, Tim Kientzle wrote: >> >>>> I have not gone through the process scheduler code of Free BSD. Hence, I >>>> am not yet aware about the current support for Multicore Architectures. >>> >>> Since you posted to a lot of different lists, I think you probably don't >>> already use FreeBSD. (If you did, why would you post to NetBSD and >>> DragonflyBSD lists?) ?Scheduler work is quite complex and interacts heavily >>> with the rest of the system; it may not be a good choice for someone who >>> doesn't already have a lot of experience with FreeBSD. >> >> All the things you say are true, but let's not be too hard on the new guy, >> however -- many of our GSoC students don't have previous FreeBSD >> kernel-hacking experience. ?However, it does mean that they have to pick >> project ideas that are well-suited to a significant warmup and investigation >> period on the front end of the project. > > I apologize to Siddharth and others if I came off overly > harsh. ?My intention was to caution him that he should > plan for a lot of work prior to GSoC if he wants to > tackle something that's at the core of the OS like this. > >> I'm also not convinced that a scheduler project along these lines would be >> the most successful, but I wonder if a more experimental-spin proposal for >> looking at how to investigate poor scheduling decisions using dtrace, >> instrumentation and metrics to help us understand performance on NUMA >> systems, and exploring the impact of heuristics might go a long way. > > That's a good idea. ?The thing that's always impressed > me about scheduling work is how very difficult it is to > test. ?It's easy to change the scheduler code; it's > much harder to measure whether those changes have > made the scheduler better or not. > > Some testing support would help. ?Ideally, something > non-intrusive that could be easily run on a lot > of different machines so as to collect better information > about the impacts of scheduler changes: > ?* Load balancing: ?How effectively are all cores being used? > ?* CPU switching: ?What percentage of the time does a thread > stay on the same core? > ?* NUMA statistics: ?How often does a thread get scheduled on a different > processor from it's allocated memory? > ?* Priority inversion: ?How often is a higher-priority thread > idle while a lower-priority thread is running? > > A student who built such a tool and then ran some tests > with a variety of hardware and workloads could really > do a lot to advance scheduler development. ?Eventually, > turning such a tool into something that anyone could run > and upload data to a central collection site could be > a huge advance. Speaking from experience, this is the way to go. If you don't do this you'll end up producing a potential black hole in terms of time and resources, which doesn't help your reputation on the project. Some food for thought. Cheers, -Garrett From ed at 80386.nl Thu Feb 26 10:07:58 2009 From: ed at 80386.nl (Ed Schouten) Date: Thu Feb 26 10:08:05 2009 Subject: Renaming all symbols in libmp(3) Message-ID: <20090226180756.GX19161@hoeg.nl> Hello all, As you may have read some days ago, work has started to make the FreeBSD base system compile better with clang, the BSD licensed C compiler from the LLVM project. One of the reasons why we can't compile the base system yet, is because some applications in the base system (keyserv, newkey, chkey, libtelnet) won't compile, because a library they depend (libmp)on has a function called pow(). By default, LLVM has a built-in prototype of pow(), similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The manual page also mentions this issue. After some talking on IRC, I think the best solution would be to rename all functions in libmp. Even though this may sound very awful at first, it seems the competition has done this as well: http://docs.sun.com/app/docs/doc/819-2246/mp-3mp?a=view I'm not saying that if the Solaris developers jump off a cliff, we should do the same, but looking at the amount of applications that depend on libmp (almost none), I think it's the only definitive solution. So this is the patch I propose to commit to SVN one of these days: http://80386.nl/pub/mp.diff Because this also reduces some warnings when compiling stuff with GCC, I've increased WARNS in various Makefiles. I've also increased the SHLIB_MAJOR and __FreeBSD_version. Any comments? -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090226/6e87bab1/attachment.pgp From peterjeremy at optushome.com.au Thu Feb 26 10:13:01 2009 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Thu Feb 26 10:13:07 2009 Subject: x11 status In-Reply-To: <20090225065308.GO19161@hoeg.nl> References: <49A4B9ED.5040705@telenix.org> <20090225065308.GO19161@hoeg.nl> Message-ID: <20090226181225.GA3540@server.vk2pj.dyndns.org> On 2009-Feb-25 07:53:08 +0100, Ed Schouten wrote: >The XFree86 project has been dying ever since almost all the active >development moved to the Xorg-project. Xorg has many new features that >XFree86 doesn't have, like hardware compositing and improved device >detection. And along the way, they've dropped things like integration testing, avoiding regressions and avoiding POLA violations. >> latest cvs image from Xfree86, and it built FAR easier that xorg, far >> faster, far simpler to configure ... > >Why should it matter how easy it is to build a piece of software? You >can just run `make -C /usr/ports/x11/xorg install clean' or `pkg_add -r >xorg'. Note that Chuck also mentioned faster (the conversion from imake to configure added something like 30% to the time to build X.org for absolutely no benefit - some pieces of X.org now take 4 times as long to configure as to build) and easier to configure. Whilst the ease of building a port doesn't really affect the end user, it does affect the port maintainer - a port that needs lots of tender care and feeding will lead to more rapid maintainer burnout. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090226/7b71e23a/attachment.pgp From deischen at freebsd.org Thu Feb 26 10:18:40 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Thu Feb 26 10:18:48 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090226180756.GX19161@hoeg.nl> References: <20090226180756.GX19161@hoeg.nl> Message-ID: On Thu, 26 Feb 2009, Ed Schouten wrote: > Hello all, > > As you may have read some days ago, work has started to make the FreeBSD > base system compile better with clang, the BSD licensed C compiler from > the LLVM project. > > One of the reasons why we can't compile the base system yet, is because > some applications in the base system (keyserv, newkey, chkey, libtelnet) > won't compile, because a library they depend (libmp)on has a function > called pow(). By default, LLVM has a built-in prototype of pow(), > similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The > manual page also mentions this issue. > > After some talking on IRC, I think the best solution would be to rename > all functions in libmp. Even though this may sound very awful at first, > it seems the competition has done this as well: > > http://docs.sun.com/app/docs/doc/819-2246/mp-3mp?a=view > > I'm not saying that if the Solaris developers jump off a cliff, we > should do the same, but looking at the amount of applications that > depend on libmp (almost none), I think it's the only definitive > solution. > > So this is the patch I propose to commit to SVN one of these days: > > http://80386.nl/pub/mp.diff > > Because this also reduces some warnings when compiling stuff with GCC, > I've increased WARNS in various Makefiles. I've also increased the > SHLIB_MAJOR and __FreeBSD_version. Why don't you add symbol versioning to libmp, so that old binaries will still work, but new ones will get the new symbols by default. Hmm, will that work without bumping SHLIB_MAJOR? You might want to play around with it and see. -- DE From ed at 80386.nl Thu Feb 26 10:25:44 2009 From: ed at 80386.nl (Ed Schouten) Date: Thu Feb 26 10:25:51 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: References: <20090226180756.GX19161@hoeg.nl> Message-ID: <20090226182543.GY19161@hoeg.nl> * Daniel Eischen wrote: > Why don't you add symbol versioning to libmp, so that old > binaries will still work, but new ones will get the new > symbols by default. Hmm, will that work without bumping > SHLIB_MAJOR? You might want to play around with it and > see. Well, even without symbol versioning this could be done, by just making a __strong_reference() between the symbols, but I decided not to do so. I think solutions like these are perfect when just renaming/removing a couple of symbols, but because we're basically touching everything, I thought we'd better just use the old approach. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090226/11b0edd6/attachment.pgp From deischen at freebsd.org Thu Feb 26 10:31:15 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Thu Feb 26 10:31:22 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090226182543.GY19161@hoeg.nl> References: <20090226180756.GX19161@hoeg.nl> <20090226182543.GY19161@hoeg.nl> Message-ID: On Thu, 26 Feb 2009, Ed Schouten wrote: > * Daniel Eischen wrote: >> Why don't you add symbol versioning to libmp, so that old >> binaries will still work, but new ones will get the new >> symbols by default. Hmm, will that work without bumping >> SHLIB_MAJOR? You might want to play around with it and >> see. > > Well, even without symbol versioning this could be done, by just making > a __strong_reference() between the symbols, but I decided not to do so. > I think solutions like these are perfect when just renaming/removing a > couple of symbols, but because we're basically touching everything, I > thought we'd better just use the old approach. Well, as long as you're in there, maybe you should add symbol versioning anyway, even after a library version bump. Seems like it would be easy since there aren't that many symbols. -- DE From ed at 80386.nl Thu Feb 26 10:40:23 2009 From: ed at 80386.nl (Ed Schouten) Date: Thu Feb 26 10:42:11 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: References: <20090226180756.GX19161@hoeg.nl> <20090226182543.GY19161@hoeg.nl> Message-ID: <20090226184021.GZ19161@hoeg.nl> * Daniel Eischen wrote: > Well, as long as you're in there, maybe you should add > symbol versioning anyway, even after a library version > bump. Seems like it would be easy since there aren't > that many symbols. I assume I should just mark all symbols with version FBSD_1.1? If so, the following patch should be good enough: http://80386.nl/pub/mp.diff -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090226/eddb79c0/attachment.pgp From alex-goncharov at comcast.net Thu Feb 26 12:10:58 2009 From: alex-goncharov at comcast.net (Alex Goncharov) Date: Thu Feb 26 12:11:05 2009 Subject: x11 status In-Reply-To: <20090226181225.GA3540@server.vk2pj.dyndns.org> (message from Peter Jeremy on Fri, 27 Feb 2009 05:12:25 +1100) References: <49A4B9ED.5040705@telenix.org> <20090225065308.GO19161@hoeg.nl> <20090226181225.GA3540@server.vk2pj.dyndns.org> Message-ID: ,--- You/Peter (Fri, 27 Feb 2009 05:12:25 +1100) ----* | And along the way, they've dropped things like integration testing, | avoiding regressions and avoiding POLA violations. Since I don't care about the new X anymore, I can afford to express my opinion bluntly: the Xorg we have in ports now is a disaster. This opinion was arrived to at the cost of more than two weeks undoing the damage to my systems under the circumstances not conductive for such activities. | >> latest cvs image from Xfree86, and it built FAR easier that xorg, | >> faster, far simpler to configure ... | > | >Why should it matter how easy it is to build a piece of software? You | >can just run `make -C /usr/ports/x11/xorg install clean' or `pkg_add -r | >xorg'. | | Note that Chuck also mentioned faster (the conversion from imake to | configure added something like 30% to the time to build X.org for | absolutely no benefit - some pieces of X.org now take 4 times as long | to configure as to build) and easier to configure. | | Whilst the ease of building a port doesn't really affect the end | user, I strongly disagree: a FreeBSD user is almost by definition somebody who ultimately turns from using built packages to building ports from source, with options of personal preference. So, how things are done build-wise, does affect me, big time. If the only option to get the contributed (ports) software were using packages, I'd be using Debian, not FreeBSD. I have no desire to have HAL on my systems, for example, and FreeBSD had been giving me the option of not having it -- just build it with an appropriate option in /etc/make.conf. And other such things. Struggling with the Xorg 1.5 unfortunate upgrade, and examining the ports make files for Xorg/Gnome moved me to the ultimate decision: don't try to comprehend this mess and stick with something that works. I rolled back everything x11 to "xorg-server 1.4.2" and have no plans to upgrade it -- ever, on any of my systems: the old one works, perhaps not perfectly, but predictably, and why would I need a new one? So, for the last month I've been doing weekly rebuilds of ports with everything upgraded -- other that the /usr/ports/x11* trees, which will be frozen until I see that other people stop reporting serious Xorg problems, which may well never happen. This is a testament to the greatness of the ports system and an illustration to my claim: building a port does affect the end user. -- Alex -- alex-goncharov@comcast.net -- From das at FreeBSD.ORG Thu Feb 26 12:40:37 2009 From: das at FreeBSD.ORG (David Schultz) Date: Thu Feb 26 12:40:43 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090226180756.GX19161@hoeg.nl> References: <20090226180756.GX19161@hoeg.nl> Message-ID: <20090226204243.GA96251@zim.MIT.EDU> On Thu, Feb 26, 2009, Ed Schouten wrote: > One of the reasons why we can't compile the base system yet, is because > some applications in the base system (keyserv, newkey, chkey, libtelnet) > won't compile, because a library they depend (libmp)on has a function > called pow(). By default, LLVM has a built-in prototype of pow(), > similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The > manual page also mentions this issue. I think most apps that used to use libmp have transitioned to libgmp, so I don't have much of an opinion on this change... However, if the compiler as a builtin for the math.h-style pow() function, and the builtin causes it to choke even when math.h isn't #included, that's a bug in the compiler. The people who are proposing that we make the base system LLVM-compatible should be more forceful in insisting that LLVM be fixed. ;-) What do the LLVM folks propose to do about all the (perfectly legal) programs out there that have a variable called 'exp'? From christoph.mallon at gmx.de Thu Feb 26 12:50:30 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Feb 26 12:50:36 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090226204243.GA96251@zim.MIT.EDU> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> Message-ID: <49A70092.6030601@gmx.de> David Schultz schrieb: > On Thu, Feb 26, 2009, Ed Schouten wrote: >> One of the reasons why we can't compile the base system yet, is because >> some applications in the base system (keyserv, newkey, chkey, libtelnet) >> won't compile, because a library they depend (libmp)on has a function >> called pow(). By default, LLVM has a built-in prototype of pow(), >> similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The >> manual page also mentions this issue. > > I think most apps that used to use libmp have transitioned to > libgmp, so I don't have much of an opinion on this change... > > However, if the compiler as a builtin for the math.h-style pow() > function, and the builtin causes it to choke even when math.h > isn't #included, that's a bug in the compiler. The people who are > proposing that we make the base system LLVM-compatible should be > more forceful in insisting that LLVM be fixed. ;-) What do the LLVM > folks propose to do about all the (perfectly legal) programs out > there that have a variable called 'exp'? No, it's invalid code to have a function named pow() in a hosted environment which is not /The/ pow(). Having a *local* variable named exp is fine, because this declaration shadows the function. A *global* variable on the other hand is not allowed. From christoph.mallon at gmx.de Thu Feb 26 13:02:32 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Feb 26 13:02:39 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <49A70092.6030601@gmx.de> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> Message-ID: <49A70364.1020707@gmx.de> Christoph Mallon schrieb: > David Schultz schrieb: >> On Thu, Feb 26, 2009, Ed Schouten wrote: >>> One of the reasons why we can't compile the base system yet, is because >>> some applications in the base system (keyserv, newkey, chkey, libtelnet) >>> won't compile, because a library they depend (libmp)on has a function >>> called pow(). By default, LLVM has a built-in prototype of pow(), >>> similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The >>> manual page also mentions this issue. >> >> I think most apps that used to use libmp have transitioned to >> libgmp, so I don't have much of an opinion on this change... >> >> However, if the compiler as a builtin for the math.h-style pow() >> function, and the builtin causes it to choke even when math.h >> isn't #included, that's a bug in the compiler. The people who are >> proposing that we make the base system LLVM-compatible should be >> more forceful in insisting that LLVM be fixed. ;-) What do the LLVM >> folks propose to do about all the (perfectly legal) programs out >> there that have a variable called 'exp'? > > No, it's invalid code to have a function named pow() in a hosted > environment which is not /The/ pow(). > Having a *local* variable named exp is fine, because this declaration > shadows the function. A *global* variable on the other hand is not allowed. Just a hint, why you really mustn't use global names which are reserved by the standard: double pow(double x, double y) { abort(); } int main(void) { printf("%f\n", pow(10., 2.)); return 0; } %gcc -w bla.c %./a.out 100.000000 % Yes, GCC does that and it's perfectly valid. From deischen at freebsd.org Thu Feb 26 13:06:09 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Thu Feb 26 13:06:16 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090226184021.GZ19161@hoeg.nl> References: <20090226180756.GX19161@hoeg.nl> <20090226182543.GY19161@hoeg.nl> <20090226184021.GZ19161@hoeg.nl> Message-ID: On Thu, 26 Feb 2009, Ed Schouten wrote: > * Daniel Eischen wrote: >> Well, as long as you're in there, maybe you should add >> symbol versioning anyway, even after a library version >> bump. Seems like it would be easy since there aren't >> that many symbols. > > I assume I should just mark all symbols with version FBSD_1.1? If so, > the following patch should be good enough: > > http://80386.nl/pub/mp.diff Yeah, looks about right. That was easy :-) -- DE From das at FreeBSD.ORG Thu Feb 26 14:11:55 2009 From: das at FreeBSD.ORG (David Schultz) Date: Thu Feb 26 14:12:01 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <49A70092.6030601@gmx.de> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> Message-ID: <20090226221403.GA96580@zim.MIT.EDU> On Thu, Feb 26, 2009, Christoph Mallon wrote: > David Schultz schrieb: > >On Thu, Feb 26, 2009, Ed Schouten wrote: > >>One of the reasons why we can't compile the base system yet, is because > >>some applications in the base system (keyserv, newkey, chkey, libtelnet) > >>won't compile, because a library they depend (libmp)on has a function > >>called pow(). By default, LLVM has a built-in prototype of pow(), > >>similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The > >>manual page also mentions this issue. > > > >I think most apps that used to use libmp have transitioned to > >libgmp, so I don't have much of an opinion on this change... > > > >However, if the compiler as a builtin for the math.h-style pow() > >function, and the builtin causes it to choke even when math.h > >isn't #included, that's a bug in the compiler. The people who are > >proposing that we make the base system LLVM-compatible should be > >more forceful in insisting that LLVM be fixed. ;-) What do the LLVM > >folks propose to do about all the (perfectly legal) programs out > >there that have a variable called 'exp'? > > No, it's invalid code to have a function named pow() in a hosted > environment which is not /The/ pow(). > Having a *local* variable named exp is fine, because this declaration > shadows the function. A *global* variable on the other hand is not allowed. The C standard makes no guarantees because it doesn't want to say much about system libraries or how apps are actually linked, but in practice people have found it very useful to do things like link against libraries with special debugging versions of malloc(), and in the past, developers have tried to ensure that this continues to work. (You can find some old threads in the archives.) Of course, people who do this need to take adequate precautions (e.g., linking their programs in the right order, using weak symbols where appropriate), but it does work, and it would be nice if the compiler didn't break it. As for gcc's math builtins, most of them are buggy. They fail to respect the dynamic rounding mode, fail to generate exceptions where appropriate, fail to respect FENV_ACCESS and other pragmas, etc. Also, the complex builtins use simplified formulas that don't get the right answers for complex numbers with inf/nan components. Try running some of the tests in tools/regression/lib/msun without -fno-builtin and see what happens ;-) From christoph.mallon at gmx.de Thu Feb 26 14:26:40 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Feb 26 14:27:38 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090226221403.GA96580@zim.MIT.EDU> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <20090226221403.GA96580@zim.MIT.EDU> Message-ID: <49A7171D.1060401@gmx.de> David Schultz schrieb: > On Thu, Feb 26, 2009, Christoph Mallon wrote: >> David Schultz schrieb: >>> On Thu, Feb 26, 2009, Ed Schouten wrote: >>>> One of the reasons why we can't compile the base system yet, is because >>>> some applications in the base system (keyserv, newkey, chkey, libtelnet) >>>> won't compile, because a library they depend (libmp)on has a function >>>> called pow(). By default, LLVM has a built-in prototype of pow(), >>>> similar to GCC. Unlike GCC, LLVM raises a compiler error by default. The >>>> manual page also mentions this issue. >>> I think most apps that used to use libmp have transitioned to >>> libgmp, so I don't have much of an opinion on this change... >>> >>> However, if the compiler as a builtin for the math.h-style pow() >>> function, and the builtin causes it to choke even when math.h >>> isn't #included, that's a bug in the compiler. The people who are >>> proposing that we make the base system LLVM-compatible should be >>> more forceful in insisting that LLVM be fixed. ;-) What do the LLVM >>> folks propose to do about all the (perfectly legal) programs out >>> there that have a variable called 'exp'? >> No, it's invalid code to have a function named pow() in a hosted >> environment which is not /The/ pow(). >> Having a *local* variable named exp is fine, because this declaration >> shadows the function. A *global* variable on the other hand is not allowed. > > The C standard makes no guarantees because it doesn't want to say > much about system libraries or how apps are actually linked, but The C standard is very explicit which identifiers are reserved, see ?7.1.3 > in practice people have found it very useful to do things like > link against libraries with special debugging versions of > malloc(), and in the past, developers have tried to ensure that > this continues to work. (You can find some old threads in the > archives.) Of course, people who do this need to take adequate > precautions (e.g., linking their programs in the right order, > using weak symbols where appropriate), but it does work, and it > would be nice if the compiler didn't break it. > > As for gcc's math builtins, most of them are buggy. They fail to > respect the dynamic rounding mode, fail to generate exceptions > where appropriate, fail to respect FENV_ACCESS and other pragmas, > etc. Also, the complex builtins use simplified formulas that don't > get the right answers for complex numbers with inf/nan components. > Try running some of the tests in tools/regression/lib/msun without > -fno-builtin and see what happens ;-) pow() is just an example. The compiler may do magic with any call which has semantics defined by the C standard in a hosted environment. From neldredge at math.ucsd.edu Thu Feb 26 21:25:48 2009 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu Feb 26 21:25:55 2009 Subject: portupgrade spurious skips Message-ID: Hi folks, In the past few months I've noticed a bug in portupgrade. When I update my ports tree and do `portupgrade -a', often a few ports will be skipped, supposedly because another port on which they depend failed to install. However, the apparently failed port actually did not fail, and if I rerun `portupgrade -a', some of the skipped ports will install successfully without complaint. After enough iterations I can eventually get all of them. I'd like to file a PR about this, but it's a little bit tricky coming up with a test case, since the behavior depends on having outdated packages installed, and on the dependencies between them. Moreover, after I run `portupgrade -a' and notice the problem, the state of the installed packages has changed and the same packages aren't skipped the next time. So my question is whether anyone has ideas about how to construct a reasonable test case that could help me make this reproducible and easier to investigate. Any thoughts? Thanks in advance. -- Nate Eldredge neldredge@math.ucsd.edu From das at FreeBSD.ORG Thu Feb 26 21:26:11 2009 From: das at FreeBSD.ORG (David Schultz) Date: Thu Feb 26 21:26:17 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <49A7171D.1060401@gmx.de> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <20090226221403.GA96580@zim.MIT.EDU> <49A7171D.1060401@gmx.de> Message-ID: <20090227052821.GA98527@zim.MIT.EDU> On Thu, Feb 26, 2009, Christoph Mallon wrote: > David Schultz schrieb: > >As for gcc's math builtins, most of them are buggy. They fail to > >respect the dynamic rounding mode, fail to generate exceptions > >where appropriate, fail to respect FENV_ACCESS and other pragmas, > >etc. Also, the complex builtins use simplified formulas that don't > >get the right answers for complex numbers with inf/nan components. > >Try running some of the tests in tools/regression/lib/msun without > >-fno-builtin and see what happens ;-) > > pow() is just an example. > The compiler may do magic with any call which has semantics defined by > the C standard in a hosted environment. Okay, so first, the world doesn't revolve around the strictest possible interpretation of the C standard. For example, FreeBSD has an extension such that `printf("%s", NULL)' prints "(null)" instead of having undefined behavior. But gcc translates this into `puts(NULL)', which once again has undefined behavior. You can put on your lawyer hat and counter that the beloved standard doesn't guarantee that any such thing will work, but that's a very contrarian attitude; the bottom line is that it's a POLA violation, and the extension worked just fine for many years. Second, the problems with the math builtins I cited above violate even a strict interpretation of the C standard. Third, this is a digression, and this is the last I'm going to say about it. :-) From dnelson at allantgroup.com Thu Feb 26 22:13:03 2009 From: dnelson at allantgroup.com (Dan Nelson) Date: Thu Feb 26 22:13:10 2009 Subject: portupgrade spurious skips In-Reply-To: References: Message-ID: <20090227055737.GF45976@dan.emsphone.com> In the last episode (Feb 26), Nate Eldredge said: > In the past few months I've noticed a bug in portupgrade. When I update > my ports tree and do `portupgrade -a', often a few ports will be skipped, > supposedly because another port on which they depend failed to install. > However, the apparently failed port actually did not fail, and if I rerun > `portupgrade -a', some of the skipped ports will install successfully > without complaint. After enough iterations I can eventually get all of > them. > > I'd like to file a PR about this, but it's a little bit tricky coming up > with a test case, since the behavior depends on having outdated packages > installed, and on the dependencies between them. Moreover, after I run > `portupgrade -a' and notice the problem, the state of the installed > packages has changed and the same packages aren't skipped the next time. > So my question is whether anyone has ideas about how to construct a > reasonable test case that could help me make this reproducible and easier > to investigate. Any thoughts? "me too".. It seems to happen frequently when doing > 20 or so ports. Every week or so I upgrade old ports and don't have problems, but after a gnome or xorg update that ends up bumping the portrevision for half my installed ports, it always takes three or four "portupgrade -a" loops for everything to buid. If you've got ZFS, you can snapshot your filesystems, and if portupgrade fails, roll back to the snapshot and do it again to see if it happens on the same port a second time. Or if you know ruby, you could instrument the code that checks for port build errors and see if it's got a bug in it... -- Dan Nelson dnelson@allantgroup.com From christoph.mallon at gmx.de Thu Feb 26 22:49:31 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Feb 26 22:49:38 2009 Subject: Renaming all symbols in libmp(3) In-Reply-To: <20090227052821.GA98527@zim.MIT.EDU> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <20090226221403.GA96580@zim.MIT.EDU> <49A7171D.1060401@gmx.de> <20090227052821.GA98527@zim.MIT.EDU> Message-ID: <49A78CF7.8090606@gmx.de> David Schultz schrieb: > On Thu, Feb 26, 2009, Christoph Mallon wrote: >> David Schultz schrieb: >>> As for gcc's math builtins, most of them are buggy. They fail to >>> respect the dynamic rounding mode, fail to generate exceptions >>> where appropriate, fail to respect FENV_ACCESS and other pragmas, >>> etc. Also, the complex builtins use simplified formulas that don't >>> get the right answers for complex numbers with inf/nan components. >>> Try running some of the tests in tools/regression/lib/msun without >>> -fno-builtin and see what happens ;-) >> pow() is just an example. >> The compiler may do magic with any call which has semantics defined by >> the C standard in a hosted environment. > > Okay, so first, the world doesn't revolve around the strictest > possible interpretation of the C standard. For example, FreeBSD So you haven't seen enough crashes and security problems due to sloppy coding in the past decades, yet? I don't agree with everything the standard says (e.g. it tells you that it's a bad idea to call a function strange_quark()), but there's no reason not to avoid a simple name clash. > has an extension such that `printf("%s", NULL)' prints "(null)" > instead of having undefined behavior. But gcc translates this into This *is* perfectly valid undefined behaviour. > `puts(NULL)', which once again has undefined behavior. You can put And this is, too. I prefer the crash variant (or the one with the demons flying out of the programmers nose), because I've seen quite some programs which showed me "(null)" where they should have printed something sensible and bugs (sadly) only seem to have a real chance of being fixed in a timely manner, when they are hard crashes. A program limping along with invalid data is very bad and could be a security problem, too - imagine a "(null)" in an sprintf()ed path or something. > on your lawyer hat and counter that the beloved standard doesn't > guarantee that any such thing will work, but that's a very > contrarian attitude; the bottom line is that it's a POLA > violation, and the extension worked just fine for many years. It's a POLA violation in the first place, to have a function named pow(), which does not the "real" pow() thing. > Second, the problems with the math builtins I cited above violate > even a strict interpretation of the C standard. GCC is buggy, but that's a totally different story. > Third, this is a digression, and this is the last I'm going to say > about it. :-) Me, too. From perryh at pluto.rain.com Fri Feb 27 00:24:38 2009 From: perryh at pluto.rain.com (perryh@pluto.rain.com) Date: Fri Feb 27 00:24:45 2009 Subject: hosted, or not (Re: Renaming all symbols in libmp(3)) In-Reply-To: <49A70092.6030601@gmx.de> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> Message-ID: <49a79a4e.jj/fvw29lwxNJgz+%perryh@pluto.rain.com> > >> By default, LLVM has a built-in prototype of pow(), similar to > >> GCC. Unlike GCC, LLVM raises a compiler error by default ... > ... it's invalid code to have a function named pow() > in a hosted environment which is not /The/ pow(). ^^^^^^^^^^^^^^^^^^^^^^^ I don't suppose LLVM supports a commmand-line switch to use embedded mode instead of hosted? From rdivacky at freebsd.org Fri Feb 27 00:27:49 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Fri Feb 27 00:27:56 2009 Subject: hosted, or not (Re: Renaming all symbols in libmp(3)) In-Reply-To: <49a79a4e.jj/fvw29lwxNJgz+%perryh@pluto.rain.com> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <49a79a4e.jj/fvw29lwxNJgz+%perryh@pluto.rain.com> Message-ID: <20090227082417.GA55567@freebsd.org> On Thu, Feb 26, 2009 at 11:46:22PM -0800, perryh@pluto.rain.com wrote: > > >> By default, LLVM has a built-in prototype of pow(), similar to > > >> GCC. Unlike GCC, LLVM raises a compiler error by default ... > > > ... it's invalid code to have a function named pow() > > in a hosted environment which is not /The/ pow(). > ^^^^^^^^^^^^^^^^^^^^^^^ > > I don't suppose LLVM supports a commmand-line switch to use embedded > mode instead of hosted? of course it does.... -ffreestanding From Alexander at Leidinger.net Fri Feb 27 04:19:48 2009 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Fri Feb 27 04:25:54 2009 Subject: portupgrade spurious skips In-Reply-To: <20090227055737.GF45976@dan.emsphone.com> References: <20090227055737.GF45976@dan.emsphone.com> Message-ID: <20090227131931.18306giseysouk8w@webmail.leidinger.net> Quoting Dan Nelson (from Thu, 26 Feb 2009 23:57:37 -0600): > In the last episode (Feb 26), Nate Eldredge said: >> In the past few months I've noticed a bug in portupgrade. When I update >> my ports tree and do `portupgrade -a', often a few ports will be skipped, >> supposedly because another port on which they depend failed to install. >> However, the apparently failed port actually did not fail, and if I rerun >> `portupgrade -a', some of the skipped ports will install successfully >> without complaint. After enough iterations I can eventually get all of >> them. >> >> I'd like to file a PR about this, but it's a little bit tricky coming up >> with a test case, since the behavior depends on having outdated packages >> installed, and on the dependencies between them. Moreover, after I run >> `portupgrade -a' and notice the problem, the state of the installed >> packages has changed and the same packages aren't skipped the next time. >> So my question is whether anyone has ideas about how to construct a >> reasonable test case that could help me make this reproducible and easier >> to investigate. Any thoughts? > > "me too".. It seems to happen frequently when doing > 20 or so ports. > Every week or so I upgrade old ports and don't have problems, but after a > gnome or xorg update that ends up bumping the portrevision for half my > installed ports, it always takes three or four "portupgrade -a" loops for > everything to buid. > > If you've got ZFS, you can snapshot your filesystems, and if portupgrade > fails, roll back to the snapshot and do it again to see if it happens on the > same port a second time. Or if you know ruby, you could instrument the code > that checks for port build errors and see if it's got a bug in it... There's a more easy solution, pipe the output of portupgrade to a logfile. This way you can have a look what happened with the port which was reported as broken. Maybe there's a dependency missing, and after updating other ports after the failure, this dependency was satisfied so that the next run succeeds. Bye, Alexander. -- A squeegee by any other name wouldn't sound as funny. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From perryh at pluto.rain.com Sat Feb 28 01:11:09 2009 From: perryh at pluto.rain.com (perryh@pluto.rain.com) Date: Sat Feb 28 01:11:16 2009 Subject: hosted, or not (Re: Renaming all symbols in libmp(3)) In-Reply-To: <20090227082417.GA55567@freebsd.org> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <49a79a4e.jj/fvw29lwxNJgz+%perryh@pluto.rain.com> <20090227082417.GA55567@freebsd.org> Message-ID: <49a8fdb4.47pe/BeTfmPBIr7P%perryh@pluto.rain.com> Roman Divacky wrote: > On Thu, Feb 26, 2009 at 11:46:22PM -0800, perryh@pluto.rain.com wrote: > > > >> By default, LLVM has a built-in prototype of pow(), similar to > > > >> GCC. Unlike GCC, LLVM raises a compiler error by default ... > > > > > ... it's invalid code to have a function named pow() > > > in a hosted environment which is not /The/ pow(). > > ^^^^^^^^^^^^^^^^^^^^^^^ > > > > I don't suppose LLVM supports a commmand-line switch to use > > embedded mode instead of hosted? > > of course it does.... -ffreestanding So perhaps one solution would be to compile libmp with -ffreestanding? From ed at 80386.nl Sat Feb 28 01:24:58 2009 From: ed at 80386.nl (Ed Schouten) Date: Sat Feb 28 01:25:05 2009 Subject: hosted, or not (Re: Renaming all symbols in libmp(3)) In-Reply-To: <49a8fdb4.47pe/BeTfmPBIr7P%perryh@pluto.rain.com> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <49a79a4e.jj/fvw29lwxNJgz+%perryh@pluto.rain.com> <20090227082417.GA55567@freebsd.org> <49a8fdb4.47pe/BeTfmPBIr7P%perryh@pluto.rain.com> Message-ID: <20090228092456.GJ19161@hoeg.nl> * perryh@pluto.rain.com wrote: > So perhaps one solution would be to compile libmp with -ffreestanding? And all applications that use . -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090228/0a38e7de/attachment.pgp From rdivacky at freebsd.org Sat Feb 28 01:47:08 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Sat Feb 28 01:47:15 2009 Subject: hosted, or not (Re: Renaming all symbols in libmp(3)) In-Reply-To: <20090228092456.GJ19161@hoeg.nl> References: <20090226180756.GX19161@hoeg.nl> <20090226204243.GA96251@zim.MIT.EDU> <49A70092.6030601@gmx.de> <49a79a4e.jj/fvw29lwxNJgz+%perryh@pluto.rain.com> <20090227082417.GA55567@freebsd.org> <49a8fdb4.47pe/BeTfmPBIr7P%perryh@pluto.rain.com> <20090228092456.GJ19161@hoeg.nl> Message-ID: <20090228094347.GA61328@freebsd.org> On Sat, Feb 28, 2009 at 10:24:56AM +0100, Ed Schouten wrote: > * perryh@pluto.rain.com wrote: > > So perhaps one solution would be to compile libmp with -ffreestanding? > > And all applications that use . which is a nonsense... please move forward From boris at brooknet.com.au Fri Feb 27 20:44:25 2009 From: boris at brooknet.com.au (Sam Lawrance) Date: Sat Feb 28 04:12:54 2009 Subject: Fwd: Kernel conference Australia References: <20090227201947.GB24240@bcd.geek.com.au> Message-ID: <8662C4CC-DE22-4D48-BF57-0CE19ECFEABD@brooknet.com.au> Early days, this could be cool. Interested speakers might consider approaching their employers or the FreeBSD Foundation to obtain assistance. Begin forwarded message: > From: Daniel Carosone > Date: 28 February 2009 7:19:47 AM > To: current-users@netbsd.org, regional-au@netbsd.org > Subject: Kernel conference Australia > > Just spotted this go by my RSS, in case others are interested. > > http://ln-s.net/2vsw > > -- > Dan.