From julian at elischer.org Thu May 1 00:11:28 2008 From: julian at elischer.org (Julian Elischer) Date: Thu May 1 00:11:32 2008 Subject: maybe_preempt_in_ksegrp In-Reply-To: References: <4818E40F.9070004@elischer.org> <4818F7F8.6020602@elischer.org> Message-ID: <48190AB2.4010102@elischer.org> Murty, Ravi wrote: > Sorry I wish I was part of the development effort. I am just coming on > board with FreeBSD work. > > I guess ksegrps were implemented for the purpose of PROCESS_SCOPE > threads and like you said avoiding a process from hogging the CPU. > > If every thread in the system has it's own ksegrp (SYSTEM_SCOPE) I don't > see this call (maybe_preempt_in_ksegrp) ever getting called :). which is why the default was process scope. > > Thanks > ravi > > > -----Original Message----- > From: Julian Elischer [mailto:julian@elischer.org] > Sent: Wednesday, April 30, 2008 3:52 PM > To: Murty, Ravi > Cc: freebsd-hackers@freebsd.org > Subject: Re: maybe_preempt_in_ksegrp > > Murty, Ravi wrote: >> Julian, >> >> Apologies for sticking to 6.x, I checked and looks like this function >> and several others are out in 7.x. It's just that we've been using 6.x >> for a while and continue to look at it. :) >> >> >> Coming back, I was thinking of the problem the other way around. The >> thread gets put on the ksegrp runq, but we don't know if it gets put > at >> the head of the queue. All we know is we either find a slot or not. If >> we do, great sched_add is called which will add it to a CPU runq and >> check if it can preempt some thread on the target CPU. If we can't > find >> a slot, it checks if it can steal (preempt) some other thread (of the >> same ksegrp) from a cpu. Let's consider the UP case to keep this > simple. >> One of the checks is the priority of the newly runnable thread and the >> curthread on the CPU and the fact that they are part of the same > KSEGRP. >> If both pass, I think it should say "run me" since we just established >> that I am higher priority than what's running on the CPU. >> >> Ravi >> > > > Quite possibly.. > where were you when we needed more > man-power on this :-) > > this was part of the attempt to make a 'fair' scheduler > which would not gove a person 10,000 times the cpu just because > he had 10000 threads :-) > > > It was eventually removed as being too complicated, too resource > intensive, and not solving a problem that people were seeing. > > From keramida at ceid.upatras.gr Fri May 2 02:14:53 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Fri May 2 02:15:01 2008 Subject: indent(1) support for gcc(1) 0b prefix In-Reply-To: <200804270201.53271.max@love2party.net> (Max Laier's message of "Sun, 27 Apr 2008 02:01:53 +0200") References: <20080426213557.GA88577@marvin.blogreen.org> <200804270201.53271.max@love2party.net> Message-ID: <8763txlaj6.fsf@kobe.laptop> On Sun, 27 Apr 2008 02:01:53 +0200, Max Laier wrote: > On Saturday 26 April 2008 23:35:57 Romain Tarti=8Are wrote: >> Hello FreeBSD hackers! >> >> I'm using avr-gcc from the ports and relying on the 0b prefix notation >> for binary constants, that is: >> >> foo =3D 0b00101010; >> >> Thanks to /usr/ports/devel/avr-gcc/files/patch-0b-constants this is >> possible :-) >> >> But I would like to use indent(1) to reformat contributed code >> automatically. Unfortunately, the 0b notation is not supported by that >> program, and the resulting code looks like this: >> >> foo =3D 0 b00101010; >> >> ... then compilation fails, bla bla bla... > > I can't think of a case (outside of "0x...." context) where "...0b..." > would be valid C code, let alone better formated as "...0 b...". > Hence I see no harm in adding your patch to the base indent(1). > > Does anyone have an example where "...0 b..." is valid C code? The only case I can think of is when the "b..." is an existing macro, i.e. something like: 1 #include 2 3 #define b0101 + 3 4 5 int 6 main(void) 7 { 8 printf("%u\n", 0 b0101); 9 return 0; 10 } But that's a rather contrived example. Making the "0b...." support tunable through a command-line option seems ok for me. From keramida at ceid.upatras.gr Fri May 2 02:14:53 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Fri May 2 02:15:02 2008 Subject: indent(1) support for gcc(1) 0b prefix In-Reply-To: <200804270201.53271.max@love2party.net> (Max Laier's message of "Sun, 27 Apr 2008 02:01:53 +0200") References: <20080426213557.GA88577@marvin.blogreen.org> <200804270201.53271.max@love2party.net> Message-ID: <8763txlaj6.fsf@kobe.laptop> On Sun, 27 Apr 2008 02:01:53 +0200, Max Laier wrote: > On Saturday 26 April 2008 23:35:57 Romain Tarti=8Are wrote: >> Hello FreeBSD hackers! >> >> I'm using avr-gcc from the ports and relying on the 0b prefix notation >> for binary constants, that is: >> >> foo =3D 0b00101010; >> >> Thanks to /usr/ports/devel/avr-gcc/files/patch-0b-constants this is >> possible :-) >> >> But I would like to use indent(1) to reformat contributed code >> automatically. Unfortunately, the 0b notation is not supported by that >> program, and the resulting code looks like this: >> >> foo =3D 0 b00101010; >> >> ... then compilation fails, bla bla bla... > > I can't think of a case (outside of "0x...." context) where "...0b..." > would be valid C code, let alone better formated as "...0 b...". > Hence I see no harm in adding your patch to the base indent(1). > > Does anyone have an example where "...0 b..." is valid C code? The only case I can think of is when the "b..." is an existing macro, i.e. something like: 1 #include 2 3 #define b0101 + 3 4 5 int 6 main(void) 7 { 8 printf("%u\n", 0 b0101); 9 return 0; 10 } But that's a rather contrived example. Making the "0b...." support tunable through a command-line option seems ok for me. From kientzle at freebsd.org Fri May 2 04:19:07 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Fri May 2 04:19:10 2008 Subject: indent(1) support for gcc(1) 0b prefix In-Reply-To: <8763txlaj6.fsf@kobe.laptop> References: <20080426213557.GA88577@marvin.blogreen.org> <200804270201.53271.max@love2party.net> <8763txlaj6.fsf@kobe.laptop> Message-ID: <481A959C.5000704@freebsd.org> >>>But I would like to use indent(1) to reformat contributed code >>>automatically. Unfortunately, the 0b notation is not supported by that >>>program, and the resulting code looks like this: >>> >>> foo = 0 b00101010; >>> >>>... then compilation fails, bla bla bla... >> >>I can't think of a case (outside of "0x...." context) where "...0b..." >>would be valid C code, let alone better formated as "...0 b...". >>Hence I see no harm in adding your patch to the base indent(1). >> >>Does anyone have an example where "...0 b..." is valid C code? > > The only case I can think of is when the "b..." is an existing macro, > i.e. something like: > > 1 #include > 2 > 3 #define b0101 + 3 In all of these cases, though, adding a space between the '0' and the 'b' changes the meaning, so is wrong. Indent can change/insert whitespace, but should never do so in a way that changes the meaning of the program. In all of these cases, having indent recognize "0b..." as a single token is the correct behavior. So I don't see any point in having this recognition be tunable. indent already has too many switches. Tim Kientzle From kientzle at freebsd.org Fri May 2 04:57:49 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Fri May 2 04:57:54 2008 Subject: indent(1) support for gcc(1) 0b prefix In-Reply-To: <8763txlaj6.fsf@kobe.laptop> References: <20080426213557.GA88577@marvin.blogreen.org> <200804270201.53271.max@love2party.net> <8763txlaj6.fsf@kobe.laptop> Message-ID: <481A959C.5000704@freebsd.org> >>>But I would like to use indent(1) to reformat contributed code >>>automatically. Unfortunately, the 0b notation is not supported by that >>>program, and the resulting code looks like this: >>> >>> foo = 0 b00101010; >>> >>>... then compilation fails, bla bla bla... >> >>I can't think of a case (outside of "0x...." context) where "...0b..." >>would be valid C code, let alone better formated as "...0 b...". >>Hence I see no harm in adding your patch to the base indent(1). >> >>Does anyone have an example where "...0 b..." is valid C code? > > The only case I can think of is when the "b..." is an existing macro, > i.e. something like: > > 1 #include > 2 > 3 #define b0101 + 3 In all of these cases, though, adding a space between the '0' and the 'b' changes the meaning, so is wrong. Indent can change/insert whitespace, but should never do so in a way that changes the meaning of the program. In all of these cases, having indent recognize "0b..." as a single token is the correct behavior. So I don't see any point in having this recognition be tunable. indent already has too many switches. Tim Kientzle From lists at mschuette.name Fri May 2 21:16:43 2008 From: lists at mschuette.name (=?ISO-8859-15?Q?Martin_Sch=FCtte?=) Date: Fri May 2 21:16:47 2008 Subject: Improving Syslog Message-ID: <481B7ED4.3020208@mschuette.name> Hello, I am taking part in this year's Google Summer of Code for NetBSD and want to implement the upcoming IETF Syslog standards for BSD's syslogd(8). The most important improvements will be an extended message format, TLS network transport, and digital signatures. I hope these new functions will later be ported to benefit all BSD variants (as their syslogds are similar). So everyone interested is invited to take a look at the project pages now and then. Naturally any feedback is welcome. The official project's homepage is at http://netbsd-soc.sourceforge.net/projects/syslogd/ and for developing I maintain a Trac at http://barney.cs.uni-potsdam.de/trac/syslogd/ -- Martin From delphij at delphij.net Fri May 2 21:26:28 2008 From: delphij at delphij.net (Xin LI) Date: Fri May 2 21:26:31 2008 Subject: Improving Syslog In-Reply-To: <481B7ED4.3020208@mschuette.name> References: <481B7ED4.3020208@mschuette.name> Message-ID: <481B86F6.7050207@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin Sch?tte wrote: | Hello, | I am taking part in this year's Google Summer of Code for NetBSD and | want to implement the upcoming IETF Syslog standards for BSD's | syslogd(8). The most important improvements will be an extended message | format, TLS network transport, and digital signatures. | | I hope these new functions will later be ported to benefit all BSD | variants (as their syslogds are similar). | So everyone interested is invited to take a look at the project pages | now and then. Naturally any feedback is welcome. | | The official project's homepage is at | http://netbsd-soc.sourceforge.net/projects/syslogd/ | and for developing I maintain a Trac at | http://barney.cs.uni-potsdam.de/trac/syslogd/ That's cool. Is there any commit-mail style stuff that we can subscribe so we can gradually incorporate your work into FreeBSD? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgbhvUACgkQi+vbBBjt66DEnwCfZbFNXV8svbv4i31uXlJNsglS 4/EAoKogxgrOvyEKbr2DKvKncGcvh+rP =4W6A -----END PGP SIGNATURE----- From sanjeevfiles at yahoo.com Sat May 3 08:42:00 2008 From: sanjeevfiles at yahoo.com (Sanjeev Kumar.S) Date: Sat May 3 08:42:04 2008 Subject: Saving entire console sessions In-Reply-To: Message-ID: <12198.11591.qm@web57712.mail.re3.yahoo.com> Thank you this serves my purpose very well. Pranav Peshwe wrote: On Thu, Apr 24, 2008 at 5:57 PM, Sanjeev Kumar.S wrote: > Hi, > Is there any way to save entire console > sessions ? > Screen asks me to copy and paste what > I want. I don't want to do that I need to save > entire sessions. like a start till I do a exit or > quit the application. Is there a utility like > that ? > Did you try enabling logging for a screen window ? `screen -L` may be of help. Best regards, Pranav -------------------------------------------------------------------------- Blessed are the pessimists, for they take backups!! _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --------------------------------- Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. From sanjeevfiles at yahoo.com Sat May 3 08:42:51 2008 From: sanjeevfiles at yahoo.com (Sanjeev Kumar.S) Date: Sat May 3 08:42:59 2008 Subject: Saving entire console sessions In-Reply-To: <20080424123047.GF38660@rink.nu> Message-ID: <607443.11591.qm@web57712.mail.re3.yahoo.com> Sorry, did not mean to. Thank you very much for the suggestion. Suits my requirement very well. Regards, Sanjeev. Rink Springer wrote: Hi, On Thu, Apr 24, 2008 at 05:27:36AM -0700, Sanjeev Kumar.S wrote: > Is there any way to save entire console > sessions ? > Screen asks me to copy and paste what > I want. I don't want to do that I need to save > entire sessions. like a start till I do a exit or > quit the application. Is there a utility like > that ? Please, don't hijack the thread next time :-) Either way, have a look at script(1) - I think that is what you are looking for. Regards, -- Rink P.W. Springer - http://rink.nu "Anyway boys, this is America. Just because you get more votes doesn't mean you win." - Fox Mulder _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --------------------------------- Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. From sanjeevfiles at yahoo.com Sat May 3 08:42:51 2008 From: sanjeevfiles at yahoo.com (Sanjeev Kumar.S) Date: Sat May 3 08:43:00 2008 Subject: Saving entire console sessions In-Reply-To: <20080424123047.GF38660@rink.nu> Message-ID: <607443.11591.qm@web57712.mail.re3.yahoo.com> Sorry, did not mean to. Thank you very much for the suggestion. Suits my requirement very well. Regards, Sanjeev. Rink Springer wrote: Hi, On Thu, Apr 24, 2008 at 05:27:36AM -0700, Sanjeev Kumar.S wrote: > Is there any way to save entire console > sessions ? > Screen asks me to copy and paste what > I want. I don't want to do that I need to save > entire sessions. like a start till I do a exit or > quit the application. Is there a utility like > that ? Please, don't hijack the thread next time :-) Either way, have a look at script(1) - I think that is what you are looking for. Regards, -- Rink P.W. Springer - http://rink.nu "Anyway boys, this is America. Just because you get more votes doesn't mean you win." - Fox Mulder _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --------------------------------- Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. From ap00 at mail.ru Sat May 3 08:54:32 2008 From: ap00 at mail.ru (Anthony Pankov) Date: Sat May 3 08:54:35 2008 Subject: Improving Syslog In-Reply-To: <481B7ED4.3020208@mschuette.name> References: <481B7ED4.3020208@mschuette.name> Message-ID: <1723583765.20080503123027@mail.ru> Hello Martin, It is not pleasant to me that most comprehensible unix subsystem - syslog - will grow to multipurpose monster. More pleasant to here about careful redesign of syslogd that make available various plugins|additional daemons each of which adds well-defined functionality. I dislike many features living in one binary file. Especially when this binary will do one thing for me - write messages to file. Sorry, but when i imagine one more software to scrub from cool features i become more sad. Saturday, May 03, 2008, 12:51:32 AM, you wrote: MS> Hello, MS> I am taking part in this year's Google Summer of Code for NetBSD and MS> want to implement the upcoming IETF Syslog standards for BSD's MS> syslogd(8). The most important improvements will be an extended message MS> format, TLS network transport, and digital signatures. MS> I hope these new functions will later be ported to benefit all BSD MS> variants (as their syslogds are similar). MS> So everyone interested is invited to take a look at the project pages MS> now and then. Naturally any feedback is welcome. MS> The official project's homepage is at MS> http://netbsd-soc.sourceforge.net/projects/syslogd/ MS> and for developing I maintain a Trac at MS> http://barney.cs.uni-potsdam.de/trac/syslogd/ -- Best regards, Anthony mailto:ap00@mail.ru From edwin at mavetju.org Sat May 3 09:32:21 2008 From: edwin at mavetju.org (Edwin Groothuis) Date: Sat May 3 09:32:24 2008 Subject: Improving Syslog In-Reply-To: <1723583765.20080503123027@mail.ru> References: <481B7ED4.3020208@mschuette.name> <1723583765.20080503123027@mail.ru> Message-ID: <20080503090723.GA3159@k7.mavetju> On Sat, May 03, 2008 at 12:30:27PM +0400, Anthony Pankov wrote: > It is not pleasant to me that most comprehensible unix subsystem - > syslog - will grow to multipurpose monster. > > More pleasant to here about careful redesign of syslogd that make > available various plugins|additional daemons each of which adds > well-defined functionality. > > I dislike many features living in one binary file. Especially when > this binary will do one thing for me - write messages to file. > > Sorry, but when i imagine one more software to scrub from cool > features i become more sad. Even if the voices are opposing it, we can always make a port out of it so that the people who are interested in it can use it :-) Edwin -- Edwin Groothuis | Personal website: http://www.mavetju.org edwin@mavetju.org | Weblog: http://www.mavetju.org/weblog/ From lists at mschuette.name Sat May 3 12:38:38 2008 From: lists at mschuette.name (=?ISO-8859-15?Q?Martin_Sch=FCtte?=) Date: Sat May 3 12:38:43 2008 Subject: Improving Syslog In-Reply-To: <481B86F6.7050207@delphij.net> References: <481B7ED4.3020208@mschuette.name> <481B86F6.7050207@delphij.net> Message-ID: <481C5CB7.4000004@mschuette.name> Xin LI schrieb: > | and for developing I maintain a Trac at > | http://barney.cs.uni-potsdam.de/trac/syslogd/ > That's cool. Is there any commit-mail style stuff that we can subscribe > so we can gradually incorporate your work into FreeBSD? You could use the RSS feed (Browse Source -> Revision Log) to follow all my SVN commits to certain paths (e.g. a stable branch). But instead of following every single step, I guess it would be more time-efficient to wait and later port/merge larger units at once. For the implementation I will have to concentrate on one codebase, but I am certainly interested in porting/helping to port everything to the other syslogd versions later on. -- Martin From lists at mschuette.name Sat May 3 12:39:31 2008 From: lists at mschuette.name (=?ISO-8859-15?Q?Martin_Sch=FCtte?=) Date: Sat May 3 12:39:35 2008 Subject: Improving Syslog In-Reply-To: <1723583765.20080503123027@mail.ru> References: <481B7ED4.3020208@mschuette.name> <1723583765.20080503123027@mail.ru> Message-ID: <481C5CF9.1090705@mschuette.name> Anthony Pankov schrieb: > It is not pleasant to me that most comprehensible unix subsystem - > syslog - will grow to multipurpose monster. I would like to discuss this point further (so this mail got a bit longer) because I also thought about it and it basically depends on where you draw the line between basic functionality and uncommonly used feature or when to extend a grown system and when to completely refactor it. 1. Drawing the line in big binaries: On the one hand I do not want to introduce the feature list of syslog-ng (custom filters, regexp support, different message formats) into syslogd. That would clearly be the 'multipurpose monster' that belongs in the ports tree for those who need it. On the other hand I think TLS support is a basic functionality. We are not in the 1980s anymore and having TLS in the standard syslogd is IMHO no bloat but desirable. That would leave syslog-sign in the middle. I am really undecided about this, because it potentially has the most configuration options and settings and it could just be implemented as a proxy or a filter. But then again it does not introduce new dependencies, neither does it require any configuration for default usage, and having 5-10 processes (for every log destination) also seems excessive to me. I want to defer my judgement here until I studied at least the existing code for syslog-sec (http://sourceforge.net/projects/syslog-sec/), a preliminary implementation by Albert Mietus. 2. Redesign the syslog subsystem: To change the architecture would also be interesting. The result could be a whole chain of small programs somewhat like Postfix (http://www.postfix.org/big-picture.html) with a design similar to rsyslog (http://www.rsyslog.com/doc-generic_design.html) It would require - a set of collectors (kernel log, local sockets, UDP, TLS) - a set of destinations (UDP, TLS, file, pipe, console/tty/wall message, memory buffer) - some core elements (central dispatcher, memory queue) I only wonder if that would not be the bigger and more drastic change that would prevent adoption; just like FreeBSD keeps Sendmail instead of adopting Postfix in its base system. On a more pragmatic level I am also afraid this would break my schedule for the summer; so I will keep it in mind as a reminder to keep everything modular, but not persue it with high priority. (Or I might start with seperate threads in order to persue the design but not spent too much time with IPC details.) If there is consensus that this is the right way, then it would make a nice follow up project. -- Martin From edwin at mavetju.org Sat May 3 12:54:26 2008 From: edwin at mavetju.org (Edwin Groothuis) Date: Sat May 3 12:54:30 2008 Subject: Improving Syslog In-Reply-To: <481C5CF9.1090705@mschuette.name> References: <481B7ED4.3020208@mschuette.name> <1723583765.20080503123027@mail.ru> <481C5CF9.1090705@mschuette.name> Message-ID: <20080503125423.GA74315@k7.mavetju> On Sat, May 03, 2008 at 02:39:21PM +0200, Martin Sch?tte wrote: > I only wonder if that would not be the bigger and more drastic change > that would prevent adoption; just like FreeBSD keeps Sendmail instead of > adopting Postfix in its base system. There are other reasons for this. And the way the mail system is setup makes it is very simple to replace the default MTA with an other MTA in the ports collection. Edwin -- Edwin Groothuis | Personal website: http://www.mavetju.org edwin@mavetju.org | Weblog: http://www.mavetju.org/weblog/ From lists at mschuette.name Sat May 3 13:29:35 2008 From: lists at mschuette.name (=?ISO-8859-1?Q?Martin_Sch=FCtte?=) Date: Sat May 3 13:29:40 2008 Subject: Improving Syslog In-Reply-To: <20080503125423.GA74315@k7.mavetju> References: <481B7ED4.3020208@mschuette.name> <1723583765.20080503123027@mail.ru> <481C5CF9.1090705@mschuette.name> <20080503125423.GA74315@k7.mavetju> Message-ID: <481C68B5.3020200@mschuette.name> Edwin Groothuis schrieb: >> that would prevent adoption; just like FreeBSD keeps Sendmail instead of >> adopting Postfix in its base system. > There are other reasons for this. And the way the mail system is > setup makes it is very simple to replace the default MTA with an > other MTA in the ports collection. Certainly, it was just an example. My point is only that old, slowly evolving, well-known, and tested programs (legitimately) build up a certain amount of trust and inertia. Thus making it difficult to replace them because the new program has to be so much better to justify the replacement. -- Martin From fgleiser at cactus.fi.uba.ar Mon May 5 01:57:36 2008 From: fgleiser at cactus.fi.uba.ar (Fer Gleiser) Date: Mon May 5 01:57:41 2008 Subject: problems with CTF on -STABLE Message-ID: <481E6988.10200@cactus.fi.uba.ar> We're facing some problems with CTF on -STABLE. We need that so we can have DTrace runing on FreeBSD 7 The tools and the libs build fine, but I can't ctfconvert anything. In some cases get the following error: ctfconvert -L VERSION -g tdfx_linux.o ERROR: ctfconvert: die 37: base type without name and in some others ctfconvert just dies and dumps core. I all cases, xstrdup is the culprit: (gdb) where #0 0x281bbfa9 in strlen () from /lib/libc.so.7 #1 0x28147187 in strdup (str=0x0) at /usr/src/lib/libc/string/strdup.c:47 #2 0x08050f22 in xstrdup () #3 0x0804dfde in die_through_create () #4 0x0804d712 in die_create_one () #5 0x0804d769 in die_create () #6 0x0804ecb4 in dw_read () #7 0x0804c12b in main () now, if I copy both the .o and the ctfconvert executable to a -CURRENT box, it runs fine. From the error (base type without name) it seems that some info that ctfconvert needs is not available on -STABLE, or maybe it's on a different place or in a format ctfconvert doesn't understand. Maybe libdwarf is built on a different way, but so far I couln't find what the problem is. Any pointers/help would be greatly apreciated Fer From fgleiser at cactus.fi.uba.ar Mon May 5 02:16:29 2008 From: fgleiser at cactus.fi.uba.ar (Fer Gleiser) Date: Mon May 5 02:16:33 2008 Subject: soc2008 DTrace toolkit In-Reply-To: References: Message-ID: <481E65AF.7020009@cactus.fi.uba.ar> LiQun Li wrote: > Hi, all, > My name is LiQun Li, originally from China, now study in University of > Iowa. I am a graduate student, majored in Computer Science. > I have used Open Source Software for a while, most time under Linux, a > little bit under FreeBSD. > DTrace is a really cool feature for Solaris, it will be great if OSS > has this one too. > This sumer, Google sponsors this SOC, and FreeBSD has this project. I > am going to have a try. > Since my courses will end after the middle of May, I would not spend > too much time on this project before that. > I'm also working on porting some DTrace providers to FreeBSD (a subset of sched, vminfo and sysinfo). John Birrel is leading the main effort an we're basing our work on his latest sources. What parts of DTrace are you planing to do? Fer From carl.shapiro at gmail.com Mon May 5 13:05:23 2008 From: carl.shapiro at gmail.com (Carl Shapiro) Date: Mon May 5 13:05:27 2008 Subject: binary compatibility query Message-ID: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> FreeBSD Hackers, I have a general question about the compatibility of FreeBSD binaries within major releases. If I build a binary for a given release of FreeBSD can I make a reasonable guarantee that the binary will run on both previous and subsequent minor releases of the same major release? In other words, if I build on FreeBSD 6.3 and do not rely on anything unique to 6.3 (such as the presence of specific version strings) how certain can I be that the code will or will not run on 6.2, 6.1 etc.? Also, is this documented anywhere on the FreeBSD web site? The closest thing I could find is the following guidance for driver vendors which falls just short of answering my question: http://wiki.freebsd.org/VendorInformation (Too bad the fancy illustration is missing.) Regards, Carl From kris at FreeBSD.org Mon May 5 13:37:58 2008 From: kris at FreeBSD.org (Kris Kennaway) Date: Mon May 5 13:38:10 2008 Subject: binary compatibility query In-Reply-To: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> Message-ID: <481F0DB3.9070505@FreeBSD.org> Carl Shapiro wrote: > FreeBSD Hackers, > > I have a general question about the compatibility of FreeBSD binaries > within major releases. If I build a binary for a given release of > FreeBSD can I make a reasonable guarantee that the binary will run on > both previous and subsequent minor releases of the same major release? > In other words, if I build on FreeBSD 6.3 and do not rely on anything > unique to 6.3 (such as the presence of specific version strings) how > certain can I be that the code will or will not run on 6.2, 6.1 etc.? > > Also, is this documented anywhere on the FreeBSD web site? The > closest thing I could find is the following guidance for driver > vendors which falls just short of answering my question: > > http://wiki.freebsd.org/VendorInformation > > (Too bad the fancy illustration is missing.) Binaries compiled on a certain version of FreeBSD will continue to run on later versions, but are not guaranteed to run on earlier versions (and in fact *will* not run depending on the binary). This is because over time the system libraries and kernel grow new features which may be used by applications, so they will therefore fail to run if executed on old systems that do not provide these features. If your goal is to provide an application that runs on a range of FreeBSD versions, then either build it for the oldest of these versions, or provide multiple versions if there is a reason to do so (e.g. if there have been major improvements in the OS that are relevant to your application). Kris From eischen at vigrid.com Mon May 5 14:26:21 2008 From: eischen at vigrid.com (Daniel Eischen) Date: Mon May 5 14:26:27 2008 Subject: binary compatibility query In-Reply-To: <481F0DB3.9070505@FreeBSD.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> Message-ID: On Mon, 5 May 2008, Kris Kennaway wrote: > Carl Shapiro wrote: >> FreeBSD Hackers, >> >> I have a general question about the compatibility of FreeBSD binaries >> within major releases. If I build a binary for a given release of >> FreeBSD can I make a reasonable guarantee that the binary will run on >> both previous and subsequent minor releases of the same major release? >> In other words, if I build on FreeBSD 6.3 and do not rely on anything >> unique to 6.3 (such as the presence of specific version strings) how >> certain can I be that the code will or will not run on 6.2, 6.1 etc.? >> >> Also, is this documented anywhere on the FreeBSD web site? The >> closest thing I could find is the following guidance for driver >> vendors which falls just short of answering my question: >> >> http://wiki.freebsd.org/VendorInformation >> >> (Too bad the fancy illustration is missing.) > > Binaries compiled on a certain version of FreeBSD will continue to run on > later versions, but are not guaranteed to run on earlier versions (and in > fact *will* not run depending on the binary). This is because over time the > system libraries and kernel grow new features which may be used by > applications, so they will therefore fail to run if executed on old systems > that do not provide these features. For 7.0 and onwards, we will hopefully have a tool that you can run on your application to do abi checking. One of the things it should be able to do is tell under what releases it will run, including previous releases. For 6.x, you may be able to manually check the symbols your application uses against the libraries from 6.0, 6.1, etc. This should give a good idea as to whether your application will run on those releases. I think checking symbols is good enough for releases within a branch since we don't change ABIs. -- DE From kris at FreeBSD.org Mon May 5 14:39:35 2008 From: kris at FreeBSD.org (Kris Kennaway) Date: Mon May 5 14:39:38 2008 Subject: binary compatibility query In-Reply-To: References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> Message-ID: <481F1C24.3030405@FreeBSD.org> Daniel Eischen wrote: >> Binaries compiled on a certain version of FreeBSD will continue to run >> on later versions, but are not guaranteed to run on earlier versions >> (and in fact *will* not run depending on the binary). This is because >> over time the system libraries and kernel grow new features which may >> be used by applications, so they will therefore fail to run if >> executed on old systems that do not provide these features. > > For 7.0 and onwards, we will hopefully have a tool that you can > run on your application to do abi checking. One of the things > it should be able to do is tell under what releases it will run, > including previous releases. That sounds like a good idea! > For 6.x, you may be able to manually check the symbols your > application uses against the libraries from 6.0, 6.1, etc. > This should give a good idea as to whether your application > will run on those releases. I think checking symbols is > good enough for releases within a branch since we don't > change ABIs. Yeah, this should be enough for almost all cases. There are some aspects of the ABI that are not covered by library symbols though, such as file formats, the sysctl MIB, device ioctls, probably others. Kris From nayak_purushotham at yahoo.com Sat May 3 16:44:01 2008 From: nayak_purushotham at yahoo.com (Purushotham Nayak) Date: Mon May 5 16:22:05 2008 Subject: Gprof C++ support Message-ID: <689122.62781.qm@web56102.mail.re3.yahoo.com> Hi All, I've been trying to use gprof on some C++ code and appears it doesn't demangle C++ function names. I was wondering if anyone is working on it? I would like to contribute/help in adding this. Purushotham _________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. [1]Try it now. References 1. http://us.rd.yahoo.com/evt=51733/*http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ From julian at elischer.org Mon May 5 17:50:38 2008 From: julian at elischer.org (Julian Elischer) Date: Mon May 5 17:50:42 2008 Subject: binary compatibility query In-Reply-To: <481F0DB3.9070505@FreeBSD.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> Message-ID: <481F48EE.3050806@elischer.org> Kris Kennaway wrote: > Carl Shapiro wrote: >> FreeBSD Hackers, >> >> I have a general question about the compatibility of FreeBSD binaries >> within major releases. If I build a binary for a given release of >> FreeBSD can I make a reasonable guarantee that the binary will run on >> both previous and subsequent minor releases of the same major release? >> In other words, if I build on FreeBSD 6.3 and do not rely on anything >> unique to 6.3 (such as the presence of specific version strings) how >> certain can I be that the code will or will not run on 6.2, 6.1 etc.? >> >> Also, is this documented anywhere on the FreeBSD web site? The >> closest thing I could find is the following guidance for driver >> vendors which falls just short of answering my question: >> >> http://wiki.freebsd.org/VendorInformation >> >> (Too bad the fancy illustration is missing.) > > Binaries compiled on a certain version of FreeBSD will continue to run > on later versions, but are not guaranteed to run on earlier versions > (and in fact *will* not run depending on the binary). This is because > over time the system libraries and kernel grow new features which may be > used by applications, so they will therefore fail to run if executed on > old systems that do not provide these features. > > If your goal is to provide an application that runs on a range of > FreeBSD versions, then either build it for the oldest of these versions, > or provide multiple versions if there is a reason to do so (e.g. if > there have been major improvements in the OS that are relevant to your > application). I agree in general, however we do make an attempt to keep ABI compatibility within a release line, so that there is a high probability that a binary compiled on a later one will run on an earlier one as long as it does not rely on new features. It's not guaranteed, but if you test it on the oldest of the line it should work on all intermmediate releases too. > > Kris > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From kris at FreeBSD.org Mon May 5 18:16:15 2008 From: kris at FreeBSD.org (Kris Kennaway) Date: Mon May 5 18:16:18 2008 Subject: binary compatibility query In-Reply-To: <481F48EE.3050806@elischer.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> Message-ID: <481F4EED.2030300@FreeBSD.org> Julian Elischer wrote: > Kris Kennaway wrote: >> Carl Shapiro wrote: >>> FreeBSD Hackers, >>> >>> I have a general question about the compatibility of FreeBSD binaries >>> within major releases. If I build a binary for a given release of >>> FreeBSD can I make a reasonable guarantee that the binary will run on >>> both previous and subsequent minor releases of the same major release? >>> In other words, if I build on FreeBSD 6.3 and do not rely on anything >>> unique to 6.3 (such as the presence of specific version strings) how >>> certain can I be that the code will or will not run on 6.2, 6.1 etc.? >>> >>> Also, is this documented anywhere on the FreeBSD web site? The >>> closest thing I could find is the following guidance for driver >>> vendors which falls just short of answering my question: >>> >>> http://wiki.freebsd.org/VendorInformation >>> >>> (Too bad the fancy illustration is missing.) >> >> Binaries compiled on a certain version of FreeBSD will continue to run >> on later versions, but are not guaranteed to run on earlier versions >> (and in fact *will* not run depending on the binary). This is because >> over time the system libraries and kernel grow new features which may >> be used by applications, so they will therefore fail to run if >> executed on old systems that do not provide these features. >> >> If your goal is to provide an application that runs on a range of >> FreeBSD versions, then either build it for the oldest of these >> versions, or provide multiple versions if there is a reason to do so >> (e.g. if there have been major improvements in the OS that are >> relevant to your application). > > I agree in general, however we do make an attempt to keep ABI > compatibility within a release line, so that there is a high > probability that a binary compiled on a later one will run on > an earlier one as long as it does not rely on new features. Actually we don't attempt to keep this form of ABI compatibility (running 6.3 binaries on 6.0, for example), because it basically precludes ever adding new functions to libc within a branch, or new syscalls to the kernel. You are correct that often binaries will not notice these accumulated changes though, or can be carefully constructed to avoid them. Kris From julian at elischer.org Mon May 5 18:24:33 2008 From: julian at elischer.org (Julian Elischer) Date: Mon May 5 18:24:37 2008 Subject: binary compatibility query In-Reply-To: <481F4EED.2030300@FreeBSD.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> Message-ID: <481F50E1.3010902@elischer.org> Kris Kennaway wrote: > Julian Elischer wrote: >> Kris Kennaway wrote: >>> Carl Shapiro wrote: >>>> FreeBSD Hackers, >>>> >>>> I have a general question about the compatibility of FreeBSD binaries >>>> within major releases. If I build a binary for a given release of >>>> FreeBSD can I make a reasonable guarantee that the binary will run on >>>> both previous and subsequent minor releases of the same major release? >>>> In other words, if I build on FreeBSD 6.3 and do not rely on anything >>>> unique to 6.3 (such as the presence of specific version strings) how >>>> certain can I be that the code will or will not run on 6.2, 6.1 etc.? >>>> >>>> Also, is this documented anywhere on the FreeBSD web site? The >>>> closest thing I could find is the following guidance for driver >>>> vendors which falls just short of answering my question: >>>> >>>> http://wiki.freebsd.org/VendorInformation >>>> >>>> (Too bad the fancy illustration is missing.) >>> >>> Binaries compiled on a certain version of FreeBSD will continue to >>> run on later versions, but are not guaranteed to run on earlier >>> versions (and in fact *will* not run depending on the binary). This >>> is because over time the system libraries and kernel grow new >>> features which may be used by applications, so they will therefore >>> fail to run if executed on old systems that do not provide these >>> features. >>> >>> If your goal is to provide an application that runs on a range of >>> FreeBSD versions, then either build it for the oldest of these >>> versions, or provide multiple versions if there is a reason to do so >>> (e.g. if there have been major improvements in the OS that are >>> relevant to your application). >> >> I agree in general, however we do make an attempt to keep ABI >> compatibility within a release line, so that there is a high >> probability that a binary compiled on a later one will run on >> an earlier one as long as it does not rely on new features. > > Actually we don't attempt to keep this form of ABI compatibility > (running 6.3 binaries on 6.0, for example), because it basically > precludes ever adding new functions to libc within a branch, or new > syscalls to the kernel. You are correct that often binaries will not > notice these accumulated changes though, or can be carefully constructed > to avoid them. that's all that I said.. I said that because we keep a stable ABI there is a HIGH PROBABLILITY of stuff working, but not a guarantee. > > Kris > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From carl.shapiro at gmail.com Mon May 5 18:32:30 2008 From: carl.shapiro at gmail.com (Carl Shapiro) Date: Mon May 5 18:32:33 2008 Subject: binary compatibility query In-Reply-To: <481F4EED.2030300@FreeBSD.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> Message-ID: <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> Kris & Julian Thank you for clarifiying the compatibility situation. This information was exactly what I was looking for. I have a follow-up question based on this remark... On 5/5/08, Kris Kennaway wrote: > Actually we don't attempt to keep this form of ABI compatibility (running > 6.3 binaries on 6.0, for example), because it basically precludes ever > adding new functions to libc within a branch, or new syscalls to the kernel. > You are correct that often binaries will not notice these accumulated > changes though, or can be carefully constructed to avoid them. If my binary only executes system calls indirectly through libc interfaces, as far as libc and libm are concerned, are new symbols the only thing I need to worry about? Carl From kris at FreeBSD.org Mon May 5 19:18:19 2008 From: kris at FreeBSD.org (Kris Kennaway) Date: Mon May 5 19:18:22 2008 Subject: binary compatibility query In-Reply-To: <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> Message-ID: <481F5D79.4030203@FreeBSD.org> Carl Shapiro wrote: > If my binary only executes system calls indirectly through libc > interfaces, as far as libc and libm are concerned, are new symbols the > only thing I need to worry about? I think so, yes. Kris From ghelmer at palisadesys.com Mon May 5 19:48:12 2008 From: ghelmer at palisadesys.com (Guy Helmer) Date: Mon May 5 19:48:18 2008 Subject: binary compatibility query In-Reply-To: <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> Message-ID: <481F5D1D.6050703@palisadesys.com> Carl Shapiro wrote: > Kris & Julian > > Thank you for clarifiying the compatibility situation. This > information was exactly what I was looking for. > > I have a follow-up question based on this remark... > > On 5/5/08, Kris Kennaway wrote: > >> Actually we don't attempt to keep this form of ABI compatibility (running >> 6.3 binaries on 6.0, for example), because it basically precludes ever >> adding new functions to libc within a branch, or new syscalls to the kernel. >> You are correct that often binaries will not notice these accumulated >> changes though, or can be carefully constructed to avoid them. >> > > If my binary only executes system calls indirectly through libc > interfaces, as far as libc and libm are concerned, are new symbols the > only thing I need to worry about? > Not necessarily. For example, the DNS resolver library was updated between FreeBSD 6.1 and 6.2 to bind 9's resolver library, resulting in the disappearance of the _res symbol in 6.2. As long as you depend only on public interfaces, though, you should be fine. Guy -- Guy Helmer, Ph.D. Chief System Architect Palisade Systems, Inc. From julian at elischer.org Mon May 5 20:09:52 2008 From: julian at elischer.org (Julian Elischer) Date: Mon May 5 20:09:56 2008 Subject: binary compatibility query In-Reply-To: <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> Message-ID: <481F6990.9010007@elischer.org> Carl Shapiro wrote: > Kris & Julian > > Thank you for clarifiying the compatibility situation. This > information was exactly what I was looking for. > > I have a follow-up question based on this remark... > > On 5/5/08, Kris Kennaway wrote: >> Actually we don't attempt to keep this form of ABI compatibility (running >> 6.3 binaries on 6.0, for example), because it basically precludes ever >> adding new functions to libc within a branch, or new syscalls to the kernel. >> You are correct that often binaries will not notice these accumulated >> changes though, or can be carefully constructed to avoid them. > > If my binary only executes system calls indirectly through libc > interfaces, as far as libc and libm are concerned, are new symbols the > only thing I need to worry about? > > Carl basically if you rely only on the standard posix interfaces and don't do anything exotic then you will "probably" be safe. the really safe way of course it to make a 6.0 chroot on your machine and compile your app there. From rdivacky at freebsd.org Mon May 5 20:44:23 2008 From: rdivacky at freebsd.org (Roman Divacky) Date: Mon May 5 20:44:27 2008 Subject: hashinit versus phashinit Message-ID: <20080505204350.GA45321@freebsd.org> hi when we want to use a hash table in kernel we call "hashinit" which initializes a hash table with power-of-2 size. There's also "phashinit" that creates hash table of size that is a prime number. This was added in 1995 by davidg@ but it is not used anywhere in the kernel. phk@ commited rev. 1.30 of vfs_cache.c replacing phashinit with hashinit stating that it's better because it replaces a division with logical and. is this reason still valid today? (the commit was done almost 11 years ago) is there still any reason why not use the phashinit instead of hashinit? I believe using prime-sized hash table might have positive performance impact... do you have comments? roman From zbeeble at gmail.com Mon May 5 21:46:42 2008 From: zbeeble at gmail.com (Zaphod Beeblebrox) Date: Mon May 5 21:46:45 2008 Subject: binary compatibility query In-Reply-To: <5f67a8c40805051351i19930db7le0a1bcce7c6276b8@mail.gmail.com> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <5f67a8c40805051351i19930db7le0a1bcce7c6276b8@mail.gmail.com> Message-ID: <5f67a8c40805051352x6c7ed348v4c6648e708dfdba5@mail.gmail.com> FreeBSD Hackers, > > > > > > I have a general question about the compatibility of FreeBSD binaries > > > within major releases. If I build a binary for a given release of > > > FreeBSD can I make a reasonable guarantee that the binary will run on > > > > As a datapoint, I have been using cistron-radiusd for some time. Recently, I got a new system to run it and compiled the port on it. The compiled port didn't work on amd64 (or my configuration didn't jive with the new version of the daemon). So I grabbed the i386 binary that was built likely around 4.6 to 4.8 FreeBSD (I don't even remember --- it was so long ago). and dropped it on the amd64 7.0-RELEASE system. The binary is dynamically linked (albeit not depending on other packaged libraries) and I had the compat4, compat5 and compat6 ports installed on the new machine. It runs (and is still running) just fine. That's pretty good for compatibility. From das at FreeBSD.ORG Tue May 6 07:02:19 2008 From: das at FreeBSD.ORG (David Schultz) Date: Tue May 6 07:02:23 2008 Subject: hashinit versus phashinit In-Reply-To: <20080505204350.GA45321@freebsd.org> References: <20080505204350.GA45321@freebsd.org> Message-ID: <20080506062556.GA68171@zim.MIT.EDU> On Mon, May 05, 2008, Roman Divacky wrote: > hi > > when we want to use a hash table in kernel we call "hashinit" which > initializes a hash table with power-of-2 size. There's also "phashinit" > that creates hash table of size that is a prime number. This was > added in 1995 by davidg@ but it is not used anywhere in the kernel. > > phk@ commited rev. 1.30 of vfs_cache.c replacing phashinit with hashinit > stating that it's better because it replaces a division with logical and. > is this reason still valid today? (the commit was done almost 11 years ago) > > is there still any reason why not use the phashinit instead of hashinit? > I believe using prime-sized hash table might have positive performance > impact... There's a tradeoff. The argument for using powers of 2 is that division takes many times longer than a logical AND. The argument for using primes is that if your hash function isn't as good as you thought it was, or if the data has some regularity you weren't expecting, you can get screwed a lot more easily with a power of 2 hash table. With a prime-sized hash table, you only get screwed if lots of your data is congruent modulo the prime, which is very rare. Most general-purpose hash implementations I've used (e.g., GNU libstdc++, Sun JDK, Microsoft .NET) use prime table sizes, probably to make it less likely that programmers will shoot themselves in the foot with pathological data or bad hash functions. From rdivacky at FreeBSD.ORG Tue May 6 07:49:03 2008 From: rdivacky at FreeBSD.ORG (Roman Divacky) Date: Tue May 6 07:49:06 2008 Subject: hashinit versus phashinit In-Reply-To: <20080506062556.GA68171@zim.MIT.EDU> References: <20080505204350.GA45321@freebsd.org> <20080506062556.GA68171@zim.MIT.EDU> Message-ID: <20080506074830.GA70848@freebsd.org> On Tue, May 06, 2008 at 02:25:56AM -0400, David Schultz wrote: > On Mon, May 05, 2008, Roman Divacky wrote: > > hi > > > > when we want to use a hash table in kernel we call "hashinit" which > > initializes a hash table with power-of-2 size. There's also "phashinit" > > that creates hash table of size that is a prime number. This was > > added in 1995 by davidg@ but it is not used anywhere in the kernel. > > > > phk@ commited rev. 1.30 of vfs_cache.c replacing phashinit with hashinit > > stating that it's better because it replaces a division with logical and. > > is this reason still valid today? (the commit was done almost 11 years ago) > > > > is there still any reason why not use the phashinit instead of hashinit? > > I believe using prime-sized hash table might have positive performance > > impact... > > There's a tradeoff. > > The argument for using powers of 2 is that division takes many > times longer than a logical AND. > > The argument for using primes is that if your hash function isn't > as good as you thought it was, or if the data has some regularity > you weren't expecting, you can get screwed a lot more easily with > a power of 2 hash table. With a prime-sized hash table, you only > get screwed if lots of your data is congruent modulo the prime, > which is very rare. > > Most general-purpose hash implementations I've used (e.g., GNU > libstdc++, Sun JDK, Microsoft .NET) use prime table sizes, > probably to make it less likely that programmers will shoot > themselves in the foot with pathological data or bad hash functions. yes... a division takes roughly 40 cycles on modern i386 hw, while and is just 1, but does it matter compared to the access times of memory these days? the ratio between cpu-speed/mem-speed has changed a lot. I am not arguing if it's still true that avoiding the division helps the performance these days... From adrian at freebsd.org Tue May 6 12:35:02 2008 From: adrian at freebsd.org (Adrian Chadd) Date: Tue May 6 12:35:07 2008 Subject: hashinit versus phashinit In-Reply-To: <20080506074830.GA70848@freebsd.org> References: <20080505204350.GA45321@freebsd.org> <20080506062556.GA68171@zim.MIT.EDU> <20080506074830.GA70848@freebsd.org> Message-ID: 2008/5/6 Roman Divacky : > > Most general-purpose hash implementations I've used (e.g., GNU > > libstdc++, Sun JDK, Microsoft .NET) use prime table sizes, > > probably to make it less likely that programmers will shoot > > themselves in the foot with pathological data or bad hash functions. > > yes... a division takes roughly 40 cycles on modern i386 hw, while > and is just 1, but does it matter compared to the access times of > memory these days? > > the ratio between cpu-speed/mem-speed has changed a lot. I am not > arguing if it's still true that avoiding the division helps the performance > these days... My 2c - I'd poke the assembler, a book or two on current architectures, and combine it with some benchmarking followed by logical reasoning. Modern microprocessors don't execute instructions like they used to before I was born; "divide versus logical shift/operator" speed may be secondary to pipeline and IU effects, memory access (as mentioned before), etc. (I know its not much - but I'm a firm believer in "Science!", and this is one of those questions which can be understood with a little bit of it..) Adrian -- Adrian Chadd - adrian@freebsd.org From max at love2party.net Tue May 6 13:05:04 2008 From: max at love2party.net (Max Laier) Date: Tue May 6 13:05:11 2008 Subject: hashinit versus phashinit In-Reply-To: <20080506074830.GA70848@freebsd.org> References: <20080505204350.GA45321@freebsd.org> <20080506062556.GA68171@zim.MIT.EDU> <20080506074830.GA70848@freebsd.org> Message-ID: <200805061500.33538.max@love2party.net> On Tuesday 06 May 2008 09:48:30 Roman Divacky wrote: > On Tue, May 06, 2008 at 02:25:56AM -0400, David Schultz wrote: > > On Mon, May 05, 2008, Roman Divacky wrote: > > > hi > > > > > > when we want to use a hash table in kernel we call "hashinit" which > > > initializes a hash table with power-of-2 size. There's also > > > "phashinit" that creates hash table of size that is a prime number. > > > This was added in 1995 by davidg@ but it is not used anywhere in > > > the kernel. > > > > > > phk@ commited rev. 1.30 of vfs_cache.c replacing phashinit with > > > hashinit stating that it's better because it replaces a division > > > with logical and. is this reason still valid today? (the commit was > > > done almost 11 years ago) > > > > > > is there still any reason why not use the phashinit instead of > > > hashinit? I believe using prime-sized hash table might have > > > positive performance impact... > > > > There's a tradeoff. > > > > The argument for using powers of 2 is that division takes many > > times longer than a logical AND. > > > > The argument for using primes is that if your hash function isn't > > as good as you thought it was, or if the data has some regularity > > you weren't expecting, you can get screwed a lot more easily with > > a power of 2 hash table. With a prime-sized hash table, you only > > get screwed if lots of your data is congruent modulo the prime, > > which is very rare. > > > > Most general-purpose hash implementations I've used (e.g., GNU > > libstdc++, Sun JDK, Microsoft .NET) use prime table sizes, > > probably to make it less likely that programmers will shoot > > themselves in the foot with pathological data or bad hash functions. > > yes... a division takes roughly 40 cycles on modern i386 hw, while > and is just 1, but does it matter compared to the access times of > memory these days? > > the ratio between cpu-speed/mem-speed has changed a lot. I am not > arguing if it's still true that avoiding the division helps the > performance these days... requests/s * div_overhead - [avoided_]collisions/s * collision_overhead ~= -([avoided_]collisions/requests * collision_overhead) assuming the collision_overhead (requiring memory operations) greatly dominates the div_overhead. So if there is a high collision rate and you can reasonably assert that you will lower that significantly by using a prime sized hash table, the div_overhead doesn't matter. At least that's what I've come up with off the top of my head now. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From sigtrm at gmail.com Tue May 6 13:11:36 2008 From: sigtrm at gmail.com (Lukasz Jaroszewski) Date: Tue May 6 13:11:40 2008 Subject: utmp.h: UT_HOSTSIZE Message-ID: Hi, just by curiosity, why #define UT_HOSTSIZE is 16, not 256, like in OtherBSDs and some unix-like-like OSes? :) Regards LVJ From jbucht at gmail.com Tue May 6 13:30:38 2008 From: jbucht at gmail.com (Johan Bucht) Date: Tue May 6 13:30:41 2008 Subject: hashinit versus phashinit In-Reply-To: References: <20080505204350.GA45321@freebsd.org> <20080506062556.GA68171@zim.MIT.EDU> <20080506074830.GA70848@freebsd.org> Message-ID: <947010c30805060606l1a92d56ayfc7e08e179baf5cc@mail.gmail.com> On Tue, May 6, 2008 at 5:08 AM, Adrian Chadd wrote: > 2008/5/6 Roman Divacky : > > > > > > Most general-purpose hash implementations I've used (e.g., GNU > > > libstdc++, Sun JDK, Microsoft .NET) use prime table sizes, > > > probably to make it less likely that programmers will shoot > > > themselves in the foot with pathological data or bad hash functions. > > > > yes... a division takes roughly 40 cycles on modern i386 hw, while > > and is just 1, but does it matter compared to the access times of > > memory these days? > > > > the ratio between cpu-speed/mem-speed has changed a lot. I am not > > arguing if it's still true that avoiding the division helps the performance > > these days... > > My 2c - I'd poke the assembler, a book or two on current > architectures, and combine it with some benchmarking followed by > logical reasoning. > > Modern microprocessors don't execute instructions like they used to > before I was born; "divide versus logical shift/operator" speed may be > secondary to pipeline and IU effects, memory access (as mentioned > before), etc. > > (I know its not much - but I'm a firm believer in "Science!", and this > is one of those questions which can be understood with a little bit of > it..) > > If a hash algorithm relies on prime sized tables to produce uniform results it's a bad hash algorithm. =) http://burtleburtle.net/bob/hash/index.html is a recommended read, especially the article for Dr Dobb's. /Johan From marcolz at stack.nl Tue May 6 14:51:33 2008 From: marcolz at stack.nl (Marc Olzheim) Date: Tue May 6 14:51:37 2008 Subject: utmp.h: UT_HOSTSIZE In-Reply-To: References: Message-ID: <20080506145131.GA24882@zlo.nu> On Tue, May 06, 2008 at 03:11:35PM +0200, Lukasz Jaroszewski wrote: > just by curiosity, why #define UT_HOSTSIZE is 16, not 256, like in > OtherBSDs and some unix-like-like OSes? :) Probably historical reasons. It's very much possible to change it though. We've defined it to 64 since FreeBSD 2.05 and never really had any trouble. Marc From mark at crosscutmedia.com Tue May 6 00:40:22 2008 From: mark at crosscutmedia.com (Mark Bucciarelli) Date: Tue May 6 15:58:44 2008 Subject: posix semaphores Message-ID: How reliable is the Posix semaphore support in FreeBSD 7 for interprocess synchronization? Is it not on by default because no one uses it, or because the code isn't trustworthy, or some other reason? Is this what you would recommend for synchronizing access to a shared resource between processes on FreeBSD? Thanks, m From ed at 80386.nl Tue May 6 16:18:58 2008 From: ed at 80386.nl (Ed Schouten) Date: Tue May 6 16:19:00 2008 Subject: utmp.h: UT_HOSTSIZE In-Reply-To: <20080506145131.GA24882@zlo.nu> References: <20080506145131.GA24882@zlo.nu> Message-ID: <20080506161856.GJ1181@hoeg.nl> * Marc Olzheim wrote: > Probably historical reasons. It's very much possible to change it > though. We've defined it to 64 since FreeBSD 2.05 and never really had > any trouble. When increasing it, we shouldn't forget to increase UT_LINESIZE to 16 as well. Using the UNIX 98 PTY naming, we can't go beyond pts/999, because it is eight bytes, including the null byte. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080506/e8ec6674/attachment.pgp From rwatson at FreeBSD.org Tue May 6 16:21:46 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Tue May 6 16:21:51 2008 Subject: Improving Syslog In-Reply-To: <481B7ED4.3020208@mschuette.name> References: <481B7ED4.3020208@mschuette.name> Message-ID: <20080506171525.O52391@fledge.watson.org> On Fri, 2 May 2008, Martin Sch?tte wrote: > I am taking part in this year's Google Summer of Code for NetBSD and want to > implement the upcoming IETF Syslog standards for BSD's syslogd(8). The most > important improvements will be an extended message format, TLS network > transport, and digital signatures. > > I hope these new functions will later be ported to benefit all BSD variants > (as their syslogds are similar). So everyone interested is invited to take a > look at the project pages now and then. Naturally any feedback is welcome. > > The official project's homepage is at > http://netbsd-soc.sourceforge.net/projects/syslogd/ and for developing I > maintain a Trac at http://barney.cs.uni-potsdam.de/trac/syslogd/ Dear Martin, This sounds like an exciting project -- while I recognize the concerns other have expressed about complexity, I think that complexity can be managed if done carefully. I'm not sure if you've looked at Apple's extended syslog, which among other things, includes a binary log file format making it more mechanically searched and managed, do take a look if you haven't. thanks, Robert N M Watson Computer Laboratory University of Cambridge From zbeeble at gmail.com Tue May 6 16:41:25 2008 From: zbeeble at gmail.com (Zaphod Beeblebrox) Date: Tue May 6 16:41:29 2008 Subject: Improving Syslog In-Reply-To: <20080506171525.O52391@fledge.watson.org> References: <481B7ED4.3020208@mschuette.name> <20080506171525.O52391@fledge.watson.org> Message-ID: <5f67a8c40805060941n7d2882aayea7d72d0a76cd89d@mail.gmail.com> On Tue, May 6, 2008 at 12:21 PM, Robert Watson wrote: > > This sounds like an exciting project -- while I recognize the concerns > other have expressed about complexity, I think that complexity can be > managed if done carefully. I'm not sure if you've looked at Apple's > extended syslog, which among other things, includes a binary log file format > making it more mechanically searched and managed, do take a look if you > haven't. ... and I'm not just saying this to be ornery, but what about test log file formats is not mechanically searchable? Note that I'm not playing the XML card here (I'm not an XML fan) but the only real draw of a binary format (to me) is a small amount of innate compression (numbers and dates in binary form) and the ability to have field separators that are not part of the printable character set. UN*X has a strong tradition of text files that work around these two limitations in a variety of ways --- and UN*X tools are built with these assumptions. There's a strong set of reasons to consider retaining text formats and continuing to improve our tools around them. One way to strengthen the syslog format is to have syslog enforce a format _and_ enforce that whatever field separator is chosen cannot be written within a field. From rnoland at 2hip.net Tue May 6 17:22:41 2008 From: rnoland at 2hip.net (Robert Noland) Date: Tue May 6 17:22:46 2008 Subject: Improving Syslog In-Reply-To: <5f67a8c40805060941n7d2882aayea7d72d0a76cd89d@mail.gmail.com> References: <481B7ED4.3020208@mschuette.name> <20080506171525.O52391@fledge.watson.org> <5f67a8c40805060941n7d2882aayea7d72d0a76cd89d@mail.gmail.com> Message-ID: <1210092425.40346.2.camel@squirrel.corp.cox.com.cox.net> On Tue, 2008-05-06 at 12:41 -0400, Zaphod Beeblebrox wrote: > On Tue, May 6, 2008 at 12:21 PM, Robert Watson wrote: > > > > > This sounds like an exciting project -- while I recognize the concerns > > other have expressed about complexity, I think that complexity can be > > managed if done carefully. I'm not sure if you've looked at Apple's > > extended syslog, which among other things, includes a binary log file format > > making it more mechanically searched and managed, do take a look if you > > haven't. > > > ... and I'm not just saying this to be ornery, but what about test log file > formats is not mechanically searchable? Note that I'm not playing the XML > card here (I'm not an XML fan) but the only real draw of a binary format (to > me) is a small amount of innate compression (numbers and dates in binary > form) and the ability to have field separators that are not part of the > printable character set. UN*X has a strong tradition of text files that > work around these two limitations in a variety of ways --- and UN*X tools > are built with these assumptions. There's a strong set of reasons to > consider retaining text formats and continuing to improve our tools around > them. I don't think anyone has suggested replacing text with a binary format. Just providing an alternative for those who choose to use it. robert. > One way to strengthen the syslog format is to have syslog enforce a format > _and_ enforce that whatever field separator is chosen cannot be written > within a field. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From lists at mschuette.name Tue May 6 22:31:53 2008 From: lists at mschuette.name (=?ISO-8859-1?Q?Martin_Sch=FCtte?=) Date: Tue May 6 22:31:58 2008 Subject: Improving Syslog In-Reply-To: <20080506171525.O52391@fledge.watson.org> References: <481B7ED4.3020208@mschuette.name> <20080506171525.O52391@fledge.watson.org> Message-ID: <4820DC52.9070901@mschuette.name> Robert Watson schrieb: > managed if done carefully. I'm not sure if you've looked at Apple's > extended syslog, which among other things, includes a binary log file > format making it more mechanically searched and managed, do take a look > if you haven't. I read the asl man pages (http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPages/man3/asl.3.html) because I am looking for an API to support the new fields in syslog-protocol. I liked the approach of adding different attributes to a message object, I think that would work well for structured data parametern. But the asl functions would have to be extended for syslog-protocol because they do not use namespaces for the key=value pairs. The log storage and filter mechanisms are interesting, but IMHO out of scope for a basic syslogd. If desired one could probably write a small tool to use an indexed BDB for storage and attach it with a pipe to syslogd. -- Martin From rmgls at free.fr Wed May 7 15:59:20 2008 From: rmgls at free.fr (rmgls@free.fr) Date: Wed May 7 15:59:26 2008 Subject: general i/o question Message-ID: <20080507153903.41F6C322858@smtp7-g19.free.fr> Hi all, Sorry if its a FAQ but i don't find any answer for this topic. i need to test (NOWAIT), the presence of keypressed/depressed on a terminal and then read the scan code, like for a piano pc keyboard. my questions are as follows: 1. is it a general C function which may scan a terminal without waiting? 2. how to get the scancodes? of course i can poll tje (0x64) keyboard port on a i386 architecture, but this is not a general method, and it is suited for a distant terminal for instance. Any hint would be appreciated. thanks in advance. rmgls From yanefbsd at gmail.com Wed May 7 18:14:02 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Wed May 7 18:14:07 2008 Subject: general i/o question In-Reply-To: <20080507153903.41F6C322858@smtp7-g19.free.fr> References: <20080507153903.41F6C322858@smtp7-g19.free.fr> Message-ID: On May 7, 2008, at 8:39 AM, rmgls@free.fr wrote: > > Hi all, > > Sorry if its a FAQ but i don't find any answer for this topic. > > i need to test (NOWAIT), the presence of keypressed/depressed on a > terminal > and then read the scan code, like for a piano pc keyboard. > > my questions are as follows: > > 1. is it a general C function which may scan a terminal without > waiting? > > 2. how to get the scancodes? > > of course i can poll tje (0x64) keyboard port on a i386 architecture, > but this is not a general method, and it is suited for a distant > terminal > for instance. > > Any hint would be appreciated. > > thanks in advance. > > rmgls getc(3), or were you looking for something a bit more specific? -Garret From dwmalone at maths.tcd.ie Wed May 7 19:06:34 2008 From: dwmalone at maths.tcd.ie (David Malone) Date: Wed May 7 19:06:37 2008 Subject: general i/o question In-Reply-To: <20080507153903.41F6C322858@smtp7-g19.free.fr> References: <20080507153903.41F6C322858@smtp7-g19.free.fr> Message-ID: <20080507190631.GA90936@walton.maths.tcd.ie> On Wed, May 07, 2008 at 05:39:00PM +0200, rmgls@free.fr wrote: > i need to test (NOWAIT), the presence of keypressed/depressed on a terminal > and then read the scan code, like for a piano pc keyboard. > > my questions are as follows: > > 1. is it a general C function which may scan a terminal without waiting? There isn't a general way of doing this, as even if there is a terminal attached, the terminal may actaully be an ssh session, and which does not transmit key up/down events. If you look at the Unix TTY stuff, you can get the stream of keys pressed, but this may not be enough for a piano type application. If you're only targeting the FreeBSD console there may be other options, but it will be more specific that TTY calls. If you are trageting X11, then it does support key up/down events and a book or tutorial on X programming should help. David. From koitsu at freebsd.org Wed May 7 21:40:58 2008 From: koitsu at freebsd.org (Jeremy Chadwick) Date: Wed May 7 21:41:02 2008 Subject: general i/o question In-Reply-To: <20080507153903.41F6C322858@smtp7-g19.free.fr> References: <20080507153903.41F6C322858@smtp7-g19.free.fr> Message-ID: <20080507214057.GA74435@eos.sc1.parodius.com> On Wed, May 07, 2008 at 05:39:00PM +0200, rmgls@free.fr wrote: > i need to test (NOWAIT), the presence of keypressed/depressed on a terminal > and then read the scan code, like for a piano pc keyboard. > > my questions are as follows: > > 1. is it a general C function which may scan a terminal without waiting? > > 2. how to get the scancodes? It depends on if you're wanting the actual hard-wired keyboard (on the console), or if you're taking input from a tty/pty. You won't get a true scancode from a tty/pty, but you will get a character (0x00 to 0xFF). Regarding I/O without waiting: there is not a general libc function for this. Garrett mentioned getc(), which blocks (waits). You might want to consider seeing if the kqueue/kevent stuff on the BSDs will work with pty/tty input. You can use that to set up an event in the kernel which tells the kernel "run function XYZ when I/O is seen on this fh/fd". It's like select() in that respect, but is faster. > of course i can poll tje (0x64) keyboard port on a i386 architecture, > but this is not a general method, and it is suited for a distant terminal > for instance. I agree. This method is also very old, and I'd be surprised if it would work with USB keyboards. I would assume it would also interfere with any existing keyboard I/O handler the OS has, but I'm not sure. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From mwm-keyword-freebsdhackers2.e313df at mired.org Wed May 7 22:31:26 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Wed May 7 22:31:28 2008 Subject: general i/o question In-Reply-To: <20080507214057.GA74435@eos.sc1.parodius.com> References: <20080507153903.41F6C322858@smtp7-g19.free.fr> <20080507214057.GA74435@eos.sc1.parodius.com> Message-ID: <20080507180441.008acfe9@bhuda.mired.org> On Wed, 7 May 2008 14:40:57 -0700 Jeremy Chadwick wrote: > On Wed, May 07, 2008 at 05:39:00PM +0200, rmgls@free.fr wrote: > > i need to test (NOWAIT), the presence of keypressed/depressed on a terminal > > and then read the scan code, like for a piano pc keyboard. > > > > my questions are as follows: > > > > 1. is it a general C function which may scan a terminal without waiting? > Regarding I/O without waiting: there is not a general libc function for > this. Garrett mentioned getc(), which blocks (waits). getc won't tell you whether a key is pressed or not in any case, just that a key *has been* pressed when it returns. This question is normally tied back to some DOS getc-like call that didn't block, but returned null if a key hadn't been pressed. The standard way to do that is select() (now poll). The problem with both of those is that - again - they won't tell you whether or not a key *is* pressed, but only whether one *has been* pressed. > You might want to consider seeing if the kqueue/kevent stuff on the BSDs > will work with pty/tty input. You can use that to set up an event in > the kernel which tells the kernel "run function XYZ when I/O is seen on > this fh/fd". It's like select() in that respect, but is faster. Yes, they'll work with it. The question is - can you get the information about whether or not a key is currently pressed that way? Since X gets the information, its' there - but it's not clear how close to the hardware X has to get to get it. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From rmgls at free.fr Thu May 8 06:06:57 2008 From: rmgls at free.fr (rmgls@free.fr) Date: Thu May 8 06:07:03 2008 Subject: general i/o question Message-ID: <20080508060655.9C4BC3227F6@smtp7-g19.free.fr> On Wed, 7 May 2008 14:40:57 Jeremy Chadwick wrote: > On Wed, May 07, 2008 at 05:39:00PM +0200, rmgls@free.fr wrote: > > i need to test (NOWAIT), the presence of keypressed/depressed on a terminal > > and then read the scan code, like for a piano pc keyboard. > > > > my questions are as follows: > > > > 1. is it a general C function which may scan a terminal without waiting? > > > > 2. how to get the scancodes? > > It depends on if you're wanting the actual hard-wired keyboard (on the > console), or if you're taking input from a tty/pty. You won't get a > true scancode from a tty/pty, but you will get a character (0x00 to > 0xFF). Hi Jeremy, after all, i don't need the scancode, i can work with characters, this avoid the nls translation, which is already done! > Regarding I/O without waiting: there is not a general libc function for > this. Garrett mentioned getc(), which blocks (waits). Attilio Rao pointed me tje termios struct, and looking to the doc i found the O_NONBLOCK open(2) flag... which means: NOWAIT i was not aware of it! and in this case getc() works fine as expected. > You might want to consider seeing if the kqueue/kevent stuff on the BSDs > will work with pty/tty input. You can use that to set up an event in > the kernel which tells the kernel "run function XYZ when I/O is seen on > this fh/fd". It's like select() in that respect, but is faster. Thanks for this point i will investigate now. BUT don't you think it would be a desirable behaviour addition to the read() libc function??? > > of course i can poll tje (0x64) keyboard port on a i386 architecture, > > but this is not a general method, and it is suited for a distant terminal > > for instance. > > I agree. This method is also very old, and I'd be surprised if it would > work with USB keyboards. I would assume it would also interfere with > any existing keyboard I/O handler the OS has, but I'm not sure. Yes, i agree it would be a bad idea, kqueue/kevent is better. i will see that. Best regards raoul rmgls@free.fr From mathieu.prevot at gmail.com Thu May 8 14:23:52 2008 From: mathieu.prevot at gmail.com (Mathieu Prevot) Date: Thu May 8 14:23:56 2008 Subject: binary file within a shell script Message-ID: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> Hi there, I would like to use one exec file from a shellscript but I would like it to be incorporated in the same file, like Nvidia do for its FreeBSD drivers. How can I do this in a convenient way ? Mathieu From viaprog at gmail.com Thu May 8 14:39:14 2008 From: viaprog at gmail.com (Igor A. Valcov) Date: Thu May 8 14:39:18 2008 Subject: do not work nested unnamed anchor Message-ID: Hello. For example: ==== pf.conf ==== ext_if="xl0" ip_world="nn.nn.nn.nn" # Filter rules block log all anchor in on $ext_if { pass quick proto tcp to $ip_world port 22 keep state # SSH pass quick proto tcp to $ip_world port 25 keep state # SMTP pass quick proto tcp to $ip_world port 110 keep state # POP3 anchor { pass quick proto tcp to $ip_world port 995 keep state # POP3S } } ============ nmap results: PORT STATE SERVICE VERSION 22/tcp open ssh OpenSSH 4.5p1 (FreeBSD 20061110; protocol 2.0) 25/tcp open smtp? 110/tcp open pop3 Openwall popa3d I can not understand what the problem... FreeBSD-7.0-RELEASE-p1 i386 -- Igor A. Valcov From jille at quis.cx Thu May 8 15:21:08 2008 From: jille at quis.cx (Jille) Date: Thu May 8 15:21:33 2008 Subject: binary file within a shell script In-Reply-To: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> References: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> Message-ID: <48231693.10805@quis.cx> It's not exactly what you are looking for: But you could take a look at shar(1). I don't even know for sure whether it can archive binaries. shar gives you a shellscript, to which you could prefix your own script, and when you run it, it'll extract the incorporated file, and you can exec it :) -- Jille (Resend from right email-adres) Mathieu Prevot wrote: > Hi there, > > I would like to use one exec file from a shellscript but I would like > it to be incorporated in the same file, like Nvidia do for its FreeBSD > drivers. How can I do this in a convenient way ? > > Mathieu > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From teemu at rinta-aho.org Thu May 8 15:35:15 2008 From: teemu at rinta-aho.org (Teemu Rinta-aho) Date: Thu May 8 15:35:17 2008 Subject: vm_map_insert fails, vm_map->max_offset==0? Message-ID: <48231DB8.1020705@rinta-aho.org> Hi, is it possible to insert a new map entry to a process vm_map from a kernel module? I've tried to use vm_map_insert, vm_map_fixed, vm_map_findspace, but they all fail as the vm_map->max_offset is zero (0). I have looked at vm_mmap etc. but I don't get a clue what I'm doing wrong... BR, Teemu From dnelson at allantgroup.com Thu May 8 16:22:53 2008 From: dnelson at allantgroup.com (Dan Nelson) Date: Thu May 8 16:22:56 2008 Subject: binary file within a shell script In-Reply-To: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> References: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> Message-ID: <20080508161246.GA2922@dan.emsphone.com> In the last episode (May 08), Mathieu Prevot said: > Hi there, > > I would like to use one exec file from a shellscript but I would like > it to be incorporated in the same file, like Nvidia do for its FreeBSD > drivers. How can I do this in a convenient way ? Take a look at the file generated by /usr/bin/gzexe; that's one way to do it (basically, determine the number of lines in your shell script, append your binary file to the end of the script, and use tail to extract only the binary file to a tempfile). -- Dan Nelson dnelson@allantgroup.com From teemu at rinta-aho.org Thu May 8 15:29:00 2008 From: teemu at rinta-aho.org (Teemu Rinta-aho) Date: Thu May 8 17:25:58 2008 Subject: vm_map_insert fails, vm_map->max_offset==0? Message-ID: <48231C3F.3080906@rinta-aho.org> Hi, is it possible to insert a new map entry to a process vm_map from a kernel module? I've tried to use vm_map_insert, vm_map_fixed, vm_map_findspace, but they all fail as the vm_map->max_offset is zero (0). I have looked at vm_mmap etc. but I don't get a clue what I'm doing wrong... BR, Teemu From edelkind-freebsd-hackers at episec.com Thu May 8 18:08:23 2008 From: edelkind-freebsd-hackers at episec.com (ari edelkind) Date: Thu May 8 18:08:27 2008 Subject: binary file within a shell script In-Reply-To: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> References: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> Message-ID: <20080508174142.GZ79355@episec.com> mathieu.prevot@gmail.com wrote: > I would like to use one exec file from a shellscript but I would like > it to be incorporated in the same file, like Nvidia do for its FreeBSD > drivers. How can I do this in a convenient way ? I haven't looked at nvidia's driver packaging, but you can embed binaries into shell scripts using uuencode or base64. Example: ------------------------ % cat >test.sh #!/bin/ksh echo "*** generating ls..." file=`mktemp /tmp/ls.XXXXXX` [[ $? -eq 0 ]] || exit 1 uudecode -o $lsfile <<'__EOM__' ^D % uuencode ls >test.sh % cat >>test.sh __EOM__ chmod +x $lsfile echo "*** running $lsfile ..." $lsfile echo "*** cleaning up" rm -f $lsfile ^D ------------------------ Note that i used single quotes in the here-document initialization, so there won't be any shell expansion of the uuencoded data. A few commonly-installed programs that may suit your needs: - uuencode / uudecode - base64 - b64 - openssl base64 If relying on one of the above is infeasible: You can't portably use inline binary data in a shell script without preprocessing it (as with one of the above programs), since most shells can't handle binary zeros; shar(1) fails in this case. You could, theoretically, write a small, clever wrapper function to account for the issue. You'd also have to ensure that regexp("^__EOM__$") (in the above example) doesn't exist within the file contents, and note that excessively long lines may not be handled efficiently by the shell. You'll need to account for files that do or don't end in a newline, possibly by always appending an extra newline, then stripping it upon extraction. Lastly, you'll need to consider whether you must account for CR/LF conversion in file transfers or while editing your script. You probably won't want to deal with all of this, and would be better off leaving it for out-of-band extraction, such as with dan's gzexe suggestion. You'll still need to determine whether CR/LF conversion may be an issue for you. ari From ravi.murty at intel.com Thu May 8 20:33:22 2008 From: ravi.murty at intel.com (Murty, Ravi) Date: Thu May 8 20:33:25 2008 Subject: SW_PREEMPT and cpu runq Message-ID: Hi, When a thread is being switched out and it is being preempted (e.g. time quantum expires), why does sched_switch hold it on the current cpu? i.e. why does the code see that it was preempted and put it back on the same queue? In other cases it looks to see if it can be migrated and the thread goes back some place else. If a thread is being kicked out and there is a perfectly idle CPU some where on the system, wouldn't it make sense to migrate the thread? Thanks Ravi From edelkind-freebsd-hackers at episec.com Thu May 8 20:55:25 2008 From: edelkind-freebsd-hackers at episec.com (ari edelkind) Date: Thu May 8 20:55:28 2008 Subject: binary file within a shell script In-Reply-To: <20080508174142.GZ79355@episec.com> References: <3e473cc60805080659h721db611s886b80d213f9a2f3@mail.gmail.com> <20080508174142.GZ79355@episec.com> Message-ID: <20080508205524.GA79355@episec.com> edelkind-freebsd-hackers@episec.com wrote: > echo "*** generating ls..." > file=`mktemp /tmp/ls.XXXXXX` > [[ $? -eq 0 ]] || exit 1 Er, s/^file/lsfile/, obviously. ari From julian at elischer.org Thu May 8 22:27:45 2008 From: julian at elischer.org (Julian Elischer) Date: Thu May 8 22:27:50 2008 Subject: SW_PREEMPT and cpu runq In-Reply-To: References: Message-ID: <48237E60.9040007@elischer.org> Murty, Ravi wrote: > Hi, > > > > When a thread is being switched out and it is being preempted (e.g. time > quantum expires), why does sched_switch hold it on the current cpu? i.e. > why does the code see that it was preempted and put it back on the same > queue? > > In other cases it looks to see if it can be migrated and the thread goes > back some place else. If a thread is being kicked out and there is a > perfectly idle CPU some where on the system, wouldn't it make sense to > migrate the thread? it shouldn't be held.. why do you think it is? (and is this in 6.x still?) > > > > Thanks > Ravi > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From ravi.murty at intel.com Thu May 8 22:48:14 2008 From: ravi.murty at intel.com (Murty, Ravi) Date: Thu May 8 22:48:17 2008 Subject: SW_PREEMPT and cpu runq In-Reply-To: <48237E60.9040007@elischer.org> References: <48237E60.9040007@elischer.org> Message-ID: I guess two places. 1. maybe_preempt() - I've decided to preempt a thread on a cpu and the outgoing thread is held (SW_PREEMPT) on the same cpu. 2. timer expires and thread is out of its slice (ULE), in this case I remove the load and re-add it back to the same (current) cpu. Sorry Julian, yes this is 6.2 Thanks much, Ravi -----Original Message----- From: Julian Elischer [mailto:julian@elischer.org] Sent: Thursday, May 08, 2008 3:28 PM To: Murty, Ravi Cc: freebsd-hackers@freebsd.org Subject: Re: SW_PREEMPT and cpu runq Murty, Ravi wrote: > Hi, > > > > When a thread is being switched out and it is being preempted (e.g. time > quantum expires), why does sched_switch hold it on the current cpu? i.e. > why does the code see that it was preempted and put it back on the same > queue? > > In other cases it looks to see if it can be migrated and the thread goes > back some place else. If a thread is being kicked out and there is a > perfectly idle CPU some where on the system, wouldn't it make sense to > migrate the thread? it shouldn't be held.. why do you think it is? (and is this in 6.x still?) > > > > Thanks > Ravi > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From ravi.murty at intel.com Thu May 8 23:02:08 2008 From: ravi.murty at intel.com (Murty, Ravi) Date: Thu May 8 23:02:12 2008 Subject: SW_PREEMPT and cpu runq References: <48237E60.9040007@elischer.org> Message-ID: Oh, I find this happens only in ULE -- during sched_switch(), it sets KEF_HOLD and then calls setrunqueue(). This ensures that the thread does not migrate on preemptions. Ravi -----Original Message----- From: Murty, Ravi Sent: Thursday, May 08, 2008 3:48 PM To: 'Julian Elischer' Cc: freebsd-hackers@freebsd.org Subject: RE: SW_PREEMPT and cpu runq I guess two places. 1. maybe_preempt() - I've decided to preempt a thread on a cpu and the outgoing thread is held (SW_PREEMPT) on the same cpu. 2. timer expires and thread is out of its slice (ULE), in this case I remove the load and re-add it back to the same (current) cpu. Sorry Julian, yes this is 6.2 Thanks much, Ravi -----Original Message----- From: Julian Elischer [mailto:julian@elischer.org] Sent: Thursday, May 08, 2008 3:28 PM To: Murty, Ravi Cc: freebsd-hackers@freebsd.org Subject: Re: SW_PREEMPT and cpu runq Murty, Ravi wrote: > Hi, > > > > When a thread is being switched out and it is being preempted (e.g. time > quantum expires), why does sched_switch hold it on the current cpu? i.e. > why does the code see that it was preempted and put it back on the same > queue? > > In other cases it looks to see if it can be migrated and the thread goes > back some place else. If a thread is being kicked out and there is a > perfectly idle CPU some where on the system, wouldn't it make sense to > migrate the thread? it shouldn't be held.. why do you think it is? (and is this in 6.x still?) > > > > Thanks > Ravi > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From andenore at start.no Fri May 9 11:54:06 2008 From: andenore at start.no (Anders Nore) Date: Fri May 9 11:54:11 2008 Subject: Adding .db support to pkg_tools Message-ID: Hi, I'm working on adding .db support to the pkg_tools( i.e. pkg_add, pkg_info, etc. ) as part of SoC 2008. The database api used is BerkeleyDB that comes with the base system (/usr/src/include/db.h). BerkeleyDB is not you're typical relational db, and can only save key/value pairs. The way I'm thinking of storing information to the .db is to name the keys as the directory names in /var/db/pkg. And save the +* files in the directories to the value element in the db, separated with a special character or similar. One problem lies with the +* files which is scripts (e.g., +INSTALL, +DEINSTALL). I've gotten some input that it's bad to save scripts in the db, but if it's not going to be saved there, then where? Isn't it possible to execute a script without saving a file to disk? Like using "sh -c 'string'". In my personal opinion it should not be a hybrid solution where you save the script files in an old fashion way, for example /var/db/pkg/someport-1.2_1/+INSTALL, and the rest of the information lies in the .db file. Because then you have redundancy and that could lead to inconsistencies. Let me hear your opinions =) - Anders Nore From joerg at britannica.bec.de Fri May 9 12:43:07 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Fri May 9 12:43:10 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: Message-ID: <20080509124308.GA596@britannica.bec.de> On Fri, May 09, 2008 at 01:52:46PM +0200, Anders Nore wrote: > I'm working on adding .db support to the pkg_tools( i.e. pkg_add, pkg_info, > etc. ) as part of SoC 2008. The database api used is BerkeleyDB that comes > with the base system (/usr/src/include/db.h). BerkeleyDB is not you're > typical relational db, and can only save key/value pairs. The way I'm > thinking of storing information to the .db is to name the keys as the > directory names in /var/db/pkg. And save the +* files in the directories to > the value element in the db, separated with a special character or similar. As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I would *strongly* advisy you against storing the files only in a bdb file. The change of major and complete corruption with bdb185 is high, consider pulling the plug in the middle of a long update. Secondly, I would also advisy against just storing all meta data in a single key/value pair. For example, +CONTENTS can be extremely large. Check texmf for a good example. Joerg From viaprog at gmail.com Fri May 9 12:55:24 2008 From: viaprog at gmail.com (Igor A. Valcov) Date: Fri May 9 12:55:28 2008 Subject: do not work nested unnamed anchor Message-ID: Hello. For example: ==== pf.conf ==== ext_if="xl0" ip_world="nn.nn.nn.nn" # Filter rules block log all anchor in on $ext_if { pass quick proto tcp to $ip_world port 22 keep state # SSH pass quick proto tcp to $ip_world port 25 keep state # SMTP pass quick proto tcp to $ip_world port 110 keep state # POP3 anchor { pass quick proto tcp to $ip_world port 995 keep state # POP3S } } ============ nmap results: PORT STATE SERVICE VERSION 22/tcp open ssh OpenSSH 4.5p1 (FreeBSD 20061110; protocol 2.0) 25/tcp open smtp? 110/tcp open pop3 Openwall popa3d I can not understand what the problem... FreeBSD-7.0-RELEASE-p1 i386 -- Igor A. Valcov From fbsd.hackers at rachie.is-a-geek.net Fri May 9 12:56:53 2008 From: fbsd.hackers at rachie.is-a-geek.net (Mel) Date: Fri May 9 12:56:58 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: Message-ID: <200805091440.36202.fbsd.hackers@rachie.is-a-geek.net> On Friday 09 May 2008 13:52:46 Anders Nore wrote: > I'm working on adding .db support to the pkg_tools( i.e. pkg_add, > pkg_info, etc. ) as part of SoC 2008. Is this gonna be optional? > One problem lies with the +* files which is scripts (e.g., +INSTALL, > +DEINSTALL). I've gotten some input that it's bad to save scripts in the > db, but if it's not going to be saved there, then where? Isn't it possible > to execute a script without saving a file to disk? Like using "sh -c > 'string'". In my personal opinion it should not be a hybrid solution where > you save the script files in an old fashion way, for example > /var/db/pkg/someport-1.2_1/+INSTALL, and the rest of the information lies > in the .db file. Because then you have redundancy and that could lead to > inconsistencies. Don't know what the reasons are for people to say it's bad to save scripts in the db, but for me it would be that I can't inspect and/or modify them, though if there were support in the ports for PKG(DE)INSTALL_LOCAL, I suppose I could do without the modification part. -- Mel From mail25 at bzerk.org Fri May 9 12:51:21 2008 From: mail25 at bzerk.org (Ruben de Groot) Date: Fri May 9 14:14:46 2008 Subject: Panics in kern_timeout.c (RELENG_7) Message-ID: <20080509122229.GA11892@ei.bzerk.org> Hi, After upgrading to 7-stable with sources of about 2 weeks ago I'm seeing panics about once every 2 days, see below. I've posted to stable@ two days ago, but got no reactions. Nobody else seems to have this issue according to google. Any ideas? Ruben Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x736e77 fault code = supervisor read, page not present instruction pointer = 0x20:0xc05f095f stack pointer = 0x28:0xd4ceec80 frame pointer = 0x28:0xd4ceecbc code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 13 (swi4: clock sio) trap number = 12 panic: page fault cpuid = 0 Uptime: 1d15h8m59s Physical memory: 503 MB Dumping 78 MB: 63 47 31 15 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) list *0xc05f095f 0xc05f095f is in softclock (/usr/src/sys/kern/kern_timeout.c:203). 198 curticks = softticks; 199 bucket = &callwheel[curticks & callwheelmask]; 200 c = TAILQ_FIRST(bucket); 201 while (c) { 202 depth++; 203 if (c->c_time != curticks) { 204 c = TAILQ_NEXT(c, c_links.tqe); 205 ++steps; 206 if (steps >= MAX_SOFTCLOCK_STEPS) { 207 nextsoftcheck = c; (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc05de987 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc05dec49 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:572 #3 0xc08961cc in trap_fatal (frame=0xd4ceec40, eva=7564919) at /usr/src/sys/i386/i386/trap.c:899 #4 0xc0896b4f in trap (frame=0xd4ceec40) at /usr/src/sys/i386/i386/trap.c:280 #5 0xc087cb8b in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #6 0xc05f095f in softclock (dummy=0x0) at /usr/src/sys/kern/kern_timeout.c:202 #7 0xc05bf55b in ithread_loop (arg=0xc2997280) at /usr/src/sys/kern/kern_intr.c:1036 #8 0xc05bc339 in fork_exit (callout=0xc05bf3b0 , arg=0xc2997280, frame=0xd4ceed38) at /usr/src/sys/kern/kern_fork.c:783 #9 0xc087cc00 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:205 From romain at blogreen.org Fri May 9 14:35:14 2008 From: romain at blogreen.org (Romain =?iso-8859-1?Q?Tarti=E8re?=) Date: Fri May 9 14:35:24 2008 Subject: indent(1) support for gcc(1) 0b prefix In-Reply-To: <481A959C.5000704@freebsd.org> References: <20080426213557.GA88577@marvin.blogreen.org> <200804270201.53271.max@love2party.net> <8763txlaj6.fsf@kobe.laptop> <481A959C.5000704@freebsd.org> Message-ID: <20080509143512.GA69792@marvin.blogreen.org> On Thu, May 01, 2008 at 09:16:28PM -0700, Tim Kientzle wrote: > In all of these cases, though, adding a space between the '0' and > the 'b' changes the meaning, so is wrong. Indent can > change/insert whitespace, but should never do so in a way that > changes the meaning of the program. In all of these cases, > having indent recognize "0b..." as a single token is the > correct behavior. > > So I don't see any point in having this recognition be > tunable. indent already has too many switches. I oppened PR bin/123553 concerning this: http://www.freebsd.org/cgi/query-pr.cgi?pr=123553 Regards -- Romain Tarti?re http://romain.blogreen.org/ pgp: 8DAB A124 0DA4 7024 F82A E748 D8E9 A33F FF56 FF43 (ID: 0xFF56FF43) (plain text =non-HTML= PGP/GPG encrypted/signed e-mail much appreciated) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080509/5d1a4b67/attachment.pgp From romain at blogreen.org Fri May 9 14:35:14 2008 From: romain at blogreen.org (Romain =?iso-8859-1?Q?Tarti=E8re?=) Date: Fri May 9 14:35:25 2008 Subject: indent(1) support for gcc(1) 0b prefix In-Reply-To: <481A959C.5000704@freebsd.org> References: <20080426213557.GA88577@marvin.blogreen.org> <200804270201.53271.max@love2party.net> <8763txlaj6.fsf@kobe.laptop> <481A959C.5000704@freebsd.org> Message-ID: <20080509143512.GA69792@marvin.blogreen.org> On Thu, May 01, 2008 at 09:16:28PM -0700, Tim Kientzle wrote: > In all of these cases, though, adding a space between the '0' and > the 'b' changes the meaning, so is wrong. Indent can > change/insert whitespace, but should never do so in a way that > changes the meaning of the program. In all of these cases, > having indent recognize "0b..." as a single token is the > correct behavior. > > So I don't see any point in having this recognition be > tunable. indent already has too many switches. I oppened PR bin/123553 concerning this: http://www.freebsd.org/cgi/query-pr.cgi?pr=123553 Regards -- Romain Tarti?re http://romain.blogreen.org/ pgp: 8DAB A124 0DA4 7024 F82A E748 D8E9 A33F FF56 FF43 (ID: 0xFF56FF43) (plain text =non-HTML= PGP/GPG encrypted/signed e-mail much appreciated) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080509/5d1a4b67/attachment-0001.pgp From andenore at start.no Fri May 9 16:51:25 2008 From: andenore at start.no (Anders Nore) Date: Fri May 9 16:51:30 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <20080509124308.GA596@britannica.bec.de> References: <20080509124308.GA596@britannica.bec.de> Message-ID: On Fri, 09 May 2008 14:43:08 +0200, Joerg Sonnenberger wrote: > On Fri, May 09, 2008 at 01:52:46PM +0200, Anders Nore wrote: >> I'm working on adding .db support to the pkg_tools( i.e. pkg_add, >> pkg_info, >> etc. ) as part of SoC 2008. The database api used is BerkeleyDB that >> comes >> with the base system (/usr/src/include/db.h). BerkeleyDB is not you're >> typical relational db, and can only save key/value pairs. The way I'm >> thinking of storing information to the .db is to name the keys as the >> directory names in /var/db/pkg. And save the +* files in the >> directories to >> the value element in the db, separated with a special character or >> similar. > > As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I would > *strongly* advisy you against storing the files only in a bdb file. > The change of major and complete corruption with bdb185 is high, > consider pulling the plug in the middle of a long update. Yes that would probably be bad for the database, but I'm sure one can manage to get around this problem by copying it before changing the db and delete the copy if it doesn't fail. At the next time executed it will check for a copy, use that and assume that the last run was unsuccessful. Note that this is not a replacement for the existing tools, although it might be in the future. It's a standalone replica of the old tools with bdb support and other smaller improvements. And for compatibility reasons the tools will use a hybrid method (saving to both .db and flat db), but reading and querying will only be done on the .db file. > Secondly, I would also advisy against just storing all meta data in a > single key/value pair. For example, +CONTENTS can be extremely large. > Check texmf for a good example. When it comes to storing large values in a key/value pair, I think that's what bdb was designed for, handling large amounts of data (in the orders of gigabytes even in key's) fast. -Anders From joerg at britannica.bec.de Fri May 9 17:06:32 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Fri May 9 17:06:37 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: <20080509124308.GA596@britannica.bec.de> Message-ID: <20080509170633.GB3571@britannica.bec.de> On Fri, May 09, 2008 at 06:50:10PM +0200, Anders Nore wrote: > Yes that would probably be bad for the database, but I'm sure one can > manage to get around this problem by copying it before changing the db and > delete the copy if it doesn't fail. At the next time executed it will check > for a copy, use that and assume that the last run was unsuccessful. /var/db/pkg contains 10MB for the various packages installed on my laptop and 10MB for the cache of +CONTENTS. You don't want to copy that around all the time. >> Secondly, I would also advisy against just storing all meta data in a >> single key/value pair. For example, +CONTENTS can be extremely large. >> Check texmf for a good example. > > When it comes to storing large values in a key/value pair, I think that's > what bdb was designed for, handling large amounts of data (in the orders of > gigabytes even in key's) fast. No, actually that is exactly what it was *not* designed for. Having billions of keys is fine, but data that exceeds the size of a database page is going to slow down. While it might not be a problem of you are copying the data to a new file anyway, it also means that fragmentation in the database will be more problematic. My main point is that for the interesting operations you want to actually look up with fine grained keys and that's what is not possible if you store the meta data as blob. In fact, storing the meta data as blob is not faster than just using the filesystem. Joerg From roberto at keltia.freenix.fr Fri May 9 17:35:10 2008 From: roberto at keltia.freenix.fr (Ollivier Robert) Date: Fri May 9 17:35:13 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: Message-ID: <20080509173507.GB35511@keltia.freenix.fr> According to Anders Nore: > with the base system (/usr/src/include/db.h). BerkeleyDB is not you're > typical relational db, and can only save key/value pairs. The way I'm > thinking of storing information to the .db is to name the keys as the > directory names in /var/db/pkg. And save the +* files in the directories to > the value element in the db, separated with a special character or similar. Please make it compatible with the existing infrastructure in /var/db/pkg maintained by portinstall/portupgrade. If it can be improved as a side-effect, that would be even better. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr Darwin sidhe.keltia.net Version 9.2.0: Tue Feb 5 16:13:22 PST 2008; i386 From roberto at keltia.freenix.fr Fri May 9 17:36:28 2008 From: roberto at keltia.freenix.fr (Ollivier Robert) Date: Fri May 9 17:36:31 2008 Subject: sshd patch to avoid DNS lookups when using 'UseDNS no' or -u0 In-Reply-To: <4818F72C.90300@cran.org.uk> References: <4818F72C.90300@cran.org.uk> Message-ID: <20080509173626.GC35511@keltia.freenix.fr> According to Bruce Cran: > I've attached a patch which implements this (the change to loginrec.c > reverts it back to the default OpenSSH code) and was wondering if someone > could take a look at it. If you have not already done so, please use send-pr to record it in GNATS, that will help not forgetting about it, thanks! -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr Darwin sidhe.keltia.net Version 9.2.0: Tue Feb 5 16:13:22 PST 2008; i386 From roberto at keltia.freenix.fr Fri May 9 17:36:28 2008 From: roberto at keltia.freenix.fr (Ollivier Robert) Date: Fri May 9 17:36:31 2008 Subject: sshd patch to avoid DNS lookups when using 'UseDNS no' or -u0 In-Reply-To: <4818F72C.90300@cran.org.uk> References: <4818F72C.90300@cran.org.uk> Message-ID: <20080509173626.GC35511@keltia.freenix.fr> According to Bruce Cran: > I've attached a patch which implements this (the change to loginrec.c > reverts it back to the default OpenSSH code) and was wondering if someone > could take a look at it. If you have not already done so, please use send-pr to record it in GNATS, that will help not forgetting about it, thanks! -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr Darwin sidhe.keltia.net Version 9.2.0: Tue Feb 5 16:13:22 PST 2008; i386 From andenore at start.no Fri May 9 17:55:55 2008 From: andenore at start.no (Anders Nore) Date: Fri May 9 17:55:59 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <20080509170633.GB3571@britannica.bec.de> References: <20080509124308.GA596@britannica.bec.de> <20080509170633.GB3571@britannica.bec.de> Message-ID: On Fri, 09 May 2008 19:06:33 +0200, Joerg Sonnenberger wrote: > On Fri, May 09, 2008 at 06:50:10PM +0200, Anders Nore wrote: >> Yes that would probably be bad for the database, but I'm sure one can >> manage to get around this problem by copying it before changing the db >> and >> delete the copy if it doesn't fail. At the next time executed it will >> check >> for a copy, use that and assume that the last run was unsuccessful. > > /var/db/pkg contains 10MB for the various packages installed on my > laptop and 10MB for the cache of +CONTENTS. You don't want to copy that > around all the time. > >>> Secondly, I would also advisy against just storing all meta data in a >>> single key/value pair. For example, +CONTENTS can be extremely large. >>> Check texmf for a good example. >> >> When it comes to storing large values in a key/value pair, I think >> that's >> what bdb was designed for, handling large amounts of data (in the >> orders of >> gigabytes even in key's) fast. > > No, actually that is exactly what it was *not* designed for. Having > billions of keys is fine, but data that exceeds the size of a database > page is going to slow down. While it might not be a problem of you are > copying the data to a new file anyway, it also means that fragmentation > in the database will be more problematic. > > My main point is that for the interesting operations you want to > actually look up with fine grained keys and that's what is not possible > if you store the meta data as blob. In fact, storing the meta data as > blob is not faster than just using the filesystem. > You are probably right, but how would you store the key's? Is storing the key as e.g., 'portname-1.2_3+CONTENT' a good solution? From stevefranks at ieee.org Fri May 9 18:17:29 2008 From: stevefranks at ieee.org (Steve Franks) Date: Fri May 9 18:17:32 2008 Subject: CLOCK_REALTIME undefined on my system Message-ID: <539c60b90805091117l57ed9966h99d6ae5f45925cdf@mail.gmail.com> The manpage (http://www.freebsd.org/cgi/man.cgi?query=clock_gettime&sektion=2&apropos=0&manpath=FreeBSD+7.0-RELEASE) for clock_gettime() specifies the correct header as , which I am using, and I don't see any errors on clock_gettime(), but the param I need, listed in the manpage, CLOCK_REALTIME is undefined. Any ideas how this could be? I've recently cvsup'd standard-supfile, so you'd think I'm up to date... I'm also getting an implicit declaration of isnormal(), and math.h is clearly included... Steve /* * fclock.c * * Copyright (C) 2005 Hein Roehrig * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License * as published by the Free Software Foundation; either version 2 * of the License, or (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA * 02111-1307, USA. * */ #define _ISOC99_SOURCE #define _POSIX_C_SOURCE 199309L #include #include #include #include #ifndef __MINGW32__ #include #endif #include #include /* ------------------------------------------------------------------ */ #ifdef __APPLE__ #include #include #include long double frealtime() { long double result; static uint64_t start_mat; static long double start_time; static double multiplier; mach_timebase_info_data_t mtid; struct timeval tv; if(!mtid.denom == 0) { mach_timebase_info(&mtid); multiplier = (double)mtid.numer / (double)mtid.denom; gettimeofday(&tv, NULL); start_time = (long double)tv.tv_sec + (long double)tv.tv_usec * 1000.0; start_mat = mach_absolute_time(); } result = start_time + (mach_absolute_time() - start_mat) * multiplier; assert(isnormal(result)); assert(result > 0); return result; } #else /* def __APPLE__ */ /* ------------------------------------------------------------------ */ #ifdef _POSIX_TIMERS long double frealtime() { long double result; struct timespec t; if (clock_gettime(CLOCK_REALTIME, &t)==-1) { perror("frealtime (clock_gettime)"); exit(EXIT_FAILURE); } result = (long double)t.tv_sec + (long double)t.tv_nsec*(long double)1e-9; assert(isnormal(result)); assert(result > 0); return result; } #else /* def _POSIX_TIMERS */ /* ------------------------------------------------------------------ */ #ifdef __MINGW32__ #include long double frealtime() { long double result; struct timeb t; ftime(&t); result = (long double)t.time + (long double)t.millitm * (long double)1e-3; assert(isnormal(result)); assert(result > 0); return result; } #else /* def __MINGW32__ */ /* ------------------------------------------------------------------ */ #ifndef CLK_TCK static clock_t CLK_TCK = 0; static void set_clk_tck(void) __attribute__ ((constructor)); static void set_clk_tck(void) { long v = sysconf(_SC_CLK_TCK); if (v == -1) { perror("sysconf(_SC_CLK_TCK)"); exit(EXIT_FAILURE); } CLK_TCK = v; } #endif long double frealtime() { long double result; struct tms t; clock_t c=times(&t); if (c==(clock_t)-1) { perror("frealtime (times)"); exit(EXIT_FAILURE); } result = (long double)c/CLK_TCK; assert(isnormal(result)); assert(result > 0); return result; } #endif #endif #endif From joerg at britannica.bec.de Fri May 9 18:22:37 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Fri May 9 18:22:41 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: <20080509124308.GA596@britannica.bec.de> <20080509170633.GB3571@britannica.bec.de> Message-ID: <20080509182147.GA998@britannica.bec.de> On Fri, May 09, 2008 at 07:54:40PM +0200, Anders Nore wrote: > You are probably right, but how would you store the key's? Is storing the > key as e.g., 'portname-1.2_3+CONTENT' a good solution? I'd just use a different db file. I am not sure how much the following applies to FreeBSD as pkg_install has diverted a lot. The most expensive operations during pkg_add and pkg_info are scans for conflicts (explicit via @pkgcfl or implicit due to overlapping file lists) as they need to compare the to-be-installed package with all existing ones. After that come directory scans to resolve dependencies. Everything else is really just "open this small file and extract some data from it", where small usually means less than one block. Putting that into a database can help or not, but I don't think it is relevant. So the most important operations to support a btree of all files (implemented in NetBSD/pkgsrc) and a btree of all @pkgcfl/@pkgdb (not implemented yet). Joerg From stevefranks at ieee.org Fri May 9 18:32:33 2008 From: stevefranks at ieee.org (Steve Franks) Date: Fri May 9 18:32:37 2008 Subject: correct #define in source to specify FBSD vs. linux? Message-ID: <539c60b90805091103u2c8907a6q83cdca69fb45888e@mail.gmail.com> Seems there is a more appropriate list for my earlier question to freebsd-questions: On and on I charge porting linux engineering tools. Major pita. I see a bunch of #ifdef __APPLE__ lines to pull in alternate headers; what's the equiv for FreeBSD? Thanks, Steve From delphij at delphij.net Fri May 9 18:33:31 2008 From: delphij at delphij.net (Xin LI) Date: Fri May 9 18:33:35 2008 Subject: CLOCK_REALTIME undefined on my system In-Reply-To: <539c60b90805091117l57ed9966h99d6ae5f45925cdf@mail.gmail.com> References: <539c60b90805091117l57ed9966h99d6ae5f45925cdf@mail.gmail.com> Message-ID: <482498F2.8010201@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steve Franks wrote: | The manpage (http://www.freebsd.org/cgi/man.cgi?query=clock_gettime&sektion=2&apropos=0&manpath=FreeBSD+7.0-RELEASE) | for clock_gettime() specifies the correct header as , which I | am using, and I don't see any errors on clock_gettime(), but the param | I need, listed in the manpage, CLOCK_REALTIME is undefined. Any ideas | how this could be? I've recently cvsup'd standard-supfile, so you'd | think I'm up to date... | | I'm also getting an implicit declaration of isnormal(), and math.h is | clearly included... I think with #define _POSIX_C_SOURCE 199309L These will be masked... Could you please try if changing it to 200112 would work? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkgkmPIACgkQi+vbBBjt66D/bgCfd4jBteEBrPdQT272TcxY0bLF LwIAoIkcfwxlCBog7+1tyJr84Uns6jbJ =AS7o -----END PGP SIGNATURE----- From delphij at delphij.net Fri May 9 18:36:36 2008 From: delphij at delphij.net (Xin LI) Date: Fri May 9 18:36:38 2008 Subject: correct #define in source to specify FBSD vs. linux? In-Reply-To: <539c60b90805091103u2c8907a6q83cdca69fb45888e@mail.gmail.com> References: <539c60b90805091103u2c8907a6q83cdca69fb45888e@mail.gmail.com> Message-ID: <482499AB.5000609@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steve Franks wrote: | Seems there is a more appropriate list for my earlier question to | freebsd-questions: | | On and on I charge porting linux engineering tools. Major pita. I | see a bunch of #ifdef __APPLE__ lines to pull in alternate headers; | what's the equiv for FreeBSD? #ifdef __FreeBSD__? - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkgkmaoACgkQi+vbBBjt66C30ACeIbw6P7CuwErAIvzcUjX4d4Gk 7G0An3+S2hM9YctAkKsWDVzkEoZIMhOH =AzxS -----END PGP SIGNATURE----- From julian at elischer.org Fri May 9 18:44:04 2008 From: julian at elischer.org (Julian Elischer) Date: Fri May 9 18:44:08 2008 Subject: correct #define in source to specify FBSD vs. linux? In-Reply-To: <539c60b90805091103u2c8907a6q83cdca69fb45888e@mail.gmail.com> References: <539c60b90805091103u2c8907a6q83cdca69fb45888e@mail.gmail.com> Message-ID: <48249B71.3050804@elischer.org> Steve Franks wrote: > Seems there is a more appropriate list for my earlier question to > freebsd-questions: > > On and on I charge porting linux engineering tools. Major pita. I > see a bunch of #ifdef __APPLE__ lines to pull in alternate headers; > what's the equiv for FreeBSD? __FreeBSD__ but it is more fine grained than that.. http://www.freebsd.org/doc/en/books/porters-handbook/porting-versions.html (and the next page) maybe you should read the entire porter's handbook :-) > > Thanks, > Steve > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From rwatson at FreeBSD.org Sat May 10 13:08:36 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Sat May 10 13:08:42 2008 Subject: binary compatibility query In-Reply-To: <481F6990.9010007@elischer.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> <481F6990.9010007@elischer.org> Message-ID: <20080510134833.J63808@fledge.watson.org> On Mon, 5 May 2008, Julian Elischer wrote: > basically if you rely only on the standard posix interfaces and don't do > anything exotic then you will "probably" be safe. > > the really safe way of course it to make a 6.0 chroot on your machine and > compile your app there. For "raw" UNIX applications, this rule of thumb works well, but not for applications that depend on third-party libraries, languages, or daemons. For example, Java binaries built against 6.0 using packages shipped with 6.0 can't run on 6.1 due to incompatible changes in third-party libraries it depends on. While we try to be pretty careful with the base system, we have no control over third party applications, and as far as I know, we perform no testing (nor even have policies) for addressing that sort of incompatibility. The safety of depending on those third-party libraries pretty much corresponds to the carefulness of the thirdy-party library authors and their attention to those same sorts of details. Robert N M Watson Computer Laboratory University of Cambridge From gabor at FreeBSD.org Sat May 10 18:17:38 2008 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Sat May 10 18:17:42 2008 Subject: My SoC2008 project: Porting BSD-licensed Text-Processing Tools from OpenBSD Message-ID: <4825E2E2.2030605@FreeBSD.org> Hello, I've been actively working on FreeBSD as a ports and doc committer since December 2006 and has participated 2 times in the Summer of Code program, but my current project is from a different area thus I would like to summarize my goals and introduce myself for those who don't know me yet. I'm a Hungarian student of 20 years studying Computer Engineering at the Technical University of Budapest. So far I've been involved with the Ports Collection and with my two last SoC project I've written some enhancements for the Ports Collection infrastructure. I've been also working on the documentation set. I've started the Hungarian Documentation Project in which I've translated the webpages and some articles and have coordinated the volunteer efforts of other translators. One of them, pgj@ has translated the Handbook and he is my mentee now. We are reviewing the translation at the moment so we will commit it in the near future. With my current project I want to get some experience in new areas, I intend to deepen my knowledges of C programming and study some POSIX API. My project aims to port the text-processing tools from OpenBSD: grep, sort, diff. You can read my proposal here: http://kovesdan.org/doc/en_US.ISO8859-1/soc2008/soc2008.pdf The current progress can be seen on my wiki page: http://wiki.freebsd.org/G%C3%A1borSoC2008 And the actual changes made by me can be tracked in my Perforce branch: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/soc2008/gabor%5ftextproc&HIDEDEL=NO Comments are welcome! Regards, -- Gabor Kovesdan EMAIL: gabor@FreeBSD.org WWW: http://www.kovesdan.org From mwm-keyword-freebsdhackers2.e313df at mired.org Sat May 10 19:26:53 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sat May 10 19:26:56 2008 Subject: binary compatibility query In-Reply-To: <20080510134833.J63808@fledge.watson.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> <481F6990.9010007@elischer.org> <20080510134833.J63808@fledge.watson.org> Message-ID: <20080510152718.788d638f@bhuda.mired.org> On Sat, 10 May 2008 13:50:48 +0100 (BST) Robert Watson wrote: > On Mon, 5 May 2008, Julian Elischer wrote: > > basically if you rely only on the standard posix interfaces and don't do > > anything exotic then you will "probably" be safe. > For "raw" UNIX applications, this rule of thumb works well, but not for > applications that depend on third-party libraries, languages, or daemons. You can lose the dependency on third party libraries by compiling your application to a static binary. That will also help with the few breakages that occur with the system libraries, and with compatibility across major releases using the backwards compatX packages. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From rwatson at FreeBSD.org Sat May 10 20:35:31 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Sat May 10 20:35:36 2008 Subject: binary compatibility query In-Reply-To: <20080510152718.788d638f@bhuda.mired.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> <481F6990.9010007@elischer.org> <20080510134833.J63808@fledge.watson.org> <20080510152718.788d638f@bhuda.mired.org> Message-ID: <20080510213323.E35578@fledge.watson.org> On Sat, 10 May 2008, Mike Meyer wrote: > On Sat, 10 May 2008 13:50:48 +0100 (BST) > Robert Watson wrote: >> On Mon, 5 May 2008, Julian Elischer wrote: >>> basically if you rely only on the standard posix interfaces and don't do >>> anything exotic then you will "probably" be safe. >> For "raw" UNIX applications, this rule of thumb works well, but not for >> applications that depend on third-party libraries, languages, or daemons. > > You can lose the dependency on third party libraries by compiling your > application to a static binary. That will also help with the few breakages > that occur with the system libraries, and with compatibility across major > releases using the backwards compatX packages. However, if you do that, we'll probably shoot you in the foot by removing kernel support for the thread library you are linking against. Since the removal of KSE, it appears our ABI compatibility promise is at the libc or libpthread layer, and not at the system call layer. We haven't done the necessary compat work to let 6.x binaries using libkse work on an 8.x system yet, but will presumably do that with a backport of libthr to a 6.x userspace (i.e., 6.x library version). On almost all (serious) OS's I'm aware of, static linking for applications is essentially forbidden for that reason -- vendors promise library compatibility, not system call compatibility, and rely on being able to change the implementations of system calls. Robert N M Watson Computer Laboratory University of Cambridge From deischen at freebsd.org Sun May 11 01:18:03 2008 From: deischen at freebsd.org (Daniel Eischen) Date: Sun May 11 01:18:07 2008 Subject: binary compatibility query In-Reply-To: <20080510213323.E35578@fledge.watson.org> References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com> <481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org> <481F4EED.2030300@FreeBSD.org> <4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com> <481F6990.9010007@elischer.org> <20080510134833.J63808@fledge.watson.org> <20080510152718.788d638f@bhuda.mired.org> <20080510213323.E35578@fledge.watson.org> Message-ID: On Sat, 10 May 2008, Robert Watson wrote: > On Sat, 10 May 2008, Mike Meyer wrote: > >> On Sat, 10 May 2008 13:50:48 +0100 (BST) >> Robert Watson wrote: >>> On Mon, 5 May 2008, Julian Elischer wrote: >>>> basically if you rely only on the standard posix interfaces and don't do >>>> anything exotic then you will "probably" be safe. >>> For "raw" UNIX applications, this rule of thumb works well, but not for >>> applications that depend on third-party libraries, languages, or daemons. >> >> You can lose the dependency on third party libraries by compiling your >> application to a static binary. That will also help with the few breakages >> that occur with the system libraries, and with compatibility across major >> releases using the backwards compatX packages. > > However, if you do that, we'll probably shoot you in the foot by removing > kernel support for the thread library you are linking against. Since the > removal of KSE, it appears our ABI compatibility promise is at the libc or > libpthread layer, and not at the system call layer. We haven't done the > necessary compat work to let 6.x binaries using libkse work on an 8.x system > yet, but will presumably do that with a backport of libthr to a 6.x userspace > (i.e., 6.x library version). > > On almost all (serious) OS's I'm aware of, static linking for applications is > essentially forbidden for that reason -- vendors promise library > compatibility, not system call compatibility, and rely on being able to > change the implementations of system calls. I've brought this up before - we should seriously look at removing our static libraries, or at least make the statement that static binaries are not guaranteed to be ABI compatible. -- DE From yanefbsd at gmail.com Sun May 11 13:37:28 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Sun May 11 13:37:33 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <20080509124308.GA596@britannica.bec.de> References: <20080509124308.GA596@britannica.bec.de> Message-ID: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> On May 9, 2008, at 5:43 AM, Joerg Sonnenberger wrote: > On Fri, May 09, 2008 at 01:52:46PM +0200, Anders Nore wrote: >> I'm working on adding .db support to the pkg_tools( i.e. pkg_add, >> pkg_info, >> etc. ) as part of SoC 2008. The database api used is BerkeleyDB >> that comes >> with the base system (/usr/src/include/db.h). BerkeleyDB is not >> you're >> typical relational db, and can only save key/value pairs. The way I'm >> thinking of storing information to the .db is to name the keys as the >> directory names in /var/db/pkg. And save the +* files in the >> directories to >> the value element in the db, separated with a special character or >> similar. > > As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I would > *strongly* advisy you against storing the files only in a bdb file. > The change of major and complete corruption with bdb185 is high, > consider pulling the plug in the middle of a long update. +1. BDB is quite easy to corrupt... -Garrett From yanefbsd at gmail.com Sun May 11 13:40:56 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Sun May 11 13:41:02 2008 Subject: correct #define in source to specify FBSD vs. linux? In-Reply-To: <482499AB.5000609@delphij.net> References: <539c60b90805091103u2c8907a6q83cdca69fb45888e@mail.gmail.com> <482499AB.5000609@delphij.net> Message-ID: On May 9, 2008, at 11:36 AM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Steve Franks wrote: > | Seems there is a more appropriate list for my earlier question to > | freebsd-questions: > | > | On and on I charge porting linux engineering tools. Major pita. I > | see a bunch of #ifdef __APPLE__ lines to pull in alternate headers; > | what's the equiv for FreeBSD? > > #ifdef __FreeBSD__? > > - -- > Xin LI http://www.delphij.net/ > FreeBSD - The Power to Serve! > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.9 (FreeBSD) > > iEYEARECAAYFAkgkmaoACgkQi+vbBBjt66C30ACeIbw6P7CuwErAIvzcUjX4d4Gk > 7G0An3+S2hM9YctAkKsWDVzkEoZIMhOH > =AzxS > -----END PGP SIGNATURE----- #ifdef __linux__ for Linux as well. See: http://lists.debian.org/debian-devel/2001/01/msg00971.html (same applies for all platforms I would assume).. -Garrett From bart at it-ss.be Sun May 11 18:49:23 2008 From: bart at it-ss.be (Bart Van Kerckhove) Date: Sun May 11 18:49:27 2008 Subject: Xenu PV References: <4dcb5abd0805050540m292b319aw52aa2cb8ba018e12@mail.gmail.com><481F0DB3.9070505@FreeBSD.org> <481F48EE.3050806@elischer.org><481F4EED.2030300@FreeBSD.org><4dcb5abd0805051132o77d68e36u3f0ad38630a02afd@mail.gmail.com><481F6990.9010007@elischer.org><20080510134833.J63808@fledge.watson.org><20080510152718.788d638f@bhuda.mired.org><20080510213323.E35578@fledge.watson.org> Message-ID: <007001c8b386$528dbf70$020b000a@bartwrkstxp> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear list, I'm interested in checking out the work being done on XenU PV support in the freebsd kernel. However, probably because of some stupid oversight on my part, I can't seem to check out the source from eiter cvsup10 or 18 (as stated on http://wiki.freebsd.org/FreeBSD/Xen ). I've been asking around and sending mails for read access on p4, but to no avail. Could you guys give me a nudge in the right direction? How do I get to the sources that are being worked on? Cheers! With kind regards, Bart Van Kerckhove bart@it-ss.be http://friet.net/pgp.txt -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (MingW32) - WinPT 1.0.1 iD8DBQFIJyJ1CggVyEEzQEoRAtcHAJ9A/wIMtPz9wutZfKS1gyFZSEFPjQCeLjjS w8BgGgHRk607aNK+xqd/VAk= =Kbwb -----END PGP SIGNATURE----- From jos at catnook.com Sun May 11 19:34:02 2008 From: jos at catnook.com (Jos Backus) Date: Sun May 11 19:34:06 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> Message-ID: <20080511190741.GB47383@lizzy.catnook.local> On Sun, May 11, 2008 at 06:38:25AM -0700, Garrett Cooper wrote: > +1. BDB is quite easy to corrupt... If we're going to use a binary file format, please consider using SQLite instead. It has the right license, a nice API (transactions!) and is robust enough for yum to use it for a similar purpose. -- Jos Backus jos at catnook.com From avg at icyb.net.ua Sun May 11 20:50:34 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Sun May 11 20:50:37 2008 Subject: devctl (alike?) for devfs In-Reply-To: <20080425143646.GF18958@deviant.kiev.zoral.com.ua> References: <480E4269.2090604@icyb.net.ua> <480FBAB9.1000904@icyb.net.ua> <48103F36.6060707@icyb.net.ua> <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> Message-ID: <48275C0C.2040601@icyb.net.ua> Kostik, John, Warner, thank you for your guidance and suggestions. I am currently testing the patch attached and I am using a kernel with WITNESS and INVARIANTS enabled. Scope of my testing is plugging/unplugging of UMASS devices. I get CREATE notifications all right. I do not get any panics/complaints from the kernel, good. Unfortunately I do not get any DESTROY notifications either. Could you please look through the patch? Is there any control flow path that I missed or something even more obvious? I hope that we do not have any cdev leaks. I am testing the patch with RELENG_7 as of May 6. -- Andriy Gapon -------------- next part -------------- diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c index 1db25f8..0245253 100644 --- a/sys/kern/kern_conf.c +++ b/sys/kern/kern_conf.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -99,6 +100,9 @@ dev_unlock_and_free(void) mtx_unlock(&devmtx); while ((cdp = TAILQ_FIRST(&cdp_free)) != NULL) { + if (!cold) + devctl_notify("DEVFS", cdp->cdp_c.si_name, "DESTROY", NULL); + TAILQ_REMOVE(&cdp_free, cdp, cdp_list); devfs_free(&cdp->cdp_c); } @@ -172,8 +176,12 @@ dev_rel(struct cdev *dev) flag = 1; } dev_unlock(); - if (flag) + if (flag) { + if (!cold) + devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); + devfs_free(dev); + } } struct cdevsw * @@ -706,6 +714,10 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, devfs_create(dev); clean_unrhdrl(devfs_inos); dev_unlock_and_free(); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } @@ -794,6 +806,10 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) clean_unrhdrl(devfs_inos); dev_unlock(); dev_depends(pdev, dev); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } From kostikbel at gmail.com Sun May 11 21:48:44 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sun May 11 21:48:49 2008 Subject: devctl (alike?) for devfs In-Reply-To: <48275C0C.2040601@icyb.net.ua> References: <480E4269.2090604@icyb.net.ua> <480FBAB9.1000904@icyb.net.ua> <48103F36.6060707@icyb.net.ua> <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> Message-ID: <20080511214833.GB18958@deviant.kiev.zoral.com.ua> On Sun, May 11, 2008 at 11:50:20PM +0300, Andriy Gapon wrote: > > Kostik, John, Warner, > > thank you for your guidance and suggestions. > I am currently testing the patch attached and I am using a kernel with > WITNESS and INVARIANTS enabled. > Scope of my testing is plugging/unplugging of UMASS devices. > I get CREATE notifications all right. > I do not get any panics/complaints from the kernel, good. > Unfortunately I do not get any DESTROY notifications either. > > Could you please look through the patch? Is there any control flow path > that I missed or something even more obvious? > I hope that we do not have any cdev leaks. > > I am testing the patch with RELENG_7 as of May 6. No, we do not have a leak, but we have somewhat non-obvious behaviour. The cdev structure is freed only after the last reference to cdev is gone. Typical holder of the reference is the devfs vnode. In the normal usage, the vnode is present until both the device is destroyed _and_ devfs_populate_loop() run is performed. This function actually reclaim the vnodes for destroyed devices, that causes last reference to cdev to be dropped and memory freed. The populate loop is called syncronously from the upper levels. The easiest method to trigger it is to do ls /dev, since it is called from the devfs_lookupx(). > > -- > Andriy Gapon > diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c > index 1db25f8..0245253 100644 > --- a/sys/kern/kern_conf.c > +++ b/sys/kern/kern_conf.c > @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); > #include > #include > #include > +#include > #include > #include > #include > @@ -99,6 +100,9 @@ dev_unlock_and_free(void) > mtx_unlock(&devmtx); > > while ((cdp = TAILQ_FIRST(&cdp_free)) != NULL) { > + if (!cold) > + devctl_notify("DEVFS", cdp->cdp_c.si_name, "DESTROY", NULL); > + > TAILQ_REMOVE(&cdp_free, cdp, cdp_list); > devfs_free(&cdp->cdp_c); > } > @@ -172,8 +176,12 @@ dev_rel(struct cdev *dev) > flag = 1; > } > dev_unlock(); > - if (flag) > + if (flag) { > + if (!cold) > + devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); > + > devfs_free(dev); > + } > } > > struct cdevsw * > @@ -706,6 +714,10 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, > devfs_create(dev); > clean_unrhdrl(devfs_inos); > dev_unlock_and_free(); > + > + if (!cold) > + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); > + > return (dev); > } > > @@ -794,6 +806,10 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) > clean_unrhdrl(devfs_inos); > dev_unlock(); > dev_depends(pdev, dev); > + > + if (!cold) > + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); > + > return (dev); > } > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080511/a5621dbd/attachment.pgp From bruce at cran.org.uk Mon May 12 01:45:39 2008 From: bruce at cran.org.uk (Bruce Cran) Date: Mon May 12 01:45:42 2008 Subject: sshd patch to avoid DNS lookups when using 'UseDNS no' or -u0 In-Reply-To: <20080509173626.GC35511@keltia.freenix.fr> References: <4818F72C.90300@cran.org.uk> <20080509173626.GC35511@keltia.freenix.fr> Message-ID: <4827A129.4000205@cran.org.uk> Ollivier Robert wrote: > According to Bruce Cran: >> I've attached a patch which implements this (the change to loginrec.c >> reverts it back to the default OpenSSH code) and was wondering if someone >> could take a look at it. > > If you have not already done so, please use send-pr to record it in GNATS, > that will help not forgetting about it, thanks! The patch has been attached to PR bin/97499 - http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/97499 -- Bruce From elijah.buck at gmail.com Mon May 12 03:09:38 2008 From: elijah.buck at gmail.com (Elijah Buck) Date: Mon May 12 03:09:42 2008 Subject: 4bsd fuzzy runq Message-ID: <303eddcb0805111943t7c456c79l56f427c526d0b454@mail.gmail.com> Hi, I'm looking at the code for 4bsd fuzzy run queues in kern_switch.c The relevant bit: if (fuzz > 1) { int count = fuzz; int cpu = PCPU_GET(cpuid); struct thread *td2; td2 = td = TAILQ_FIRST(rqh); while (count-- && td2) { if (td->td_lastcpu == cpu) { td = td2; break; } td2 = TAILQ_NEXT(td2, td_runq); } ...return(td) The purpose of this code appears to be to look through the runq to a depth defined by fuzz for a thread that was last run on the current cpu. Here are the cases I see: 1.) if (td_lastcpu == cpu) on the first iteration then TAILQ_FIRST(rqh) is the selected thread (because of the break). 2.) if (td_lastcpu != cpu) on the first iteration then td is never set again, and so (td_lastcpu != cpu) is always true. So, again, TAILQ_FIRST(rqh) will be selected (because count will reach 0). Which doesn't seem right (since that's what the else clause does). What am I missing here? Thanks, Elijah From julian at elischer.org Mon May 12 06:10:52 2008 From: julian at elischer.org (Julian Elischer) Date: Mon May 12 06:10:57 2008 Subject: 4bsd fuzzy runq In-Reply-To: <303eddcb0805111943t7c456c79l56f427c526d0b454@mail.gmail.com> References: <303eddcb0805111943t7c456c79l56f427c526d0b454@mail.gmail.com> Message-ID: <4827DF6B.6050101@elischer.org> Elijah Buck wrote: > Hi, > > I'm looking at the code for 4bsd fuzzy run queues in kern_switch.c > > The relevant bit: > > if (fuzz > 1) { > int count = fuzz; > int cpu = PCPU_GET(cpuid); > struct thread *td2; > td2 = td = TAILQ_FIRST(rqh); > while (count-- && td2) { > if (td->td_lastcpu == cpu) { this should be td2. The bug is mine.. and I wondered why the code commited never had the same result as that I tested in another tree. > td = td2; > break; > } > td2 = TAILQ_NEXT(td2, td_runq); > } > ...return(td) > The purpose of this code appears to be to look through the runq to a depth > defined by fuzz for a thread that was last run on the current cpu. > > Here are the cases I see: > 1.) if (td_lastcpu == cpu) on the first iteration then TAILQ_FIRST(rqh) is > the selected thread (because of the break). > 2.) if (td_lastcpu != cpu) on the first iteration then td is never set > again, and so (td_lastcpu != cpu) is always true. So, again, > TAILQ_FIRST(rqh) will be selected (because count will reach 0). > > Which doesn't seem right (since that's what the else clause does). What am I > missing here? > > Thanks, > Elijah > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From ap00 at mail.ru Mon May 12 08:54:33 2008 From: ap00 at mail.ru (Anthony Pankov) Date: Mon May 12 08:54:35 2008 Subject: BDB corrupt In-Reply-To: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> Message-ID: <15336578.20080512123806@mail.ru> Please, can anybody explain what is the problem with BDB (1.86). Is there known caveats of using BDB? Is there some rules which guarantee from curruption or it is fully undesirable to use BDB under high load? It is important for me because of using BDB in my project. > On Fri, May 09, 2008 at 01:52:46PM +0200, Joerg Sonnenberger wrote: >> >> As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I would >> *strongly* advisy you against storing the files only in a bdb file. >> The change of major and complete corruption with bdb185 is high, >> consider pulling the plug in the middle of a long update. >Sunday, May 11, 2008, 5:38:25 PM, you wrote: GC> +1. BDB is quite easy to corrupt... -- Best regards, Anthony mailto:ap00@mail.ru From jonathan+freebsd-hackers at hst.org.za Mon May 12 10:19:22 2008 From: jonathan+freebsd-hackers at hst.org.za (Jonathan McKeown) Date: Mon May 12 10:19:26 2008 Subject: BDB corrupt In-Reply-To: <15336578.20080512123806@mail.ru> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> Message-ID: <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> On Monday 12 May 2008 10:38, Anthony Pankov wrote: > Please, can anybody explain what is the problem with BDB (1.86). > > Is there known caveats of using BDB? Is there some rules which > guarantee from curruption or it is fully undesirable to use BDB under > high load? > > It is important for me because of using BDB in my project. Interesting. I would have thought that the two processes "find out advantages and problems of proposed solutions" and "choose a solution" had a natural ordering other than the one you seem to be using. Jonathan From avg at icyb.net.ua Mon May 12 14:26:03 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon May 12 14:26:08 2008 Subject: openbsd solution to mounted umass removal Message-ID: <48285372.9060103@icyb.net.ua> I've just came back from a good 2 week vacation and catching up on news. In release notes for OpenBSD 4.3 I see the following: http://openbsd.org/43.html Filesystems on USB devices are automatically dismounted if the device is disconnected. Does anybody have more [technical] details on this? Thanks. -- Andriy Gapon From avg at icyb.net.ua Mon May 12 14:34:38 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon May 12 14:34:43 2008 Subject: i386 cpu_reset_real: code/comment mismatch Message-ID: <4828557B.9000506@icyb.net.ua> This is not a real issue, just a code clarification. First a snippet from sys/i386/i386/vm_machdep.c, cpu_reset_real() /* * Attempt to force a reset via the Reset Control register at * I/O port 0xcf9. Bit 2 forces a system reset when it is * written as 1. Bit 1 selects the type of reset to attempt: * 0 selects a "soft" reset, and 1 selects a "hard" reset. We * try to do a "soft" reset first, and then a "hard" reset. */ outb(0xcf9, 0x2); outb(0xcf9, 0x6); I think that the comment is correct up to but not including the last sentence. Writing 0x2 sets bit 1 to 1 (thus selecting hard reset), and writing 0x6 sets both bits 2 and 1 to 1 (thus performing hard reset). So we always just do a hard reset, no trying of soft reset (would it even make sense to do the last line of the comment says). -- Andriy Gapon From andenore at start.no Mon May 12 15:14:13 2008 From: andenore at start.no (Anders Nore) Date: Mon May 12 15:14:17 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <20080511190741.GB47383@lizzy.catnook.local> References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <20080511190741.GB47383@lizzy.catnook.local> Message-ID: On Sun, 11 May 2008 21:07:41 +0200, Jos Backus wrote: > On Sun, May 11, 2008 at 06:38:25AM -0700, Garrett Cooper wrote: >> +1. BDB is quite easy to corrupt... > > If we're going to use a binary file format, please consider using SQLite > instead. It has the right license, a nice API (transactions!) and is > robust > enough for yum to use it for a similar purpose. > One of the reasons for using BDB is that it is in the base system, SQLite however is not. From koitsu at freebsd.org Mon May 12 15:47:53 2008 From: koitsu at freebsd.org (Jeremy Chadwick) Date: Mon May 12 15:47:58 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <20080511190741.GB47383@lizzy.catnook.local> Message-ID: <20080512154753.GA16847@eos.sc1.parodius.com> On Mon, May 12, 2008 at 05:12:56PM +0200, Anders Nore wrote: > On Sun, 11 May 2008 21:07:41 +0200, Jos Backus wrote: > >> On Sun, May 11, 2008 at 06:38:25AM -0700, Garrett Cooper wrote: >>> +1. BDB is quite easy to corrupt... >> >> If we're going to use a binary file format, please consider using SQLite >> instead. It has the right license, a nice API (transactions!) and is >> robust >> enough for yum to use it for a similar purpose. >> > > One of the reasons for using BDB is that it is in the base system, SQLite > however is not. Interesting discussion, and there's some irony in it (SQLite uses a very BDB-like a storage medium, despite being a relational database system). SQLite's license is compatible/friendly, its API is decent, the source is fairly small and binary footprint (disk and memory both) are small, and it includes tools for dumping databases to ASCII so developers aren't forced to write DB dumping tools and users don't have to resort to using hexdump. A few things worth pointing out: With regards to DB1.x being easily corrupted, this is true, and I have heard claims of it from a couple developer-folk I worked with many years ago at Best Internet. On the other hand, SQLite appears to have a history of this as well, although those bugs have been fixed each and every time: http://www.sqlite.org/cvstrac/wiki?p=DatabaseCorruption Secondly, the following FAQ entry and documentation from Mozilla is of concern, specifically the last paragraph of the FAQ entry, since there is ongoing work in the ports collection to support parallel building, which would surely use fork() or locks): http://www.sqlite.org/faq.html#q6 http://developer.mozilla.org/en/docs/Storage#SQLite_Locking Thirdly, there is the concern of disk space. There are a substantial number of users who have FreeBSD on embedded systems or systems with little or no space, who use binary packages (pkg_add -r), which of course still makes use of /var/db/pkg. SQLite DBs grow over time, and don't shrink in size unless you use auto-vacuum mode or execute the VACUUM query: http://www.sqlite.org/faq.html#q12 Fourthly, it doesn't sound as if storage SQLite databases on an NFS mount is a good idea, and I've a feeling some people have /var NFS mounted (diskless systems for example): http://www.sqlite.org/faq.html#q5 Finally, there's the concern about the SQLite file format changing. Sometimes they're backwards-compatible, other times not. http://www.sqlite.org/formatchng.html http://www.sqlite.org/fileformat.html Just stuff to consider. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From joerg at britannica.bec.de Mon May 12 16:03:32 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Mon May 12 16:03:38 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <20080512154753.GA16847@eos.sc1.parodius.com> References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <20080511190741.GB47383@lizzy.catnook.local> <20080512154753.GA16847@eos.sc1.parodius.com> Message-ID: <20080512160240.GA19040@britannica.bec.de> On Mon, May 12, 2008 at 08:47:53AM -0700, Jeremy Chadwick wrote: > Secondly, the following FAQ entry and documentation from Mozilla is of > concern, specifically the last paragraph of the FAQ entry, since there > is ongoing work in the ports collection to support parallel building, > which would surely use fork() or locks): I don't think this is applicable. > Thirdly, there is the concern of disk space. There are a substantial > number of users who have FreeBSD on embedded systems or systems with > little or no space, who use binary packages (pkg_add -r), which of > course still makes use of /var/db/pkg. SQLite DBs grow over time, and > don't shrink in size unless you use auto-vacuum mode or execute the > VACUUM query: It is trivial to recreate the database, so I don't think this is a concern either. I should point again that it will result in a non-trivial increase in size. > Fourthly, it doesn't sound as if storage SQLite databases on an NFS > mount is a good idea, and I've a feeling some people have /var NFS > mounted (diskless systems for example): Same applies to BDB and basically anything that depends on fcntl to work. Basically, as long as you have /var per host (you don't share /var, do you?), it will just work as the *local* locks are working. The problem with NFS comes form the fact that multiple hosts might not share them. Joerg From ale at FreeBSD.org Mon May 12 16:32:03 2008 From: ale at FreeBSD.org (Alex Dupre) Date: Mon May 12 16:32:05 2008 Subject: openbsd solution to mounted umass removal In-Reply-To: <48285372.9060103@icyb.net.ua> References: <48285372.9060103@icyb.net.ua> Message-ID: <482870FC.5050806@FreeBSD.org> Andriy Gapon wrote: > Filesystems on USB devices are automatically dismounted if the device is > disconnected. > > Does anybody have more [technical] details on this? Modified files: sys/dev/usb : umass.c sys/scsi : sdvar.h sd.c sys/kern : vfs_default.c vfs_subr.c Log message: Allow to pull out an usb stick with ffs filesystem while mounted and a file is written onto the stick. Without these fixes the machine panics or hangs. The usb fix calls the callback when the stick is pulled out to free the associated buffers. Otherwise we have busy buffers for ever and the automatic unmount will panic. The change in the scsi layer prevents passing down further dirty buffers to usb after the stick has been deactivated. In vfs the automatic unmount has moved from the function vgonel() to vop_generic_revoke(). Both are called when the sd device's vnode is removed. In vgonel() the VXLOCK is already held which can cause a deadlock. So call dounmount() earlier. -- Alex Dupre From jos at catnook.com Mon May 12 16:58:32 2008 From: jos at catnook.com (Jos Backus) Date: Mon May 12 16:58:52 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <20080511190741.GB47383@lizzy.catnook.local> Message-ID: <20080512165853.GA11324@lizzy.catnook.local> On Mon, May 12, 2008 at 05:12:56PM +0200, Anders Nore wrote: > One of the reasons for using BDB is that it is in the base system, SQLite > however is not. I'm aware of that. But I believe that the pain and suffering of importing and maintaining SQLite in the base (that is, the cost) is outweighed by its richer feature set and higher-level API, facilitating easier tools development. -- Jos Backus jos at catnook.com From fbsd06 at mlists.homeunix.com Mon May 12 17:39:17 2008 From: fbsd06 at mlists.homeunix.com (RW) Date: Mon May 12 17:39:23 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <4828557B.9000506@icyb.net.ua> References: <4828557B.9000506@icyb.net.ua> Message-ID: <20080512182328.09a8a173@gumby.homeunix.com.> On Mon, 12 May 2008 17:34:35 +0300 Andriy Gapon wrote: > > This is not a real issue, just a code clarification. > > First a snippet from sys/i386/i386/vm_machdep.c, cpu_reset_real() > /* > * Attempt to force a reset via the Reset Control register at > * I/O port 0xcf9. Bit 2 forces a system reset when it is > * written as 1. Bit 1 selects the type of reset to attempt: > * 0 selects a "soft" reset, and 1 selects a "hard" reset. We > * try to do a "soft" reset first, and then a "hard" reset. > */ > outb(0xcf9, 0x2); > outb(0xcf9, 0x6); > > I think that the comment is correct up to but not including the last > sentence. Writing 0x2 sets bit 1 to 1 (thus selecting hard reset), and > writing 0x6 sets both bits 2 and 1 to 1 (thus performing hard reset). > So we always just do a hard reset, no trying of soft reset (would it > even make sense to do the last line of the comment says). > It looks to me as if the comment was added retrospectively by someone who got the two bits mixed-up when reading the source. If bits 1 and 2 were the other way around, it would be code for a soft-reset followed by a hard-reset. From ap00 at mail.ru Mon May 12 18:33:09 2008 From: ap00 at mail.ru (Anthony Pankov) Date: Mon May 12 18:33:13 2008 Subject: BDB corrupt In-Reply-To: <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> Message-ID: <1663320218.20080512223531@mail.ru> So, can anyone make clear about BDB 1.86 (which is a part of base system). When 1. there is no need for SQL 2. processes are sharing db file in concurrent mode (key=>value pair) 3. reading/writing = 60%/40% the first idea is to use BDB. Because BDB: 1. do not need additional installation 2. is part of base system which mean it is mature, reliable and stable (otherwise why BDB is still a part of FreeBSD?) Discussion "Adding .db support to pkg_tools" reveal BDB ability to corrupt data. Can anyone suggest BDB alternative (not GPLed)? Monday, May 12, 2008, 1:53:00 PM, you wrote: JM> On Monday 12 May 2008 10:38, Anthony Pankov wrote: >> Please, can anybody explain what is the problem with BDB (1.86). >> >> Is there known caveats of using BDB? Is there some rules which >> guarantee from curruption or it is fully undesirable to use BDB under >> high load? >> >> It is important for me because of using BDB in my project. JM> Interesting. I would have thought that the two processes "find out advantages JM> and problems of proposed solutions" and "choose a solution" had a natural JM> ordering other than the one you seem to be using. JM> Jonathan -- Best regards, Anthony mailto:ap00@mail.ru From mwm at mired.org Mon May 12 19:24:35 2008 From: mwm at mired.org (Mike Meyer) Date: Mon May 12 20:09:35 2008 Subject: BDB corrupt In-Reply-To: <1663320218.20080512223531@mail.ru> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> Message-ID: <20080512152430.3720683e@mbook.local> On Mon, 12 May 2008 22:35:31 +0400 Anthony Pankov wrote: > Because BDB: > 1. do not need additional installation > 2. is part of base system which mean it is mature, reliable and stable BDB in the base system is mature, reliable and stable *for what it's used for in the base system.* So long as your requirements are covered by that usage, you'll be ok. The uses I know of for BDB in the base system all consist of databases of relatively small items that are changed infrequently, and usually with a locking mechanism. From what you've said, this doesn't describe your requirements. More importantly, from what other people are saying, your requirements are ones for which it's known that BDB is *not* reliable, or otherwise unsuitable. In particular, an effort is underway to allow parallel ports builds, which implies concurrent access to the database, which is a known source of problems for BDB. Skipped content of type multipart/mixed-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080512/492cf94e/attachment.pgp From reg at FreeBSD.ORG Mon May 12 21:05:32 2008 From: reg at FreeBSD.ORG (Jeremy Lea) Date: Mon May 12 21:05:37 2008 Subject: Adding .db support to pkg_tools In-Reply-To: References: Message-ID: <20080512204558.GA44114@flint.openpave.org> Hi, On Fri, May 09, 2008 at 01:52:46PM +0200, Anders Nore wrote: > I'm working on adding .db support to the pkg_tools I hope that you know that you're stepping into a hotly debated part of the project... Good luck. My advice to you is to ignore any calls for a 'complete rewrite' and to focus on releasing small patches from your project rather than one big patch at the end. Back each patch up with performance numbers (I suggest pkg_add gnome2-x (from a local package source), pkg_version and pkg_delete -r glib-2.x). > The way I'm thinking of storing information to the .db is to name the > keys as the directory names in /var/db/pkg. And save the +* files in > the directories to the value element in the db, separated with a > special character or similar. I think this is a really bad idea. The problem with the tools is not with the files. It is that the files need to be parsed on each run, often recursively, and your solution would not help at all. The database(s) should just be a cache of the information stored in the files. This is what the existing databases used by portupgrade and NetBSD's tools do. They store the files and which ports installed them, so that it is easy to do conflict resolution, and some pkg_info operations. In addition, what you need is a cache which stores {pkg_name,pkg_origin} pairs, where both are unique keys with a 1:1 relationship, and then a {pkg_name,pkg_dep_name} directed graph, which stores the dependency network between ports (aslo replacing +REQUIRED_BY). Both can be reconstructed if needed from the +CONTENTS files. You could do both in BDB, but you'll be better off loading both into memory completely (as needed), using C to do the queries, and just maintaining the database when things change (on pkg_add or pkg_delete). SQLite would make this job easier in some ways, but hurt in others. You could use queries, which would make what was going on much more obvious, but then it would be slower. You also have to fight the anti-SQLite lobby. Ignore people who talk about disk space on embedded systems. They're just trolling. ;-) If they don't have the disk space for /var/db/pkg, then they don't have the space for the ports themselves. Besides, removing the redundant copy of +MTREE_DIRS for each package would save more space (and is as easy as checking the CVS revision of the new file in each package and updating a single master copy (even /etc/mtree/BSD.local.dist), if it is newer). With regards to locking, which someone raised, the package tools are a long way away from parallel modifications to /var/db/pkg. People are talking about parallel ports builds, but those have small windows where the package database is being updated, and those updates need to be sync'ed via a lock. Regards, -Jeremy -- FreeBSD - Because the best things in life are free... http://www.freebsd.org/ From yanefbsd at gmail.com Mon May 12 22:45:02 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Mon May 12 22:45:06 2008 Subject: BDB corrupt In-Reply-To: <15336578.20080512123806@mail.ru> References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> Message-ID: On May 12, 2008, at 1:38 AM, Anthony Pankov wrote: > > Please, can anybody explain what is the problem with BDB (1.86). > > Is there known caveats of using BDB? Is there some rules which > guarantee from curruption or it is fully undesirable to use BDB under > high load? > > It is important for me because of using BDB in my project. > > >> On Fri, May 09, 2008 at 01:52:46PM +0200, Joerg Sonnenberger wrote: >>> >>> As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I >>> would >>> *strongly* advisy you against storing the files only in a bdb file. >>> The change of major and complete corruption with bdb185 is high, >>> consider pulling the plug in the middle of a long update. > >> Sunday, May 11, 2008, 5:38:25 PM, you wrote: > > GC> +1. BDB is quite easy to corrupt... BDB isn't ATOMic, like SQL or other DB backends. -Garrett From lidl at pix.net Tue May 13 02:18:08 2008 From: lidl at pix.net (Kurt Lidl) Date: Tue May 13 02:18:11 2008 Subject: BDB corrupt In-Reply-To: References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> Message-ID: <4828F55E.6020407@pix.net> Garrett Cooper wrote: > On May 12, 2008, at 1:38 AM, Anthony Pankov wrote: > >> >> Please, can anybody explain what is the problem with BDB (1.86). >> >> Is there known caveats of using BDB? Is there some rules which >> guarantee from curruption or it is fully undesirable to use BDB under >> high load? >> >> It is important for me because of using BDB in my project. >> >> >>> On Fri, May 09, 2008 at 01:52:46PM +0200, Joerg Sonnenberger wrote: >>>> >>>> As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I would >>>> *strongly* advisy you against storing the files only in a bdb file. >>>> The change of major and complete corruption with bdb185 is high, >>>> consider pulling the plug in the middle of a long update. >> >>> Sunday, May 11, 2008, 5:38:25 PM, you wrote: >> >> GC> +1. BDB is quite easy to corrupt... > > BDB isn't ATOMic, like SQL or other DB backends. You mean ACID probably. And there are plenty of SQL databases that aren't ACID either. (e.g. Mysql 4.x, Mysql 5.x w/o the right kind of backing store) -Kurt From zbeeble at gmail.com Tue May 13 02:55:08 2008 From: zbeeble at gmail.com (Zaphod Beeblebrox) Date: Tue May 13 02:55:11 2008 Subject: ZFS and mmap() datapoint. Message-ID: <5f67a8c40805121955t3269b1d5we302dda607e21573@mail.gmail.com> I don't know exactly where this fits in the discussion, but I was using "ghost" v 14 to backup my XP box over SMB to a 7.0-RELEASE system using ZFS. After doing one backup, the second backup wouldn't proceed, so I erased the backup files and started fresh. The next attempt, I turned on the verify option in ghost. It claimed that the backup it was creating was corrupt. So I upgraded the FreeBSD machine to 7-STABLE. Started the backup again, same problem. So I checked into samba's options ... and it has an option that claims to turn off the mmap() usage of the daemon. It certainly slows the daemon, so it's doing something. Anyways... the backup was still corrupt. Obviously... this is testing at a high layer. Does the option in smb.conf turn off the use of mmap(). Is that moot because C libraries might use mmap() anyways? Is ghost the culprit here? I dunno. The ZFS array does handle linear reads and writes of large media files without detectable error. From kientzle at freebsd.org Tue May 13 04:20:42 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Tue May 13 04:20:47 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <20080512204558.GA44114@flint.openpave.org> References: <20080512204558.GA44114@flint.openpave.org> Message-ID: <48291719.5010801@freebsd.org> > I think this is a really bad idea. The problem with the tools is not > with the files. It is that the files need to be parsed on each run, > often recursively, and your solution would not help at all. Parsing one file isn't expensive; parsing several hundred files to find one bit of information is expensive. > The database(s) should just be a cache of the information stored in the > files. Bingo! As long as the .db version can be easily recreated from scratch from the master data stored in the same files as always, it doesn't really matter if the BDB is occasionally corrupted, as long as it can be rebuilt fairly quickly. So, the key points are: * Use the .db file for lookups that can't be done quickly against the existing data. In particular, look for operations where the pkg tools have to scan a lot of files to verify a single fact. * Ensure the .db file can be deleted and rebuilt from scratch from the data in the regular files. Tim Kientzle From ap00 at mail.ru Tue May 13 11:41:42 2008 From: ap00 at mail.ru (Anthony Pankov) Date: Tue May 13 11:41:46 2008 Subject: BDB corrupt In-Reply-To: <20080512152430.3720683e@mbook.local> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> Message-ID: <2117635718.20080513154406@mail.ru> Monday, Mike Meyer May 12, 2008, 11:24:30 PM, you wrote: MM> On Mon, 12 May 2008 22:35:31 +0400 Anthony Pankov wrote: >> Because BDB: >> 1. do not need additional installation >> 2. is part of base system which mean it is mature, reliable and stable MM> BDB in the base system is mature, reliable and stable *for what it's MM> used for in the base system.* So long as your requirements are covered MM> by that usage, you'll be ok. MM> The uses I know of for BDB in the base system all consist of databases MM> of relatively small items that are changed infrequently, and usually MM> with a locking mechanism. From what you've said, this doesn't describe MM> your requirements. MM> More importantly, from what other people are saying, your requirements MM> are ones for which it's known that BDB is *not* reliable, or otherwise MM> unsuitable. In particular, an effort is underway to allow parallel MM> ports builds, which implies concurrent access to the database, which MM> is a known source of problems for BDB. MM> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> Message-ID: <20080513121452.GA70860@eos.sc1.parodius.com> On Tue, May 13, 2008 at 03:44:06PM +0400, Anthony Pankov wrote: > If concurrency is the only problem then: > 1. ?an data corruption be avoided? Or this is impossible? > 2. How? Use Sleepycat/Oracle DB instead? The libc DB1.x, despite being "mature", really should be deprecated in some manner. I'm sure there are others I've forgotten, but the only thing I can think of in the base system which relies on DB1.x is sendmail (which IMHO should really be removed from the base system and replaced with a small standalone mailer -- but that's been discussed in a previous thread in the past). Even "simple" ports like postgrey pull in db41, even though they could technically "work fine" with DB1.x. > If all BDB readers would use O_SHLOCK and all writers O_EXLOCK is it > guarantee for data integrity? The corruption I've seen in the past results in DB operations failing for no particular reason ("what do you mean those are all the records? No! I just inserted a bunch more!"). It turns out some of the data in the actual .db file is corrupt -- even when locking is used everywhere. It's as if the code has some weird bug where it'll write bogus data to the file for some reason. I'll ask you this: is there some particular reason you can't just write to a file yourself, using your own/documented file format? Why does DB have to be used? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From joerg at britannica.bec.de Tue May 13 13:00:54 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Tue May 13 13:00:58 2008 Subject: BDB corrupt In-Reply-To: <2117635718.20080513154406@mail.ru> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> Message-ID: <20080513125956.GD397@britannica.bec.de> On Tue, May 13, 2008 at 03:44:06PM +0400, Anthony Pankov wrote: > 3. reading/writing = 60%/40% I don't know where you get those numbers from, but they feel *very* wrong from the perspective of someone who actually dealt a lot with those tools. Writing is only a very small part of the operations and typically done a single large batch. For the same reason a single reader/writer lock is perfective fine. Joerg From yanefbsd at gmail.com Tue May 13 15:50:54 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Tue May 13 15:50:58 2008 Subject: BDB corrupt In-Reply-To: <4828F55E.6020407@pix.net> References: <20080509124308.GA596@britannica.bec.de> <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <4828F55E.6020407@pix.net> Message-ID: <48B3E83F-A165-4680-8F29-4EE6FB60D079@gmail.com> On May 12, 2008, at 6:56 PM, Kurt Lidl wrote: > Garrett Cooper wrote: >> On May 12, 2008, at 1:38 AM, Anthony Pankov wrote: >>> >>> Please, can anybody explain what is the problem with BDB (1.86). >>> >>> Is there known caveats of using BDB? Is there some rules which >>> guarantee from curruption or it is fully undesirable to use BDB >>> under >>> high load? >>> >>> It is important for me because of using BDB in my project. >>> >>> >>>> On Fri, May 09, 2008 at 01:52:46PM +0200, Joerg Sonnenberger wrote: >>>>> >>>>> As one of the persons hacking on pkg_install in pkgsrc/NetBSD, I >>>>> would >>>>> *strongly* advisy you against storing the files only in a bdb >>>>> file. >>>>> The change of major and complete corruption with bdb185 is high, >>>>> consider pulling the plug in the middle of a long update. >>> >>>> Sunday, May 11, 2008, 5:38:25 PM, you wrote: >>> >>> GC> +1. BDB is quite easy to corrupt... >> BDB isn't ATOMic, like SQL or other DB backends. > > You mean ACID probably. And there are plenty of SQL databases > that aren't ACID either. (e.g. Mysql 4.x, Mysql 5.x w/o the > right kind of backing store) > > -Kurt Yes, you're right. Atomicity is the A in ACID. -Garrett From lidl at pix.net Tue May 13 16:03:07 2008 From: lidl at pix.net (Kurt J. Lidl) Date: Tue May 13 16:03:14 2008 Subject: BDB corrupt In-Reply-To: <2117635718.20080513154406@mail.ru> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> Message-ID: <20080513154840.GB28842@pix.net> On Tue, May 13, 2008 at 03:44:06PM +0400, Anthony Pankov wrote: > My requirements is > 1. there is no need for SQL > 2. processes are sharing db file in concurrent mode > 3. reading/writing = 60%/40% > > With BDB > clause 1 - satisfied > clause 3 - satisfied (databases of relatively small items that are > changed infrequently). > > Is there a problem with concurrent access? And, most important, is > this the ONLY problem? (I still don't understand because of blurred > mention of "data corruption".) > > If concurrency is the only problem then: > 1. Can data corruption be avoided? Or this is impossible? > 2. How? > > If all BDB readers would use O_SHLOCK and all writers O_EXLOCK is it > guarantee for data integrity? This is probably good enough, if your writers also do a fflush() before releasing the lock. If the writing process ever dies horribly, you can still have database corruption. You could turn the thing that does the actual I/O into the database into daemon that fullfils read/write requests from a unix domain socket. In fact, there is a sample application that does exactly that for the newer Sleepycat BDB code. It could probably be adapted or re-written for the 1.8x series code. If it were me, I'd just punt on using the 1.8x code, drop straight into using the BDB 4.6 codebase, and use their transactioning code. It's mature, robust, etc, etc, for a much large set of operations than the 1.8x code ever was. But there's still that licensing issue. -Kurt From lidl at pix.net Tue May 13 16:03:08 2008 From: lidl at pix.net (Kurt J. Lidl) Date: Tue May 13 16:03:14 2008 Subject: BDB corrupt In-Reply-To: <20080513121452.GA70860@eos.sc1.parodius.com> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> Message-ID: <20080513154137.GA28842@pix.net> On Tue, May 13, 2008 at 05:14:52AM -0700, Jeremy Chadwick wrote: > On Tue, May 13, 2008 at 03:44:06PM +0400, Anthony Pankov wrote: > > If concurrency is the only problem then: > > 1. ?an data corruption be avoided? Or this is impossible? > > 2. How? > > Use Sleepycat/Oracle DB instead? The libc DB1.x, despite being > "mature", really should be deprecated in some manner. This catapults back into the arena of "stuff that isn't in the base system". Not to mention I'm not sure that the Oracle BDB license would allow bundling in the OS as a binary. I doubt it, but that's a different bikeshed to paint :-) > I'm sure there are others I've forgotten, but the only thing I can think > of in the base system which relies on DB1.x is sendmail (which IMHO > should really be removed from the base system and replaced with a small > standalone mailer -- but that's been discussed in a previous thread in > the past). Even "simple" ports like postgrey pull in db41, even though > they could technically "work fine" with DB1.x. Well, on a 7.0 machine, it looks like the following: /etc/mail/aliases.db /etc/{s,}pwd.db /usr/share/misc/{termcap,vgrindefs}.db And of course, nvi uses the db code's recno interface to do all its work. > > If all BDB readers would use O_SHLOCK and all writers O_EXLOCK is it > > guarantee for data integrity? > > The corruption I've seen in the past results in DB operations failing > for no particular reason ("what do you mean those are all the records? > No! I just inserted a bunch more!"). It turns out some of the data in > the actual .db file is corrupt -- even when locking is used everywhere. > It's as if the code has some weird bug where it'll write bogus data to > the file for some reason. There are known problems with certain keys corrupting the DB 1.8x series code. In fact, the "release" of the 1.86 was an attempt to solve this problem when the KerberosV people at MIT found a repeatable key insert sequence that would corrupt things. (Or at least that's what I remember, it was a long time ago, and I might have the details wrong.) -Kurt From romain at blogreen.org Tue May 13 16:13:54 2008 From: romain at blogreen.org (Romain =?iso-8859-1?Q?Tarti=E8re?=) Date: Tue May 13 16:14:01 2008 Subject: Adaptec RAID 3805 and FreeBSD 7.0 Message-ID: <20080513161351.GA10097@marvin.blogreen.org> Hi! I am trying to have an Adaptec RAID 3805 controller working on FreeBSD 7.0. According to the release note [1], it's okay. Unfortunately, the driver fails to detect everything as expected. The following is displayed at boot-time: > aac0: mem 0xff200000-0xff3fffff irq 16 at device 14.0 on pci3 > aac0: couldn't allocate register window 0 > device_attach: aac0 attach returned 6 The CD provided with the controller include kernel modules for FreeBSD 5 and 6, but not 7 (I however tried to kldload it (Yes, I am _very_ optimistic by nature) but link_elf fails with "symbol msleep undefined"). The Adaptec website does not provide drivers for FreeBSD 7.0, but has a much more recent version of the driver than the one on the CD, so I fetched it, and tried to compile it (AACRAID 32- and 64-bit Driver Build 15727 for FreeBSD 6.1 and 6.2). I fixed the source the "lazy way", that is fixing that makes the compilation fail untill it pass. The result was not much better: > aacu0: mem 0xff200000-0xff3fffff irq 16 at device 14.0 on pci3 > aacu0: couldn't allocate register window 0 > device_attach: aacu0 attach returned 12 So I took the aac(4) source code of FreeBSD 6.2-STABLE, 7.0-STABLE and the version of Adaptec and tried to merge changes from 6.2 to 7.0 in the Adaptec code... The result is however almost the same :-( Has anybody tried / succeeded in using such a controller on FreeBSD 7.0-CURRENT? Has somebody this controller working on FreeBSD 6.3 (I can consider to downgrade to the legacy release if required since the machine is not yet in production)? Thank you! Romain References: 1. http://www.freebsd.org/releases/7.0R/hardware.html#DISK -- Romain Tarti?re http://romain.blogreen.org/ pgp: 8DAB A124 0DA4 7024 F82A E748 D8E9 A33F FF56 FF43 (ID: 0xFF56FF43) (plain text =non-HTML= PGP/GPG encrypted/signed e-mail much appreciated) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080513/f4b6621f/attachment.pgp From avg at icyb.net.ua Tue May 13 19:05:05 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 13 19:05:08 2008 Subject: devctl (alike?) for devfs In-Reply-To: <20080511214833.GB18958@deviant.kiev.zoral.com.ua> References: <480E4269.2090604@icyb.net.ua> <480FBAB9.1000904@icyb.net.ua> <48103F36.6060707@icyb.net.ua> <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> <20080511214833.GB18958@deviant.kiev.zoral.com.ua> Message-ID: <4829E658.3000605@icyb.net.ua> on 12/05/2008 00:48 Kostik Belousov said the following: > No, we do not have a leak, but we have somewhat non-obvious behaviour. > > The cdev structure is freed only after the last reference to cdev is > gone. Typical holder of the reference is the devfs vnode. In the normal > usage, the vnode is present until both the device is destroyed _and_ > devfs_populate_loop() run is performed. This function actually reclaim > the vnodes for destroyed devices, that causes last reference to cdev to > be dropped and memory freed. > > The populate loop is called syncronously from the upper levels. The > easiest method to trigger it is to do ls /dev, since it is called from > the devfs_lookupx(). Thank you for the explanation! ls did trigger DESTROY notifications. But this arbitrary delay between device entry going away and notification about about it is a little bit "uncool". So I re-wrote the patch to post notifications about deleted devices with si_refcount > 0 in a fashion similar to how si_refcount=0 devices are freed. I put those cdev-s onto a special list (re-/ab-using si_siblings LIST link field) and bump their si_refcount to prevent them from getting destroyed before the notification is sent (it would need si_name). Then, in dev_unlock_and_free() I send notifications and call dev_rel on the devices. I am not entirely satisfied with the code: 1. I don't like the way I move LIST elements from one head to the other, this should be a macro in queue.h 2. I am not sure about the names I picked 3. I slightly don't like a fact that parent-child destroy notifications are sent in reverse order because of my simplistic LIST usage. -- Andriy Gapon -------------- next part -------------- diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c index 1db25f8..0245253 100644 --- a/sys/kern/kern_conf.c +++ b/sys/kern/kern_conf.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -99,6 +100,9 @@ dev_unlock_and_free(void) mtx_unlock(&devmtx); while ((cdp = TAILQ_FIRST(&cdp_free)) != NULL) { + if (!cold) + devctl_notify("DEVFS", cdp->cdp_c.si_name, "DESTROY", NULL); + TAILQ_REMOVE(&cdp_free, cdp, cdp_list); devfs_free(&cdp->cdp_c); } @@ -172,8 +176,12 @@ dev_rel(struct cdev *dev) flag = 1; } dev_unlock(); - if (flag) + if (flag) { + if (!cold) + devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); + devfs_free(dev); + } } struct cdevsw * @@ -706,6 +714,10 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, devfs_create(dev); clean_unrhdrl(devfs_inos); dev_unlock_and_free(); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } @@ -794,6 +806,10 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) clean_unrhdrl(devfs_inos); dev_unlock(); dev_depends(pdev, dev); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } From avg at icyb.net.ua Tue May 13 19:09:34 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 13 19:09:39 2008 Subject: devctl (alike?) for devfs In-Reply-To: <4829E658.3000605@icyb.net.ua> References: <480E4269.2090604@icyb.net.ua> <480FBAB9.1000904@icyb.net.ua> <48103F36.6060707@icyb.net.ua> <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> <20080511214833.GB18958@deviant.kiev.zoral.com.ua> <4829E658.3000605@icyb.net.ua> Message-ID: <4829E76B.8010204@icyb.net.ua> on 13/05/2008 22:04 Andriy Gapon said the following: > on 12/05/2008 00:48 Kostik Belousov said the following: >> No, we do not have a leak, but we have somewhat non-obvious behaviour. >> >> The cdev structure is freed only after the last reference to cdev is >> gone. Typical holder of the reference is the devfs vnode. In the normal >> usage, the vnode is present until both the device is destroyed _and_ >> devfs_populate_loop() run is performed. This function actually reclaim >> the vnodes for destroyed devices, that causes last reference to cdev to >> be dropped and memory freed. >> >> The populate loop is called syncronously from the upper levels. The >> easiest method to trigger it is to do ls /dev, since it is called from >> the devfs_lookupx(). > > Thank you for the explanation! ls did trigger DESTROY notifications. > But this arbitrary delay between device entry going away and > notification about about it is a little bit "uncool". > > So I re-wrote the patch to post notifications about deleted devices with > si_refcount > 0 in a fashion similar to how si_refcount=0 devices are > freed. I put those cdev-s onto a special list (re-/ab-using si_siblings > LIST link field) and bump their si_refcount to prevent them from getting > destroyed before the notification is sent (it would need si_name). > Then, in dev_unlock_and_free() I send notifications and call dev_rel on > the devices. > > I am not entirely satisfied with the code: > 1. I don't like the way I move LIST elements from one head to the other, > this should be a macro in queue.h > 2. I am not sure about the names I picked > 3. I slightly don't like a fact that parent-child destroy notifications > are sent in reverse order because of my simplistic LIST usage. > Oops! I attached the older patch. New patch is here. -- Andriy Gapon -------------- next part -------------- diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c index 1db25f8..7106182 100644 --- a/sys/kern/kern_conf.c +++ b/sys/kern/kern_conf.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -63,6 +64,8 @@ static struct cdev_priv_list cdevp_free_list = TAILQ_HEAD_INITIALIZER(cdevp_free_list); static SLIST_HEAD(free_cdevsw, cdevsw) cdevsw_gt_post_list = SLIST_HEAD_INITIALIZER(); +static LIST_HEAD(cdev_notif, cdev) cdev_notif_list = + LIST_HEAD_INITIALIZER(cdev_notif_list); void dev_lock(void) @@ -82,8 +85,10 @@ dev_unlock_and_free(void) { struct cdev_priv_list cdp_free; struct free_cdevsw csw_free; + struct cdev_notif cd_notif; struct cdev_priv *cdp; struct cdevsw *csw; + struct cdev *cd; mtx_assert(&devmtx, MA_OWNED); @@ -95,10 +100,23 @@ dev_unlock_and_free(void) TAILQ_CONCAT(&cdp_free, &cdevp_free_list, cdp_list); csw_free = cdevsw_gt_post_list; SLIST_INIT(&cdevsw_gt_post_list); + /* XXX How to do the following nicely? */ + cd_notif = cdev_notif_list; + if (!LIST_EMPTY(&cd_notif)) + LIST_FIRST(&cd_notif)->si_siblings.le_prev = &LIST_FIRST(&cd_notif); + LIST_INIT(&cdev_notif_list); mtx_unlock(&devmtx); + while ((cd = LIST_FIRST(&cd_notif)) != NULL) { + if (!cold) + devctl_notify("DEVFS", cd->si_name, "DESTROY", NULL); + LIST_REMOVE(cd, si_siblings); + dev_rel(cd); + } while ((cdp = TAILQ_FIRST(&cdp_free)) != NULL) { + if (!cold) + devctl_notify("DEVFS", cdp->cdp_c.si_name, "DESTROY", NULL); TAILQ_REMOVE(&cdp_free, cdp, cdp_list); devfs_free(&cdp->cdp_c); } @@ -706,6 +724,10 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, devfs_create(dev); clean_unrhdrl(devfs_inos); dev_unlock_and_free(); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } @@ -794,6 +816,10 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) clean_unrhdrl(devfs_inos); dev_unlock(); dev_depends(pdev, dev); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } @@ -861,6 +887,9 @@ destroy_devl(struct cdev *dev) if (dev->si_refcount > 0) { LIST_INSERT_HEAD(&dead_cdevsw.d_devs, dev, si_list); + /* (re-/ab-)use si_siblings for destroy notification list */ + LIST_INSERT_HEAD(&cdev_notif_list, dev, si_siblings); + dev_refl(dev); /* Avoid race with dev_rel() */ } else { dev_free_devlocked(dev); } From kostikbel at gmail.com Tue May 13 19:16:24 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue May 13 19:16:29 2008 Subject: devctl (alike?) for devfs In-Reply-To: <4829E658.3000605@icyb.net.ua> References: <480FBAB9.1000904@icyb.net.ua> <48103F36.6060707@icyb.net.ua> <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> <20080511214833.GB18958@deviant.kiev.zoral.com.ua> <4829E658.3000605@icyb.net.ua> Message-ID: <20080513191610.GK18958@deviant.kiev.zoral.com.ua> On Tue, May 13, 2008 at 10:04:56PM +0300, Andriy Gapon wrote: > on 12/05/2008 00:48 Kostik Belousov said the following: > >No, we do not have a leak, but we have somewhat non-obvious behaviour. > > > >The cdev structure is freed only after the last reference to cdev is > >gone. Typical holder of the reference is the devfs vnode. In the normal > >usage, the vnode is present until both the device is destroyed _and_ > >devfs_populate_loop() run is performed. This function actually reclaim > >the vnodes for destroyed devices, that causes last reference to cdev to > >be dropped and memory freed. > > > >The populate loop is called syncronously from the upper levels. The > >easiest method to trigger it is to do ls /dev, since it is called from > >the devfs_lookupx(). > > Thank you for the explanation! ls did trigger DESTROY notifications. > But this arbitrary delay between device entry going away and > notification about about it is a little bit "uncool". > > So I re-wrote the patch to post notifications about deleted devices with > si_refcount > 0 in a fashion similar to how si_refcount=0 devices are > freed. I put those cdev-s onto a special list (re-/ab-using si_siblings > LIST link field) and bump their si_refcount to prevent them from getting > destroyed before the notification is sent (it would need si_name). > Then, in dev_unlock_and_free() I send notifications and call dev_rel on > the devices. > > I am not entirely satisfied with the code: > 1. I don't like the way I move LIST elements from one head to the other, > this should be a macro in queue.h > 2. I am not sure about the names I picked > 3. I slightly don't like a fact that parent-child destroy notifications > are sent in reverse order because of my simplistic LIST usage. I looked at your previous patch, and it seems it is much simpler to do drop the devmtx once more then to try to abuse free lists. In the destroy_devl(), after the while (dev->si_threadcount != 0) { /* Use unique dummy wait ident */ msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); } loop, add mtx_unlock(&devmtx); if (!cold) devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); mtx_lock(&devmtx); -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080513/76f9c0f8/attachment.pgp From avg at icyb.net.ua Tue May 13 19:26:13 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 13 19:26:15 2008 Subject: /dev/mem wrap-around behavior Message-ID: <4829EB53.3010804@icyb.net.ua> This is just out of curiosity, not a real issue. I use i386 RELENG_7. I recently played with dd-ing memory contents out of /dev/mem and examining various stuff. I "intuitively" expected dd to reach EOF at 0xFFFFFFFF (i.e. maximum possible address for i386 non-PAE), but it continued on. Apparently this happens because off_t is somewhere assigned to caddr_t (or something like that) without any extra checks. So, I am curious if this intentional, unintentional or it just happened to be this way and nobody really cares. -- Andriy Gapon From avg at icyb.net.ua Tue May 13 20:40:23 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 13 20:40:29 2008 Subject: devctl (alike?) for devfs In-Reply-To: <20080513191610.GK18958@deviant.kiev.zoral.com.ua> References: <480FBAB9.1000904@icyb.net.ua> <48103F36.6060707@icyb.net.ua> <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> <20080511214833.GB18958@deviant.kiev.zoral.com.ua> <4829E658.3000605@icyb.net.ua> <20080513191610.GK18958@deviant.kiev.zoral.com.ua> Message-ID: <4829FCB5.8030308@icyb.net.ua> on 13/05/2008 22:16 Kostik Belousov said the following: > I looked at your previous patch, and it seems it is much simpler to > do drop the devmtx once more then to try to abuse free lists. > In the destroy_devl(), after the > > while (dev->si_threadcount != 0) { > /* Use unique dummy wait ident */ > msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); > } > > loop, add > > mtx_unlock(&devmtx); > if (!cold) > devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); > mtx_lock(&devmtx); Thank you again! This is simply perfect. -- Andriy Gapon -------------- next part -------------- diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c index 1db25f8..f90e469 100644 --- a/sys/kern/kern_conf.c +++ b/sys/kern/kern_conf.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -706,6 +707,10 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, devfs_create(dev); clean_unrhdrl(devfs_inos); dev_unlock_and_free(); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } @@ -794,6 +799,10 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) clean_unrhdrl(devfs_inos); dev_unlock(); dev_depends(pdev, dev); + + if (!cold) + devctl_notify("DEVFS", dev->si_name, "CREATE", NULL); + return (dev); } @@ -842,6 +851,12 @@ destroy_devl(struct cdev *dev) msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); } + if (!cold) { + mtx_unlock(&devmtx); + devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); + mtx_lock(&devmtx); + } + dev->si_drv1 = 0; dev->si_drv2 = 0; bzero(&dev->__si_u, sizeof(dev->__si_u)); From mwm-keyword-freebsdhackers2.e313df at mired.org Tue May 13 20:48:35 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Tue May 13 20:48:44 2008 Subject: BDB corrupt In-Reply-To: <2117635718.20080513154406@mail.ru> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> Message-ID: <20080513164900.35f89e47@bhuda.mired.org> On Tue, 13 May 2008 15:44:06 +0400 Anthony Pankov wrote: > > Monday, Mike Meyer May 12, 2008, 11:24:30 PM, you wrote: > > MM> On Mon, 12 May 2008 22:35:31 +0400 Anthony Pankov wrote: > >> Because BDB: > >> 1. do not need additional installation > >> 2. is part of base system which mean it is mature, reliable and stable > > MM> BDB in the base system is mature, reliable and stable *for what it's > MM> used for in the base system.* So long as your requirements are covered > MM> by that usage, you'll be ok. > > MM> The uses I know of for BDB in the base system all consist of databases > MM> of relatively small items that are changed infrequently, and usually > MM> with a locking mechanism. From what you've said, this doesn't describe > MM> your requirements. > > MM> More importantly, from what other people are saying, your requirements > MM> are ones for which it's known that BDB is *not* reliable, or otherwise > MM> unsuitable. In particular, an effort is underway to allow parallel > MM> ports builds, which implies concurrent access to the database, which > MM> is a known source of problems for BDB. > > MM> > My requirements is > 1. there is no need for SQL > 2. processes are sharing db file in concurrent mode > 3. reading/writing = 60%/40% > > With BDB > clause 1 - satisfied > clause 3 - satisfied (databases of relatively small items that are > changed infrequently). Actually, I don't think you get #3, because my wording was poor. The things in the base system are databases of small items, where the *database* is changed infrequently. I think what you have is a database of small items where the *items* are changed infrequently, but the database itself changes frequently. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From james at mansionfamily.plus.com Tue May 13 21:49:36 2008 From: james at mansionfamily.plus.com (James Mansion) Date: Tue May 13 21:49:41 2008 Subject: BDB corrupt In-Reply-To: <20080513154137.GA28842@pix.net> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> Message-ID: <482A02CD.7040308@mansionfamily.plus.com> Kurt J. Lidl wrote: > This catapults back into the arena of "stuff that isn't in the > base system". Not to mention I'm not sure that the Oracle BDB > license would allow bundling in the OS as a binary. I doubt it, > but that's a different bikeshed to paint :-) > Is the LGPL of QDBM and TokyoCabinet also a problem? Could even try grovelling with Mikio? (Partially joking there. I assume he chose LGPL because he wants it that way, but people have been known to change licenses for a base system - like this http://blogs.sun.com/aalok/entry/lzma_on_opensolaris) And is the objection to SQL such the sqlite is really out of the running? Anyway, in this case, would writing an RPC server to own the data kill the performance? It should be easier to write something that can save the database atomically and index it in-core. It could be started on demand and shut down after a short inactivity, a bit like tibco's rvd. > There are known problems with certain keys corrupting the DB 1.8x > series code. In fact, the "release" of the 1.86 was an attempt > to solve this problem when the KerberosV people at MIT found > a repeatable key insert sequence that would corrupt things. > (Or at least that's what I remember, it was a long time ago, and > I might have the details wrong.) > > Have to say its a little concerning that such 'mature' code is actually problematic. Particularly since I'm not aware of a non-LGPL alternative. Do you have anything by way of a pointer? Google didn't help me here. James From msaad at datapipe.com Tue May 13 23:47:50 2008 From: msaad at datapipe.com (Mark Saad) Date: Tue May 13 23:47:56 2008 Subject: : Re: Socket leak (Was: Re: What triggers "No BufferSpace), ?Available" Message-ID: <482A2639.7000401@datapipe.com> Hello All This issue goes back some time, but I do not see a solution. Sorry about the cross post not sure where this belongs. Here is an overview of my issue which is similar and I hope someone can point me in the direction of a solution. I have experiencing an odd socket related issue on a few servers i manage. They are fairly large ftp servers for popular north american news agency. They handle 1000's of ftp transactions per hour. Currently they are running FreeBSD 6.3-Release-p1 . I have verified this happened on FreeBSD 6.1-Release 6.1-Stable 6.2-Release 6.2-Stable and 7.0-Release all 32bit installs and in both SMP an UP kernels. Oddly this issue did not happen on FreeBSD 4.x . I have a similar setup that has a 1400+ Day uptime running FreeBSD 4.x-Release. The issue is after 7 to 14 days the servers lock up and will not create any new tcp sockets. The system used proftp with mysql for authentication of the ftp accounts. The system is also running Apache 2.2.x , Postfix, Cyrus, clam-av, Diablo JDK 1.5 for Resin Appserver and daemontools . The only sysctls that seem to help are kern.ipc.maxsockets and kern.maxusers . Currently they are set to 65535 and 1024 . Changing kern.ipc.maxsockbuf did not have any effect I tried bumping this up to 2Meg, In any case I started work on logging everything we could think of to see what was happening. I started logging the values of kern.ipc.numopensockets and I noticed that something is leaking sockets. Here is a sample of the log 2008-04-29--15:04.10 ____ kern.ipc.numopensockets: 1501 2008-04-29--16:04.01 ____ kern.ipc.numopensockets: 1535 2008-04-29--17:04.00 ____ kern.ipc.numopensockets: 1617 2008-04-29--18:04.00 ____ kern.ipc.numopensockets: 1710 This continues until kern.ipc.maxsockets its reached or the box is rebooted. The other thing we looked at was the output from vmstat -z The first thing was the high amount of malloc 128 bucket failures 128 Bucket: 524, 0, 2489, 80, 8364, 23055239 I also logged the mbuf clusters, we never reached the max mbuf clusters Its almost like there are stale sockets. Here is a snapshot of the server now ewr# sockstat -4u |wc -l 139 ewr# sysctl kern.ipc.numopensockets kern.ipc.numopensockets: 13935 ewr# uptime 7:30PM up 6 days, 26 mins, 3 users, load averages: 0.18, 0.25, 0.17 My questions. 1. If I can not identify who / what is consuming all my tcp sockets what will happen if I double or triple the value of kern.ipc.maxsockets ? 2. Could this be an issue with a low kern.maxusers . Its currently set to 1024 . Also at times when I can not create a new socket I am not pinned on mbuf clusters . I was able to verify this in the past. 3. I installed a debugging kernel, which I built on the server. I was able to get a coredump of the server at the point in time we last had an issue. But I am not sure what I can do with this, kernel debugging is way beyond what I am capable of doing . Do I want to even pursue this ? 4. Does anyone have any system tunings you could recommend for a high volume ftp site ? What does ftp.freebsd.org have ? -- Mark Saad msaad@datapipe.com From msaad at datapipe.com Tue May 13 23:48:26 2008 From: msaad at datapipe.com (Mark Saad) Date: Tue May 13 23:48:29 2008 Subject: : Re: Socket leak (Was: Re: What triggers "No BufferSpace), ?Available" Message-ID: <482A2668.8070209@datapipe.com> Hello All This issue goes back some time, but I do not see a solution. Sorry about the cross post not sure where this belongs. Here is an overview of my issue which is similar and I hope someone can point me in the direction of a solution. I have experiencing an odd socket related issue on a few servers i manage. They are fairly large ftp servers for popular north american news agency. They handle 1000's of ftp transactions per hour. Currently they are running FreeBSD 6.3-Release-p1 . I have verified this happened on FreeBSD 6.1-Release 6.1-Stable 6.2-Release 6.2-Stable and 7.0-Release all 32bit installs and in both SMP an UP kernels. Oddly this issue did not happen on FreeBSD 4.x . I have a similar setup that has a 1400+ Day uptime running FreeBSD 4.x-Release. The issue is after 7 to 14 days the servers lock up and will not create any new tcp sockets. The system used proftp with mysql for authentication of the ftp accounts. The system is also running Apache 2.2.x , Postfix, Cyrus, clam-av, Diablo JDK 1.5 for Resin Appserver and daemontools . The only sysctls that seem to help are kern.ipc.maxsockets and kern.maxusers . Currently they are set to 65535 and 1024 . Changing kern.ipc.maxsockbuf did not have any effect I tried bumping this up to 2Meg, In any case I started work on logging everything we could think of to see what was happening. I started logging the values of kern.ipc.numopensockets and I noticed that something is leaking sockets. Here is a sample of the log 2008-04-29--15:04.10 ____ kern.ipc.numopensockets: 1501 2008-04-29--16:04.01 ____ kern.ipc.numopensockets: 1535 2008-04-29--17:04.00 ____ kern.ipc.numopensockets: 1617 2008-04-29--18:04.00 ____ kern.ipc.numopensockets: 1710 This continues until kern.ipc.maxsockets its reached or the box is rebooted. The other thing we looked at was the output from vmstat -z The first thing was the high amount of malloc 128 bucket failures 128 Bucket: 524, 0, 2489, 80, 8364, 23055239 I also logged the mbuf clusters, we never reached the max mbuf clusters Its almost like there are stale sockets. Here is a snapshot of the server now ewr# sockstat -4u |wc -l 139 ewr# sysctl kern.ipc.numopensockets kern.ipc.numopensockets: 13935 ewr# uptime 7:30PM up 6 days, 26 mins, 3 users, load averages: 0.18, 0.25, 0.17 My questions. 1. If I can not identify who / what is consuming all my tcp sockets what will happen if I double or triple the value of kern.ipc.maxsockets ? 2. Could this be an issue with a low kern.maxusers . Its currently set to 1024 . Also at times when I can not create a new socket I am not pinned on mbuf clusters . I was able to verify this in the past. 3. I installed a debugging kernel, which I built on the server. I was able to get a coredump of the server at the point in time we last had an issue. But I am not sure what I can do with this, kernel debugging is way beyond what I am capable of doing . Do I want to even pursue this ? 4. Does anyone have any system tunings you could recommend for a high volume ftp site ? What does ftp.freebsd.org have ? -- Mark Saad msaad@datapipe.com From bu7cher at yandex.ru Wed May 14 04:54:31 2008 From: bu7cher at yandex.ru (Andrey V. Elsukov) Date: Wed May 14 04:54:35 2008 Subject: Adaptec RAID 3805 and FreeBSD 7.0 In-Reply-To: <20080513161351.GA10097@marvin.blogreen.org> References: <20080513161351.GA10097@marvin.blogreen.org> Message-ID: <482A6760.5020704@yandex.ru> Romain Tarti?re wrote: > I am trying to have an Adaptec RAID 3805 controller working on FreeBSD > 7.0. According to the release note [1], it's okay. Unfortunately, the > driver fails to detect everything as expected. The following is > displayed at boot-time: >> aac0: mem 0xff200000-0xff3fffff irq 16 at device 14.0 on pci3 >> aac0: couldn't allocate register window 0 >> device_attach: aac0 attach returned 6 > The result was not much better: >> aacu0: mem 0xff200000-0xff3fffff irq 16 at device 14.0 on pci3 >> aacu0: couldn't allocate register window 0 >> device_attach: aacu0 attach returned 12 > Has anybody tried / succeeded in using such a controller on FreeBSD > 7.0-CURRENT? Has somebody this controller working on FreeBSD 6.3 (I can > consider to downgrade to the legacy release if required since the > machine is not yet in production)? Did you try last 8.0-CURRENT snapshot? I remember that Ed made many changes in aac(4) driver in CURRENT.. If it will not work then fill problem report. -- WBR, Andrey V. Elsukov From ap00 at mail.ru Wed May 14 06:51:11 2008 From: ap00 at mail.ru (Anthony Pankov) Date: Wed May 14 06:51:14 2008 Subject: BDB corrupt In-Reply-To: <482A02CD.7040308@mansionfamily.plus.com> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> Message-ID: <394875078.20080514105335@mail.ru> Thanks everyone for responding. I'll use QDBM as most attractive from my point of view. Oracle BDB is to complex for my task and have drastic "free" license with unknown price for commercial use. Licensing is not a main issue for me now, but i'll beware it to be on the safe side. I think that FreeBSD (ideally) must have a few PRIMARY datastore tools which together cover most tasks and able for horizontal and vertical scaling: 1. BDB1 = QDBM 2. CDB 3. SQLite 4. Firebird 5. PostgreSQL I'm absolutely sure that tradition of programmers to write own datastore, memory list, etc has to be dropped. I think that some warning like 'may corrupt data' has to be committed to 'man db'. -- Best regards, Anthony mailto:ap00@mail.ru From peterjeremy at optushome.com.au Wed May 14 07:17:34 2008 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Wed May 14 07:17:39 2008 Subject: BDB corrupt In-Reply-To: <482A02CD.7040308@mansionfamily.plus.com> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> Message-ID: <20080514071728.GP64804@server.vk2pj.dyndns.org> On 2008-May-13 22:06:21 +0100, James Mansion wrote: >And is the objection to SQL such the sqlite is really out of the running? There is no specific objection to SQL. There is a general objection to adding more utilities to the base system unless a _very_ good case can be made for including them. So far, no-one has made a compelling reason to include SQLite (or other SQL engine) into the base system. SQLite does appear to be a potential candidate for the base system - the license is acceptable and it is relatively small. On the downside, SQLite is currently undergoing rapid development - at a rate much faster than FreeBSD can comfortably accomodate. >Anyway, in this case, would writing an RPC server to own the data kill the >performance? That depends on how everything is implemented. It doesn't really solve the underlying problem that a database is needed. -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080514/8861345c/attachment.pgp From yanefbsd at gmail.com Wed May 14 07:24:11 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Wed May 14 07:24:15 2008 Subject: BDB corrupt In-Reply-To: <482A02CD.7040308@mansionfamily.plus.com> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> Message-ID: On May 13, 2008, at 2:06 PM, James Mansion wrote: > Kurt J. Lidl wrote: >> This catapults back into the arena of "stuff that isn't in the >> base system". Not to mention I'm not sure that the Oracle BDB >> license would allow bundling in the OS as a binary. I doubt it, >> but that's a different bikeshed to paint :-) >> > Is the LGPL of QDBM and TokyoCabinet also a problem? Could even try > grovelling > with Mikio? (Partially joking there. I assume he chose LGPL because > he wants > it that way, but people have been known to change licenses for a > base system - like > this http://blogs.sun.com/aalok/entry/lzma_on_opensolaris) > > And is the objection to SQL such the sqlite is really out of the > running? > > Anyway, in this case, would writing an RPC server to own the data > kill the performance? > It should be easier to write something that can save the database > atomically and index > it in-core. It could be started on demand and shut down after a > short inactivity, a bit > like tibco's rvd. >> There are known problems with certain keys corrupting the DB 1.8x >> series code. In fact, the "release" of the 1.86 was an attempt >> to solve this problem when the KerberosV people at MIT found >> a repeatable key insert sequence that would corrupt things. >> (Or at least that's what I remember, it was a long time ago, and >> I might have the details wrong.) >> >> > Have to say its a little concerning that such 'mature' code is > actually problematic. > Particularly since I'm not aware of a non-LGPL alternative. > > Do you have anything by way of a pointer? Google didn't help me here. > > James Most of the complaints about other DBs is licensing related, but SQLite's complaint was also the fact that the past stability record was a bit rocky. HTH, -Garrett From doconnor at gsoft.com.au Wed May 14 08:44:54 2008 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Wed May 14 08:44:58 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <48291719.5010801@freebsd.org> References: <20080512204558.GA44114@flint.openpave.org> <48291719.5010801@freebsd.org> Message-ID: <200805141755.35892.doconnor@gsoft.com.au> On Tue, 13 May 2008, Tim Kientzle wrote: > > I think this is a really bad idea. The problem with the tools is > > not with the files. It is that the files need to be parsed on each > > run, often recursively, and your solution would not help at all. > > Parsing one file isn't expensive; parsing several hundred files > to find one bit of information is expensive. > > > The database(s) should just be a cache of the information stored in > > the files. > > Bingo! As long as the .db version can be easily recreated > from scratch from the master data stored in the same files > as always, it doesn't really matter if the BDB is occasionally > corrupted, as long as it can be rebuilt fairly quickly. So long as you can tell it is corrupted.. It's also a drag from a user POV when the tool crashes because the DB is hosed (seen in portupgrade a number of times) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080514/1723b1fb/attachment.pgp From romain at blogreen.org Wed May 14 10:17:31 2008 From: romain at blogreen.org (Romain =?iso-8859-1?Q?Tarti=E8re?=) Date: Wed May 14 10:17:37 2008 Subject: Adaptec RAID 3805 and FreeBSD 7.0 In-Reply-To: <482A6760.5020704@yandex.ru> References: <20080513161351.GA10097@marvin.blogreen.org> <482A6760.5020704@yandex.ru> Message-ID: <20080514101729.GB77349@marvin.blogreen.org> On Wed, May 14, 2008 at 08:15:28AM +0400, Andrey V. Elsukov wrote: > Romain Tarti?re wrote: > > I am trying to have an Adaptec RAID 3805 controller working on > > FreeBSD 7.0. > Did you try last 8.0-CURRENT snapshot? Just tried with April snapshot. Same result as with 7.0-STABLE :-( . > I remember that Ed made many changes in aac(4) driver in CURRENT.. > If it will not work then fill problem report. PR kern/123666 sent. http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/123666 Regards, Romain -- Romain Tarti?re http://romain.blogreen.org/ pgp: 8DAB A124 0DA4 7024 F82A E748 D8E9 A33F FF56 FF43 (ID: 0xFF56FF43) (plain text =non-HTML= PGP/GPG encrypted/signed e-mail much appreciated) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080514/cb751fb3/attachment.pgp From to.my.trociny at gmail.com Wed May 14 10:42:26 2008 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Wed May 14 10:42:53 2008 Subject: Socket leak In-Reply-To: <482A2639.7000401@datapipe.com> (Mark Saad's message of "Tue\, 13 May 2008 19\:37\:29 -0400") References: <482A2639.7000401@datapipe.com> Message-ID: <81zlqtfazy.fsf@zhuzha.ua1> On Tue, 13 May 2008 19:37:29 -0400 Mark Saad wrote: MS> I started logging the values of kern.ipc.numopensockets and I noticed MS> that something is leaking sockets. Here is a sample of the log MS> 2008-04-29--15:04.10 ____ kern.ipc.numopensockets: 1501 MS> 2008-04-29--16:04.01 ____ kern.ipc.numopensockets: 1535 MS> 2008-04-29--17:04.00 ____ kern.ipc.numopensockets: 1617 MS> 2008-04-29--18:04.00 ____ kern.ipc.numopensockets: 1710 MS> This continues until kern.ipc.maxsockets its reached or the box is MS> rebooted. MS> The other thing we looked at was the output from vmstat -z MS> The first thing was the high amount of malloc 128 bucket failures MS> 128 Bucket: 524, 0, 2489, 80, 8364, 23055239 MS> I also logged the mbuf clusters, we never reached the max mbuf clusters MS> Its almost like there are stale sockets. Here is a snapshot of the server now MS> ewr# sockstat -4u |wc -l MS> 139 MS> ewr# sysctl kern.ipc.numopensockets MS> kern.ipc.numopensockets: 13935 MS> ewr# uptime MS> 7:30PM up 6 days, 26 mins, 3 users, load averages: 0.18, 0.25, 0.17 We had the same problem on one of hosts running 6.2-RELEASE-p11. The situation was complicated by the fact that I didn't have root access to the host and there were problems with getting more debugging or running tcpdump. Eventually, it appeared that problem was caused by proftpd. One of our clients connected to ftp server every five minutes looking for new file to download. When there was the file everything was good. But when there wasn't, some strange things happened. In proftpd logs we had: FTP session opened. mod_delay/0.5: delaying for 28 usecs user fake authenticated by mod_auth_pam.c USER fake: Login successful. Preparing to chroot to directory '/var/ftp/fake' Environment successfully chroot()ed. mod_delay/0.5: delaying for 621 usecs Entering Passive Mode (XX,YY,ZZ,213,241,70). FTP session closed. i.e. the client connected to server, had login successful, created new DATA connection in passive mode and then exited. But although proftpd reported that connection closed and proftpd process exited we still had this orphaned connection in our system reported by netstat in ESTABLISHED state. sockstat did not display this connections. Some of these connections could be in CLOSE_WAIT mode instead of ESTABLISHED. Such connection was seen by netstat for several hours and then disappeared but I suspect that the socket buffer was not freed and numopensockets counter did not decrease. Unfortunately, I did not managed to persuade admin to increase DebugLevel in proftpd.conf and run tcpdump to investigate more what was going on. It turned out that we had proftpd built for FREEBSD5_4: Compile-time Settings: Version: 1.3.0 Platform: FREEBSD5 (FREEBSD5_4) Built With: configure --localstatedir=/var/run --sysconfdir=/usr/local/etc --disable-sendfile --disable-ipv6 --with-modules=mod_ratio:mod_readme:mod_rewrite:mod_wrap:mod_ifsession --prefix=/usr/local i386-portbld-freebsd5.4 Upgrade to more recent proftpd built for proper platform resolved the problem. So I would recommend to look for process that could cause this leak. In my case careful investigation of netstat output history and comparing with sockstat output helped to find guilty. May be it would help to restart daemons one by one and see if sockets are freed. You can surely increase kern.ipc.maxsockets as workaround until you identify what causes the problem. -- Mikolaj Golub From stefan.lambrev at moneybookers.com Wed May 14 10:59:48 2008 From: stefan.lambrev at moneybookers.com (Stefan Lambrev) Date: Wed May 14 10:59:51 2008 Subject: 53 ECONNABORTED Software caused connection abort. A connection abort was caused internal to your host machine. Message-ID: <482AC2D5.1010800@moneybookers.com> Greetings, I'm running web balancer pound-2.3.2 on FreeBSD 6.x and I see strange error message: HTTP accept: Software caused connection abort. The code that triggers this is: for(lstn = listeners, i = 0; lstn; lstn = lstn->next, i++) { if(polls[i].revents & (POLLIN | POLLPRI)) { memset(&clnt_addr, 0, sizeof(clnt_addr)); clnt_length = sizeof(clnt_addr); if((clnt = accept(lstn->sock, (struct sockaddr *)&clnt_addr, (socklen_t *)&clnt_length)) < 0) { logmsg(LOG_WARNING, "HTTP accept: %s", strerror(errno)); ... Any ideas where should I look to understand better the error message and why errno 53 was returned? -- Best Wishes, Stefan Lambrev ICQ# 24134177 From kostikbel at gmail.com Wed May 14 12:19:36 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Wed May 14 12:19:43 2008 Subject: devctl (alike?) for devfs In-Reply-To: <4829FCB5.8030308@icyb.net.ua> References: <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> <20080511214833.GB18958@deviant.kiev.zoral.com.ua> <4829E658.3000605@icyb.net.ua> <20080513191610.GK18958@deviant.kiev.zoral.com.ua> <4829FCB5.8030308@icyb.net.ua> Message-ID: <20080514121920.GR18958@deviant.kiev.zoral.com.ua> On Tue, May 13, 2008 at 11:40:21PM +0300, Andriy Gapon wrote: > on 13/05/2008 22:16 Kostik Belousov said the following: > >I looked at your previous patch, and it seems it is much simpler to > >do drop the devmtx once more then to try to abuse free lists. > >In the destroy_devl(), after the > > > > while (dev->si_threadcount != 0) { > > /* Use unique dummy wait ident */ > > msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); > > } > > > >loop, add > > > > mtx_unlock(&devmtx); > > if (!cold) > > devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); > > mtx_lock(&devmtx); > > Thank you again! This is simply perfect. > > > -- > Andriy Gapon > diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c > index 1db25f8..f90e469 100644 I do not like abusing the subsystem to communicate the data. What about this instead ? The created/destroyed cdev name is available in the $cdev variable, example clause for devd.conf looks like this: notify 1000 { system "DEVFS"; subsystem "CDEV"; action "echo $type $cdev >/dev/console"; }; diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c index e911913..a5ed25c 100644 --- a/sys/kern/kern_conf.c +++ b/sys/kern/kern_conf.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD: src/sys/kern/kern_conf.c,v 1.211 2008/04/02 11:11:58 kib Exp #include #include #include +#include #include #include #include @@ -526,6 +527,37 @@ unit2minor(int unit) return ((unit & 0xff) | ((unit << 8) & ~0xffff)); } +static void +notify(struct cdev *dev, const char *ev) +{ + static const char prefix[] = "cdev="; + char *data; + int namelen; + + if (cold) + return; + namelen = strlen(dev->si_name); + data = malloc(namelen + sizeof(prefix), M_TEMP, M_WAITOK); + memcpy(data, prefix, sizeof(prefix) - 1); + memcpy(data + sizeof(prefix) - 1, dev->si_name, namelen + 1); + devctl_notify("DEVFS", "CDEV", ev, data); + free(data, M_TEMP); +} + +static void +notify_create(struct cdev *dev) +{ + + notify(dev, "CREATE"); +} + +static void +notify_destroy(struct cdev *dev) +{ + + notify(dev, "DESTROY"); +} + static struct cdev * newdev(struct cdevsw *csw, int y, struct cdev *si) { @@ -706,6 +738,9 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, devfs_create(dev); clean_unrhdrl(devfs_inos); dev_unlock_and_free(); + + notify_create(dev); + return (dev); } @@ -794,6 +829,9 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) clean_unrhdrl(devfs_inos); dev_unlock(); dev_depends(pdev, dev); + + notify_create(dev); + return (dev); } @@ -842,6 +880,10 @@ destroy_devl(struct cdev *dev) msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); } + mtx_unlock(&devmtx); + notify_destroy(dev); + mtx_lock(&devmtx); + dev->si_drv1 = 0; dev->si_drv2 = 0; bzero(&dev->__si_u, sizeof(dev->__si_u)); -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080514/d30820c0/attachment.pgp From avg at icyb.net.ua Wed May 14 13:19:10 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed May 14 13:19:19 2008 Subject: devctl (alike?) for devfs In-Reply-To: <20080514121920.GR18958@deviant.kiev.zoral.com.ua> References: <200804240811.26183.jhb@freebsd.org> <4810FD1E.70602@icyb.net.ua> <20080425095009.GD18958@deviant.kiev.zoral.com.ua> <4811E6BC.4060306@icyb.net.ua> <20080425143646.GF18958@deviant.kiev.zoral.com.ua> <48275C0C.2040601@icyb.net.ua> <20080511214833.GB18958@deviant.kiev.zoral.com.ua> <4829E658.3000605@icyb.net.ua> <20080513191610.GK18958@deviant.kiev.zoral.com.ua> <4829FCB5.8030308@icyb.net.ua> <20080514121920.GR18958@deviant.kiev.zoral.com.ua> Message-ID: <482AE6C9.7090505@icyb.net.ua> on 14/05/2008 15:19 Kostik Belousov said the following: > On Tue, May 13, 2008 at 11:40:21PM +0300, Andriy Gapon wrote: >> on 13/05/2008 22:16 Kostik Belousov said the following: >>> I looked at your previous patch, and it seems it is much simpler to >>> do drop the devmtx once more then to try to abuse free lists. >>> In the destroy_devl(), after the >>> >>> while (dev->si_threadcount != 0) { >>> /* Use unique dummy wait ident */ >>> msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); >>> } >>> >>> loop, add >>> >>> mtx_unlock(&devmtx); >>> if (!cold) >>> devctl_notify("DEVFS", dev->si_name, "DESTROY", NULL); >>> mtx_lock(&devmtx); >> Thank you again! This is simply perfect. >> >> >> -- >> Andriy Gapon > >> diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c >> index 1db25f8..f90e469 100644 > > I do not like abusing the subsystem to communicate the data. > > What about this instead ? The created/destroyed cdev name is available > in the $cdev variable, example clause for devd.conf looks like this: Kostik, I do not have a strong opinion on the exact format of notification. Your code definitely looks cleaner and friendlier, so I am all for it. P.S. the only non-obvious thing in "strcat-ing" is that space for '\0' is accounted for in sizeof(prefix) whereas actual '\0' is copied from si_name. > notify 1000 { > system "DEVFS"; > subsystem "CDEV"; > action "echo $type $cdev >/dev/console"; > }; > > diff --git a/sys/kern/kern_conf.c b/sys/kern/kern_conf.c > index e911913..a5ed25c 100644 > --- a/sys/kern/kern_conf.c > +++ b/sys/kern/kern_conf.c > @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD: src/sys/kern/kern_conf.c,v 1.211 2008/04/02 11:11:58 kib Exp > #include > #include > #include > +#include > #include > #include > #include > @@ -526,6 +527,37 @@ unit2minor(int unit) > return ((unit & 0xff) | ((unit << 8) & ~0xffff)); > } > > +static void > +notify(struct cdev *dev, const char *ev) > +{ > + static const char prefix[] = "cdev="; > + char *data; > + int namelen; > + > + if (cold) > + return; > + namelen = strlen(dev->si_name); > + data = malloc(namelen + sizeof(prefix), M_TEMP, M_WAITOK); > + memcpy(data, prefix, sizeof(prefix) - 1); > + memcpy(data + sizeof(prefix) - 1, dev->si_name, namelen + 1); > + devctl_notify("DEVFS", "CDEV", ev, data); > + free(data, M_TEMP); > +} > + > +static void > +notify_create(struct cdev *dev) > +{ > + > + notify(dev, "CREATE"); > +} > + > +static void > +notify_destroy(struct cdev *dev) > +{ > + > + notify(dev, "DESTROY"); > +} > + > static struct cdev * > newdev(struct cdevsw *csw, int y, struct cdev *si) > { > @@ -706,6 +738,9 @@ make_dev_credv(int flags, struct cdevsw *devsw, int minornr, > devfs_create(dev); > clean_unrhdrl(devfs_inos); > dev_unlock_and_free(); > + > + notify_create(dev); > + > return (dev); > } > > @@ -794,6 +829,9 @@ make_dev_alias(struct cdev *pdev, const char *fmt, ...) > clean_unrhdrl(devfs_inos); > dev_unlock(); > dev_depends(pdev, dev); > + > + notify_create(dev); > + > return (dev); > } > > @@ -842,6 +880,10 @@ destroy_devl(struct cdev *dev) > msleep(&csw, &devmtx, PRIBIO, "devdrn", hz / 10); > } > > + mtx_unlock(&devmtx); > + notify_destroy(dev); > + mtx_lock(&devmtx); > + > dev->si_drv1 = 0; > dev->si_drv2 = 0; > bzero(&dev->__si_u, sizeof(dev->__si_u)); -- Andriy Gapon From msaad at datapipe.com Wed May 14 13:46:37 2008 From: msaad at datapipe.com (Mark Saad) Date: Wed May 14 13:46:47 2008 Subject: Socket leak In-Reply-To: <81zlqtfazy.fsf@zhuzha.ua1> References: <482A2639.7000401@datapipe.com> <81zlqtfazy.fsf@zhuzha.ua1> Message-ID: <482AED3B.1020307@datapipe.com> Mikolaj Thanks for the input, did you change any of the options for TimeoutLinger or TimeoutIdle ? The Proftpd I am running is build for 6.3-RELEASE here are the build options Compile-time Settings: Version: 1.3.0a Platform: FREEBSD6 (FREEBSD6_3) Built With: configure CPPFLAGS=-DHAVE_OPENSSL --localstatedir=/var/run --disable-sendfile --disable-ipv6 --with-modules=mod_sql:mod_sql_mysql:mod_check_mysql:mod_check_digest --prefix=/usr/local --with-includes=/usr/local/include/mysql:/usr/include/openssl --with-libraries=/usr/local/lib/mysql This is not built from ports, one of the other S/A who works on this setup built proftpd with MySQL support, not sure why this is not from ports , from what I understand this mysql patch is not in ports. Mikolaj Golub wrote: > On Tue, 13 May 2008 19:37:29 -0400 Mark Saad wrote: > > MS> I started logging the values of kern.ipc.numopensockets and I noticed > MS> that something is leaking sockets. Here is a sample of the log > > MS> 2008-04-29--15:04.10 ____ kern.ipc.numopensockets: 1501 > MS> 2008-04-29--16:04.01 ____ kern.ipc.numopensockets: 1535 > MS> 2008-04-29--17:04.00 ____ kern.ipc.numopensockets: 1617 > MS> 2008-04-29--18:04.00 ____ kern.ipc.numopensockets: 1710 > > MS> This continues until kern.ipc.maxsockets its reached or the box is > MS> rebooted. > > MS> The other thing we looked at was the output from vmstat -z > MS> The first thing was the high amount of malloc 128 bucket failures > > MS> 128 Bucket: 524, 0, 2489, 80, 8364, 23055239 > > MS> I also logged the mbuf clusters, we never reached the max mbuf clusters > > MS> Its almost like there are stale sockets. Here is a snapshot of the server now > > MS> ewr# sockstat -4u |wc -l > MS> 139 > MS> ewr# sysctl kern.ipc.numopensockets > MS> kern.ipc.numopensockets: 13935 > > MS> ewr# uptime > MS> 7:30PM up 6 days, 26 mins, 3 users, load averages: 0.18, 0.25, 0.17 > > We had the same problem on one of hosts running 6.2-RELEASE-p11. The situation > was complicated by the fact that I didn't have root access to the host and > there were problems with getting more debugging or running tcpdump. > > Eventually, it appeared that problem was caused by proftpd. One of our clients > connected to ftp server every five minutes looking for new file to > download. When there was the file everything was good. But when there wasn't, > some strange things happened. In proftpd logs we had: > > FTP session opened. > mod_delay/0.5: delaying for 28 usecs > user fake authenticated by mod_auth_pam.c > USER fake: Login successful. > Preparing to chroot to directory '/var/ftp/fake' > Environment successfully chroot()ed. > mod_delay/0.5: delaying for 621 usecs > Entering Passive Mode (XX,YY,ZZ,213,241,70). > FTP session closed. > > i.e. the client connected to server, had login successful, created new DATA > connection in passive mode and then exited. But although proftpd reported that > connection closed and proftpd process exited we still had this orphaned > connection in our system reported by netstat in ESTABLISHED state. sockstat > did not display this connections. Some of these connections could be in > CLOSE_WAIT mode instead of ESTABLISHED. Such connection was seen by netstat > for several hours and then disappeared but I suspect that the socket buffer > was not freed and numopensockets counter did not decrease. > > Unfortunately, I did not managed to persuade admin to increase DebugLevel in > proftpd.conf and run tcpdump to investigate more what was going on. It turned > out that we had proftpd built for FREEBSD5_4: > > Compile-time Settings: > Version: 1.3.0 > Platform: FREEBSD5 (FREEBSD5_4) > Built With: > configure --localstatedir=/var/run --sysconfdir=/usr/local/etc --disable-sendfile --disable-ipv6 --with-modules=mod_ratio:mod_readme:mod_rewrite:mod_wrap:mod_ifsession --prefix=/usr/local i386-portbld-freebsd5.4 > > Upgrade to more recent proftpd built for proper platform resolved the problem. > > So I would recommend to look for process that could cause this leak. In my > case careful investigation of netstat output history and comparing with > sockstat output helped to find guilty. May be it would help to restart daemons > one by one and see if sockets are freed. > > You can surely increase kern.ipc.maxsockets as workaround until you identify > what causes the problem. > > -- > Mikolaj Golub > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Mark Saad msaad@datapipe.com From lidl at pix.net Wed May 14 13:50:54 2008 From: lidl at pix.net (Kurt J. Lidl) Date: Wed May 14 13:51:10 2008 Subject: BDB corrupt In-Reply-To: References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> Message-ID: <20080514135051.GA21370@pix.net> On Wed, May 14, 2008 at 12:25:16AM -0700, Garrett Cooper wrote: > Most of the complaints about other DBs is licensing related, but SQLite's > complaint was also the fact that the past stability record was a bit rocky. One other thing to watch for in SQLite is the lack of atomicity in updates. It's not ACID, just like BDB 1.8x isn't ACID. Without a write-ahead log, you cannot be sure that the data written actually made it to stable storage, and as such, you cannot be sure that your database didn't get corrupted when the process stops in a non-optimal way. I view SQLite as adding syntactic sugar to your data access language. It doesn't give you real tranactions, it doesn't give you a write-ahead log, it doesn't give you referential integrity. Heck, the last time I looked (admitted, a while ago), it didn't even enforce column type checking on tables. (Put this string in the INT column? No problem!) -Kurt From lidl at pix.net Wed May 14 14:05:01 2008 From: lidl at pix.net (Kurt J. Lidl) Date: Wed May 14 14:05:07 2008 Subject: BDB corrupt In-Reply-To: <482A02CD.7040308@mansionfamily.plus.com> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> Message-ID: <20080514140458.GB21370@pix.net> On Tue, May 13, 2008 at 10:06:21PM +0100, James Mansion wrote: > Kurt J. Lidl wrote: >> There are known problems with certain keys corrupting the DB 1.8x >> series code. In fact, the "release" of the 1.86 was an attempt >> to solve this problem when the KerberosV people at MIT found >> a repeatable key insert sequence that would corrupt things. >> (Or at least that's what I remember, it was a long time ago, and >> I might have the details wrong.) >> > Have to say its a little concerning that such 'mature' code is actually > problematic. > Particularly since I'm not aware of a non-LGPL alternative. > > Do you have anything by way of a pointer? Google didn't help me here. This is somewhat alluded to here: http://www.mail-archive.com/kerberos@mit.edu/msg01560.html You might want to read the entire thread on that issue. There is the comment in the Oracle web pages about 1.86 vs 1.85: http://www.oracle.com/technology/software/products/berkeley-db/db/index.html "Do not upgrade to the 1.86 release other than to fix specific hash access method problems found in the 1.85 release." And in the "Berkeley DB: A Retrospective" paper (http://sites.computer.org/debull/A07Sept/seltzer.pdf), Margo notes: "Db-1.85 enjoyed widespread adoption after its release with 4.4BSD. In particular, both the MIT Kerberos project and the University of Michigan LDAP project incorporated it. A group at Harvard released a minor patch version db-1.86 for the Kerberos team in 1996." So, I think my recollection is correct. -Kurt From xorquewasp at googlemail.com Wed May 14 14:25:06 2008 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Wed May 14 14:25:09 2008 Subject: BDB corrupt In-Reply-To: <20080514135051.GA21370@pix.net> References: <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514135051.GA21370@pix.net> Message-ID: <20080514135822.GA4873@logik.internal.network> On 20080514 09:50:52, Kurt J. Lidl wrote: > Heck, the last time I looked (admitted, a while ago), it didn't > even enforce column type checking on tables. (Put this string in > the INT column? No problem!) They consider it a feature. Nobody knows why. From mwm at mired.org Wed May 14 14:24:17 2008 From: mwm at mired.org (Mike Meyer) Date: Wed May 14 14:42:28 2008 Subject: BDB corrupt In-Reply-To: <20080514071728.GP64804@server.vk2pj.dyndns.org> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514071728.GP64804@server.vk2pj.dyndns.org> Message-ID: <20080514102410.639f16ef@mbook.local> On Wed, 14 May 2008 17:17:28 +1000 Peter Jeremy wrote: > On 2008-May-13 22:06:21 +0100, James Mansion wrote: > >And is the objection to SQL such the sqlite is really out of the running? > > There is no specific objection to SQL. There is a general objection > to adding more utilities to the base system unless a _very_ good case > can be made for including them. So far, no-one has made a compelling > reason to include SQLite (or other SQL engine) into the base system. Just out of curiosity - there seems to be an unspoken assumption that the ports system can only use tools that are part of the base system. But this is clearly false - the ports system currently includes a couple of directories full of software that's not in the base system. Adding compiled code to those tools would mean that installing the ports system gets a bit more complex - you have to run "make install" after extracting the tarball. Is that so bad it's not going to happen? References: <20080514120011.9131F10656C4@hub.freebsd.org> Message-ID: <482AFC9A.7050807@hesiod.org> Some years ago I mailed patches out to someone regarding Berkeley DB 1.85 btree problems. The two issues which come to mind are... 1) The page split position is improperly computed. This can cause corruption when a very full page has an item which is very large inserted onto it. The split then happens but because the split isn't in the right place the inserted item will not fit and I think it overruns where it should on the page. (I think 1.86 tried to fix this, but the fix is about recovering late i.e. when it sees an item doesn't fit, the proper fix is to split the page right in the first place.) 2) The record put code has a "last page put to" member which speeds up sequential inserts, however if that last page put to gets deleted, that variable is not cleared causing, if you are unlucky, a put might try to put data onto a deleted page which is then woven into both the tree and the free page list. This causes terrible problems with record scanning and eventual corruption. I'm going to have to dig up these fixes, but presuming I do, who should be alerted? Just file a bug? Recreation is extremely difficult. Jeff Anton From fabonacci at yahoo.com Wed May 14 15:34:44 2008 From: fabonacci at yahoo.com (John Timony) Date: Wed May 14 15:34:49 2008 Subject: A prob to Installing FreeBSD7.0 in VMWare 5.0 Message-ID: <163189.78306.qm@web46306.mail.sp1.yahoo.com> BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; =FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}Hi,I use the CD-ROM to =install FreeBSD7.O=20in VMWare 5.0,But when i start the Virtual Mathine,I get that ="elf32_loadimage=20failed"... By the way,,,I use a =FreeBSD7.0.iso=20mounted in CD-ROM... Any ideas? BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}Hi,I use the CD-ROM to install FreeBSD7.O in VMWare 5.0,But when i start the Virtual Mathine,I get that "elf32_loadimage failed"... By the way,,,I use a FreeBSD7.0.iso mounted in CD-ROM... Any ideas? From kientzle at freebsd.org Wed May 14 15:48:53 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Wed May 14 15:48:58 2008 Subject: BDB corrupt - patches In-Reply-To: <482AFC9A.7050807@hesiod.org> References: <20080514120011.9131F10656C4@hub.freebsd.org> <482AFC9A.7050807@hesiod.org> Message-ID: <482B09E5.5010302@freebsd.org> Jeff Anton wrote: > Some years ago I mailed patches out to someone regarding > Berkeley DB 1.85 btree problems. The two issues which come > to mind are... > > 1) The page split position is improperly computed. ... > 2) The record put code has a "last page put to" member ... > > I'm going to have to dig up these fixes, but presuming > I do, who should be alerted? Just file a bug? Recreation > is extremely difficult. Definitely file a PR and include the patches. If nothing else, that will help ensure they don't get lost. When you do file that PR, notify this list. Given how heavily the base system uses DB 1.85, I think there will be a lot of interest in anything that improves the stability and reliability of that code. Cheers, Tim Kientzle From delphij at delphij.net Wed May 14 17:51:56 2008 From: delphij at delphij.net (Xin LI) Date: Wed May 14 17:52:03 2008 Subject: BDB corrupt - patches In-Reply-To: <482AFC9A.7050807@hesiod.org> References: <20080514120011.9131F10656C4@hub.freebsd.org> <482AFC9A.7050807@hesiod.org> Message-ID: <482B26AF.3050302@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Anton wrote: [...] | I'm going to have to dig up these fixes, but presuming | I do, who should be alerted? Just file a bug? Recreation | is extremely difficult. I think Oracle is maintaining a webpage about their known bug/fixes here: http://www.oracle.com/technology/software/products/berkeley-db/db/index.html FreeBSD has applied a number of fixes IIRC, if it was a BerkeleyDB bug I'd suggest that you also give Oracle developers a headsup. Cheers, - -- ** Help China's quake relief at http://www.redcross.org.cn/ |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkgrJq4ACgkQi+vbBBjt66CB3gCfQ/hhQXVsyItDVmbP/j4lbI4v 8k8AnRgqtHZeg7lw9aacSeXNb8uYo9Ae =dWBS -----END PGP SIGNATURE----- From hartzell at alerce.com Wed May 14 18:03:58 2008 From: hartzell at alerce.com (George Hartzell) Date: Wed May 14 18:22:12 2008 Subject: BDB corrupt In-Reply-To: <20080514135051.GA21370@pix.net> References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514135051.GA21370@pix.net> Message-ID: <18475.9225.878567.877705@almost.alerce.com> Kurt J. Lidl writes: > On Wed, May 14, 2008 at 12:25:16AM -0700, Garrett Cooper wrote: > > Most of the complaints about other DBs is licensing related, but SQLite's > > complaint was also the fact that the past stability record was a bit rocky. > > One other thing to watch for in SQLite is the lack of atomicity > in updates. It's not ACID, just like BDB 1.8x isn't ACID. > > Without a write-ahead log, you cannot be sure that the data written > actually made it to stable storage, and as such, you cannot be sure > that your database didn't get corrupted when the process stops in a > non-optimal way. In what way is SQLite not atomic? The documentation, Atomic Commit In SQLite, suggests that it is: http://www.sqlite.org/atomiccommit.html I don't know that it supports fully ACID (atomic, consist, isolated, durable) transactions or how it handles the various SQL standard transaction isolation levels (Read Uncommitted, Read Committed, Repeatable Read, Serializable) but I believe that updates are atomic and that it does as well as any db (in the face of lying synch. operations, etc...) to handle "non-optimal" stops. g. From to.my.trociny at gmail.com Wed May 14 19:18:52 2008 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Wed May 14 19:18:55 2008 Subject: Socket leak In-Reply-To: <482AED3B.1020307@datapipe.com> (Mark Saad's message of "Wed\, 14 May 2008 09\:46\:35 -0400") References: <482A2639.7000401@datapipe.com> <81zlqtfazy.fsf@zhuzha.ua1> <482AED3B.1020307@datapipe.com> Message-ID: <81y76c7kyy.fsf@zhuzha.ua1> On Wed, 14 May 2008 09:46:35 -0400 Mark Saad wrote: MS> Mikolaj MS> Thanks for the input, did you change any of the options for MS> TimeoutLinger or TimeoutIdle ? No, I didn't MS> The Proftpd I am running is build for 6.3-RELEASE here are the build MS> options MS> Compile-time Settings: MS> Version: 1.3.0a MS> Platform: FREEBSD6 (FREEBSD6_3) MS> Built With: MS> configure CPPFLAGS=-DHAVE_OPENSSL --localstatedir=/var/run MS> --disable-sendfile --disable-ipv6 MS> --with-modules=mod_sql:mod_sql_mysql:mod_check_mysql:mod_check_digest MS> --prefix=/usr/local MS> --with-includes=/usr/local/include/mysql:/usr/include/openssl MS> --with-libraries=/usr/local/lib/mysql It might be that it is not proftpd but other application that cause the leak. Anyway, to check if it is proftpd, look in its logs for entries like these: Entering Passive Mode (192,168,0,213,241,70). FTP session closed. Convert the last two numbers to port (241*256+70) and check by netstat if you still have this connection. If you have, then it is likely this is the same situation as in my case and the proftpd is a problem. Upgrade to 1.3.1 from ports then. If proftpd is ok, look for other applications. Search for connections reported by netstat as ESTABLISHED but not displayed by sockstat utility. You could run something like this: netstat -an | grep ESTABL | while read b l a local remote state; do echo -n "$local $remote: " sockstat | sed -e 's/:/./g' | grep -c "$local *$remote" done Look for sockets with 0 count. These are suspicious ones. Observe these sockets by netstat and try to figure out what application they could belong and dig in that direction. -- Mikolaj Golub From peterjeremy at optushome.com.au Wed May 14 19:20:33 2008 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Wed May 14 19:20:36 2008 Subject: BDB corrupt In-Reply-To: <20080514135051.GA21370@pix.net> References: <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514135051.GA21370@pix.net> Message-ID: <20080514192026.GY64804@server.vk2pj.dyndns.org> On 2008-May-14 09:50:52 -0400, "Kurt J. Lidl" wrote: >One other thing to watch for in SQLite is the lack of atomicity >in updates. It's not ACID, just like BDB 1.8x isn't ACID. This isn't true. SQLite does provide full ACID. One difference from (eg) Oracle is that you need to explicitly begin a transaction, rather than a transaction implicitly commencing with the first DML statement. (I don't know what the SQL standard requires). >Without a write-ahead log, It does have a log to record incomplete transactions. > it doesn't give you referential integrity. To quote http://www.sqlite.org/omitted.html: "FOREIGN KEY constraints are parsed but are not enforced. However, the equivalent constraint enforcement can be achieved using triggers." >I looked (admitted, a while ago), it didn't even enforce column type >checking on tables. (Put this string in the INT column? No problem!) As noted, this is seen as a feature, not a bug. No-one is claiming SQLite is perfect. It does, however, provide a very usable SQL engine, that provides most of SQL92, with a very small footprint and a friendly license. If FreeBSD does need something with capabilities than BDB1.x, SQLite is an obvious contender. -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080514/8574fa78/attachment.pgp From peterjeremy at optushome.com.au Wed May 14 19:45:34 2008 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Wed May 14 19:45:39 2008 Subject: BDB corrupt In-Reply-To: <20080514102410.639f16ef@mbook.local> References: <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514071728.GP64804@server.vk2pj.dyndns.org> <20080514102410.639f16ef@mbook.local> Message-ID: <20080514194529.GB64804@server.vk2pj.dyndns.org> On 2008-May-14 10:24:10 -0400, Mike Meyer wrote: >Just out of curiosity - there seems to be an unspoken assumption that >the ports system can only use tools that are part of the base >system. There have been suggestions that the ports/package infrastructure (pkg_* tools, portsnap etc) be unbundled from the base OS. The difficulty comes when you want to upgrade those components. I know, from experience, that portugrading portupgrade or ruby usually fails as the running portupgrade unexpectedly trips over changed bits of itself. > But this is clearly false - the ports system currently >includes a couple of directories full of software that's not in the >base system. There is a directory full of Makefile includes and another directory full of optional tools but pkg_* sits in the base system. What are you alluding to here. >Adding compiled code to those tools would mean that installing the >ports system gets a bit more complex - you have to run "make install" >after extracting the tarball. Is that so bad it's not going to happen? The problem is not the initial install so much as managing packages and upgrades. I see no problem with having the ports/package infrastructure be part of the ports system as long as: a) A user can install/uninstall/audid (and preferably upgrade) packages without needing to compile anything b) The ports system knows how to upgrade itself without tripping over itself in the process. -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080514/07bdc4f1/attachment.pgp From reg at FreeBSD.ORG Wed May 14 20:04:03 2008 From: reg at FreeBSD.ORG (Jeremy Lea) Date: Wed May 14 20:04:07 2008 Subject: Adding .db support to pkg_tools In-Reply-To: <200805141755.35892.doconnor@gsoft.com.au> References: <20080512204558.GA44114@flint.openpave.org> <48291719.5010801@freebsd.org> <200805141755.35892.doconnor@gsoft.com.au> Message-ID: <20080514200353.GA77403@flint.openpave.org> Hi, On Wed, May 14, 2008 at 05:55:33PM +0930, Daniel O'Connor wrote: > So long as you can tell it is corrupted.. > It's also a drag from a user POV when the tool crashes because the DB is > hosed (seen in portupgrade a number of times) The simplest way to achieve some measure of protection (provided the database itself is not buggy), is to use a single lock file in /var/db/pkg. The ports tools can create and lock this as needed, and if the file exists, but they are able to obtain an exclusive lock on it, then it's a dangling lock. In that case, the tools crashed, and the database caches should be rebuilt. This is not fool proof, but it would reduce the number of FAQs on the mailing lists. This kills two birds with one stone, since it would serialize access to /var/db/pkg, so that various tools don't stomp on one another. An exclusive lock can be held by pkg_create, pkg_add and pkg_delete as needed. pkg_info and pkg_version can downgrade the lock to a shared lock, and just continue if they can get a shared lock, or block if the file is exclusively locked. One problem is the pkg_add calls itself recursively, so it would have to drop the lock around those calls. That could be hard. If I recall correctly, it was not that hard to remove this recursive behaviour. Also, since bsd.port.mk grubs around in /var/db/pkg, it would need to learn to participate in the locking. Regards, -Jeremy -- FreeBSD - Because the best things in life are free... http://www.freebsd.org/ From lidl at pix.net Wed May 14 21:02:45 2008 From: lidl at pix.net (Kurt J. Lidl) Date: Wed May 14 21:02:49 2008 Subject: BDB corrupt In-Reply-To: <20080514192026.GY64804@server.vk2pj.dyndns.org> References: <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514135051.GA21370@pix.net> <20080514192026.GY64804@server.vk2pj.dyndns.org> Message-ID: <20080514210240.GB28827@pix.net> On Thu, May 15, 2008 at 05:20:26AM +1000, Peter Jeremy wrote: > On 2008-May-14 09:50:52 -0400, "Kurt J. Lidl" wrote: > >One other thing to watch for in SQLite is the lack of atomicity > >in updates. It's not ACID, just like BDB 1.8x isn't ACID. > > This isn't true. SQLite does provide full ACID. One difference from > (eg) Oracle is that you need to explicitly begin a transaction, rather > than a transaction implicitly commencing with the first DML statement. > (I don't know what the SQL standard requires). Generally, you get either implicit transactions, or you need to put your database handle into explicit transaction mode, typically by bracketing your sql with: begin transaction; stuff; stuff; commit; if (error) { rollback; whine() } > >Without a write-ahead log, > > It does have a log to record incomplete transactions. Well, thanks for the various pointers. I see that it grew a transaction log since the last time I bothered to look at it in depth. That's a very good thing. I'll retract my assertion that it doesn't have a commit log. -Kurt From msaad at datapipe.com Wed May 14 21:03:39 2008 From: msaad at datapipe.com (Mark Saad) Date: Wed May 14 21:03:43 2008 Subject: Socket leak In-Reply-To: <81y76c7kyy.fsf@zhuzha.ua1> References: <482A2639.7000401@datapipe.com> <81zlqtfazy.fsf@zhuzha.ua1> <482AED3B.1020307@datapipe.com> <81y76c7kyy.fsf@zhuzha.ua1> Message-ID: <482B5364.7080406@datapipe.com> Mikolaj I looked at netstat and I do not have this many sockets TCP or UNIX. Wed May 14 16:58:37 EDT 2008 ewr# sysctl kern.ipc.numopensockets && netstat -an -p tcp | wc -l && sockstat -u |wc -l kern.ipc.numopensockets: 15903 261 60 ewr# sockstat -46lu | wc -l 82 Running your script I can only find 1 matching 0 count socket . I also shutdown proftpd and left it down for 10 mins and I did not see the number of sockets drop at all. Any ideas ? Mikolaj Golub wrote: > On Wed, 14 May 2008 09:46:35 -0400 Mark Saad wrote: > > MS> Mikolaj > MS> Thanks for the input, did you change any of the options for > MS> TimeoutLinger or TimeoutIdle ? > > No, I didn't > > MS> The Proftpd I am running is build for 6.3-RELEASE here are the build > MS> options > > MS> Compile-time Settings: > MS> Version: 1.3.0a > MS> Platform: FREEBSD6 (FREEBSD6_3) > MS> Built With: > MS> configure CPPFLAGS=-DHAVE_OPENSSL --localstatedir=/var/run > MS> --disable-sendfile --disable-ipv6 > MS> --with-modules=mod_sql:mod_sql_mysql:mod_check_mysql:mod_check_digest > MS> --prefix=/usr/local > MS> --with-includes=/usr/local/include/mysql:/usr/include/openssl > MS> --with-libraries=/usr/local/lib/mysql > > It might be that it is not proftpd but other application that cause the leak. > Anyway, to check if it is proftpd, look in its logs for entries like these: > > Entering Passive Mode (192,168,0,213,241,70). > FTP session closed. > > Convert the last two numbers to port (241*256+70) and check by netstat if you > still have this connection. If you have, then it is likely this is the same > situation as in my case and the proftpd is a problem. Upgrade to 1.3.1 from > ports then. > > If proftpd is ok, look for other applications. Search for connections reported > by netstat as ESTABLISHED but not displayed by sockstat utility. You could run > something like this: > > netstat -an | grep ESTABL | > while read b l a local remote state; do > echo -n "$local $remote: " > sockstat | > sed -e 's/:/./g' | > grep -c "$local *$remote" > done > > Look for sockets with 0 count. These are suspicious ones. Observe these > sockets by netstat and try to figure out what application they could belong > and dig in that direction. > > -- > Mikolaj Golub > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Mark Saad Managed UNIX Support DataPipe Managed Global IT Services msaad@datapipe.com 1.201.792.4847 (international) 1.888.749.5821 (toll free) () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/emaildisclaimer.aspx for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. From mwm-keyword-freebsdhackers2.e313df at mired.org Wed May 14 21:33:09 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Wed May 14 21:33:13 2008 Subject: BDB corrupt In-Reply-To: <20080514194529.GB64804@server.vk2pj.dyndns.org> References: <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> <20080513154137.GA28842@pix.net> <482A02CD.7040308@mansionfamily.plus.com> <20080514071728.GP64804@server.vk2pj.dyndns.org> <20080514102410.639f16ef@mbook.local> <20080514194529.GB64804@server.vk2pj.dyndns.org> Message-ID: <20080514173334.11bbae5c@bhuda.mired.org> On Thu, 15 May 2008 05:45:29 +1000 Peter Jeremy wrote: > On 2008-May-14 10:24:10 -0400, Mike Meyer wrote: > >Just out of curiosity - there seems to be an unspoken assumption that > >the ports system can only use tools that are part of the base > >system. > There have been suggestions that the ports/package infrastructure > (pkg_* tools, portsnap etc) be unbundled from the base OS. The > difficulty comes when you want to upgrade those components. I know, > from experience, that portugrading portupgrade or ruby usually fails > as the running portupgrade unexpectedly trips over changed bits of > itself. Having the ports system depend on packages that are maintained via the ports system does make things fairly complex. On the other hand, unbundling can be done. OS X has N (for some N > 1) ports systems that run outside the base system. Notably, they don't treat the code for manipulating the ports system as a port, and provide distinct commands for updating an installed port, the ports, and the port system software. However, I don't know that that's necessary for what I asked. > > But this is clearly false - the ports system currently > >includes a couple of directories full of software that's not in the > >base system. > There is a directory full of Makefile includes and another directory > full of optional tools but pkg_* sits in the base system. What are > you alluding to here. I was thinking of those two directories. I wasn't thinking about the pkg_* tools, because I pretty much never use them. > >Adding compiled code to those tools would mean that installing the > >ports system gets a bit more complex - you have to run "make install" > >after extracting the tarball. Is that so bad it's not going to happen? > The problem is not the initial install so much as managing packages > and upgrades. I see no problem with having the ports/package > infrastructure be part of the ports system as long as: > a) A user can install/uninstall/audid (and preferably upgrade) > packages without needing to compile anything > b) The ports system knows how to upgrade itself without tripping over > itself in the process. You could do what I suggested by adding the desired db library to /usr/src/usr.sbin/pkg_install, and linking it statically into the pkg* tools? Yeah, it's not really pretty, but is it any worse than having a BDB in the base system that we recommend be avoided? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From avg at icyb.net.ua Wed May 14 21:47:54 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed May 14 21:47:57 2008 Subject: using git for freebsd development Message-ID: <482B5E08.9000309@icyb.net.ua> I finally found some time and inspiration to write about some things I did about a month ago. I decided to use GIT for local FreeBSD development and I looked for the tools to do initial conversion of FreeBSD CVS src repository to GIT (I wanted to get as much history as possible) and also to do one-way sync-ing from CVS to GIT for subsequent updates. Please note that I just wanted to achieve my goal, I didn't attempt to do objective comparisons, benchmarking, etc. So whatever performance data I give below are very imprecise. The following can be considered as a followup to the excellent FreeBSD/GIT wiki page: http://wiki.freebsd.org/GitConversion So my first task was to do the initial conversion. My research showed that there were the following most popular options: git-cvsimport which is a part of GIT suite parsecvs http://gitweb.freedesktop.org/?p=users/keithp/parsecvs.git fromcvs/togit http://www.selenic.com/mercurial/wiki/index.cgi/fromcvs tailor http://progetti.arstecnica.it/tailor/browser/README.rst All of the tools either required source CVS repository to be available locally or worked much faster in that case, so the first thing to do was to get src-all from my local cvsup mirror. Easy. The first tool I tried was cvsimport because it came with git. It failed. After working for a short while it went into infinite loop on some file, it first complained that version X is before Y on branch Z and then that version Y is before X on the same branch and so on and on. Moving away that file didn't help as there were more troublesome ones. Maybe it had to do with repocopying. Next one was parsecvs. I think that this is the best one for initial import. It worked for about 6 hours on a very old machine: 512MB RAM, 450 MHz Pentium III. Resulting GIT repo took about 8G of space. Subsequent git repack took about 12 hours and reduced the size to ~500M. Quite nice. I should note that during the whole process of conversion parsecvs did not use more than 300M of RAM, this is by far the most conservative of all the tools that I tried (and that worked). There were some warning messages during conversion. Unfortunately parsecvs does not provide any option for keyword handling control and it doesn't expand any keywords. There are reasons to prefer this behavior, but personally I would prefer them to be expanded. I think that this is something that should be very easy to tweak in parsecvs source code. Also, quite unfortunately, parsecvs can only do full repository conversion and doesn't support incremental import. Because of the above, although I already had a converted git FreeBSD repo, I decided to give a try to some other tools - thinking that maybe using the same tool for both tasks would be somehow better. Thus I tried fromcvs/togit. It required me to install couple of ruby packages available via ports and two custom ones - rcsparse and fromcvs. It was quite easy to setup and run. This time I executed conversion on a modern system with Athlon XP 4800+ two core processor and 2GB of RAM. And that was needed - fromcvs worked for about two hours, peak memory usage was around 1.5G. There were some warning messages during conversion. Unfortunately, detailed examination showed that there were some issues with the conversion. Some files that were never changed on some branches in CVS were not to be found on the corresponding GIT branches. What's strange is that when I tried to convert only sys/ subdirectory everything went very well, no issues. Only on the complete src repository this problem did happen. Author of fromcvs (Simon 'corecode' Schubert) is aware of the issue and encountered it himself, so I hope it will be resolved soon. But so far no go. Then I decided to try tailor. I must admit that I had some difficulties understanding its documentation and that's probably the cause of what happened next. I provided what I thought were good options to tailor and it generated its config file. Then I executed it with the config, it worked for about two hours being the most resource hungry of everything I tried - using swap on the mentioned 2GB machine. Then it produced some error that looked like a complaint about configuration problem and then I gave up. Summary: only parsecvs worked good enough for me. Part two, doing incremental updates. I updated my copy of FreeBSD CVS repository with cvsup and proceeded. BTW, csup supported only checkout mode, so it could not be used instead of cvsup. By tradition I tried git-cvsimport first. It went into infinite loop again (maybe not infinite, but too long for me). This time it didn't produce any errors, just consumed 100% CPU, didn't make any system calls at all (ktrace to witness). I waited for about 3 hours (on the modern machine). parsecvs, as I said before, doesn't do incremental imports. tailor, on it I gave up. So I finally tried fromcvs and it worked, and it worked fast and it worked good. At least, so far I do not see any issues in incremental updates that it performs. So my conclusion is that at this time parsecvs is the best tool for initial import and fromcvs is the best tool for incremental imports. One small quirk is that parsecvs imported keywords unexpanded, but fromcvs expands them in incremental updates. Another small quirk: couple of commit messages in CVS contain extended Latin symbols from ISO8859-1. It seems that parsecvs copied them as is to GIT log history, I think they should have been converted to UTF-8. E.g. "H?rnquist ?strand" in history of Makefile.inc1 As a concluding word: I decided to clone the converted repository and to create my topic and "integration" and other branches in the cloned repository. Some of the branches are tracking branches, so that it is easy, for example, to synchronize my RELENG_7-specific changes with what is going on in CVS. So I first get CVS updates with cvsup. Then do fromcvs incremental import into "pristine" GIT repository. And then do 'git pull' into the working GIT repo. Hope that this will be interesting and/or useful to the community. -- Andriy Gapon From murray at FreeBSD.org Wed May 14 22:18:08 2008 From: murray at FreeBSD.org (Murray Stokely) Date: Wed May 14 22:18:34 2008 Subject: A prob to Installing FreeBSD7.0 in VMWare 5.0 In-Reply-To: <163189.78306.qm@web46306.mail.sp1.yahoo.com> References: <163189.78306.qm@web46306.mail.sp1.yahoo.com> Message-ID: <2a7894eb0805141518k1c50be3yf517afc3a159b11e@mail.gmail.com> On Wed, May 14, 2008 at 8:21 AM, John Timony wrote: > BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}Hi,I use the CD-ROM to install FreeBSD7.O in VMWare 5.0,But when i start the Virtual Mathine,I get that "elf32_loadimage failed"... > > By the way,,,I use a FreeBSD7.0.iso mounted in CD-ROM... Are you installing a 32-bit (i386) FreeBSD 7.0 iso on a 32-bit virtual machine? Or an amd64 ISO on a 64-bit virtual machine? I would make sure you are using the right ISO for the virtual machine setup you are using. VMWare Fusion on a Mac at least purports to support both 32bit and 64bit FreeBSD. I had no problems installing Ivan Voras's latest FreeBSD 7.0 + finstall on a 32bit virtual machine last night. - Murray From edelkind-freebsd-hackers at episec.com Wed May 14 23:39:39 2008 From: edelkind-freebsd-hackers at episec.com (ari edelkind) Date: Wed May 14 23:39:43 2008 Subject: using git for freebsd development In-Reply-To: <482B5E08.9000309@icyb.net.ua> References: <482B5E08.9000309@icyb.net.ua> Message-ID: <20080514231258.GB79355@episec.com> avg@icyb.net.ua wrote: > The following can be considered as a followup to the excellent > FreeBSD/GIT wiki page: > http://wiki.freebsd.org/GitConversion > [...] > All of the tools either required source CVS repository to be available > locally or worked much faster in that case, so the first thing to do was > to get src-all from my local cvsup mirror. Easy. Everyone who tries converting freebsd's cvs repository to joe-random-RCS attempts to import the entire source tree as a single project. Honestly, it covers too broad a spectrum. Separate these projects into the cvsup collections: src-sys, src-bin, src-lib, etc.. You won't be able to simply tag a single branch into a full system release without a wrapper script to handle your collections, but that's a small price to pay for the added robustness, separation of privilege, and smaller-scale potential for conflict. In fact, even src-bin may be too broad, and it may make sense to have separate projects within the collection hierarchy. This would, at least, make reparenting projects (say, from src-bin to src-usrbin) easier. ari From sbruno at miralink.com Thu May 15 02:21:33 2008 From: sbruno at miralink.com (Sean Bruno) Date: Thu May 15 02:21:36 2008 Subject: USB Install Method Message-ID: <482B96F5.8030800@miralink.com> Does anyone have a pointer to a FreeBSD 7 install method for those of us without CD/DVD Rom drives? The easiest way for me would be to boot off of a USB stick and then install across the network, but I'm open to suggestions. Sean From toyj at union.edu Thu May 15 06:07:12 2008 From: toyj at union.edu (james toy) Date: Thu May 15 06:07:18 2008 Subject: USB Install Method In-Reply-To: <482B96F5.8030800@miralink.com> References: <482B96F5.8030800@miralink.com> Message-ID: <7bb8f98f0805142240x344bad54s7ef671b4d33832c2@mail.gmail.com> Sean, How about a net-boot? That would be my next logical suggestion -- as far as I'm concerned its relatively hard to install via USB key at least i'm not aware of any current ways to do so. respectfully, jt On Wed, May 14, 2008 at 9:50 PM, Sean Bruno wrote: > Does anyone have a pointer to a FreeBSD 7 install method for those of us > without CD/DVD Rom drives? > > The easiest way for me would be to boot off of a USB stick and then install > across the network, but I'm open to suggestions. > > Sean > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From doconnor at gsoft.com.au Thu May 15 08:52:34 2008 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Thu May 15 08:52:42 2008 Subject: USB Install Method In-Reply-To: <482B96F5.8030800@miralink.com> References: <482B96F5.8030800@miralink.com> Message-ID: <200805151822.43236.doconnor@gsoft.com.au> On Thu, 15 May 2008, Sean Bruno wrote: > Does anyone have a pointer to a FreeBSD 7 install method for those of > us without CD/DVD Rom drives? > > The easiest way for me would be to boot off of a USB stick and then > install across the network, but I'm open to suggestions. The simplest way would be to get a USB CDROM drive. Note that it is likely you will need a version of FreeBSD released after this commit.. http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/boot/i386/btx/btx/btx.S?rev=1.46 Unfortunately as of today it will only be in a -current snapshot :( eg ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/200805 You might be lucky and it will work but if not you will find the loader crashes (eg spins really fast writing register dumps to the screen..) It is fairly straight forward to create a USB key with FreeBSD on it but it requires a FreeBSD system to create it.. http://typo.submonkey.net/articles/2006/4/13/installing-freebsd-on-usb-stick-episode-2 (I haven't actually tried this but it looks OK - I have done it with FreesBIE though) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080515/0929f06e/attachment.pgp From teemu at rinta-aho.org Thu May 15 10:20:36 2008 From: teemu at rinta-aho.org (Teemu Rinta-aho) Date: Thu May 15 10:20:48 2008 Subject: copy-on-write anonymous memory? Message-ID: <482C0E70.4020305@rinta-aho.org> Hi all, is it possible to create a memory object that represents anonymous memory pages *and* is copy-on-write? I have this code in a kernel module: object = vm_object_allocate(OBJT_DEFAULT, 1); result = vm_map_find(vmmap_proc, object, 0, &addr, len, TRUE, VM_PROT_ALL, VM_PROT_ALL, MAP_COPY_ON_WRITE); Then I pass the addr to the user space, but when I write to the addr, I see no shadow objects created, i.e. the changes are written to the original memory pages no matter if I have the map entry set as copy-on-write or not... I am assuming a write fault would create a new page and hang it to a shadow object thus leaving the original memory untouched. I'd appreciate any kind of help here. Best regards, Teemu Rinta-aho From kostikbel at gmail.com Thu May 15 11:18:56 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Thu May 15 11:19:06 2008 Subject: copy-on-write anonymous memory? In-Reply-To: <482C0E70.4020305@rinta-aho.org> References: <482C0E70.4020305@rinta-aho.org> Message-ID: <20080515111847.GX18958@deviant.kiev.zoral.com.ua> On Thu, May 15, 2008 at 01:20:32PM +0300, Teemu Rinta-aho wrote: > Hi all, > > is it possible to create a memory object that represents > anonymous memory pages *and* is copy-on-write? > > I have this code in a kernel module: > > object = vm_object_allocate(OBJT_DEFAULT, 1); > > result = vm_map_find(vmmap_proc, > object, > 0, > &addr, > len, > TRUE, > VM_PROT_ALL, > VM_PROT_ALL, > MAP_COPY_ON_WRITE); > > Then I pass the addr to the user space, but when > I write to the addr, I see no shadow objects created, > i.e. the changes are written to the original memory > pages no matter if I have the map entry set as > copy-on-write or not... I am assuming a write fault would > create a new page and hang it to a shadow object thus > leaving the original memory untouched. > > I'd appreciate any kind of help here. I cannot get a complete handle on your problem without full code, but I guess that you have only one reference to the backing object through the vm map entry. If this is the case, then the shadow copying actually does not make sense since no other users of the object need it. Look at the vm_object_shadow(), check at the start of the function. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080515/26d0fc41/attachment.pgp From teemu at rinta-aho.org Thu May 15 12:33:02 2008 From: teemu at rinta-aho.org (Teemu Rinta-aho) Date: Thu May 15 12:33:13 2008 Subject: copy-on-write anonymous memory? In-Reply-To: <20080515111847.GX18958@deviant.kiev.zoral.com.ua> References: <482C0E70.4020305@rinta-aho.org> <20080515111847.GX18958@deviant.kiev.zoral.com.ua> Message-ID: <482C2D7C.5070703@rinta-aho.org> Kostik Belousov wrote: > On Thu, May 15, 2008 at 01:20:32PM +0300, Teemu Rinta-aho wrote: >> Hi all, >> >> is it possible to create a memory object that represents >> anonymous memory pages *and* is copy-on-write? >> >> I have this code in a kernel module: >> >> object = vm_object_allocate(OBJT_DEFAULT, 1); >> >> result = vm_map_find(vmmap_proc, >> object, >> 0, >> &addr, >> len, >> TRUE, >> VM_PROT_ALL, >> VM_PROT_ALL, >> MAP_COPY_ON_WRITE); >> >> Then I pass the addr to the user space, but when >> I write to the addr, I see no shadow objects created, >> i.e. the changes are written to the original memory >> pages no matter if I have the map entry set as >> copy-on-write or not... I am assuming a write fault would >> create a new page and hang it to a shadow object thus >> leaving the original memory untouched. >> >> I'd appreciate any kind of help here. > > I cannot get a complete handle on your problem without full code, but > I guess that you have only one reference to the backing object through > the vm map entry. If this is the case, then the shadow copying actually > does not make sense since no other users of the object need it. > > Look at the vm_object_shadow(), check at the start of the function. Hi Kostik, thanks, that was a good point. Adding a reference helps! :-) Thanks for your quick reply, Teemu From teemu at rinta-aho.org Thu May 15 14:51:32 2008 From: teemu at rinta-aho.org (Teemu Rinta-aho) Date: Thu May 15 14:51:45 2008 Subject: copy-on-write anonymous memory? In-Reply-To: <482C2D7C.5070703@rinta-aho.org> References: <482C0E70.4020305@rinta-aho.org> <20080515111847.GX18958@deviant.kiev.zoral.com.ua> <482C2D7C.5070703@rinta-aho.org> Message-ID: <482C4DF3.8030709@rinta-aho.org> Teemu Rinta-aho wrote: > Kostik Belousov wrote: >> On Thu, May 15, 2008 at 01:20:32PM +0300, Teemu Rinta-aho wrote: >>> Hi all, >>> >>> is it possible to create a memory object that represents >>> anonymous memory pages *and* is copy-on-write? >>> >>> I have this code in a kernel module: >>> >>> object = vm_object_allocate(OBJT_DEFAULT, 1); >>> >>> result = vm_map_find(vmmap_proc, >>> object, >>> 0, >>> &addr, >>> len, >>> TRUE, >>> VM_PROT_ALL, >>> VM_PROT_ALL, >>> MAP_COPY_ON_WRITE); >>> >>> Then I pass the addr to the user space, but when >>> I write to the addr, I see no shadow objects created, >>> i.e. the changes are written to the original memory >>> pages no matter if I have the map entry set as >>> copy-on-write or not... I am assuming a write fault would >>> create a new page and hang it to a shadow object thus >>> leaving the original memory untouched. >>> >>> I'd appreciate any kind of help here. >> I cannot get a complete handle on your problem without full code, but >> I guess that you have only one reference to the backing object through >> the vm map entry. If this is the case, then the shadow copying actually >> does not make sense since no other users of the object need it. >> >> Look at the vm_object_shadow(), check at the start of the function. > > Hi Kostik, > > thanks, that was a good point. Adding a reference helps! :-) > > Thanks for your quick reply, > > Teemu New problems... Let me first describe what I try to do: I have created a kernel module that stores references to memory objects. I.e. when a process makes a syscall to the module, it will create a snapshot of the memory area, and after that the writes from the process to that memory area should create a shadow object. The next syscall should again store a pointer to the current topmost shadow object and then the next write creates yet another shadow object. Etc... When the snapshots are removed, the shadow chains may collapse normally. That's what I have assumed. Here's an illustration of what I want (first syscall OK, second one not): * * Legend: U/u = userspace K/k = kernel * * U:vm_map_entry_u -> object * * || * SYSCALL * || * \/ * * U:vm_map_entry_u -> object_shadow -> object * / * K:vm_map_entry_k ---------------- * * || * SYSCALL * || * \/ * * U:vm_map_entry_u -> object_shadow -> object_shadow -> object * / / * K:vm_map_entry_k ---------------- / * K:vm_map_entry_k -------------------------------- Now, the problem is that the first snapshot works as it should. However, the second one doesn't, and the write goes to the one and same shadow object, even if I restore MAP_ENTRY_COW and MAP_ENTRY_NEEDS_COPY manually in my handler function which is storing the snapshot. Any ideas? Teemu From jhs at berklix.org Thu May 15 18:19:06 2008 From: jhs at berklix.org (Julian Stacey) Date: Thu May 15 18:19:11 2008 Subject: Not seeing enough memory though its there. Message-ID: <200805151824.m4FIOFBm000937@fire.js.berklix.net> Hi Hackers cc Gary J, On a Compaq Pressario 2254 with 7.0-Release & Generic kernel, I'm not seeing enough RAM: A 128M DIM reports as 56M, a 256M reports as 96M. Any ideas please ? dmesg FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD-K6tm w/ multimedia extensions (233.03-MHz 586-class CPU) Origin = "AuthenticAMD" Id = 0x562 Stepping = 2 Features=0x8001bf AMD Features=0x400<> real memory = 100663296 (96 MB) avail memory = 84484096 (80 MB) pnpbios: Bad PnP BIOS data checksum # Dont know why bad With a 128M RAM Without hw.physmem= in /boot/loader.conf dmesg real memory = 58720256 (56 MB) avail memory = 43507712 (41 MB) sysctl -a | grep mem hw.physmem: 45572096 hw.usermem: 33267712 hw.realmem: 58720256 With /boot/loader.conf hw.physmem=134217728 same as above except hw.usermem: 33312768 With a 256 RAM power on Bios auto resizes to about 96M ! Without hw.physmem= dmesg real memory = 100663296 (96 MB) avail memory = 84484096 (80 MB) sysctl -a | grep mem hw.physmem: 87515136 hw.usermem: 74280960 hw.realmem: 100663296 With /boot/loader.conf hw.physmem=268435456 same as above except hw.usermem: 74141696 This Compaq has just 1 RAM slot. The RAM DIMs used: 128M RAM (original far as I know from PC in Germany) Module Paper Label: 128MB Sync 133 MHz CL3; 168 pin DIM, 13.3 x 3 cm, Slots: 1 in middle, + 1 @ 2.45 mm from end with pins 1 & 85 2 sides x 8 chips: Ramos 0131, RC28S0801B1-13AC 256M borrowed from another PC bought in UK (with 2 completely different RAM sources, unlikey both might be accidentaly or deliberately mislabeled (as occasionals occurs re CT mag article on cache chips some years back) Module Paper Label: IRL DEM DP133 064323E PC133 333 520 256MB 0137-2328-062[56] 2 sides x 8 chips: Hynix 132A, HY57V2882DAT-H I did check hw.physmem can set (reduce at least) in 7.0 /boot/loader.conf, on another laptop, after a reboot, from 512M to 256M). Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail just Ascii plain text. HTML & Base64 text are spam. From jbucht at gmail.com Thu May 15 18:40:59 2008 From: jbucht at gmail.com (Johan Bucht) Date: Thu May 15 18:41:02 2008 Subject: USB Install Method In-Reply-To: <482B96F5.8030800@miralink.com> References: <482B96F5.8030800@miralink.com> Message-ID: <947010c30805151114w73ecbb95r52f078fd1748ea69@mail.gmail.com> On Thu, May 15, 2008 at 3:50 AM, Sean Bruno wrote: > Does anyone have a pointer to a FreeBSD 7 install method for those of us > without CD/DVD Rom drives? > > The easiest way for me would be to boot off of a USB stick and then install > across the network, but I'm open to suggestions. > Try Unetbootin, I used it to install FreeBSD on my fit-PC from a USB stick, went pretty smooth. /Johan From ap00 at mail.ru Thu May 15 18:49:52 2008 From: ap00 at mail.ru (Anthony Pankov) Date: Thu May 15 18:49:56 2008 Subject: USB Install Method In-Reply-To: <482B96F5.8030800@miralink.com> References: <482B96F5.8030800@miralink.com> Message-ID: <21866640.20080515225217@mail.ru> Try the following http://frenzy.org.ua/en/ http://frenzy.org.ua/en/releases/1.0/doc/doc_en-install2flash.html Thursday, May 15, 2008, 5:50:45 AM, you wrote: SB> Does anyone have a pointer to a FreeBSD 7 install method for those of us SB> without CD/DVD Rom drives? SB> The easiest way for me would be to boot off of a USB stick and then SB> install across the network, but I'm open to suggestions. SB> Sean SB> _______________________________________________ SB> freebsd-hackers@freebsd.org mailing list SB> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers SB> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Best regards, Anthony mailto:ap00@mail.ru From stefan at fafoe.narf.at Fri May 16 08:00:29 2008 From: stefan at fafoe.narf.at (Stefan Farfeleder) Date: Fri May 16 08:00:34 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080309152712.42752293@bhuda.mired.org> References: <20080309152712.42752293@bhuda.mired.org> Message-ID: <20080516074433.GA1032@lizard.fafoe.narf.at> On Sun, Mar 09, 2008 at 03:27:12PM -0400, Mike Meyer wrote: > I've stumbled on to an obscure problem with autoconf 2.61, and I'm not > sure quite what to do with it. I've already sent mail to the autoconf > folks, but I'd like to understand what's going on. > > The problem is that, on a FreeBSD system with only /bin/sh and the > ports zsh as installed shells, if you have SHELL set to zsh when > invoking the autoconf-generated configure script, the script produces > a broken Makefile. It doesn't generate an error, it just complains > that: Can you please retry? /bin/sh now supports expanding $LINENO which was often the reason for configure not liking it. Stefan From brd at FreeBSD.org Fri May 16 13:43:51 2008 From: brd at FreeBSD.org (Brad Davis) Date: Fri May 16 13:44:02 2008 Subject: FreeBSD Status Reports for the First Quarter of 2008 Message-ID: <20080516134346.GB1844@valentine.liquidneon.com> Hi Everyone, The FreeBSD Status Reports for the First Quarter of 2008 are now available at: http://www.freebsd.org/news/status/report-2008-01-2008-03.html Regards, Brad Davis From mwm-keyword-freebsdhackers2.e313df at mired.org Fri May 16 14:11:20 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Fri May 16 14:11:58 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080516074433.GA1032@lizard.fafoe.narf.at> References: <20080309152712.42752293@bhuda.mired.org> <20080516074433.GA1032@lizard.fafoe.narf.at> Message-ID: <20080516101143.3545937c@bhuda.mired.org> On Fri, 16 May 2008 09:44:33 +0200 Stefan Farfeleder wrote: > On Sun, Mar 09, 2008 at 03:27:12PM -0400, Mike Meyer wrote: > > I've stumbled on to an obscure problem with autoconf 2.61, and I'm not > > sure quite what to do with it. I've already sent mail to the autoconf > > folks, but I'd like to understand what's going on. > > > > The problem is that, on a FreeBSD system with only /bin/sh and the > > ports zsh as installed shells, if you have SHELL set to zsh when > > invoking the autoconf-generated configure script, the script produces > > a broken Makefile. It doesn't generate an error, it just complains > > that: > > Can you please retry? /bin/sh now supports expanding $LINENO which was > often the reason for configure not liking it. Which branch, and how recently? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From stefan at fafoe.narf.at Fri May 16 14:13:32 2008 From: stefan at fafoe.narf.at (Stefan Farfeleder) Date: Fri May 16 14:13:38 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080516101143.3545937c@bhuda.mired.org> References: <20080309152712.42752293@bhuda.mired.org> <20080516074433.GA1032@lizard.fafoe.narf.at> <20080516101143.3545937c@bhuda.mired.org> Message-ID: <20080516141307.GB1032@lizard.fafoe.narf.at> On Fri, May 16, 2008 at 10:11:43AM -0400, Mike Meyer wrote: > On Fri, 16 May 2008 09:44:33 +0200 > Stefan Farfeleder wrote: > > > On Sun, Mar 09, 2008 at 03:27:12PM -0400, Mike Meyer wrote: > > > I've stumbled on to an obscure problem with autoconf 2.61, and I'm not > > > sure quite what to do with it. I've already sent mail to the autoconf > > > folks, but I'd like to understand what's going on. > > > > > > The problem is that, on a FreeBSD system with only /bin/sh and the > > > ports zsh as installed shells, if you have SHELL set to zsh when > > > invoking the autoconf-generated configure script, the script produces > > > a broken Makefile. It doesn't generate an error, it just complains > > > that: > > > > Can you please retry? /bin/sh now supports expanding $LINENO which was > > often the reason for configure not liking it. > > Which branch, and how recently? HEAD and yesterday. From avg at icyb.net.ua Fri May 16 14:37:32 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri May 16 14:37:37 2008 Subject: freebsd i386 and bios Message-ID: <482D9C28.8080805@icyb.net.ua> Does i386 FreeBSD depend for anything important on "traditional" BIOS or will it be able to load and work without it? I mean with something like coreboot. -- Andriy Gapon From teemu at rinta-aho.org Fri May 16 15:08:18 2008 From: teemu at rinta-aho.org (Teemu Rinta-aho) Date: Fri May 16 15:08:34 2008 Subject: copy-on-write anonymous memory? In-Reply-To: <200805161434.m4GEY6N3082155@casselton.net> References: <200805161434.m4GEY6N3082155@casselton.net> Message-ID: <482DA364.7000106@rinta-aho.org> Mark Tinguely wrote: > Teemu Rinta-aho wrote (some edits): > >> I have created a kernel module that stores references to memory objects. >> I.e. when a process makes a syscall to the module, it will create a >> snapshot of the memory area, and after that the writes from the process to >> that memory area should create a shadow object. The next syscall should >> again store a pointer to the current topmost shadow object and then the >> next write creates yet another shadow object. Etc... When the snapshots >> are removed, the shadow chains may collapse normally. >> >> Here's an illustration of what I want (first syscall OK, second one not): >> >> * Legend: U/u = userspace K/k = kernel >> * >> * U:vm_map_entry_u -> object >> * || >> * SYSCALL >> * || >> * \/ >> * U:vm_map_entry_u -> object_shadow -> object >> * / >> * K:vm_map_entry_k ---------------- >> * || >> * SYSCALL >> * || >> * \/ >> * U:vm_map_entry_u -> object_shadow -> object_shadow -> object >> * / / >> * K:vm_map_entry_k ---------------- / >> * K:vm_map_entry_k -------------------------------- >> >> Now, the problem is that the first snapshot works as it should. However, >> the second one doesn't, and the write goes to the one and same shadow >> object, even if I restore MAP_ENTRY_COW and MAP_ENTRY_NEEDS_COPY manually >> in my handler function which is storing the snapshot. >> >> Any ideas? > > Usually, a fork() creates the inheritance between parent and child COW memory > space. Start: > vm_map_entry -> object_shadow -> object > > fork(): > vm_map_entry -> object_shadow -\ > |-> (object_shadow*) -> object > vm_map_entry -> object_shadow -/ > > This is slightly different from your description/drawing, in the way changes > are inherited; for example: process 1 is created, process 1 writes page 0. > process 2 is created. process 1 writes p1 (or p0 again). Your > description/drawing implies that process 2 see this change from process 1. > > You are not forking over a COW memory area. Sounds like the syscall will have > manually create the inheritance. You can manually link the object_shadows > the way you want to get the desired inheritance. Process removals should > collapse the shadows automatically. > > Matt Dillion wrote a brief VM description > (http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/). > The book, "The Design and Implementation of the 4.4BSD Operating System" > is another great reference. > > Mark Tinguely Hi Mark, thanks a lot. I was just about to write to the list that I found the solution, to give future Googler's some help :-) The problem was that of course there is no page fault for every write to the same page. However, I can manually set the COW flag in the map entry, and then call pmap_protect() to set the physical page read only. Then the next write after the syscall will trigger the vm_map_lookup() and the creation of a shadow object. So, for now, this problem is solved. I'm about to head towards new problems now :-) Best regards, Teemu Rinta-aho P.S. Yes I have both the D&I of 4.4BSD and D&I of FreeBSD books, they're quite good and become understandable after browsing the code for a while :-) From jille at quis.cx Fri May 16 15:41:07 2008 From: jille at quis.cx (Jille Timmermans) Date: Fri May 16 15:41:12 2008 Subject: Trying (not) to crash with libpthread (6.3-RELEASE) Message-ID: <482DAB0E.70600@quis.cx> Hello, I'm trying to catch SIGSEGV, to be able to run 'unchecked' (possibly crashing) code. And not letting the entire program die, but just kill that thread. So I wrote some testing code. But, I ran into a problem; the signal-handler does not work when a thread is created (or something like that). The code below crashes on SIGSEGV, but should die in void sigcatcher(int); If you uncomment the crashingthread(NULL); line, it will die in the signal handler. pthread_create seems to take over signal handling. I quickly checked some pthread source, and it seems that it should call my own handler. (There might be an exception for some signals, but I didn't found that) Can anyone explain me what is happening here ? -- Jille cc -lpthread below.c #include #include #include #include #include #include int success=0; void sigcatcher(int sig) { printf("[%p] signal %d\n", pthread_self(), sig); printf("Test (probably) succeeded\n"); fflush(NULL); success=1; exit(0); } void * crashingthread(void *nada) { /* This will likely crash */ char *x=malloc(1); if(signal(SIGSEGV, sigcatcher)==SIG_ERR) err(1, "signal(SIGSEGV, catchz0r)"); x[666]=0; /* HOPEFULLY NOT REACHED (aargh! die harder!) */ int i; for(i=1; 999999>i; i++) x[i]=0; /* NOT REACHED (either killed, or exit()'ed in sigcatcher) */ abort(); } int main(int argc, char **argv) { pthread_t thr; if(signal(SIGSEGV, sigcatcher)==SIG_ERR) err(1, "signal(SIGSEGV, catchz0r)"); //crashingthread(NULL); pthread_create(&thr, NULL, crashingthread, NULL); sleep(3); return 0; } From tinguely at casselton.net Fri May 16 14:48:43 2008 From: tinguely at casselton.net (Mark Tinguely) Date: Fri May 16 16:10:01 2008 Subject: copy-on-write anonymous memory? In-Reply-To: <482C4DF3.8030709@rinta-aho.org> Message-ID: <200805161434.m4GEY6N3082155@casselton.net> Teemu Rinta-aho wrote (some edits): > I have created a kernel module that stores references to memory objects. > I.e. when a process makes a syscall to the module, it will create a > snapshot of the memory area, and after that the writes from the process to > that memory area should create a shadow object. The next syscall should > again store a pointer to the current topmost shadow object and then the > next write creates yet another shadow object. Etc... When the snapshots > are removed, the shadow chains may collapse normally. > > Here's an illustration of what I want (first syscall OK, second one not): > > * Legend: U/u = userspace K/k = kernel > * > * U:vm_map_entry_u -> object > * || > * SYSCALL > * || > * \/ > * U:vm_map_entry_u -> object_shadow -> object > * / > * K:vm_map_entry_k ---------------- > * || > * SYSCALL > * || > * \/ > * U:vm_map_entry_u -> object_shadow -> object_shadow -> object > * / / > * K:vm_map_entry_k ---------------- / > * K:vm_map_entry_k -------------------------------- > > Now, the problem is that the first snapshot works as it should. However, > the second one doesn't, and the write goes to the one and same shadow > object, even if I restore MAP_ENTRY_COW and MAP_ENTRY_NEEDS_COPY manually > in my handler function which is storing the snapshot. > > Any ideas? Usually, a fork() creates the inheritance between parent and child COW memory space. Start: vm_map_entry -> object_shadow -> object fork(): vm_map_entry -> object_shadow -\ |-> (object_shadow*) -> object vm_map_entry -> object_shadow -/ This is slightly different from your description/drawing, in the way changes are inherited; for example: process 1 is created, process 1 writes page 0. process 2 is created. process 1 writes p1 (or p0 again). Your description/drawing implies that process 2 see this change from process 1. You are not forking over a COW memory area. Sounds like the syscall will have manually create the inheritance. You can manually link the object_shadows the way you want to get the desired inheritance. Process removals should collapse the shadows automatically. Matt Dillion wrote a brief VM description (http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/). The book, "The Design and Implementation of the 4.4BSD Operating System" is another great reference. Mark Tinguely From gaspar_i at yahoo.com Fri May 16 15:57:05 2008 From: gaspar_i at yahoo.com (n4ti1us) Date: Fri May 16 16:41:29 2008 Subject: Spin locks and soft IRQs Message-ID: <17277923.post@talk.nabble.com> Hi All, I was looking at the linux kernel, and in the implementation of spinlocks, one can use different functions. One of them is spin_lock_bh(). What it does is it creates a spin lock and disables the software irqs. In FreeBSD on the other hand, the implementation of spinlocks is done by mtx_lock_spin (which is actually a macro that calls mtx_lock_spin_flags()) but the problem is that is disables all the irq vector (at the bottom of the stack it executes an "cli"). Is there any way one can implement spinlocks that only disable the bottom half (sof irq)? Alternatively, what would be the best way to manualy disable the soft irqs? And all this makes me think... Why has nobody implemented a "_bh" version so far? Is this an architectural decision, or is it just me being blind? -- View this message in context: http://www.nabble.com/Spin-locks-and-soft-IRQs-tp17277923p17277923.html Sent from the freebsd-hackers mailing list archive at Nabble.com. From deathjestr at gmail.com Fri May 16 17:23:50 2008 From: deathjestr at gmail.com (Mike) Date: Fri May 16 17:23:53 2008 Subject: Trying (not) to crash with libpthread (6.3-RELEASE) In-Reply-To: <482DAB0E.70600@quis.cx> References: <482DAB0E.70600@quis.cx> Message-ID: <44b564930805160955r76d6861fo1949c7f4d5b71ca5@mail.gmail.com> > void > sigcatcher(int sig) { > printf("[%p] signal %d\n", pthread_self(), sig); > printf("Test (probably) succeeded\n"); > fflush(NULL); > success=1; > exit(0); > } Take a look at the man page for sigaction(2). Here's an excerpt: "All functions not in the above lists are considered to be unsafe with respect to signals. That is to say, the behaviour of such functions when called from a signal handler is undefined. In general though, signal handlers should do little more than set a flag; most other actions are not safe." > void * > crashingthread(void *nada) { > /* This will likely crash */ > char *x=malloc(1); > > if(signal(SIGSEGV, sigcatcher)==SIG_ERR) > err(1, "signal(SIGSEGV, catchz0r)"); > > x[666]=0; > > /* HOPEFULLY NOT REACHED (aargh! die harder!) */ > > int i; > for(i=1; 999999>i; i++) > x[i]=0; > > /* NOT REACHED (either killed, or exit()'ed in sigcatcher) */ > abort(); > } Something like *(int*)0 = 0 is probably an easier, more reliable way to crash your program. Maybe not quite as much fun though. ;-) From mark at foster.cc Fri May 16 18:18:35 2008 From: mark at foster.cc (Mark Foster) Date: Fri May 16 18:18:40 2008 Subject: [Fwd: lwresd howto] Message-ID: <482DCBEA.3030107@foster.cc> No response from -questions, trying here. The gist is: is implementing lwresd feasible as a caching-only stub resolver? I need something I can run locally to intercept and cache DNS responses. BIND is not the answer (too heavyweight) and dnsmasq doesn't appear to cache. nscd is what I'm used to on Linux but it doesn't seem to be in FreeBSD. Something called cached but I don't see it on the systems I'm working with (5.4, 6.1, 6.2 yea I know out-of-date so don't even say it). Can I use lwresd ? The documentation for how to do so are unclear. I've found: http://www.isc.org/sw/bind/arm93/Bv9ARM.ch05.html and reviewed the lwresd(8) man page but there is not a good example of how to setup correctly. Also the ISC doc describes putting lwserver entries into resolv.conf but these are not mentioned in resolver(5). I do not see the hooks around for nsswitch.conf e.g. to put lwres in the hosts: line would have no effect because something like nss_lwres does not exist for FreeBSD. -- Some days it's just not worth chewing through the restraints... Mark D. Foster, CISSP http://mark.foster.cc/ -------------- next part -------------- An embedded message was scrubbed... From: Mark Foster Subject: lwresd howto Date: Wed, 14 May 2008 13:46:24 -0700 Size: 1445 Url: http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080516/175fac6b/lwresdhowto.eml From mark at foster.cc Fri May 16 19:01:50 2008 From: mark at foster.cc (Mark Foster) Date: Fri May 16 19:01:54 2008 Subject: [Fwd: lwresd howto] In-Reply-To: <20080516143917.5246a733@bhuda.mired.org> References: <482DCBEA.3030107@foster.cc> <20080516143917.5246a733@bhuda.mired.org> Message-ID: <482DDA1E.2030208@foster.cc> Mike Meyer wrote: > What makes you think dnsmasq doesn't cache? The documentation says > otherwise (second paragraph of the man page): > > Dnsmasq accepts DNS queries and either answers them from a small, > local, cache or forwards them to a real, recursive, DNS server. It > > it has quite a few options to control caching behavior, and certainly > seems to cache for me. > > Thanks for pointing out my error. I have no problem using dnsmasq if that is the best recommendation, in fact I quite like it but am not fond of djbdns. However lwresd *is* in the base install vs. dnsmasq which is a port, so would like to use lwresd if possible (or something else in base like cached). There is nothing I find in the man page, handbook or google that says I shouldn't or can't use lwresd, only hints that I can, yet with no clear working real-world examples. So I'm really hoping for two answers. 1. Can lwresd be used (somehow) or not 2. What are the viable/best alternatives to #1 for a caching-only stub resolver (like nscd on linux) Thanks. -- Some days it's just not worth chewing through the restraints... Mark D. Foster, CISSP http://mark.foster.cc/ From olli at lurza.secnetix.de Fri May 16 19:14:17 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri May 16 19:14:20 2008 Subject: Trying (not) to crash with libpthread (6.3-RELEASE) In-Reply-To: <44b564930805160955r76d6861fo1949c7f4d5b71ca5@mail.gmail.com> Message-ID: <200805161914.m4GJEBSD053921@lurza.secnetix.de> Mike wrote: > Jille Timmermans wrote: > > void > > sigcatcher(int sig) { > > printf("[%p] signal %d\n", pthread_self(), sig); > > printf("Test (probably) succeeded\n"); > > fflush(NULL); > > success=1; > > exit(0); > > } > > Take a look at the man page for sigaction(2). Here's an excerpt: > > "All functions not in the above lists are considered to be unsafe with > respect to signals. That is to say, the behaviour of such functions when > called from a signal handler is undefined. In general though, signal > handlers should do little more than set a flag; most other actions are > not safe." Additionally, the "success" variable should be declared as "volatile sig_atomic_t". By the way, this code ... > > char *x=malloc(1); > > [...] > > x[666]=0; .. does not necessarily cause a SIGSEGV. It may do so, but it is not guaranteed. It may as well write into a random memory area that happens to belong to your process, too. The result is undefined, i.e. it could be anything. If you need to send a SIGSEGV to yourself reliably, the best way is to kill(getpid(), SIGSEGV). Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "File names are infinite in length, where infinity is set to 255 characters." -- Peter Collinson, "The Unix File System" From mwm-keyword-freebsdhackers2.e313df at mired.org Fri May 16 19:22:00 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Fri May 16 19:22:04 2008 Subject: [Fwd: lwresd howto] In-Reply-To: <482DDA1E.2030208@foster.cc> References: <482DCBEA.3030107@foster.cc> <20080516143917.5246a733@bhuda.mired.org> <482DDA1E.2030208@foster.cc> Message-ID: <20080516152225.75d97deb@bhuda.mired.org> On Fri, 16 May 2008 12:01:50 -0700 Mark Foster wrote: > Mike Meyer wrote: > > What makes you think dnsmasq doesn't cache? The documentation says > > otherwise (second paragraph of the man page): > > > > Dnsmasq accepts DNS queries and either answers them from a small, > > local, cache or forwards them to a real, recursive, DNS server. It > > > > it has quite a few options to control caching behavior, and certainly > > seems to cache for me. > > > > > Thanks for pointing out my error. > I have no problem using dnsmasq if that is the best recommendation, in > fact I quite like it but am not fond of djbdns. I wanted a cooperating dhcp & dns server, and having them in one package is to big a win to ignore. > However lwresd *is* in the base install vs. dnsmasq which is a port, so > would like to use lwresd if possible (or something else in base like > cached). There is nothing I find in the man page, handbook or google > that says I shouldn't or can't use lwresd, only hints that I can, yet > with no clear working real-world examples. > > So I'm really hoping for two answers. > 1. Can lwresd be used (somehow) or not lwresd isn't a dns server. I suspect it's in the base system because it comes bundled with bind, rather than because it's used itself, as there's no hook to start it in /etc/rc or /etc/defaults. > 2. What are the viable/best alternatives to #1 for a caching-only stub > resolver (like nscd on linux) dnsmasq is viable. May not be the best for your needs, but it works great for me. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From sr at genyosha.net Fri May 16 20:47:56 2008 From: sr at genyosha.net (Steve Rikli) Date: Fri May 16 20:48:00 2008 Subject: [Fwd: lwresd howto] In-Reply-To: <482DDA1E.2030208@foster.cc> References: <482DCBEA.3030107@foster.cc> <20080516143917.5246a733@bhuda.mired.org> <482DDA1E.2030208@foster.cc> Message-ID: <20080516201154.GA82576@dragon.genyosha.net> On Fri, May 16, 2008 at 12:01:50PM -0700, Mark Foster wrote: > Mike Meyer wrote: > ... > I have no problem using dnsmasq if that is the best recommendation, in > fact I quite like it but am not fond of djbdns. > However lwresd *is* in the base install vs. dnsmasq which is a port, so > would like to use lwresd if possible (or something else in base like > cached). There is nothing I find in the man page, handbook or google > that says I shouldn't or can't use lwresd, only hints that I can, yet > with no clear working real-world examples. > > So I'm really hoping for two answers. > 1. Can lwresd be used (somehow) or not > 2. What are the viable/best alternatives to #1 for a caching-only stub > resolver (like nscd on linux) Wrt #2, I was able to compile & install nscd from FreeBSD src, but only on systems running fairly modern releases -- IIRC no older than 6.2 . cheers, sr. From gonzo at pbxpress.com Fri May 16 22:22:33 2008 From: gonzo at pbxpress.com (Oleksandr Tymoshenko) Date: Fri May 16 22:22:38 2008 Subject: aio_write issues Message-ID: <482E034C.2080402@pbxpress.com> It looks like aio_write is out of sync with its man page(if it ever has been). man 2 aio_write states: If O_APPEND is set for iocb->aio_fildes, aio_write() operations append to the file in the same order as the calls were made. Though simple test[1] reveals that it's not true. Order is not preserved. In addition the rest of the paragraph contradicts to SUSv3 specification of aio_write (or I got that part of man page wrong, because function behaves just as should be conforming to SUSv3). man page: If O_APPEND is not set for the file descriptor, the write operation will occur at the absolute position from the beginning of the file plus iocb->aio_offset. excerpt from IEEE 1003.1: If O_APPEND is not set for the file descriptor aio_fildes, then the requested operation shall take place at the absolute position in the file as given by aio_offset, as if lseek() were called immediately prior to the operation with an offset equal to aio_offset and a whence equal to SEEK_SET. I'd like to know if I got things right in order to avoid fixing things that are not broken. [1] http://people.freebsd.org/~gonzo/aio.c -- gonzo From bilouro at bilouro.com Sat May 17 01:34:18 2008 From: bilouro at bilouro.com (Victor Hugo Bilouro) Date: Sat May 17 01:34:22 2008 Subject: GSoC Student Message-ID: Hi, I'm Victor Hugo *Bilouro*, GSoC participant. I'll be working on tcptest. It's name came from TCP/IP Regression test suite. As a testing tool it can: *be a conformance test *test a bug already fixed (regression test - one test for each fixed bug) *test a new feature (help protocol developer) *simulate certain cases It will be build on top of pcs.sf.net (Package Construction Set) "PCS is a set of Python modules and objects that make building network protocol code easier for the protocol developer. The core of the system is the pcs module itself which provides the necessary functionality to create classes that implement packets. " My mentor is George V. Neville-Neil, no comments. :-) GSoC started a month ago, by this time, I had almost all environment done, and I'm spending several times reading and often playing with pcs. (cool) Books i bought: (*)TCP/IP Illustrated Volume 1 (far the best) Stevens (*)TCP/IP Guide Charles M.Kazierok (www.tcpipguide.com) (*)Internetworking with tcp/ip I will post on __link__ links, books, status report, futures releases information, news, everything about the project. __link__: http://wiki.freebsd.org/VictorBilouro/TCP-IP_regression_test_suite Regards -- Victor Hugo Bilouro FreeBSD From julian at elischer.org Sat May 17 02:16:06 2008 From: julian at elischer.org (Julian Elischer) Date: Sat May 17 02:16:10 2008 Subject: Trying (not) to crash with libpthread (6.3-RELEASE) In-Reply-To: <482DAB0E.70600@quis.cx> References: <482DAB0E.70600@quis.cx> Message-ID: <482E3FE8.2000509@elischer.org> Jille Timmermans wrote: > Hello, > > I'm trying to catch SIGSEGV, to be able to run 'unchecked' (possibly > crashing) code. > And not letting the entire program die, but just kill that thread. > So I wrote some testing code. > But, I ran into a problem; the signal-handler does not work when a > thread is created (or something like that). > The code below crashes on SIGSEGV, but should die in void sigcatcher(int); > If you uncomment the crashingthread(NULL); line, it will die in the > signal handler. > > pthread_create seems to take over signal handling. > I quickly checked some pthread source, and it seems that it should call > my own handler. > (There might be an exception for some signals, but I didn't found that) > > Can anyone explain me what is happening here ? yes, you are correct. I a vain attempt to give something close to the semanticcs defined by the pthread spec, the pthreads app sets a special thread to do nothing but catch signals. Originally when we started the immediate signals such as SIGSEGV were exempt from this and were delivered to the thread that caused it immediatly, however it was impossible to do this effectively and still provide correct sematics for the rest of the signals. Eventually we just gave up on SIGSEGV because we just couldn't get it to work. We could catch the signal in the signal thread, but we couldn't get it to deliver it to the errant thread reliably, (because the errant thread couldn't run any more at that point). using top -H you should be able to see the extgra thread in the kernel waiting for the signals. This was one of the reasons that 1:1 threading eventually won out in our threading evaluation period. In M:N threading the aim is to switch threads without kernel involvement, however iti turns out that to do SIGSEGV and friends properly, you need to re-do the thread signal mask for every thread with a syscall for every thread swap which defeats teh whole point. With 1:1 threads this all "just works" try libthr. it should work for you. > > -- Jille > > > cc -lpthread below.c > #include > #include > #include > #include > #include > #include > > int success=0; > > void > sigcatcher(int sig) { > printf("[%p] signal %d\n", pthread_self(), sig); > printf("Test (probably) succeeded\n"); > fflush(NULL); > success=1; > exit(0); > } > > void * > crashingthread(void *nada) { > /* This will likely crash */ > char *x=malloc(1); > > if(signal(SIGSEGV, sigcatcher)==SIG_ERR) > err(1, "signal(SIGSEGV, catchz0r)"); > > x[666]=0; > > /* HOPEFULLY NOT REACHED (aargh! die harder!) */ > > int i; > for(i=1; 999999>i; i++) > x[i]=0; > > /* NOT REACHED (either killed, or exit()'ed in sigcatcher) */ > abort(); > } > > int > main(int argc, char **argv) { > pthread_t thr; > > if(signal(SIGSEGV, sigcatcher)==SIG_ERR) > err(1, "signal(SIGSEGV, catchz0r)"); > > //crashingthread(NULL); > pthread_create(&thr, NULL, crashingthread, NULL); > > sleep(3); > return 0; > } > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From avg at icyb.net.ua Sat May 17 08:13:54 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Sat May 17 08:13:57 2008 Subject: rdmsr from userspace Message-ID: <482E93C0.4070802@icyb.net.ua> It seems that rdmsr instruction can be executed only at the highest privilege level and thus is not permitted from userland. Maybe we should provide something like Linux /dev/cpu/msr? I don't like interface of that device, I think that ioctl approach would be preferable in this case. Something like create /dev/cpuN and allow some ioctls on it: ioctl(cpu_fd, CPU_RDMSR, arg). What do you think? -- Andriy Gapon From jeremie at le-hen.org Sat May 17 09:20:35 2008 From: jeremie at le-hen.org (Jeremie Le Hen) Date: Sat May 17 09:20:42 2008 Subject: Debugging rtld Message-ID: <20080517091740.GI70896@obiwan.tataz.chchile.org> Hi there, I tried to compile my source tree with -fstack-protector-all, and it happens that rtld breaks with this: once the new rtld is installed every single problem coredumps. I tried to compile rtld-elf without SSP, but it didn't solve the problem. Then I had to compile libc_pic.a without SSP and it worked, but I don't understand the root of the problem. So I want to use the generated coredump for post-mortem analysis with gdb. I compiled world with DEBUG_FLAGS=-g. But GDB gives me a backtrace so long that it can't be real. Moreoever it doesn't seem to bring in the required symbols. I'm a GDB novice, so I'd like some help. chroot> ===> libexec/rtld-elf (install) chroot> chflags noschg /usr/libexec/ld-elf.so.1 chroot> install -s -o root -g wheel -m 555 -C -b -fschg -S ld-elf.so.1 /libexec chroot> install -o root -g wheel -m 444 rtld.1.gz /usr/share/man/man1 chroot> *** Signal 11 chroot> chroot> jarjarbinks# cd /root; ls chroot> Segmentation fault host> jarjarbinks:145# ls -l /space/chroot/root/ls.core host> -rw------- 1 root wheel 184320 May 17 10:19 /space/chroot/root/ls.core host> jarjarbinks:149# gdb -c /space/chroot/root/ls.core -e /space/chroot/bin/ls host> GNU gdb 6.1.1 [FreeBSD] host> [...] host> This GDB was configured as "i386-marcel-freebsd". host> Core was generated by `ls'. host> Program terminated with signal 11, Segmentation fault. host> #0 0x280583e4 in ?? () host> (gdb) bt host> #0 0x280583e4 in ?? () host> #1 0x00000000 in ?? () host> #2 0x00000000 in ?? () host> #3 0x00000000 in ?? () host> #4 0x00000000 in ?? () host> #5 0x00000000 in ?? () host> #6 0x00000000 in ?? () host> #7 0x00000000 in ?? () host> #8 0x00000000 in ?? () host> #9 0x00000000 in ?? () host> #10 0x00000000 in ?? () host> #11 0xffffffff in ?? () host> #12 0x00001000 in ?? () host> [...] host> #359 0x73763a68 in ?? () host> #360 0x5b455c3d in ?? () host> [...] host> #855 0x00000000 in ?? () host> [...] Any hint on how to proceed would be welcome. Thank you. Regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From kostikbel at gmail.com Sat May 17 10:13:09 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat May 17 10:13:14 2008 Subject: rdmsr from userspace In-Reply-To: <482E93C0.4070802@icyb.net.ua> References: <482E93C0.4070802@icyb.net.ua> Message-ID: <20080517101243.GH18958@deviant.kiev.zoral.com.ua> On Sat, May 17, 2008 at 11:13:52AM +0300, Andriy Gapon wrote: > > It seems that rdmsr instruction can be executed only at the highest > privilege level and thus is not permitted from userland. Maybe we should > provide something like Linux /dev/cpu/msr? > I don't like interface of that device, I think that ioctl approach would > be preferable in this case. > Something like create /dev/cpuN and allow some ioctls on it: > ioctl(cpu_fd, CPU_RDMSR, arg). > What do you think? I believe ports/sysutils/devcpu is what you need. In fact, I think this ought to be committed to the base. I do not know why Stas did not asked for this, at least at public. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080517/6bdb202b/attachment.pgp From avg at icyb.net.ua Sat May 17 10:14:54 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Sat May 17 10:14:57 2008 Subject: rdmsr from userspace In-Reply-To: <20080517101243.GH18958@deviant.kiev.zoral.com.ua> References: <482E93C0.4070802@icyb.net.ua> <20080517101243.GH18958@deviant.kiev.zoral.com.ua> Message-ID: <482EB019.3060503@icyb.net.ua> on 17/05/2008 13:12 Kostik Belousov said the following: > On Sat, May 17, 2008 at 11:13:52AM +0300, Andriy Gapon wrote: >> It seems that rdmsr instruction can be executed only at the highest >> privilege level and thus is not permitted from userland. Maybe we should >> provide something like Linux /dev/cpu/msr? >> I don't like interface of that device, I think that ioctl approach would >> be preferable in this case. >> Something like create /dev/cpuN and allow some ioctls on it: >> ioctl(cpu_fd, CPU_RDMSR, arg). >> What do you think? > > I believe ports/sysutils/devcpu is what you need. In fact, I think this > ought to be committed to the base. I do not know why Stas did not asked > for this, at least at public. Thanks a lot! This is exactly it. I agree, this would be quite useful in the OS. -- Andriy Gapon From kostikbel at gmail.com Sat May 17 10:27:08 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat May 17 10:27:12 2008 Subject: Debugging rtld In-Reply-To: <20080517091740.GI70896@obiwan.tataz.chchile.org> References: <20080517091740.GI70896@obiwan.tataz.chchile.org> Message-ID: <20080517102653.GI18958@deviant.kiev.zoral.com.ua> On Sat, May 17, 2008 at 11:17:40AM +0200, Jeremie Le Hen wrote: > Hi there, > > I tried to compile my source tree with -fstack-protector-all, and it > happens that rtld breaks with this: once the new rtld is installed every > single problem coredumps. I tried to compile rtld-elf without SSP, but > it didn't solve the problem. Then I had to compile libc_pic.a without > SSP and it worked, but I don't understand the root of the problem. > So I want to use the generated coredump for post-mortem analysis with > gdb. > > I compiled world with DEBUG_FLAGS=-g. But GDB gives me a backtrace so > long that it can't be real. Moreoever it doesn't seem to bring in the > required symbols. I'm a GDB novice, so I'd like some help. > > chroot> ===> libexec/rtld-elf (install) > chroot> chflags noschg /usr/libexec/ld-elf.so.1 > chroot> install -s -o root -g wheel -m 555 -C -b -fschg -S ld-elf.so.1 /libexec > chroot> install -o root -g wheel -m 444 rtld.1.gz /usr/share/man/man1 > chroot> *** Signal 11 > chroot> > chroot> jarjarbinks# cd /root; ls > chroot> Segmentation fault > > host> jarjarbinks:145# ls -l /space/chroot/root/ls.core > host> -rw------- 1 root wheel 184320 May 17 10:19 /space/chroot/root/ls.core > host> jarjarbinks:149# gdb -c /space/chroot/root/ls.core -e /space/chroot/bin/ls > host> GNU gdb 6.1.1 [FreeBSD] > host> [...] > host> This GDB was configured as "i386-marcel-freebsd". > host> Core was generated by `ls'. > host> Program terminated with signal 11, Segmentation fault. > host> #0 0x280583e4 in ?? () > host> (gdb) bt > host> #0 0x280583e4 in ?? () > host> #1 0x00000000 in ?? () > host> #2 0x00000000 in ?? () > host> #3 0x00000000 in ?? () > host> #4 0x00000000 in ?? () > host> #5 0x00000000 in ?? () > host> #6 0x00000000 in ?? () > host> #7 0x00000000 in ?? () > host> #8 0x00000000 in ?? () > host> #9 0x00000000 in ?? () > host> #10 0x00000000 in ?? () > host> #11 0xffffffff in ?? () > host> #12 0x00001000 in ?? () > host> [...] > host> #359 0x73763a68 in ?? () > host> #360 0x5b455c3d in ?? () > host> [...] > host> #855 0x00000000 in ?? () > host> [...] > > Any hint on how to proceed would be welcome. I usually add the CFLAGS+=-g to the rtld-elf/Makefile. Also, you do not need to bring down the whole host by the broken ld.so.1. Do not install it at all, and specify the path to the rtld by the --dynamic-linker switch, see into ld. BTW, Solaris seems to support $ORIGIN in the interpreter path, but I feel that this is an overkill. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080517/32c1feeb/attachment.pgp From kientzle at freebsd.org Sat May 17 14:40:42 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sat May 17 14:40:47 2008 Subject: Debugging rtld In-Reply-To: <20080517102653.GI18958@deviant.kiev.zoral.com.ua> References: <20080517091740.GI70896@obiwan.tataz.chchile.org> <20080517102653.GI18958@deviant.kiev.zoral.com.ua> Message-ID: <482EEE65.8070009@freebsd.org> >>I tried to compile my source tree with -fstack-protector-all, and it >>happens that rtld breaks with this ... > > I usually add the CFLAGS+=-g to the rtld-elf/Makefile. Also, you do not > need to bring down the whole host by the broken ld.so.1. Do not install > it at all, and specify the path to the rtld by the --dynamic-linker switch, > see into ld. If you do (accidentally?) break ld.so.1, remember that /rescue is all statically linked and should still work. Single-user boot to /rescue/sh is your friend. Cheers, Tim Kientzle From kientzle at freebsd.org Sat May 17 14:41:24 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sat May 17 14:41:29 2008 Subject: GSoC Student In-Reply-To: References: Message-ID: <482EEE8D.5000208@freebsd.org> Sounds interesting. I'm looking forward to hearing more about your work. Cheers, Tim Kientzle Victor Hugo Bilouro wrote: > Hi, > > I'm Victor Hugo *Bilouro*, GSoC participant. > > I'll be working on tcptest. It's name came from TCP/IP Regression test suite. > > As a testing tool it can: > *be a conformance test > *test a bug already fixed (regression test - one test for each fixed bug) > *test a new feature (help protocol developer) > *simulate certain cases > > It will be build on top of pcs.sf.net (Package Construction Set) > "PCS is a set of Python modules and objects that make building network > protocol code easier for the protocol developer. The core of the > system is the pcs module itself which provides the necessary > functionality to create classes that implement packets. " > > My mentor is George V. Neville-Neil, no comments. :-) > > GSoC started a month ago, by this time, I had almost all environment > done, and I'm spending several times reading and often playing with > pcs. (cool) > > Books i bought: > (*)TCP/IP Illustrated Volume 1 (far the best) Stevens > (*)TCP/IP Guide Charles M.Kazierok (www.tcpipguide.com) > (*)Internetworking with tcp/ip > > I will post on __link__ links, books, status report, futures releases > information, news, everything about the project. > > __link__: http://wiki.freebsd.org/VictorBilouro/TCP-IP_regression_test_suite > > > Regards From rpaulo at FreeBSD.org Sat May 17 16:05:26 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sat May 17 16:05:31 2008 Subject: rdmsr from userspace In-Reply-To: <482E93C0.4070802@icyb.net.ua> References: <482E93C0.4070802@icyb.net.ua> Message-ID: <482EFBA0.30107@FreeBSD.org> Andriy Gapon wrote: > > It seems that rdmsr instruction can be executed only at the highest > privilege level and thus is not permitted from userland. Maybe we should > provide something like Linux /dev/cpu/msr? > I don't like interface of that device, I think that ioctl approach would > be preferable in this case. > Something like create /dev/cpuN and allow some ioctls on it: > ioctl(cpu_fd, CPU_RDMSR, arg). > What do you think? > While I think this (devcpu) is good for testing and development, I prefer having a device driver to handle that specific MSR than a generic /dev/cpuN where you can issue MSRs. Both for security and reliability reasons. Regards, -- Rui Paulo From avg at icyb.net.ua Sat May 17 17:10:49 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Sat May 17 17:10:52 2008 Subject: rdmsr from userspace In-Reply-To: <482EFBA0.30107@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> Message-ID: <482F1191.70709@icyb.net.ua> on 17/05/2008 18:37 Rui Paulo said the following: > Andriy Gapon wrote: >> >> It seems that rdmsr instruction can be executed only at the highest >> privilege level and thus is not permitted from userland. Maybe we >> should provide something like Linux /dev/cpu/msr? >> I don't like interface of that device, I think that ioctl approach >> would be preferable in this case. >> Something like create /dev/cpuN and allow some ioctls on it: >> ioctl(cpu_fd, CPU_RDMSR, arg). >> What do you think? >> > > While I think this (devcpu) is good for testing and development, I > prefer having a device driver to handle that specific MSR than a generic > /dev/cpuN where you can issue MSRs. > Both for security and reliability reasons. What about /dev/pci, /dev/io? Aren't they a precedent? -- Andriy Gapon From rpaulo at FreeBSD.org Sat May 17 17:26:11 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sat May 17 17:26:15 2008 Subject: rdmsr from userspace In-Reply-To: <482F1191.70709@icyb.net.ua> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> Message-ID: <482F1529.5080409@FreeBSD.org> Andriy Gapon wrote: > on 17/05/2008 18:37 Rui Paulo said the following: >> Andriy Gapon wrote: >>> >>> It seems that rdmsr instruction can be executed only at the highest >>> privilege level and thus is not permitted from userland. Maybe we >>> should provide something like Linux /dev/cpu/msr? >>> I don't like interface of that device, I think that ioctl approach >>> would be preferable in this case. >>> Something like create /dev/cpuN and allow some ioctls on it: >>> ioctl(cpu_fd, CPU_RDMSR, arg). >>> What do you think? >>> >> >> While I think this (devcpu) is good for testing and development, I >> prefer having a device driver to handle that specific MSR than a >> generic /dev/cpuN where you can issue MSRs. >> Both for security and reliability reasons. > > What about /dev/pci, /dev/io? Aren't they a precedent? They are, but, IMHO, we should no longer continue to create this type of interfaces. Regards, -- Rui Paulo From mwm-keyword-freebsdhackers2.e313df at mired.org Sat May 17 17:30:13 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sat May 17 17:30:17 2008 Subject: rdmsr from userspace In-Reply-To: <482E93C0.4070802@icyb.net.ua> References: <482E93C0.4070802@icyb.net.ua> Message-ID: <20080517133037.3a3935db@bhuda.mired.org> On Sat, 17 May 2008 11:13:52 +0300 Andriy Gapon wrote: > It seems that rdmsr instruction can be executed only at the highest > privilege level and thus is not permitted from userland. Maybe we should > provide something like Linux /dev/cpu/msr? > I don't like interface of that device, I think that ioctl approach would > be preferable in this case. > Something like create /dev/cpuN and allow some ioctls on it: > ioctl(cpu_fd, CPU_RDMSR, arg). > What do you think? Ok, this points directly at a question I've been wondering about, but haven't been able to find an answer in the google. I've been mucking about with general access to sysctl's (a sysctl plugin for gkrellm, and a python module for accessing sysctls), and with that hammer in my hand, the nail for this problem is obviously a dev.cpu.#.msr sysctl. Except that this would be harder to use from languages that don't provide direct access to libc functions than a file, with or without an ioctl (ioctl's are at at least POSIX, which is a functionality level a lot of languages aspire to), and also easier to manipulate with standard Unix tools in general. However, I thought I'd sense some hostility towards /dev/proc-like things from the freebsd community. Which is where my questions come in: Am I imagining that? If not, is there a real basis for it - and what would that be? Thanks, http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From mwm-keyword-freebsdhackers2.e313df at mired.org Sat May 17 17:32:03 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sat May 17 17:32:07 2008 Subject: rdmsr from userspace In-Reply-To: <482F1529.5080409@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> Message-ID: <20080517133228.02c9ea5c@bhuda.mired.org> On Sat, 17 May 2008 18:26:01 +0100 Rui Paulo wrote: > Andriy Gapon wrote: > > on 17/05/2008 18:37 Rui Paulo said the following: > >> Andriy Gapon wrote: > >>> > >>> It seems that rdmsr instruction can be executed only at the highest > >>> privilege level and thus is not permitted from userland. Maybe we > >>> should provide something like Linux /dev/cpu/msr? > >>> I don't like interface of that device, I think that ioctl approach > >>> would be preferable in this case. > >>> Something like create /dev/cpuN and allow some ioctls on it: > >>> ioctl(cpu_fd, CPU_RDMSR, arg). > >>> What do you think? > >>> > >> > >> While I think this (devcpu) is good for testing and development, I > >> prefer having a device driver to handle that specific MSR than a > >> generic /dev/cpuN where you can issue MSRs. > >> Both for security and reliability reasons. > > > > What about /dev/pci, /dev/io? Aren't they a precedent? > > They are, but, IMHO, we should no longer continue to create this type of > interfaces. Ok, in relation to the question I asked about sysctl's vs. /dev/* - why not? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From jeremie at le-hen.org Sat May 17 17:39:04 2008 From: jeremie at le-hen.org (Jeremie Le Hen) Date: Sat May 17 17:39:08 2008 Subject: Debugging rtld In-Reply-To: <20080517102653.GI18958@deviant.kiev.zoral.com.ua> References: <20080517091740.GI70896@obiwan.tataz.chchile.org> <20080517102653.GI18958@deviant.kiev.zoral.com.ua> Message-ID: <20080517173525.GJ70896@obiwan.tataz.chchile.org> Hi, On Sat, May 17, 2008 at 01:26:53PM +0300, Kostik Belousov wrote: > On Sat, May 17, 2008 at 11:17:40AM +0200, Jeremie Le Hen wrote: > > I tried to compile my source tree with -fstack-protector-all, and it > > happens that rtld breaks with this: once the new rtld is installed every > > single problem coredumps. I tried to compile rtld-elf without SSP, but > > it didn't solve the problem. Then I had to compile libc_pic.a without > > SSP and it worked, but I don't understand the root of the problem. > > So I want to use the generated coredump for post-mortem analysis with > > gdb. > > > > I compiled world with DEBUG_FLAGS=-g. But GDB gives me a backtrace so > > long that it can't be real. Moreoever it doesn't seem to bring in the > > required symbols. I'm a GDB novice, so I'd like some help. > > > > chroot> ===> libexec/rtld-elf (install) > > chroot> chflags noschg /usr/libexec/ld-elf.so.1 > > chroot> install -s -o root -g wheel -m 555 -C -b -fschg -S ld-elf.so.1 /libexec > > chroot> install -o root -g wheel -m 444 rtld.1.gz /usr/share/man/man1 > > chroot> *** Signal 11 > > chroot> > > chroot> jarjarbinks# cd /root; ls > > chroot> Segmentation fault > > > > host> jarjarbinks:145# ls -l /space/chroot/root/ls.core > > host> -rw------- 1 root wheel 184320 May 17 10:19 /space/chroot/root/ls.core > > host> jarjarbinks:149# gdb -c /space/chroot/root/ls.core -e /space/chroot/bin/ls > > host> GNU gdb 6.1.1 [FreeBSD] > > host> [...] > > host> This GDB was configured as "i386-marcel-freebsd". > > host> Core was generated by `ls'. > > host> Program terminated with signal 11, Segmentation fault. > > host> #0 0x280583e4 in ?? () > > host> (gdb) bt > > host> #0 0x280583e4 in ?? () > > host> #1 0x00000000 in ?? () > > host> #2 0x00000000 in ?? () > > host> #3 0x00000000 in ?? () > > host> #4 0x00000000 in ?? () > > host> #5 0x00000000 in ?? () > > host> #6 0x00000000 in ?? () > > host> #7 0x00000000 in ?? () > > host> #8 0x00000000 in ?? () > > host> #9 0x00000000 in ?? () > > host> #10 0x00000000 in ?? () > > host> #11 0xffffffff in ?? () > > host> #12 0x00001000 in ?? () > > host> [...] > > host> #359 0x73763a68 in ?? () > > host> #360 0x5b455c3d in ?? () > > host> [...] > > host> #855 0x00000000 in ?? () > > host> [...] > > > > Any hint on how to proceed would be welcome. > > I usually add the CFLAGS+=-g to the rtld-elf/Makefile. Also, you do not > need to bring down the whole host by the broken ld.so.1. Do not install > it at all, and specify the path to the rtld by the --dynamic-linker switch, > see into ld. Thank you for this tip. However the backtrace is still unusable. I've recompiled libc_pic.a with -g, then rtld-elf with -g and finally /bin/ls with you tip and -g. I'm really brought to a standstill here. Thank you. Best regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From kostikbel at gmail.com Sat May 17 17:46:47 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat May 17 17:46:52 2008 Subject: Debugging rtld In-Reply-To: <20080517173525.GJ70896@obiwan.tataz.chchile.org> References: <20080517091740.GI70896@obiwan.tataz.chchile.org> <20080517102653.GI18958@deviant.kiev.zoral.com.ua> <20080517173525.GJ70896@obiwan.tataz.chchile.org> Message-ID: <20080517174637.GL18958@deviant.kiev.zoral.com.ua> On Sat, May 17, 2008 at 07:35:25PM +0200, Jeremie Le Hen wrote: > Hi, > > On Sat, May 17, 2008 at 01:26:53PM +0300, Kostik Belousov wrote: > > On Sat, May 17, 2008 at 11:17:40AM +0200, Jeremie Le Hen wrote: > > > I tried to compile my source tree with -fstack-protector-all, and it > > > happens that rtld breaks with this: once the new rtld is installed every > > > single problem coredumps. I tried to compile rtld-elf without SSP, but > > > it didn't solve the problem. Then I had to compile libc_pic.a without > > > SSP and it worked, but I don't understand the root of the problem. > > > So I want to use the generated coredump for post-mortem analysis with > > > gdb. > > > > > > I compiled world with DEBUG_FLAGS=-g. But GDB gives me a backtrace so > > > long that it can't be real. Moreoever it doesn't seem to bring in the > > > required symbols. I'm a GDB novice, so I'd like some help. > > > > > > chroot> ===> libexec/rtld-elf (install) > > > chroot> chflags noschg /usr/libexec/ld-elf.so.1 > > > chroot> install -s -o root -g wheel -m 555 -C -b -fschg -S ld-elf.so.1 /libexec > > > chroot> install -o root -g wheel -m 444 rtld.1.gz /usr/share/man/man1 > > > chroot> *** Signal 11 > > > chroot> > > > chroot> jarjarbinks# cd /root; ls > > > chroot> Segmentation fault > > > > > > host> jarjarbinks:145# ls -l /space/chroot/root/ls.core > > > host> -rw------- 1 root wheel 184320 May 17 10:19 /space/chroot/root/ls.core > > > host> jarjarbinks:149# gdb -c /space/chroot/root/ls.core -e /space/chroot/bin/ls > > > host> GNU gdb 6.1.1 [FreeBSD] > > > host> [...] > > > host> This GDB was configured as "i386-marcel-freebsd". > > > host> Core was generated by `ls'. > > > host> Program terminated with signal 11, Segmentation fault. > > > host> #0 0x280583e4 in ?? () > > > host> (gdb) bt > > > host> #0 0x280583e4 in ?? () > > > host> #1 0x00000000 in ?? () > > > host> #2 0x00000000 in ?? () > > > host> #3 0x00000000 in ?? () > > > host> #4 0x00000000 in ?? () > > > host> #5 0x00000000 in ?? () > > > host> #6 0x00000000 in ?? () > > > host> #7 0x00000000 in ?? () > > > host> #8 0x00000000 in ?? () > > > host> #9 0x00000000 in ?? () > > > host> #10 0x00000000 in ?? () > > > host> #11 0xffffffff in ?? () > > > host> #12 0x00001000 in ?? () > > > host> [...] > > > host> #359 0x73763a68 in ?? () > > > host> #360 0x5b455c3d in ?? () > > > host> [...] > > > host> #855 0x00000000 in ?? () > > > host> [...] > > > > > > Any hint on how to proceed would be welcome. > > > > I usually add the CFLAGS+=-g to the rtld-elf/Makefile. Also, you do not > > need to bring down the whole host by the broken ld.so.1. Do not install > > it at all, and specify the path to the rtld by the --dynamic-linker switch, > > see into ld. Hmm, ^^^^ info > > Thank you for this tip. However the backtrace is still unusable. > I've recompiled libc_pic.a with -g, then rtld-elf with -g and finally > /bin/ls with you tip and -g. > > I'm really brought to a standstill here. Looks like you have a stack corruption, that is reasonable given the matters you touching. The easiest, althought somewhat time-consuming way of searching the point where the things break is to insert some break into the code of the rtld, "int3" may be good, and moving it forward until you start hitting the breakage instead of the breakpoint. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080517/ec7755d7/attachment.pgp From kostikbel at gmail.com Sat May 17 17:53:23 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat May 17 17:53:27 2008 Subject: rdmsr from userspace In-Reply-To: <482F1529.5080409@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> Message-ID: <20080517175312.GM18958@deviant.kiev.zoral.com.ua> On Sat, May 17, 2008 at 06:26:01PM +0100, Rui Paulo wrote: > Andriy Gapon wrote: > >on 17/05/2008 18:37 Rui Paulo said the following: > >>Andriy Gapon wrote: > >>> > >>>It seems that rdmsr instruction can be executed only at the highest > >>>privilege level and thus is not permitted from userland. Maybe we > >>>should provide something like Linux /dev/cpu/msr? > >>>I don't like interface of that device, I think that ioctl approach > >>>would be preferable in this case. > >>>Something like create /dev/cpuN and allow some ioctls on it: > >>>ioctl(cpu_fd, CPU_RDMSR, arg). > >>>What do you think? > >>> > >> > >>While I think this (devcpu) is good for testing and development, I > >>prefer having a device driver to handle that specific MSR than a > >>generic /dev/cpuN where you can issue MSRs. > >>Both for security and reliability reasons. > > > >What about /dev/pci, /dev/io? Aren't they a precedent? > > They are, but, IMHO, we should no longer continue to create this type of > interfaces. Why ? Are developers some kind of the second-class users ? I would have no opinion on providing /dev/cpu by the loadable module, not compiled into GENERIC. But the interface itself is useful at least for three things: - CPU identification (see x86info or whatever it is called); - CPU tweaking for bugs workaround without patching the kernel; - updating the CPU microcode. None of these is limited to the developers only. I am interested why Stanislav still did not submitted it for inclusion into the base still. Maybe, some other reasons exist. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080517/b25f5e85/attachment.pgp From jeremie at le-hen.org Sat May 17 21:26:14 2008 From: jeremie at le-hen.org (Jeremie Le Hen) Date: Sat May 17 21:26:18 2008 Subject: do not work nested unnamed anchor In-Reply-To: References: Message-ID: <20080517212318.GK70896@obiwan.tataz.chchile.org> Hi Igor, On Fri, May 09, 2008 at 04:55:23PM +0400, Igor A. Valcov wrote: > Hello. > > For example: > > ==== pf.conf ==== > > ext_if="xl0" > ip_world="nn.nn.nn.nn" > > # Filter rules > block log all > > anchor in on $ext_if { > pass quick proto tcp to $ip_world port 22 keep state > # SSH > pass quick proto tcp to $ip_world port 25 keep state > # SMTP > pass quick proto tcp to $ip_world port 110 keep state > # POP3 > anchor { > pass quick proto tcp to $ip_world port 995 keep state > # POP3S > } > } > > ============ > > nmap results: > > PORT STATE SERVICE VERSION > 22/tcp open ssh OpenSSH 4.5p1 (FreeBSD 20061110; protocol 2.0) > 25/tcp open smtp? > 110/tcp open pop3 Openwall popa3d > > > I can not understand what the problem... > > FreeBSD-7.0-RELEASE-p1 > i386 You should ask this on pf mailing-list [1]. freebsd-hackers@ is not the right place for this, freebsd-net@ or freebsd-pf@ would have been far more better. [1] http://www.benzedrine.cx/mailinglist.html Thank you. Best regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From rpaulo at FreeBSD.org Sun May 18 15:47:50 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sun May 18 15:47:57 2008 Subject: rdmsr from userspace In-Reply-To: <20080517175312.GM18958@deviant.kiev.zoral.com.ua> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> <20080517175312.GM18958@deviant.kiev.zoral.com.ua> Message-ID: <48304F9D.9030406@FreeBSD.org> Kostik Belousov wrote: > On Sat, May 17, 2008 at 06:26:01PM +0100, Rui Paulo wrote: >> Andriy Gapon wrote: >>> on 17/05/2008 18:37 Rui Paulo said the following: >>>> Andriy Gapon wrote: >>>>> It seems that rdmsr instruction can be executed only at the highest >>>>> privilege level and thus is not permitted from userland. Maybe we >>>>> should provide something like Linux /dev/cpu/msr? >>>>> I don't like interface of that device, I think that ioctl approach >>>>> would be preferable in this case. >>>>> Something like create /dev/cpuN and allow some ioctls on it: >>>>> ioctl(cpu_fd, CPU_RDMSR, arg). >>>>> What do you think? >>>>> >>>> While I think this (devcpu) is good for testing and development, I >>>> prefer having a device driver to handle that specific MSR than a >>>> generic /dev/cpuN where you can issue MSRs. >>>> Both for security and reliability reasons. >>> What about /dev/pci, /dev/io? Aren't they a precedent? >> They are, but, IMHO, we should no longer continue to create this type of >> interfaces. > > Why ? Are developers some kind of the second-class users ? > > I would have no opinion on providing /dev/cpu by the loadable module, not > compiled into GENERIC. But the interface itself is useful at least for > three things: > - CPU identification (see x86info or whatever it is called); > - CPU tweaking for bugs workaround without patching the kernel; > - updating the CPU microcode. > None of these is limited to the developers only. Input validation is my main concern here. Regarding to your two last points, I would prefer to have a microcode driver than a microcode userland utility that relies on devcpu. Regards, -- Rui Paulo From rpaulo at FreeBSD.org Sun May 18 15:50:36 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sun May 18 15:50:41 2008 Subject: rdmsr from userspace In-Reply-To: <20080517133037.3a3935db@bhuda.mired.org> References: <482E93C0.4070802@icyb.net.ua> <20080517133037.3a3935db@bhuda.mired.org> Message-ID: <48305044.7000007@FreeBSD.org> Mike Meyer wrote: > On Sat, 17 May 2008 11:13:52 +0300 > Andriy Gapon wrote: >> It seems that rdmsr instruction can be executed only at the highest >> privilege level and thus is not permitted from userland. Maybe we should >> provide something like Linux /dev/cpu/msr? >> I don't like interface of that device, I think that ioctl approach would >> be preferable in this case. >> Something like create /dev/cpuN and allow some ioctls on it: >> ioctl(cpu_fd, CPU_RDMSR, arg). >> What do you think? > > Ok, this points directly at a question I've been wondering about, but > haven't been able to find an answer in the google. > > I've been mucking about with general access to sysctl's (a sysctl > plugin for gkrellm, and a python module for accessing sysctls), and > with that hammer in my hand, the nail for this problem is obviously a > dev.cpu.#.msr sysctl. How can you request a rdmsr within the sysctl tree? I don't think sysctl is appropriate here either. -- Rui Paulo From mwm-keyword-freebsdhackers2.e313df at mired.org Sun May 18 18:06:26 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sun May 18 18:06:29 2008 Subject: rdmsr from userspace In-Reply-To: <48305044.7000007@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <20080517133037.3a3935db@bhuda.mired.org> <48305044.7000007@FreeBSD.org> Message-ID: <20080518140650.2691af1c@bhuda.mired.org> On Sun, 18 May 2008 16:50:28 +0100 Rui Paulo wrote: > Mike Meyer wrote: > > On Sat, 17 May 2008 11:13:52 +0300 > > Andriy Gapon wrote: > >> It seems that rdmsr instruction can be executed only at the highest > >> privilege level and thus is not permitted from userland. Maybe we should > >> provide something like Linux /dev/cpu/msr? > >> I don't like interface of that device, I think that ioctl approach would > >> be preferable in this case. > >> Something like create /dev/cpuN and allow some ioctls on it: > >> ioctl(cpu_fd, CPU_RDMSR, arg). > >> What do you think? > > > > Ok, this points directly at a question I've been wondering about, but > > haven't been able to find an answer in the google. > > > > I've been mucking about with general access to sysctl's (a sysctl > > plugin for gkrellm, and a python module for accessing sysctls), and > > with that hammer in my hand, the nail for this problem is obviously a > > dev.cpu.#.msr sysctl. > > How can you request a rdmsr within the sysctl tree? I don't think sysctl > is appropriate here either. Reading (or writing) a sysctl mib can trigger a sysctl handler, which can do pretty much anything. In particular, there are already examples in the kernel where sysctl handlers use devices that don't have /dev entries to get & set their values. Look through kern/kern_cpu.c and i386/cpufreq/p4tcc.c to see the two ends of that kind of connection. In fact, the cpu frequency sysctls would seem to be an excellent model for something like the msr. ioctl, open+read/write, sysctl - they're all just interfaces to kernel handlers. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From kostikbel at gmail.com Sun May 18 18:16:05 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sun May 18 18:16:10 2008 Subject: rdmsr from userspace In-Reply-To: <48304F9D.9030406@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> <20080517175312.GM18958@deviant.kiev.zoral.com.ua> <48304F9D.9030406@FreeBSD.org> Message-ID: <20080518181549.GZ18958@deviant.kiev.zoral.com.ua> On Sun, May 18, 2008 at 04:47:41PM +0100, Rui Paulo wrote: > Kostik Belousov wrote: > >On Sat, May 17, 2008 at 06:26:01PM +0100, Rui Paulo wrote: > >>Andriy Gapon wrote: > >>>on 17/05/2008 18:37 Rui Paulo said the following: > >>>>Andriy Gapon wrote: > >>>>>It seems that rdmsr instruction can be executed only at the highest > >>>>>privilege level and thus is not permitted from userland. Maybe we > >>>>>should provide something like Linux /dev/cpu/msr? > >>>>>I don't like interface of that device, I think that ioctl approach > >>>>>would be preferable in this case. > >>>>>Something like create /dev/cpuN and allow some ioctls on it: > >>>>>ioctl(cpu_fd, CPU_RDMSR, arg). > >>>>>What do you think? > >>>>> > >>>>While I think this (devcpu) is good for testing and development, I > >>>>prefer having a device driver to handle that specific MSR than a > >>>>generic /dev/cpuN where you can issue MSRs. > >>>>Both for security and reliability reasons. > >>>What about /dev/pci, /dev/io? Aren't they a precedent? > >>They are, but, IMHO, we should no longer continue to create this type of > >>interfaces. > > > >Why ? Are developers some kind of the second-class users ? > > > >I would have no opinion on providing /dev/cpu by the loadable module, not > >compiled into GENERIC. But the interface itself is useful at least for > >three things: > >- CPU identification (see x86info or whatever it is called); > >- CPU tweaking for bugs workaround without patching the kernel; > >- updating the CPU microcode. > >None of these is limited to the developers only. > > Input validation is my main concern here. Regarding to your two last > points, I would prefer to have a microcode driver than a microcode > userland utility that relies on devcpu. Did you looked at the code ? It does exactly what you described. Driver has four basic operations: read/write msr perform cpu id work update microcode. The later is done as a whole operation, with the microcode blob supplied by the userspace. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080518/fd283f39/attachment.pgp From david at catwhisker.org Sun May 18 18:16:49 2008 From: david at catwhisker.org (David Wolfskill) Date: Sun May 18 18:16:53 2008 Subject: [Patch] Using sysctl(8) to acquire info from different systems In-Reply-To: <20080518170936.GB1797@roadrunner.spoerlein.net> References: <20080512200901.GL66703@bunrab.catwhisker.org> <20080518170936.GB1797@roadrunner.spoerlein.net> Message-ID: <20080518181648.GA7468@bunrab.catwhisker.org> On Sun, May 18, 2008 at 07:09:36PM +0200, Ulrich Spoerlein wrote: > ... > I find this functionality very useful, but the addition of another flag > as problematic. First of all, old releases don't have it. Secondly, the > behaviour you describe should be the default anyway (IMHO). Thank you for your support of the idea. I'll explain why I chose a different approach in the implementation. I agree that it would have been handy had it been the default behavior, but it hasn't been and wishing isn't likely to change that. Thus, if a change is to be seen in the older releases, *some* change will need to be made to the code for those releases. By making the change the addition of a flag, we are able to preserve the existing default behavior -- and thus avoid potentioal perception of a POLA violation. And I believe the change is easily merged from CURRENT to older releases, if someone is willing to do the commit. > So, when requesting OID a, b, and c, sysctl should print a, a warning > that it cannot find OID b (to STDERR), then print c and exit with a > return code != 0. Well, the warning should be emitted in the absence of the -q flag, I expect, but in its presence, the warning should (IMO) be suppressed. > At least, that's what I would code it to do. I'd be happy for the default to be to (at least) "carry on" after finding a request for an unknown OID. I deliberately made my patch as non-invasive as I could to improve its chances of actually getting committed -- as well as the above-cited desire to avoid perception of a POLA violation. :-} And I'm hardly claiming that the approach I took is optimal, let alone the "only" one: it is an approach that I believe to be consistent with my requirements and thus serves to more clearly illustrate those requirements. BTW, I did file a PR -- it's bin/123644: Allow sysctl(8) to ignore unknown OIDs. Peace, david -- David H. Wolfskill david@catwhisker.org I submit that "conspiracy" would be an appropriate collective noun for cats. See http://www.catwhisker.org/~david/publickey.gpg for my public key. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080518/f80a8eb9/attachment.pgp From uspoerlein at gmail.com Sun May 18 18:32:20 2008 From: uspoerlein at gmail.com (Ulrich Spoerlein) Date: Sun May 18 18:32:24 2008 Subject: [Patch] Using sysctl(8) to acquire info from different systems In-Reply-To: <20080512200901.GL66703@bunrab.catwhisker.org> References: <20080512200901.GL66703@bunrab.catwhisker.org> Message-ID: <20080518170936.GB1797@roadrunner.spoerlein.net> On Mon, 12.05.2008 at 13:09:01 -0700, David Wolfskill wrote: > In my case, I believe it would be useful to provide an ability to tell > sysctl(8) to report on everything asked for that it does know, and > ignore the OIDs it doesn't know. > > Is this percpetion so radical that I'm way off base? If so, please > educate me as to why. > > Otherwise, I'll plan on filing a PR with the attached patch, which adds > "-i" to sysctl(8)'s flags -- and which appears to work as described above: Hi David, I find this functionality very useful, but the addition of another flag as problematic. First of all, old releases don't have it. Secondly, the behaviour you describe should be the default anyway (IMHO). So, when requesting OID a, b, and c, sysctl should print a, a warning that it cannot find OID b (to STDERR), then print c and exit with a return code != 0. At least, that's what I would code it to do. Cheers, Ulrich Spoerlein -- It is better to remain silent and be thought a fool, than to speak, and remove all doubt. From rpaulo at FreeBSD.org Sun May 18 18:33:04 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sun May 18 18:33:09 2008 Subject: rdmsr from userspace In-Reply-To: <20080518181549.GZ18958@deviant.kiev.zoral.com.ua> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> <20080517175312.GM18958@deviant.kiev.zoral.com.ua> <48304F9D.9030406@FreeBSD.org> <20080518181549.GZ18958@deviant.kiev.zoral.com.ua> Message-ID: <48307658.2080502@FreeBSD.org> Kostik Belousov wrote: > On Sun, May 18, 2008 at 04:47:41PM +0100, Rui Paulo wrote: >> Kostik Belousov wrote: >>> On Sat, May 17, 2008 at 06:26:01PM +0100, Rui Paulo wrote: >>>> Andriy Gapon wrote: >>>>> on 17/05/2008 18:37 Rui Paulo said the following: >>>>>> Andriy Gapon wrote: >>>>>>> It seems that rdmsr instruction can be executed only at the highest >>>>>>> privilege level and thus is not permitted from userland. Maybe we >>>>>>> should provide something like Linux /dev/cpu/msr? >>>>>>> I don't like interface of that device, I think that ioctl approach >>>>>>> would be preferable in this case. >>>>>>> Something like create /dev/cpuN and allow some ioctls on it: >>>>>>> ioctl(cpu_fd, CPU_RDMSR, arg). >>>>>>> What do you think? >>>>>>> >>>>>> While I think this (devcpu) is good for testing and development, I >>>>>> prefer having a device driver to handle that specific MSR than a >>>>>> generic /dev/cpuN where you can issue MSRs. >>>>>> Both for security and reliability reasons. >>>>> What about /dev/pci, /dev/io? Aren't they a precedent? >>>> They are, but, IMHO, we should no longer continue to create this type of >>>> interfaces. >>> Why ? Are developers some kind of the second-class users ? >>> >>> I would have no opinion on providing /dev/cpu by the loadable module, not >>> compiled into GENERIC. But the interface itself is useful at least for >>> three things: >>> - CPU identification (see x86info or whatever it is called); >>> - CPU tweaking for bugs workaround without patching the kernel; >>> - updating the CPU microcode. >>> None of these is limited to the developers only. >> Input validation is my main concern here. Regarding to your two last >> points, I would prefer to have a microcode driver than a microcode >> userland utility that relies on devcpu. > Did you looked at the code ? It does exactly what you described. > > Driver has four basic operations: > read/write msr > perform cpu id work > update microcode. > > The later is done as a whole operation, with the microcode blob supplied > by the userspace. Yes, but I still don't like having everything mixed up in one driver. At the very least, I would like us to have two drivers. One for the microcode update and the other driver for the rest. I would like to see a microcode update utility (driver + something to parse Intel's file aka devcpu-data) in the base system, but not "the rest", though. Regards, -- Rui Paulo From rpaulo at FreeBSD.org Sun May 18 18:35:52 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sun May 18 18:35:57 2008 Subject: rdmsr from userspace In-Reply-To: <20080518140650.2691af1c@bhuda.mired.org> References: <482E93C0.4070802@icyb.net.ua> <20080517133037.3a3935db@bhuda.mired.org> <48305044.7000007@FreeBSD.org> <20080518140650.2691af1c@bhuda.mired.org> Message-ID: <48307700.70304@FreeBSD.org> Mike Meyer wrote: > On Sun, 18 May 2008 16:50:28 +0100 > Rui Paulo wrote: > >> Mike Meyer wrote: >>> On Sat, 17 May 2008 11:13:52 +0300 >>> Andriy Gapon wrote: >>>> It seems that rdmsr instruction can be executed only at the highest >>>> privilege level and thus is not permitted from userland. Maybe we should >>>> provide something like Linux /dev/cpu/msr? >>>> I don't like interface of that device, I think that ioctl approach would >>>> be preferable in this case. >>>> Something like create /dev/cpuN and allow some ioctls on it: >>>> ioctl(cpu_fd, CPU_RDMSR, arg). >>>> What do you think? >>> Ok, this points directly at a question I've been wondering about, but >>> haven't been able to find an answer in the google. >>> >>> I've been mucking about with general access to sysctl's (a sysctl >>> plugin for gkrellm, and a python module for accessing sysctls), and >>> with that hammer in my hand, the nail for this problem is obviously a >>> dev.cpu.#.msr sysctl. >> How can you request a rdmsr within the sysctl tree? I don't think sysctl >> is appropriate here either. > > Reading (or writing) a sysctl mib can trigger a sysctl handler, which > can do pretty much anything. In particular, there are already examples > in the kernel where sysctl handlers use devices that don't have /dev > entries to get & set their values. Look through kern/kern_cpu.c and > i386/cpufreq/p4tcc.c to see the two ends of that kind of > connection. In fact, the cpu frequency sysctls would seem to be an > excellent model for something like the msr. > > ioctl, open+read/write, sysctl - they're all just interfaces to kernel > handlers. > > ? I'll have to think about whether or not I like this interface. Regards, -- Rui Paulo From mwm-keyword-freebsdhackers2.e313df at mired.org Sun May 18 19:29:20 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sun May 18 19:29:23 2008 Subject: rdmsr from userspace In-Reply-To: <48307700.70304@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <20080517133037.3a3935db@bhuda.mired.org> <48305044.7000007@FreeBSD.org> <20080518140650.2691af1c@bhuda.mired.org> <48307700.70304@FreeBSD.org> Message-ID: <20080518152945.60989b9c@bhuda.mired.org> On Sun, 18 May 2008 19:35:44 +0100 Rui Paulo wrote: > Mike Meyer wrote: > > On Sun, 18 May 2008 16:50:28 +0100 > > Rui Paulo wrote: > > > >> Mike Meyer wrote: > >>> On Sat, 17 May 2008 11:13:52 +0300 > >>> Andriy Gapon wrote: > >>>> It seems that rdmsr instruction can be executed only at the highest > >>>> privilege level and thus is not permitted from userland. Maybe we should > >>>> provide something like Linux /dev/cpu/msr? > >>>> I don't like interface of that device, I think that ioctl approach would > >>>> be preferable in this case. > >>>> Something like create /dev/cpuN and allow some ioctls on it: > >>>> ioctl(cpu_fd, CPU_RDMSR, arg). > >>>> What do you think? > >>> Ok, this points directly at a question I've been wondering about, but > >>> haven't been able to find an answer in the google. > >>> > >>> I've been mucking about with general access to sysctl's (a sysctl > >>> plugin for gkrellm, and a python module for accessing sysctls), and > >>> with that hammer in my hand, the nail for this problem is obviously a > >>> dev.cpu.#.msr sysctl. > >> How can you request a rdmsr within the sysctl tree? I don't think sysctl > >> is appropriate here either. > > > > Reading (or writing) a sysctl mib can trigger a sysctl handler, which > > can do pretty much anything. In particular, there are already examples > > in the kernel where sysctl handlers use devices that don't have /dev > > entries to get & set their values. Look through kern/kern_cpu.c and > > i386/cpufreq/p4tcc.c to see the two ends of that kind of > > connection. In fact, the cpu frequency sysctls would seem to be an > > excellent model for something like the msr. > > > > ioctl, open+read/write, sysctl - they're all just interfaces to kernel > > handlers. > > > > > Yes, sure, but who do you select the MSR you want to read or write? > > dev.cpu.N. ? I don't think that would work - you'd have to register all those hexadecimal strings as sysctl names. You could do an interface like this, but the calling program would have to use sysctlnametomib to get dev.cpu.N.msr, then append the MSR number to the results. Not really very pretty. If you want to allow the user to poke at arbitrary msrs, this would be a way to do it with sysctls, but the file api is probably better. On the other hand, if you want to allow access to the fixed set of documented msr's (for each cpu model, anyway), then handing back that fixed set as an array would be a better approach, and would have the advantage of not having to deal with invalid requests (non-existent MSRs, 1-byte reads/writes of multi-byte MSRs, etc). On the other hand, it might be more useful - *especially* if the MSRs move around depending on processor types (I honestly don't know) - to provide a "named" interface: dev.cpu.0.msr.mtrr dev.cpu.0.msr.arr dev.cpu.0.msr.efer and so on. You'd register the names when the module was initialized, and would only register the ones that actually existed. You'd then never have to deal with a request for a non-existent MSR, because the sysctl call would return an error to the calling program without ever calling your handler. Of course, you can *combine* these two approaches, and have: dev.cpu.0.msr.all # returns an array of all available msrs dev.cpu.0.msr.have # an array of the available msrs but at this point it's just blue sky speculation... > I'll have to think about whether or not I like this interface. Actually, I'm more interested in why there seems to be a dislike of file-based interfaces in favor of sysctls in the freebsd community. Exploring this has been enlightening. Thank you. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From mwm-keyword-freebsdhackers2.e313df at mired.org Sun May 18 19:29:32 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sun May 18 19:29:36 2008 Subject: rdmsr from userspace In-Reply-To: <48307700.70304@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <20080517133037.3a3935db@bhuda.mired.org> <48305044.7000007@FreeBSD.org> <20080518140650.2691af1c@bhuda.mired.org> <48307700.70304@FreeBSD.org> Message-ID: <20080518152937.774cf492@bhuda.mired.org> On Sun, 18 May 2008 19:35:44 +0100 Rui Paulo wrote: > Mike Meyer wrote: > > On Sun, 18 May 2008 16:50:28 +0100 > > Rui Paulo wrote: > > > >> Mike Meyer wrote: > >>> On Sat, 17 May 2008 11:13:52 +0300 > >>> Andriy Gapon wrote: > >>>> It seems that rdmsr instruction can be executed only at the highest > >>>> privilege level and thus is not permitted from userland. Maybe we should > >>>> provide something like Linux /dev/cpu/msr? > >>>> I don't like interface of that device, I think that ioctl approach would > >>>> be preferable in this case. > >>>> Something like create /dev/cpuN and allow some ioctls on it: > >>>> ioctl(cpu_fd, CPU_RDMSR, arg). > >>>> What do you think? > >>> Ok, this points directly at a question I've been wondering about, but > >>> haven't been able to find an answer in the google. > >>> > >>> I've been mucking about with general access to sysctl's (a sysctl > >>> plugin for gkrellm, and a python module for accessing sysctls), and > >>> with that hammer in my hand, the nail for this problem is obviously a > >>> dev.cpu.#.msr sysctl. > >> How can you request a rdmsr within the sysctl tree? I don't think sysctl > >> is appropriate here either. > > > > Reading (or writing) a sysctl mib can trigger a sysctl handler, which > > can do pretty much anything. In particular, there are already examples > > in the kernel where sysctl handlers use devices that don't have /dev > > entries to get & set their values. Look through kern/kern_cpu.c and > > i386/cpufreq/p4tcc.c to see the two ends of that kind of > > connection. In fact, the cpu frequency sysctls would seem to be an > > excellent model for something like the msr. > > > > ioctl, open+read/write, sysctl - they're all just interfaces to kernel > > handlers. > > > > > Yes, sure, but who do you select the MSR you want to read or write? > > dev.cpu.N. ? I don't think that would work - you'd have to register all those hexadecimal strings as sysctl names. You could do an interface like this, but the calling program would have to use sysctlnametomib to get dev.cpu.N.msr, then append the MSR number to the results. Not really very pretty. If you want to allow the user to poke at arbitrary msrs, this would be a way to do it with sysctls, but the file api is probably better. On the other hand, if you want to allow access to the fixed set of documented msr's (for each cpu model, anyway), then handing back that fixed set as an array would be a better approach, and would have the advantage of not having to deal with invalid requests (non-existent MSRs, 1-byte reads/writes of multi-byte MSRs, etc). On the other hand, it might be more useful - *especially* if the MSRs move around depending on processor types (I honestly don't know) - to provide a "named" interface: dev.cpu.0.msr.mtrr dev.cpu.0.msr.arr dev.cpu.0.msr.efer and so on. You'd register the names when the module was initialized, and would only register the ones that actually existed. You'd then never have to deal with a request for a non-existent MSR, because the sysctl call would return an error to the calling program without ever calling your handler. Of course, you can *combine* these two approaches, and have: dev.cpu.0.msr.all # returns an array of all available msrs dev.cpu.0.msr.have # an array of the available msrs but at this point it's just blue sky speculation... > I'll have to think about whether or not I like this interface. Actually, I'm more interested in why there seems to be a dislike of file-based interfaces in favor of sysctls in the freebsd community. Exploring this has been enlightening. Thank you. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From ssouhlal at FreeBSD.org Sun May 18 20:05:56 2008 From: ssouhlal at FreeBSD.org (Suleiman Souhlal) Date: Sun May 18 20:06:01 2008 Subject: rdmsr from userspace In-Reply-To: <20080518181549.GZ18958@deviant.kiev.zoral.com.ua> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> <20080517175312.GM18958@deviant.kiev.zoral.com.ua> <48304F9D.9030406@FreeBSD.org> <20080518181549.GZ18958@deviant.kiev.zoral.com.ua> Message-ID: <821D2BD4-4C79-49B6-ADD9-2C5EA16B852A@FreeBSD.org> On May 18, 2008, at 11:15 AM, Kostik Belousov wrote: > On Sun, May 18, 2008 at 04:47:41PM +0100, Rui Paulo wrote: >> Kostik Belousov wrote: >>> On Sat, May 17, 2008 at 06:26:01PM +0100, Rui Paulo wrote: >>>> Andriy Gapon wrote: >>>>> on 17/05/2008 18:37 Rui Paulo said the following: >>>>>> Andriy Gapon wrote: >>>>>>> It seems that rdmsr instruction can be executed only at the >>>>>>> highest >>>>>>> privilege level and thus is not permitted from userland. >>>>>>> Maybe we >>>>>>> should provide something like Linux /dev/cpu/msr? >>>>>>> I don't like interface of that device, I think that ioctl >>>>>>> approach >>>>>>> would be preferable in this case. >>>>>>> Something like create /dev/cpuN and allow some ioctls on it: >>>>>>> ioctl(cpu_fd, CPU_RDMSR, arg). >>>>>>> What do you think? >>>>>>> >>>>>> While I think this (devcpu) is good for testing and >>>>>> development, I >>>>>> prefer having a device driver to handle that specific MSR than a >>>>>> generic /dev/cpuN where you can issue MSRs. >>>>>> Both for security and reliability reasons. >>>>> What about /dev/pci, /dev/io? Aren't they a precedent? >>>> They are, but, IMHO, we should no longer continue to create this >>>> type of >>>> interfaces. >>> >>> Why ? Are developers some kind of the second-class users ? >>> >>> I would have no opinion on providing /dev/cpu by the loadable >>> module, not >>> compiled into GENERIC. But the interface itself is useful at >>> least for >>> three things: >>> - CPU identification (see x86info or whatever it is called); >>> - CPU tweaking for bugs workaround without patching the kernel; >>> - updating the CPU microcode. >>> None of these is limited to the developers only. >> >> Input validation is my main concern here. Regarding to your two last >> points, I would prefer to have a microcode driver than a microcode >> userland utility that relies on devcpu. > Did you looked at the code ? It does exactly what you described. > > Driver has four basic operations: > read/write msr > perform cpu id work For what it's worth, CPUID is not a privileged instruction, so this doesn't need to be done in the kernel (as long as you have a way to pin a userland thread to a cpu, which I believe we have, now). I would personally would *really* to see stas's driver committed as well, as it's really useful. (I had a similar driver (but only for MSRs) that I was about to commit to 7.0 a few months ago, but re@ asked that I add a manpage, and I never got around to doing it: http://people.freebsd.org/~ssouhlal/testing/msr-20070707.diff It's slightly different from devcpu in that it works with lseek + read/write instead of ioctl). -- Suleiman From killing at multiplay.co.uk Sun May 18 20:13:34 2008 From: killing at multiplay.co.uk (Steven Hartland) Date: Sun May 18 20:13:38 2008 Subject: Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket Message-ID: sendmsg is not documented as ever returning EINVAL but yet when using the following code to send credentials to a remote host results in EINVAL from sendmsg. I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX sockets and not PF_INET sockets and hence the code in dbus is actually invalid. Can anyone confirm this is the case or not? [code from dbus-sysdeps-unix.c] write_credentials_byte (int server_fd, DBusError *error) { int bytes_written; char buf[1] = { '\0' }; #if defined(HAVE_CMSGCRED) union { struct cmsghdr hdr; char cred[CMSG_SPACE (sizeof (struct cmsgcred))]; } cmsg; struct iovec iov; struct msghdr msg; iov.iov_base = buf; iov.iov_len = 1; memset (&msg, 0, sizeof (msg)); msg.msg_iov = &iov; msg.msg_iovlen = 1; msg.msg_control = (caddr_t) &cmsg; msg.msg_controllen = CMSG_SPACE (sizeof (struct cmsgcred)); memset (&cmsg, 0, sizeof (cmsg)); cmsg.hdr.cmsg_len = CMSG_LEN (sizeof (struct cmsgcred)); cmsg.hdr.cmsg_level = SOL_SOCKET; cmsg.hdr.cmsg_type = SCM_CREDS; #endif _DBUS_ASSERT_ERROR_IS_CLEAR (error); again: #if defined(HAVE_CMSGCRED) bytes_written = sendmsg (server_fd, &msg, 0); #else bytes_written = write (server_fd, buf, 1); #endif if (bytes_written < 0 && errno == EINTR) goto again; if (bytes_written < 0) { dbus_set_error (error, _dbus_error_from_errno (errno), "Failed to write credentials byte: %s", _dbus_strerror (errno)); return FALSE; } else if (bytes_written == 0) { dbus_set_error (error, DBUS_ERROR_IO_ERROR, "wrote zero bytes writing credentials byte"); return FALSE; } else { _dbus_assert (bytes_written == 1); _dbus_verbose ("wrote credentials byte\n"); return TRUE; } } [/code from dbus-sysdeps-unix.c] ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From rpaulo at FreeBSD.org Sun May 18 21:01:54 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Sun May 18 21:01:59 2008 Subject: rdmsr from userspace In-Reply-To: <20080518152937.774cf492@bhuda.mired.org> References: <482E93C0.4070802@icyb.net.ua> <20080517133037.3a3935db@bhuda.mired.org> <48305044.7000007@FreeBSD.org> <20080518140650.2691af1c@bhuda.mired.org> <48307700.70304@FreeBSD.org> <20080518152937.774cf492@bhuda.mired.org> Message-ID: <4830993B.5070400@FreeBSD.org> Mike Meyer wrote: > > I don't think that would work - you'd have to register all those > hexadecimal strings as sysctl names. Yes, in theory you could also hack sysctl in a way that it doesn't walk when you do a sysctl -a, but works fine when you issue sysctl dev.cpu.N.0xffaabbcc, for example. I would be against this, though, because this will be like "sysctl on steroids", a really crude hack. > You could do an interface like > this, but the calling program would have to use sysctlnametomib to get > dev.cpu.N.msr, then append the MSR number to the results. Not really > very pretty. If you want to allow the user to poke at arbitrary msrs, > this would be a way to do it with sysctls, but the file api is > probably better. Agreed. > On the other hand, if you want to allow access to the fixed set of > documented msr's (for each cpu model, anyway), then handing back that > fixed set as an array would be a better approach, and would have the > advantage of not having to deal with invalid requests (non-existent > MSRs, 1-byte reads/writes of multi-byte MSRs, etc). > > On the other hand, it might be more useful - *especially* if the MSRs > move around depending on processor types (I honestly don't know) They do. At least on Intel (I don't know about AMD). But this "moving" is not documented. > - to > provide a "named" interface: > > dev.cpu.0.msr.mtrr > dev.cpu.0.msr.arr > dev.cpu.0.msr.efer > > and so on. You'd register the names when the module was initialized, > and would only register the ones that actually existed. You'd then > never have to deal with a request for a non-existent MSR, because the > sysctl call would return an error to the calling program without ever > calling your handler. > > Of course, you can *combine* these two approaches, and have: > > dev.cpu.0.msr.all # returns an array of all available msrs > dev.cpu.0.msr.have # an array of the available msrs > > but at this point it's just blue sky speculation... > >> I'll have to think about whether or not I like this interface. > > Actually, I'm more interested in why there seems to be a dislike > of file-based interfaces in favor of sysctls in the freebsd > community. Speaking for myself, sometimes sysctl is an interface that "just works" and is simple enough. Of course there are some types of structured data that don't belong to sysctl, but overall, I don't think we are abusing it. Creating a /dev node, handling ioctl's, etc. is much more work and error prone. That's all. At least for me. Regards, -- Rui Paulo From avg at icyb.net.ua Sun May 18 21:03:08 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Sun May 18 21:03:12 2008 Subject: rdmsr from userspace In-Reply-To: <48307658.2080502@FreeBSD.org> References: <482E93C0.4070802@icyb.net.ua> <482EFBA0.30107@FreeBSD.org> <482F1191.70709@icyb.net.ua> <482F1529.5080409@FreeBSD.org> <20080517175312.GM18958@deviant.kiev.zoral.com.ua> <48304F9D.9030406@FreeBSD.org> <20080518181549.GZ18958@deviant.kiev.zoral.com.ua> <48307658.2080502@FreeBSD.org> Message-ID: <48309983.3070900@icyb.net.ua> on 18/05/2008 21:32 Rui Paulo said the following: > > Yes, but I still don't like having everything mixed up in one driver. At > the very least, I would like us to have two drivers. One for the > microcode update and the other driver for the rest. > > I would like to see a microcode update utility (driver + something to > parse Intel's file aka devcpu-data) in the base system, but not "the > rest", though. Well, I am not sure what is a basis for such a requirement. As I pointed out before we already have /dev/pci and /dev/io and those are not going to go away, because there are quite reasonable applications that require those devices (and wide-spread too). And I think that sufficiently structured (via ioctl interface) access to CPU is also needed for some quite useful (and reasonable) userland applications. I can understand efforts to prevent foot-shooting, but I can not understand an approach of limiting abilities of a (sufficiently) privileged user. After all, he/she can rebuild a kernel and put all they need into it. -- Andriy Gapon From fabonacci at yahoo.com Mon May 19 04:40:38 2008 From: fabonacci at yahoo.com (John Timony) Date: Mon May 19 04:40:44 2008 Subject: Net Mask Prob! Message-ID: <130587.38887.qm@web46316.mail.sp1.yahoo.com> Hi,all ? I have installed Freebsd 7.0 on my Acer TravelMate 220,my router ip is 192.168.0.1,the ip of my Freebsd is 192.168.1.4,Net Mask is 255.255.0.0,Why my system can not connect wiht the router? ? any idea? From tomoyuki at pobox.com Mon May 19 06:11:07 2008 From: tomoyuki at pobox.com (Tomoyuki Murakami) Date: Mon May 19 06:11:19 2008 Subject: Net Mask Prob! In-Reply-To: <130587.38887.qm@web46316.mail.sp1.yahoo.com> References: <130587.38887.qm@web46316.mail.sp1.yahoo.com> Message-ID: <20080519.145112.193697958.tomoyuki@pobox.com> >>> On Sun, 18 May 2008 21:40:37 -0700 (PDT), >>> John Timony wrote: fabonacci> I have installed Freebsd 7.0 on my Acer TravelMate 220,my router ip is 192.168.0.1,the ip of my Freebsd is 192.168.1.4,Net Mask is 255.255.0.0,Why my system can not connect wiht the router? I guess, you have to re-check your router's netmask config. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 305 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080519/46a55ba2/attachment.pgp From koitsu at FreeBSD.org Mon May 19 06:31:34 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Mon May 19 06:31:37 2008 Subject: Net Mask Prob! In-Reply-To: <130587.38887.qm@web46316.mail.sp1.yahoo.com> References: <130587.38887.qm@web46316.mail.sp1.yahoo.com> Message-ID: <20080519063133.GA54791@eos.sc1.parodius.com> On Sun, May 18, 2008 at 09:40:37PM -0700, John Timony wrote: > Hi,all > ? > I have installed Freebsd 7.0 on my Acer TravelMate 220,my router ip is 192.168.0.1,the ip of my Freebsd is 192.168.1.4,Net Mask is 255.255.0.0,Why my system can not connect wiht the router? > ? > any idea? Chances are your router uses 255.255.255.0 as a default netmask; most residential routers do. FreeBSD doesn't have any issues/bugs relating to netmasks which would cause what you're seeing. Additionally, your request probably should have gone to -net, not -hackers. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From beech at freebsd.org Mon May 19 07:44:34 2008 From: beech at freebsd.org (Beech Rintoul) Date: Mon May 19 07:45:02 2008 Subject: Help with copytree code Message-ID: <200805182328.45822.beech@freebsd.org> This copytree code is from bsd.port.mk, and I've been asked to try and find a fix. This is very handy for installing a whole tree (like a web app) keeping everything intact. It wasn't designed to copy to a populated directory so it affects everything in that dir, not just what's being installed. We need to keep it's functionality, but a fix is more than I can come up with. So I thought I'd ask you code gurus to help. If this can be fixed many will thank you! I attached it so it won't get line wrapped. Thanks, Beech -- --------------------------------------------------------------------------------------- Beech Rintoul - FreeBSD Developer - beech@FreeBSD.org /"\ ASCII Ribbon Campaign | FreeBSD Since 4.x \ / - NO HTML/RTF in e-mail | http://www.freebsd.org X - NO Word docs in e-mail | Latest Release: / \ - http://www.FreeBSD.org/releases/7.0R/announce.html --------------------------------------------------------------------------------------- From beech at freebsd.org Mon May 19 07:55:28 2008 From: beech at freebsd.org (Beech Rintoul) Date: Mon May 19 07:55:33 2008 Subject: Help with copytree code In-Reply-To: <200805182328.45822.beech@freebsd.org> References: <200805182328.45822.beech@freebsd.org> Message-ID: <200805182355.24787.beech@freebsd.org> On Sunday 18 May 2008, Beech Rintoul said: > This copytree code is from bsd.port.mk, and I've been asked to try > and find a fix. This is very handy for installing a whole tree > (like a web app) keeping everything intact. It wasn't designed to > copy to a populated directory so it affects everything in that dir, > not just what's being installed. We need to keep it's > functionality, but a fix is more than I can come up with. So I > thought I'd ask you code gurus to help. If this can be fixed many > will thank you! > > I attached it so it won't get line wrapped. > > Thanks, > > Beech Looks like it got stripped off, so I posted it here: http://www.alaskaparadise.com/freebsd/copytree_code -- --------------------------------------------------------------------------------------- Beech Rintoul - FreeBSD Developer - beech@FreeBSD.org /"\ ASCII Ribbon Campaign | FreeBSD Since 4.x \ / - NO HTML/RTF in e-mail | http://www.freebsd.org X - NO Word docs in e-mail | Latest Release: / \ - http://www.FreeBSD.org/releases/7.0R/announce.html --------------------------------------------------------------------------------------- From yanefbsd at gmail.com Mon May 19 08:53:28 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Mon May 19 08:53:52 2008 Subject: Help with copytree code In-Reply-To: <200805182355.24787.beech@freebsd.org> References: <200805182328.45822.beech@freebsd.org> <200805182355.24787.beech@freebsd.org> Message-ID: <7d6fde3d0805190153w7440aeack9bb596cf38cceecf@mail.gmail.com> On Mon, May 19, 2008 at 12:55 AM, Beech Rintoul wrote: > On Sunday 18 May 2008, Beech Rintoul said: > > This copytree code is from bsd.port.mk, and I've been asked to try > > and find a fix. This is very handy for installing a whole tree > > (like a web app) keeping everything intact. It wasn't designed to > > copy to a populated directory so it affects everything in that dir, > > not just what's being installed. We need to keep it's > > functionality, but a fix is more than I can come up with. So I > > thought I'd ask you code gurus to help. If this can be fixed many > > will thank you! > > > > I attached it so it won't get line wrapped. > > > > Thanks, > > > > Beech > > Looks like it got stripped off, so I posted it here: > > http://www.alaskaparadise.com/freebsd/copytree_code What file patterns are you trying to copy over and what are you trying to avoid copying over? -Garrett From olli at lurza.secnetix.de Mon May 19 13:17:19 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Mon May 19 13:17:26 2008 Subject: Help with copytree code In-Reply-To: <200805182328.45822.beech@freebsd.org> Message-ID: <200805191317.m4JDHGbl000932@lurza.secnetix.de> Beech Rintoul wrote: > This copytree code is from bsd.port.mk, and I've been asked to try and > find a fix. This is very handy for installing a whole tree (like a > web app) keeping everything intact. It wasn't designed to copy to a > populated directory so it affects everything in that dir, not just > what's being installed. We need to keep it's functionality, but a fix > is more than I can come up with. Why don't you simply do the chown+chmod on the source tree instead of doing it afterwards on the target tree? I think that would solve the problem. By the way, cpio(1) does have the option -R to change owner and/or group on the target files, so you don't need to run chown(1) afterwards. Unfortunately there doesn't seem to be an option to set the permission modes, though. Finally, I recommend you use the "-print0" option of find and the -0 option of cpio, so it will also work if there are any file names that contain spaces or other special characters. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Unix gives you just enough rope to hang yourself -- and then a couple of more feet, just to be sure." -- Eric Allman From jhb at freebsd.org Mon May 19 15:01:07 2008 From: jhb at freebsd.org (John Baldwin) Date: Mon May 19 15:01:30 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <20080512182328.09a8a173@gumby.homeunix.com.> References: <4828557B.9000506@icyb.net.ua> <20080512182328.09a8a173@gumby.homeunix.com.> Message-ID: <200805191032.03134.jhb@freebsd.org> On Monday 12 May 2008 01:23:28 pm RW wrote: > On Mon, 12 May 2008 17:34:35 +0300 > > Andriy Gapon wrote: > > This is not a real issue, just a code clarification. > > > > First a snippet from sys/i386/i386/vm_machdep.c, cpu_reset_real() > > /* > > * Attempt to force a reset via the Reset Control register at > > * I/O port 0xcf9. Bit 2 forces a system reset when it is > > * written as 1. Bit 1 selects the type of reset to attempt: > > * 0 selects a "soft" reset, and 1 selects a "hard" reset. We > > * try to do a "soft" reset first, and then a "hard" reset. > > */ > > outb(0xcf9, 0x2); > > outb(0xcf9, 0x6); > > > > I think that the comment is correct up to but not including the last > > sentence. Writing 0x2 sets bit 1 to 1 (thus selecting hard reset), and > > writing 0x6 sets both bits 2 and 1 to 1 (thus performing hard reset). > > So we always just do a hard reset, no trying of soft reset (would it > > even make sense to do the last line of the comment says). > > It looks to me as if the comment was added retrospectively by someone > who got the two bits mixed-up when reading the source. If bits 1 and 2 > were the other way around, it would be code for a soft-reset followed > by a hard-reset. Or I just fubar'd the code. The comment and code were added at the same time and it should do a soft reset first. I'll have to go check the docs again to see which is wrong (comment or code). -- John Baldwin From jhb at freebsd.org Mon May 19 15:01:08 2008 From: jhb at freebsd.org (John Baldwin) Date: Mon May 19 15:01:31 2008 Subject: rdmsr from userspace In-Reply-To: <20080518152945.60989b9c@bhuda.mired.org> References: <482E93C0.4070802@icyb.net.ua> <48307700.70304@FreeBSD.org> <20080518152945.60989b9c@bhuda.mired.org> Message-ID: <200805191037.12130.jhb@freebsd.org> On Sunday 18 May 2008 03:29:45 pm Mike Meyer wrote: > On Sun, 18 May 2008 19:35:44 +0100 > > Rui Paulo wrote: > > Mike Meyer wrote: > > > On Sun, 18 May 2008 16:50:28 +0100 > > > > > > Rui Paulo wrote: > > >> Mike Meyer wrote: > > >>> On Sat, 17 May 2008 11:13:52 +0300 > > >>> > > >>> Andriy Gapon wrote: > > >>>> It seems that rdmsr instruction can be executed only at the highest > > >>>> privilege level and thus is not permitted from userland. Maybe we > > >>>> should provide something like Linux /dev/cpu/msr? > > >>>> I don't like interface of that device, I think that ioctl approach > > >>>> would be preferable in this case. > > >>>> Something like create /dev/cpuN and allow some ioctls on it: > > >>>> ioctl(cpu_fd, CPU_RDMSR, arg). > > >>>> What do you think? > > >>> > > >>> Ok, this points directly at a question I've been wondering about, but > > >>> haven't been able to find an answer in the google. > > >>> > > >>> I've been mucking about with general access to sysctl's (a sysctl > > >>> plugin for gkrellm, and a python module for accessing sysctls), and > > >>> with that hammer in my hand, the nail for this problem is obviously a > > >>> dev.cpu.#.msr sysctl. > > >> > > >> How can you request a rdmsr within the sysctl tree? I don't think > > >> sysctl is appropriate here either. > > > > > > Reading (or writing) a sysctl mib can trigger a sysctl handler, which > > > can do pretty much anything. In particular, there are already examples > > > in the kernel where sysctl handlers use devices that don't have /dev > > > entries to get & set their values. Look through kern/kern_cpu.c and > > > i386/cpufreq/p4tcc.c to see the two ends of that kind of > > > connection. In fact, the cpu frequency sysctls would seem to be an > > > excellent model for something like the msr. > > > > > > ioctl, open+read/write, sysctl - they're all just interfaces to kernel > > > handlers. > > > > > > > > > Yes, sure, but who do you select the MSR you want to read or write? > > > > dev.cpu.N. ? > > I don't think that would work - you'd have to register all those > hexadecimal strings as sysctl names. You could do an interface like > this, but the calling program would have to use sysctlnametomib to get > dev.cpu.N.msr, then append the MSR number to the results. Not really > very pretty. If you want to allow the user to poke at arbitrary msrs, > this would be a way to do it with sysctls, but the file api is > probably better. Actually, you don't have to register all of them. You can write a node handler which parses the next item in the MIB directly. Look at the all the proc sysctl's which accept a PID for example. That said, I think if someone already has a device driver done that is fine. I also think it is ok to let root request arbitrary MSR's from userland using an ioctl() or the like. -- John Baldwin From kostikbel at gmail.com Mon May 19 15:39:18 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Mon May 19 15:39:23 2008 Subject: rdmsr from userspace In-Reply-To: <200805191037.12130.jhb@freebsd.org> References: <482E93C0.4070802@icyb.net.ua> <48307700.70304@FreeBSD.org> <20080518152945.60989b9c@bhuda.mired.org> <200805191037.12130.jhb@freebsd.org> Message-ID: <20080519153902.GG18958@deviant.kiev.zoral.com.ua> On Mon, May 19, 2008 at 10:37:11AM -0400, John Baldwin wrote: > On Sunday 18 May 2008 03:29:45 pm Mike Meyer wrote: > > On Sun, 18 May 2008 19:35:44 +0100 > > > > Rui Paulo wrote: > > > Mike Meyer wrote: > > > > On Sun, 18 May 2008 16:50:28 +0100 > > > > > > > > Rui Paulo wrote: > > > >> Mike Meyer wrote: > > > >>> On Sat, 17 May 2008 11:13:52 +0300 > > > >>> > > > >>> Andriy Gapon wrote: > > > >>>> It seems that rdmsr instruction can be executed only at the highest > > > >>>> privilege level and thus is not permitted from userland. Maybe we > > > >>>> should provide something like Linux /dev/cpu/msr? > > > >>>> I don't like interface of that device, I think that ioctl approach > > > >>>> would be preferable in this case. > > > >>>> Something like create /dev/cpuN and allow some ioctls on it: > > > >>>> ioctl(cpu_fd, CPU_RDMSR, arg). > > > >>>> What do you think? > > > >>> > > > >>> Ok, this points directly at a question I've been wondering about, but > > > >>> haven't been able to find an answer in the google. > > > >>> > > > >>> I've been mucking about with general access to sysctl's (a sysctl > > > >>> plugin for gkrellm, and a python module for accessing sysctls), and > > > >>> with that hammer in my hand, the nail for this problem is obviously a > > > >>> dev.cpu.#.msr sysctl. > > > >> > > > >> How can you request a rdmsr within the sysctl tree? I don't think > > > >> sysctl is appropriate here either. > > > > > > > > Reading (or writing) a sysctl mib can trigger a sysctl handler, which > > > > can do pretty much anything. In particular, there are already examples > > > > in the kernel where sysctl handlers use devices that don't have /dev > > > > entries to get & set their values. Look through kern/kern_cpu.c and > > > > i386/cpufreq/p4tcc.c to see the two ends of that kind of > > > > connection. In fact, the cpu frequency sysctls would seem to be an > > > > excellent model for something like the msr. > > > > > > > > ioctl, open+read/write, sysctl - they're all just interfaces to kernel > > > > handlers. > > > > > > > > > > > > > Yes, sure, but who do you select the MSR you want to read or write? > > > > > > dev.cpu.N. ? > > > > I don't think that would work - you'd have to register all those > > hexadecimal strings as sysctl names. You could do an interface like > > this, but the calling program would have to use sysctlnametomib to get > > dev.cpu.N.msr, then append the MSR number to the results. Not really > > very pretty. If you want to allow the user to poke at arbitrary msrs, > > this would be a way to do it with sysctls, but the file api is > > probably better. > > Actually, you don't have to register all of them. You can write a node > handler which parses the next item in the MIB directly. Look at the all the > proc sysctl's which accept a PID for example. That said, I think if someone > already has a device driver done that is fine. I also think it is ok to let > root request arbitrary MSR's from userland using an ioctl() or the like. I started a conversation with Stas about committing the driver. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080519/47a860f8/attachment.pgp From jhs at berklix.org Mon May 19 18:55:30 2008 From: jhs at berklix.org (Julian H. Stacey) Date: Mon May 19 18:55:36 2008 Subject: Not seeing enough memory though its there. In-Reply-To: <200805151824.m4FIOFBm000937@fire.js.berklix.net> References: <200805151824.m4FIOFBm000937@fire.js.berklix.net> Message-ID: <200805191857.m4JIvde7050309@fire.js.berklix.net> I've found the answer to my RAM puzzle: Hardware design limitation. - Main board has 32M soldered on, - Single slot. (lettering on PCB, 168 pin DIMM 3.3V Unbuffered), can take 1 DIMM (though board works without). - Plugging in a 32M Ram DIMM, BIOS reports 64 MB. - Chips on main board: 1 x SB82437VX System Controller 2 x S82438VX Data Path Intel PCIset 430 VX spec claims to support 4 to 128 M DRAM - I guess board implemented as up to 64 M on the DIMM slot & [up to 64M in theory, or 32M in practice] soldered on the board. So 96M max. - Weirdly, 128 DIMM +32 on board reported as 56, Bad DIMM maybe ? (RAMOS Technology Co Ltd RM28S180TC-13AC 128MB, Sync, 133 MHz, CL3.) - I have here loose: 32M, 128 & 256. To raise my dmesg from 64 to 96, I would need to use a 256M DIMM ! - Time to swap with my other machines (though not sure of speed issues). - I also found a probable red herring: Some problem with MS @ 64M http://h10025.www1.hp.com/ewfrf/wc/softwareDownloadIndex?softwareitem=33937&lc=en&cc=us&lang=en&os=19&product=92286&dlc=en Presario Fix for Windows 95 NTKERN Error with 64 MB RAM Intended for use on systems with Windows 95 OSR1 or OSR2. Affects Compaq Presario desktops and notebooks with Windows 95 and EXACTLY 64 MB of memory (RAM). Last posting for Ref.: > Hi Hackers cc Gary J, > On a Compaq Presario 2254 with 7.0-Release & Generic kernel, I'm > not seeing enough RAM: A 128M DIM reports as 56M, a 256M reports as > 96M. Any ideas please ? > > dmesg > FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 > root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: AMD-K6tm w/ multimedia extensions (233.03-MHz 586-class CPU) > Origin = "AuthenticAMD" Id = 0x562 Stepping = 2 > Features=0x8001bf > AMD Features=0x400<> > real memory = 100663296 (96 MB) > avail memory = 84484096 (80 MB) > pnpbios: Bad PnP BIOS data checksum # Dont know why bad > > With a 128M RAM > Without hw.physmem= in /boot/loader.conf > dmesg > real memory = 58720256 (56 MB) > avail memory = 43507712 (41 MB) > sysctl -a | grep mem > hw.physmem: 45572096 > hw.usermem: 33267712 > hw.realmem: 58720256 > With /boot/loader.conf hw.physmem=134217728 > same as above except > hw.usermem: 33312768 > With a 256 RAM > power on Bios auto resizes to about 96M ! > Without hw.physmem= > dmesg > real memory = 100663296 (96 MB) > avail memory = 84484096 (80 MB) > sysctl -a | grep mem > hw.physmem: 87515136 > hw.usermem: 74280960 > hw.realmem: 100663296 > With /boot/loader.conf hw.physmem=268435456 > same as above except > hw.usermem: 74141696 Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail just Ascii plain text. HTML & Base64 text are spam. From jhb at freebsd.org Mon May 19 21:51:58 2008 From: jhb at freebsd.org (John Baldwin) Date: Mon May 19 21:52:03 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <200805191032.03134.jhb@freebsd.org> References: <4828557B.9000506@icyb.net.ua> <20080512182328.09a8a173@gumby.homeunix.com.> <200805191032.03134.jhb@freebsd.org> Message-ID: <200805191751.32336.jhb@freebsd.org> On Monday 19 May 2008 10:32:02 am John Baldwin wrote: > On Monday 12 May 2008 01:23:28 pm RW wrote: > > On Mon, 12 May 2008 17:34:35 +0300 > > > > Andriy Gapon wrote: > > > This is not a real issue, just a code clarification. > > > > > > First a snippet from sys/i386/i386/vm_machdep.c, cpu_reset_real() > > > /* > > > * Attempt to force a reset via the Reset Control register at > > > * I/O port 0xcf9. Bit 2 forces a system reset when it is > > > * written as 1. Bit 1 selects the type of reset to attempt: > > > * 0 selects a "soft" reset, and 1 selects a "hard" reset. We > > > * try to do a "soft" reset first, and then a "hard" reset. > > > */ > > > outb(0xcf9, 0x2); > > > outb(0xcf9, 0x6); > > > > > > I think that the comment is correct up to but not including the last > > > sentence. Writing 0x2 sets bit 1 to 1 (thus selecting hard reset), and > > > writing 0x6 sets both bits 2 and 1 to 1 (thus performing hard reset). > > > So we always just do a hard reset, no trying of soft reset (would it > > > even make sense to do the last line of the comment says). > > > > It looks to me as if the comment was added retrospectively by someone > > who got the two bits mixed-up when reading the source. If bits 1 and 2 > > were the other way around, it would be code for a soft-reset followed > > by a hard-reset. > > Or I just fubar'd the code. The comment and code were added at the same time > and it should do a soft reset first. I'll have to go check the docs again to > see which is wrong (comment or code). So, the comment is correct and not the code. Curiously enough, OpenSolaris does the same thing (it writes 0x2 followed by 0x6), but it has some sort of comment which implies that you have to do a write to set or clear bit 1 before setting bit 2. Linux only uses 0xcf9 on a specific x86 machine (View workstation or some such) in which case it just does a single write of 0x6. I'll test locally to make sure 0x4 and 0x6 both work and if so I will commit the fix. -- John Baldwin From stas at FreeBSD.org Mon May 19 22:25:36 2008 From: stas at FreeBSD.org (Stanislav Sedov) Date: Mon May 19 22:26:02 2008 Subject: Help with copytree code In-Reply-To: <200805182355.24787.beech@freebsd.org> References: <200805182328.45822.beech@freebsd.org> <200805182355.24787.beech@freebsd.org> Message-ID: <20080520015545.c9456ac5.stas@FreeBSD.org> On Sun, 18 May 2008 23:55:22 -0800 Beech Rintoul mentioned: > On Sunday 18 May 2008, Beech Rintoul said: > > This copytree code is from bsd.port.mk, and I've been asked to try > > and find a fix. This is very handy for installing a whole tree > > (like a web app) keeping everything intact. It wasn't designed to > > copy to a populated directory so it affects everything in that dir, > > not just what's being installed. We need to keep it's > > functionality, but a fix is more than I can come up with. So I > > thought I'd ask you code gurus to help. If this can be fixed many > > will thank you! > > > > I attached it so it won't get line wrapped. > > > > Thanks, > > > > Beech > > Looks like it got stripped off, so I posted it here: > > http://www.alaskaparadise.com/freebsd/copytree_code > I think the following will do the trick, though I have not tested it: http://mbsd.msk.ru/dist/copytree.diff -- Stanislav Sedov ST4096-RIPE From lgj at usenix.org Mon May 19 21:55:13 2008 From: lgj at usenix.org (Lionel Garth Jones) Date: Tue May 20 02:19:30 2008 Subject: Top 5 Reasons to Attend USENIX '08 Message-ID: <4831F73F.3080207@usenix.org> ---------------------------------------------- Top 5 Reasons to Attend the 2008 USENIX Annual Technical Conference June 22-27, 2008, Boston, MA http://www.usenix.org/usenix08/progb ---------------------------------------------- USENIX '08 offers a cost-effective, one-stop shop for the latest in IT training, breakthroughs, and systems research. Check out the top 5 reasons to join us in Boston, MA, June 22-27, 2008: 1. Top-notch training: Highly respected experts provide you with new information and skills you can take back to work tomorrow. Topics include: -- Bruce Potter on Botnets: Understanding and Defense -- Peter Baer Galvin on Solaris 10 Administration -- Phil Cox on Securing Virtual Environments -- Alan Robertson on Configuring and Deploying Linux-HA To view the entire training program, see: http://www.usenix.org/events/usenix08/training 2. Invited Talks that feature luminaries in computing discussing timely and important topics, such as: -- Keynote Address by David Patterson, Director of the U.C. Berkeley Parallel Computing Laboratory, on "The Parallel Revolution Has Started: Are You Part of the Solution or Part of the Problem?" -- Plenary Closing Session by Matthew Melis of the NASA Glenn Research Center on "The Columbia Accident Investigation and Returning NASA's Space Shuttle to Flight" -- Drew Endy, Cabot Assistant Professor of Biological Engineering at MIT and a co-founder of the BioBricks Foundation (BBF), on "Programming DNA: A 2-bit Language for Engineering Biology" -- And more! http://www.usenix.org/usenix08/ITs 3. You'll see it here first: -- The latest developments in cutting-edge systems research in the Refereed Papers track. http://www.usenix.org/events/usenix08/tech -- An introduction to interesting new or ongoing work at the Poster Session. http://www.usenix.org/events/usenix08/activities.html#poster 4. Answers to your toughest questions: -- Guru Is In sessions feature experts who come prepared to respond to your most burning technical questions on hot topics. The full list of topics will be announced soon! http://www.usenix.org/events/usenix08/tech 5. The chance to mingle with industry leaders: -- Evening events such as the Birds-of-a-Feather (BoF) sessions offer additional opportunities to network with peers to gain that all-important "insider" IT knowledge. http://www.usenix.org/events/usenix08/bofs.html And that's just the beginning. Visit http://www.usenix.org/usenix08/progb to see the full list of offerings. Don't forget: -- Register by June 6 and save up to $300! http://www.usenix.org/events/usenix08/registration -- Take advantage of the multiple employee discount for groups sending 5 or more: http://www.usenix.org/events/usenix08/registration/#multi -- Chat with other attendees on the USENIX '08 Facebook page: http://www.facebook.com/event.php?eid=12521903959 ---------------------------------------------- 2008 USENIX Annual Technical Conference June 22-27, 2008, Boston, MA http://www.usenix.org/usenix08/progb Early Bird Registration Deadline: June 6, 2008 ----------------------------------------------- From avg at icyb.net.ua Tue May 20 13:37:59 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 20 13:38:04 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <1211246591.00072455.1211234402@10.7.7.3> References: <1210616585.00069210.1210605002@10.7.7.3> <1210627393.00069294.1210614002@10.7.7.3> <1211221409.00072277.1211209802@10.7.7.3> <1211246591.00072455.1211234402@10.7.7.3> Message-ID: <4832D432.2050907@icyb.net.ua> on 20/05/2008 00:51 John Baldwin said the following: > So, the comment is correct and not the code. Curiously enough, OpenSolaris > does the same thing (it writes 0x2 followed by 0x6), but it has some sort of > comment which implies that you have to do a write to set or clear bit 1 > before setting bit 2. Linux only uses 0xcf9 on a specific x86 machine (View > workstation or some such) in which case it just does a single write of 0x6. > I'll test locally to make sure 0x4 and 0x6 both work and if so I will commit > the fix. And this code is most likely never reached in majority of the cases, reset via keyboard controller should just work. BTW, I understand that there is a difference between hard and soft reset in terms of hardware signals being asserted, but I don't quite understand general consequences. I.e. what is a practical difference between hard and soft reset? -- Andriy Gapon From fabonacci at yahoo.com Tue May 20 14:03:47 2008 From: fabonacci at yahoo.com (John Timony) Date: Tue May 20 14:03:54 2008 Subject: Slowly Gnome on FreeBSD 7.0 Message-ID: <171598.39316.qm@web46301.mail.sp1.yahoo.com> BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; =FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}Hi,all I have tried to post this =question to the=20right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I =install=20Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 =by=20sysinstall.. I type these =commands: #Xorg -configure #Xorg -config =xorg.conf.new #echo "exec=20gnome-session">~/.xinitrs #startx After this,I can enter gnome=20destop,but it shows me a window with a title "root",,,and then it=20disapears,,,and then apears again,,,and disapear again time and=20time... The gnome destop is slow,and =repeat the=20window again and again,,,I can not use gnome destop to do any=20thing... P.S. The Gnome destop icon =didn't=20apear... Any idea? S.O.S! By =Fabonacci. BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}Hi,all I have tried to post this question to the right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I install Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 by sysinstall.. I type these commands: #Xorg -configure #Xorg -config xorg.conf.new #echo "exec gnome-session">~/.xinitrs #startx After this,I can enter gnome destop,but it shows me a window with a title "root",,,and then it disapears,,,and then apears again,,,and disapear again time and time... The gnome destop is slow,and repeat the window again and again,,,I can not use gnome destop to do any thing... P.S. The Gnome destop icon didn't apear... Any idea? S.O.S! By Fabonacci. From fabonacci at yahoo.com Tue May 20 14:06:19 2008 From: fabonacci at yahoo.com (John Timony) Date: Tue May 20 14:06:25 2008 Subject: Slowly Gnome on FreeBSD 7.0 Message-ID: <870482.18832.qm@web46305.mail.sp1.yahoo.com> BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; =FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}Hi,all I have tried to post this =question to the=20right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I =install=20Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 =by=20sysinstall.. I type these =commands: #Xorg -configure #Xorg -config =xorg.conf.new #echo "exec=20gnome-session">~/.xinitrs #startx After this,I can enter gnome=20destop,but it shows me a window with a title "root",,,and then it=20disapears,,,and then apears again,,,and disapear again time and=20time... The gnome destop is slow,and =repeat the=20window again and again,,,I can not use gnome destop to do any=20thing... P.S. The Gnome destop icon =didn't=20apear... Any idea? S.O.S! By =Fabonacci. BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}Hi,all I have tried to post this question to the right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I install Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 by sysinstall.. I type these commands: #Xorg -configure #Xorg -config xorg.conf.new #echo "exec gnome-session">~/.xinitrs #startx After this,I can enter gnome destop,but it shows me a window with a title "root",,,and then it disapears,,,and then apears again,,,and disapear again time and time... The gnome destop is slow,and repeat the window again and again,,,I can not use gnome destop to do any thing... P.S. The Gnome destop icon didn't apear... Any idea? S.O.S! By Fabonacci. From fabonacci at yahoo.com Tue May 20 14:06:28 2008 From: fabonacci at yahoo.com (John Timony) Date: Tue May 20 14:06:33 2008 Subject: Slowly Gnome on FreeBSD 7.0 Message-ID: <810582.84518.qm@web46311.mail.sp1.yahoo.com> BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; =FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; =FONT-FAMILY: Helvetica, "Times New Roman"}Hi,all I have tried to post this =question to the=20right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I =install=20Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 =by=20sysinstall.. I type these =commands: #Xorg -configure #Xorg -config =xorg.conf.new #echo "exec=20gnome-session">~/.xinitrs #startx After this,I can enter gnome=20destop,but it shows me a window with a title "root",,,and then it=20disapears,,,and then apears again,,,and disapear again time and=20time... The gnome destop is slow,and =repeat the=20window again and again,,,I can not use gnome destop to do any=20thing... P.S. The Gnome destop icon =didn't=20apear... Any idea? S.O.S! By =Fabonacci. BlankBODY { MARGIN-TOP: 25px; FONT-SIZE: 10pt; MARGIN-LEFT: 25px; COLOR: #000000; FONT-FAMILY: Arial, Helvetica}P.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}LI.msoNormal { MARGIN-TOP: 0px; FONT-SIZE: 10pt; MARGIN-LEFT: 0px; COLOR: #ffffcc; FONT-FAMILY: Helvetica, "Times New Roman"}Hi,all I have tried to post this question to the right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I install Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 by sysinstall.. I type these commands: #Xorg -configure #Xorg -config xorg.conf.new #echo "exec gnome-session">~/.xinitrs #startx After this,I can enter gnome destop,but it shows me a window with a title "root",,,and then it disapears,,,and then apears again,,,and disapear again time and time... The gnome destop is slow,and repeat the window again and again,,,I can not use gnome destop to do any thing... P.S. The Gnome destop icon didn't apear... Any idea? S.O.S! By Fabonacci. From fabonacci at yahoo.com Tue May 20 14:07:11 2008 From: fabonacci at yahoo.com (John Timony) Date: Tue May 20 14:07:17 2008 Subject: Slowly Gnome on FreeBSD 7.0 Message-ID: <112879.11288.qm@web46306.mail.sp1.yahoo.com> Hi,all I have tried to post this =question to the=20right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I =install=20Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 =by=20sysinstall.. I type these =commands: #Xorg -configure #Xorg -config =xorg.conf.new #echo "exec=20gnome-session">~/.xinitrs #startx After this,I can enter gnome=20destop,but it shows me a window with a title "root",,,and then it=20disapears,,,and then apears again,,,and disapear again time and=20time... The gnome destop is slow,and =repeat the=20window again and again,,,I can not use gnome destop to do any=20thing... P.S. The Gnome destop icon =didn't=20apear... Any idea? S.O.S! By=20Fabonacci. Hi,all I have tried to post this question to the right mail list,but have no responds,so i try to post here. In my Acer TravelMate 220,I install Freebsd7.0 by using Minimal way,and then install Xorg,ports,and Gnome2 by sysinstall.. I type these commands: #Xorg -configure #Xorg -config xorg.conf.new #echo "exec gnome-session">~/.xinitrs #startx After this,I can enter gnome destop,but it shows me a window with a title "root",,,and then it disapears,,,and then apears again,,,and disapear again time and time... The gnome destop is slow,and repeat the window again and again,,,I can not use gnome destop to do any thing... P.S. The Gnome destop icon didn't apear... Any idea? S.O.S! By Fabonacci. From v.werth at bally-wulff.de Tue May 20 15:11:12 2008 From: v.werth at bally-wulff.de (Werth, Volker) Date: Tue May 20 15:11:20 2008 Subject: Slowly Gnome on FreeBSD 7.0 In-Reply-To: <112879.11288.qm@web46306.mail.sp1.yahoo.com> References: <112879.11288.qm@web46306.mail.sp1.yahoo.com> Message-ID: <200805201653.59940.v.werth@bally-wulff.de> Volker Werth schrieb am 20.05.2008 16:53 _____________________________________________________________________ Am Dienstag, 20. Mai 2008 16:07 schrieb John Timony: > Hi,all > > I have tried to post this =question to the=20right mail list,but have no > responds,so i try to post here. > > In my Acer TravelMate 220,I =install=20Freebsd7.0 by using Minimal way,and > then install Xorg,ports,and Gnome2 =by=20sysinstall.. > > I type these =commands: > > #Xorg -configure > #Xorg -config =xorg.conf.new > #echo "exec=20gnome-session">~/.xinitrs > #startx > > After this,I can enter gnome=20destop,but it shows me a window with a title > "root",,,and then it=20disapears,,,and then apears again,,,and disapear > again time and=20time... > > The gnome destop is slow,and =repeat the=20window again and again,,,I can > not use gnome destop to do any=20thing... > > P.S. The Gnome destop icon =didn't=20apear... > > Any idea? > > S.O.S! > > By=20Fabonacci. > > Hi,all > > I have tried to post this question to the right mail list,but have no > responds,so i try to post here. > > In my Acer TravelMate 220,I install Freebsd7.0 by using Minimal way,and > then install Xorg,ports,and Gnome2 by sysinstall.. > > I type these commands: > > #Xorg -configure > #Xorg -config xorg.conf.new > #echo "exec gnome-session">~/.xinitrs > #startx > > After this,I can enter gnome destop,but it shows me a window with a title > "root",,,and then it disapears,,,and then apears again,,,and disapear again > time and time... > > The gnome destop is slow,and repeat the window again and again,,,I can not > use gnome destop to do any thing... > > P.S. The Gnome destop icon didn't apear... > > Any idea? John, what do you think about `/usr/local/etc/rc.d/gdm start`? AFAIR it's in the Gnome documentation. Volker PS: Please - no html mail :( -- Volker Werth ph: +49(30)62002-109 Bally Wulff Entertainment GmbH, Maybachufer 48-51, 12045 Berlin, Postanschrift: Postfach 44 01 57, 12001 Berlin Tel.: 030-620 02-0 FAX: 030-620 02-200, Gesch?ftsf?hrer: Axel Herr, Amtsgericht Berlin-Charlottenburg, HRB 91532 B, UST-ID DE 234 517 998 _____________________________________________________________________ Dieses E-Mail ist nur f?r den Empf?nger bestimmt, an den es gerichtet ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht oder Meinungs- ?u?erung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder Meinung von Bally Wulff Entertainment GmbH dar. Sind Sie nicht der Empf?nger, so haben Sie diese E-Mail irrt?mlich erhalten und jegliche Verwendung, Ver?ffentlichung, Weiterleitung, Abschrift oder jeglicher Druck dieser E-Mail ist strengstens untersagt. Weder Bally Wulff Entertainment GmbH noch der Absender (Volker Werth) ?bernehmen die Haftung f?r Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren 0 Anh?nge auf Viren zu pr?fen. 0 Anh?nge: _____________________________________________________________________ Versand am 20.05.2008 16:53 von Werth Volker From ed at 80386.nl Tue May 20 17:06:40 2008 From: ed at 80386.nl (Ed Schouten) Date: Tue May 20 17:06:43 2008 Subject: FreeBSD and LLVM Message-ID: <20080520170639.GE1181@hoeg.nl> Hello everyone, First of all, for those of you who went to BSDCan, I hope you had a pleasant flight/trip back home. :-) On Saturday I went to the LLVM talk (see http://llvm.org/), which I really enjoyed. On Friday Remko Lodder and I already talked with him about the LLVM project. I was excited about the project, so I decided to give it a try at the office. At first I tried LLVM 2.2 with LLVM GCC4 4.2 from Ports, but it didn't work like expected. I won't go into many details about it. When I discussed the problems I was seeing on my system at the office, someone pointed me to the beta tarballs of the upcoming version 2.3, which I installed by patching our FreeBSD port. http://llvm.org/prereleases/2.3/ As an ideal benchmark, I decided to compile an i386 kernel using the LLVM 2.3 snapshot. I didn't expect it to happen, but it works! I was capable of successfully booting into single user mode and shutting it down safely. There is one problem however: http://llvm.org/bugs/show_bug.cgi?id=2267 For some reason, the inline asm support of LLVM is incomplete and causes compilation errors when generating some of the atomic functions in i386/include/atomic.h (lines 262 to 265). To work around this, I made the functions non-atomic. Silly, I know, but it was good enough to perform some basic tests. I think it would be nice if LLVM would once become our standard C compiler. LLVM currently uses GCC as its frontend, which proves to be somewhat compatible with the original GCC> -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080520/bfc8f59b/attachment.pgp From jhb at freebsd.org Tue May 20 18:31:18 2008 From: jhb at freebsd.org (John Baldwin) Date: Tue May 20 18:31:23 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <4832D432.2050907@icyb.net.ua> References: <1210616585.00069210.1210605002@10.7.7.3> <1211246591.00072455.1211234402@10.7.7.3> <4832D432.2050907@icyb.net.ua> Message-ID: <200805201124.23850.jhb@freebsd.org> On Tuesday 20 May 2008 09:37:54 am Andriy Gapon wrote: > on 20/05/2008 00:51 John Baldwin said the following: > > So, the comment is correct and not the code. Curiously enough, OpenSolaris > > does the same thing (it writes 0x2 followed by 0x6), but it has some sort of > > comment which implies that you have to do a write to set or clear bit 1 > > before setting bit 2. Linux only uses 0xcf9 on a specific x86 machine (View > > workstation or some such) in which case it just does a single write of 0x6. > > I'll test locally to make sure 0x4 and 0x6 both work and if so I will commit > > the fix. > > And this code is most likely never reached in majority of the cases, > reset via keyboard controller should just work. Except on boxes where it doesn't (and hence why I added it). > BTW, I understand that there is a difference between hard and soft reset > in terms of hardware signals being asserted, but I don't quite > understand general consequences. I.e. what is a practical difference > between hard and soft reset? I've no idea. It may be that we should just always do a hard reset as that is in effect what both Solaris and Linux do. I'll probably just change it to do that. -- John Baldwin From avg at icyb.net.ua Tue May 20 18:58:59 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 20 18:59:02 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <200805201124.23850.jhb@freebsd.org> References: <1210616585.00069210.1210605002@10.7.7.3> <1211246591.00072455.1211234402@10.7.7.3> <4832D432.2050907@icyb.net.ua> <200805201124.23850.jhb@freebsd.org> Message-ID: <48331F69.3070205@icyb.net.ua> on 20/05/2008 18:24 John Baldwin said the following: > On Tuesday 20 May 2008 09:37:54 am Andriy Gapon wrote: >> BTW, I understand that there is a difference between hard and soft reset >> in terms of hardware signals being asserted, but I don't quite >> understand general consequences. I.e. what is a practical difference >> between hard and soft reset? > > I've no idea. It may be that we should just always do a hard reset as that is > in effect what both Solaris and Linux do. I'll probably just change it to do > that. But I think it is already how the code works (maybe with extra port write that couldn't hurt anything). -- Andriy Gapon From jhb at freebsd.org Tue May 20 19:35:32 2008 From: jhb at freebsd.org (John Baldwin) Date: Tue May 20 19:35:37 2008 Subject: i386 cpu_reset_real: code/comment mismatch In-Reply-To: <48331F69.3070205@icyb.net.ua> References: <1210616585.00069210.1210605002@10.7.7.3> <200805201124.23850.jhb@freebsd.org> <48331F69.3070205@icyb.net.ua> Message-ID: <200805201534.42201.jhb@freebsd.org> On Tuesday 20 May 2008 02:58:49 pm Andriy Gapon wrote: > on 20/05/2008 18:24 John Baldwin said the following: > > On Tuesday 20 May 2008 09:37:54 am Andriy Gapon wrote: > >> BTW, I understand that there is a difference between hard and soft reset > >> in terms of hardware signals being asserted, but I don't quite > >> understand general consequences. I.e. what is a practical difference > >> between hard and soft reset? > > > > I've no idea. It may be that we should just always do a hard reset as that is > > in effect what both Solaris and Linux do. I'll probably just change it to do > > that. > > But I think it is already how the code works (maybe with extra port > write that couldn't hurt anything). Right, I'll probably just update the comment and remove the '0x2' write. -- John Baldwin From rdivacky at freebsd.org Wed May 21 08:39:37 2008 From: rdivacky at freebsd.org (Roman Divacky) Date: Wed May 21 08:39:42 2008 Subject: FreeBSD and LLVM In-Reply-To: <20080520170639.GE1181@hoeg.nl> References: <20080520170639.GE1181@hoeg.nl> Message-ID: <20080521082233.GA71289@freebsd.org> On Tue, May 20, 2008 at 07:06:39PM +0200, Ed Schouten wrote: > Hello everyone, > > First of all, for those of you who went to BSDCan, I hope you had a > pleasant flight/trip back home. :-) > > On Saturday I went to the LLVM talk (see http://llvm.org/), which I > really enjoyed. On Friday Remko Lodder and I already talked with him > about the LLVM project. I was excited about the project, so I decided to > give it a try at the office. > > At first I tried LLVM 2.2 with LLVM GCC4 4.2 from Ports, but it didn't > work like expected. I won't go into many details about it. > > When I discussed the problems I was seeing on my system at the office, > someone pointed me to the beta tarballs of the upcoming version 2.3, > which I installed by patching our FreeBSD port. > > http://llvm.org/prereleases/2.3/ > > As an ideal benchmark, I decided to compile an i386 kernel using the > LLVM 2.3 snapshot. I didn't expect it to happen, but it works! I was did you try clang as well? I wonder what it's able to do.... > capable of successfully booting into single user mode and shutting it > down safely. There is one problem however: > > http://llvm.org/bugs/show_bug.cgi?id=2267 > > For some reason, the inline asm support of LLVM is incomplete and causes > compilation errors when generating some of the atomic functions in > i386/include/atomic.h (lines 262 to 265). To work around this, I made > the functions non-atomic. Silly, I know, but it was good enough to > perform some basic tests. > > I think it would be nice if LLVM would once become our standard C > compiler. LLVM currently uses GCC as its frontend, which proves to be > somewhat compatible with the original GCC> yeah... thats a worthy goal :) thnx! roman From ed at 80386.nl Wed May 21 17:23:54 2008 From: ed at 80386.nl (Ed Schouten) Date: Wed May 21 17:24:01 2008 Subject: FreeBSD and LLVM In-Reply-To: <48330CDA.2080802@FreeBSD.org> References: <20080520170639.GE1181@hoeg.nl> <48330CDA.2080802@FreeBSD.org> Message-ID: <20080521172352.GJ1181@hoeg.nl> Hello Stefan, (CC'ing this back to the list) * Stefan Esser wrote: > this is great news! Can you provide your patches to upgrade the port > to 2.3-pre? I could, but I patched the ports rather poorly. The plists are incorrect. I just incremented the version numbers, updated the distinfo and made some changes to the llvm-gcc4 port to use the proper wrksrc. > Did you, by chance, record build times with llvm-gcc compared to the > system compiler? > > What about the quality of generated code (size, runtime)? > Did you manage to get a meaning-ful program built that could be used > as a benchmark? I was able to compile a kernel with SMP disabled and PAE enabled. Other combinations will not work. LLVM doesn't seem to be properly capable of dealing with the xchg instruction in inline asm statements. A non-SMP PAE kernel doesn't seem to use them :-) Below are some build stats: | llvm: | | 114.04s user 24.86s system 176% cpu 1:18.73 total ^ output of time(1) | | -rwxr-xr-x 1 ed wheel 4456839 May 21 12:59 ^ output of ls -l kernel before stripping | -rwxr-xr-x 1 ed wheel 3596008 May 21 13:16 ^ output of ls -l kernel after stripping | | text data bss dec hex | 3286323 229605 462280 3978208 3cb3e0 ^ output of size(1) after stripping | gcc: | | 119.04s user 22.64s system 178% cpu 1:19.27 total | | -rwxr-xr-x 1 ed wheel 4098317 May 21 12:33 | -rwxr-xr-x 1 ed wheel 3463448 May 21 12:57 | | text data bss dec hex | 3185574 220965 460216 3866755 3b0083 As you can see, compilation time is about the same, but the kernel image is a little bigger. I don't know anything about the quality of the generated code. > If 2.3-pre allows the kernel to boot and run, then many user-land > programs should work as well. Testing a make world (and preparing > the infrastructure to support llvm in addition to gcc and icc might > be worthwhile ...). We should really try that. Unfortunately I'm quite busy, hacking on the TTY layer. ;-) > You probably know about the clang project, which tries to completely > replace the gcc parts needed for llvm-gcc ... Yes. I haven't looked at it yet. It doesn't seem to be in Ports yet. Any takers? ;-) -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080521/501955bd/attachment.pgp From richard at vanmansom.net Wed May 21 17:46:16 2008 From: richard at vanmansom.net (Richard van Mansom) Date: Wed May 21 17:46:20 2008 Subject: Hifn 7955 doesn't work with Freebsd 7.0-release Message-ID: <001001c8bb66$c4713170$4d539450$@net> Hi, I am trying to setup two Soekris 4521 with a minipci vpn1411 (Hi/fn 7955) in a vpn. I understood that the crypto card should automatically work with only three kernel configuration file modification. So I added these three lines device crypto device cryptodev device hifn I tested with openvpn (the one release with pkg_add). I used the default cipher (I think this is BF-CBC - Blowfish 128 bit). The VPN works but I didn't notice any difference in performance (with or without the crypto card). I also tested the crypto card with AES128 but the performance only got worse (didn't have a baseline for that one) Relevant output: dmesg | grep hifn hifn0 mem 0xa0000000-0xa0000fff,0xa0002000-0xa0003fff,0xa0008000-0xa000ffff irq 10 at device 16.0 on pci0 hifn0: [ITHREAD] hifn0: Hifn 7955, rev 0, 32KB dram, pll=0x801 dmesg | grep crypto cryptosoft0: on motherboard uname -a FreeBSD Soekris 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sat May 17 10:53:38 UTC 2008 root@One:/usr/obj/usr/src/sys/C5 i386 Any help would be appreciated Richard From mwm at mired.org Wed May 21 18:39:01 2008 From: mwm at mired.org (Mike Meyer) Date: Wed May 21 18:48:38 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080516141307.GB1032@lizard.fafoe.narf.at> References: <20080309152712.42752293@bhuda.mired.org> <20080516074433.GA1032@lizard.fafoe.narf.at> <20080516101143.3545937c@bhuda.mired.org> <20080516141307.GB1032@lizard.fafoe.narf.at> Message-ID: <20080521143857.454dc869@mbook.local> On Fri, 16 May 2008 16:13:07 +0200 Stefan Farfeleder wrote: > On Fri, May 16, 2008 at 10:11:43AM -0400, Mike Meyer wrote: > > On Fri, 16 May 2008 09:44:33 +0200 > > Stefan Farfeleder wrote: > > > > > On Sun, Mar 09, 2008 at 03:27:12PM -0400, Mike Meyer wrote: > > > > I've stumbled on to an obscure problem with autoconf 2.61, and I'm not > > > > sure quite what to do with it. I've already sent mail to the autoconf > > > > folks, but I'd like to understand what's going on. > > > > > > > > The problem is that, on a FreeBSD system with only /bin/sh and the > > > > ports zsh as installed shells, if you have SHELL set to zsh when > > > > invoking the autoconf-generated configure script, the script produces > > > > a broken Makefile. It doesn't generate an error, it just complains > > > > that: > > > > > > Can you please retry? /bin/sh now supports expanding $LINENO which was > > > often the reason for configure not liking it. And autoconf seems happy to use it. Thanks, This question was previously posed of the freebsd-questions list, but with no response for a week, I'd like to try my luck here. If there's any more information I should include, please speak up: I would be glad to oblige. I would like to use smb/cifs with kerberos auth, but mount_smbfs doesn't seem to support this. Is anyone aware of an alternate means of performing a mount via smb/cifs or any patches to provide such functionality? I already have smbclient working with -k, but I am also interested in a mount. Thanks -Derek. From mike at sentex.net Wed May 21 23:58:04 2008 From: mike at sentex.net (mike@sentex.net) Date: Wed May 21 23:58:11 2008 Subject: Hifn 7955 doesn't work with Freebsd 7.0-release In-Reply-To: <001001c8bb66$c4713170$4d539450$@net> References: <001001c8bb66$c4713170$4d539450$@net> Message-ID: > >I tested with openvpn (the one release with pkg_add). I used the default >cipher (I think this is BF-CBC - Blowfish 128 bit). The VPN works but I Hi, See the man page for supported ciphers. >didn't notice any difference in performance (with or without the crypto >card). >I also tested the crypto card with AES128 but the performance only got worse >(didn't have a baseline for that one) For single crypto streams, you are not going to see any improvement really. Where it works, is when you have multiple connections. e.g. on our old backup server, we would have several dumps coming in over ssh (3des) and the card made a significant reduction in CPU usage. It doesnt really improve single crypto streams performance wise. You can also confirm its working by using hifnstats in /usr/src/tools/tools/ ---Mike From sam at freebsd.org Thu May 22 03:59:28 2008 From: sam at freebsd.org (Sam Leffler) Date: Thu May 22 03:59:31 2008 Subject: Hifn 7955 doesn't work with Freebsd 7.0-release In-Reply-To: <001001c8bb66$c4713170$4d539450$@net> References: <001001c8bb66$c4713170$4d539450$@net> Message-ID: <4834E63E.10307@freebsd.org> Richard van Mansom wrote: > Hi, > > I am trying to setup two Soekris 4521 with a minipci vpn1411 (Hi/fn 7955) in > a vpn. > > I understood that the crypto card should automatically work with only three > kernel configuration file modification. > > So I added these three lines > > device crypto > device cryptodev > device hifn > > I tested with openvpn (the one release with pkg_add). I used the default > cipher (I think this is BF-CBC - Blowfish 128 bit). The VPN works but I > didn't notice any difference in performance (with or without the crypto > card). > I also tested the crypto card with AES128 but the performance only got worse > (didn't have a baseline for that one) > > Relevant output: > dmesg | grep hifn > hifn0 mem 0xa0000000-0xa0000fff,0xa0002000-0xa0003fff,0xa0008000-0xa000ffff > irq 10 at device 16.0 on pci0 > hifn0: [ITHREAD] > hifn0: Hifn 7955, rev 0, 32KB dram, pll=0x801 > > dmesg | grep crypto > cryptosoft0: on motherboard > > uname -a > FreeBSD Soekris 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sat May 17 10:53:38 UTC > 2008 root@One:/usr/obj/usr/src/sys/C5 i386 > > Any help would be appreciated > > Unfortunately openssl doesn't use the accelerator by default. This means all apps that use openssl likewise are not automatically accelerated. I suggested a patch but it was not accepted. I can't recall how you force openssl and/or consumers to use the device. If you want to check whether the kernel support is working correctly look in src/tools/tools/crypto for cryptotest and hifnstats. Sam From fhcarron at terra.es Thu May 22 04:36:07 2008 From: fhcarron at terra.es (Fernando Herrero =?ISO-8859-1?Q?Carr=F3n?=) Date: Thu May 22 04:36:13 2008 Subject: Gprof C++ support In-Reply-To: <689122.62781.qm@web56102.mail.re3.yahoo.com> References: <689122.62781.qm@web56102.mail.re3.yahoo.com> Message-ID: <1211390720.1152.2.camel@localhost> El s?b, 03-05-2008 a las 09:17 -0700, Purushotham Nayak escribi?: > Hi All, > I've been trying to use gprof on some C++ code and appears it doesn't > demangle C++ function names. I was wondering if anyone is working on > it? I would like to contribute/help in adding this. > Purushotham You can try c++filt. From kayve at sfsu.edu Thu May 22 04:45:52 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Thu May 22 04:46:03 2008 Subject: nessus gtk yields empty scan In-Reply-To: References: <001001c8bb66$c4713170$4d539450$@net> Message-ID: [kayve@kv_bsd ~]$ uname -a FreeBSD kv_bsd 6.3-STABLE FreeBSD 6.3-STABLE #0: Wed May 7 19:40:55 PDT 2008 root@kv_bsd:/usr/obj/usr/src/sys/GENERIC i386 [kayve@kv_bsd ~]$ pkg_info | grep essus pkg_info: show_file: can't open '+COMMENT' for reading nessus-gtk2-2.2.9_1 A security scanner: looks for vulnerabilities in a given ne nessus-libnasl-2.2.9_1 Nessus Attack Scripting Language nessus-libraries-2.2.9_1 Libraries for Nessus, the security scanner [kayve@kv_bsd ~]$ when i boot there seems like there are a lot of rpm errors during the nessus loads. i think something is wrong but i don't know what. i don't know what to say i don't know what is wrong i can type faster without caps it is NOT that hard to read. the gtk GUI has a lot of plugins i think they are all selected there is a host called minkay.sfsu.edu i am supposed to scan i have a log in i put in host 10.1.1.1 like this webpage says ttp://www.securityfocus.com/infocus/1741 oops. i pasted it twice below sorry. 1.0 Introduction Nessus is a great tool designed to automate the testing and discovery of known security problems. Typically someone, a hacker group, a security company, or a researcher discovers a specific way to violate the security of a software product. The discovery may be accidental or through directed research; the vulnerability, in various levels of detail, is then released to the security community. Nessus is designed to help identify and solve these known problems, before a hacker takes advantage of them. Nessus is a great tool with lots of capabilities. However it is fairly complex and few articles exist to direct the new user through the intricacies of how to install and use it. Thus, this article shall endeavor to cover the basics of Nessus setup and configuration. The features of the current versions of Nessus (Nessus 2.0.8a and NessusWX 1.4.4) will be discussed. Future articles will cover Nessus in more depth. Nessus is a free program released under the GPL. Historically, many in the corporate world have ridiculed such public domain software as being a waste of time, instead choosing "supported" products developed by established companies. Typically these packages cost hundreds or thousands of dollars, and are often purchased using the logic that you get what you pay for. Some people are starting to realize that public domain software, such as Nessus, isn't always inferior and sometimes it is actually superior. Paid technical support for Nessus is even available from www.tenablesecurity.com. Nessus also has a great community of developers anchored by the primary author, Renaud Deraison. When allowed to fairly compete in reviews against other vulnerability scanners, Nessus has equaled or outshined products costing thousands of dollars. [ref: Information Security, Network Computing] One of the very powerful features of Nessus is its client server technology. Servers can be placed at various strategic points on a network allowing tests to be conducted from various points of view. A central client or multiple distributed clients can control all the servers. The server portion will run on most any flavor of Unix. It even runs on MAC OS X and IBM/AIX, but Linux tends to make the installation simpler. These features provide a great deal of flexibility for the penetration tester. Clients are available for both Windows and Unix. The Nessus server performs the actual testing while the client provides configuration and reporting functionality. 2.0 Installation Nessus server installation is fairly simple even for a Windows jockey like me. First an installed version of Unix is required. Secondly, prior installation of several external programs is recommended: NMAP is the industry standard for port scanners, Hydra is a weak password tester and Nikto is a cgi/.script checker. While not required, these external programs greatly enhance Nessus' scanning ability. They are included because they are the best applications in their class. If installed in the PATH$ before Nessus installation, they will automatically be available. The simplest installation method is using the Lynx automatic install. Lynx is included on many of the linux versions. The Lynx command is (logged in as a user, and not root) : lynx -source http://install.nessus.org | sh This should install the server on most platforms with no other steps necessary. Note that the latest install script can also be downloaded and run locally. Whether you install directly off the Website or using the same install script offline, either way the script will setup a temporary suid and ask for your root password when required -- if you don't like this feature you can download, compile and install the four required tarballs individually. The above command should also be used periodically to upgrade Nessus as new versions are regularly released. You will be questioned about proxy servers, a download method (www or CVS), and the branch of the development tree to use; most of the time the defaults are the best choice. This is the simplest method of installation however; you are effectively giving the install.nessus.org server temporary root privileges. Thus there is a security risk with this method albeit a low one. So if you are paranoid, and paranoid is not always a bad thing in the security field, installation can be done the old-fashioned way by downloading and compiling the source. For information on performing an install from scratch see: www.nessus.org/nessus_2_0.html. 3.0 Setup Once the server is installed, some basic setup steps are required. The first task to complete in the new install is to add a user. A new user can be added by the "nessus-adduser" command. The script will question you for the authentication method. Authentication can be performed by several means, however a password is the simplest and is recommended. The next question queries about rules to restrict the user account. When used across an enterprise, a user can be restricted and only allowed to scan specified IP addresses. However, for most uses this will be left blank, allowing the user to scan anything. A certificate also needs to be generated as well to be used to encrypt the traffic between the client and server. The nessus-mkcert command accomplishes this. 3.1 Update plug-ins Before a scan is done, the plug-ins should be updated. Nessus plug-ins are very much like virus signatures in a common virus scanner application. Each plug-in is written to test for a specific vulnerability. These can be written to actually exploit the vulnerability or just test for known vulnerable software versions. Plug-ins can be written in most any language but usually are written in the Nessus Attack Scripting Language (NASL). NASL is Nessus' own language, specifically designed for vulnerability test writing. Each plug-in is written to test for a specific known vulnerability and/or industry best practices. NASL plug-ins typically test by sending very specific code to the target and comparing the results against stored vulnerable values. There are a few built-in plug-ins that do not use NASL. These are C and Perl scripts to perform special purposes that can not easily be done in NASL. Among these is the Services plug-in which identifies port-to-program mappings. Plug-in updates should be done frequently. New vulnerabilities are being discovered and disseminated all the time. Typically after a new vulnerability is released to the public, someone in the Nessus community writes a NASL plug-in, releases it to the public and submits it to www.nessus.org. It is then reviewed by the developers and added to the approved plug-in list. For high risk, high profile vulnerabilities a plug-in is often released the same day the vulnerability information is publicly released. Updating plug-ins from the maintained list is fairly simple involving a simple command: nessus-update-plugins. This command must be done as root. By no means however, are you limited to the list of plug-ins from www.nessus.org. New and special purpose plug-ins can be written relatively easily using NASL, so you can write your own custom plug-ins as well. 3.2 Launch the daemon Nessus is now installed, updated and ready to go. The simplest way to get the server running is (as root) issue the nessusd -D command. In order to use it, one must use a client. There are three primary Nessus clients. The native Unix GUI version is installed at server install time. Alternatively, Nessus can be controlled from the command line. A third option, a Windows version also exists called NessusWX. The binaries for NessusWX can be found here . The NessusWX install is a straightforward Windows install. All three clients work well. Personally I prefer NessusWX. It is better organized, allows for easier reporting, and has a better facility for managing different sessions (groups of hosts to scan) then its Unix counterparts. To run the native Unix GUI client, run the nessus command or for NessusWX click the eye icon after installation. 3.3 Client connection Since Nessus is a client server technology, once running the client a connection must be made to the server. In the native client, enter the server IP, username and password (created with the nessus-adduser command) and hit login. The process in NessusWX is similar but uses the communications | connect menus. The client is connected to the server thru an SSL connection and a list of the currently installed plug-ins is downloaded. On the first run the SSL certificate is also downloaded and verification is requested. This verification ensures that in the future you are actually communicating with the server intended. Figures 1 and 2 shows the connection using the Unix and Windows GUI tools, respectively. Figure 3 shows user authentication using the NessusWX client. Figure 1: Starting the Nessus server and connecting with the Unix GUI Figure 1: Starting the Nessus server and connecting with the Unix GUI Figure 2: Connecting to the Nessus server with NessusWX (Windows Client) Figure 2: Connecting to the Nessus server with NessusWX (Windows Client) Figure 3: Enter in the server IP and the login and password setup with nessus_adduser Figure 3: Enter in the server IP and the login and password setup with nessus_adduser 4.0 Using Nessus Now that we have installed and connected to Nessus lets explore some of the options available. The most obvious and powerful aspect of Nessus is its plug-in feature. The choice of plug-ins is critical to the success of a scan. Most plug-ins are written very well and rarely trigger false positives or negatives; however, a few are not. One example of a poorly written plug-in is the test for the classic Windows IIS hack RFP's MSDAC /RDS vulnerability. Rain Forest Puppy (RFP) publicized this vulnerability in 1999. The vulnerability makes use of the %system%/msadc/msadcs.dll file and leads to total system compromise on un-patched IIS 4.0 servers. The problem is that the Nessus plug-in only tests for the existence of the /msadc/msadcs.dll file. It does not take into account patches, windows versions etc. Thus with this plug-in enabled, a false positive will show up on many IIS servers and must be sorted out manually. Before anyone yells out, "see the problems of public domain software," one should note that the same types of problems exist with the high priced "supported" vulnerability scanners as well. This problem is a result of the current state of the technology. The difference is that typically in purchased products you cannot easily examine the exact "proprietary" testing methodology as can be done with Nessus, thus making resolution of the false positive difficult. 4.1 Choosing dangerous/non-dangerous plug-ins Plug-ins are categorized in several different and sometimes confusing ways. One method of plug-in grouping is by category. Most importantly, some plug-ins are categorized as Dangerous/Denial of Service (DOS). These plug-ins will actually perform a DOS attack and crash systems that have these particular problems. Needless to say these should not be blindly run on production systems. They won't cause long term damage, but at least a reboot will be required. In both clients, there are buttons to "Enable all plug-ins" or just "Enable all but dangerous plug-ins" (called "Enable Non-DOS" in NessusWX). Note that the author of the plug-in decides if it is dangerous or not. Most of the time, this has been very well chosen. However there are instances (Exmple: the rpc_endpoint mapper plug-in), where the plug-in causes a DOS but it is not listed as dangerous. The native client denotes dangerous plug-ins with a caution triangle. NessusWX has no special way to notate a dangerous plug-in other then using the enable "Enable Non-DOS" button. One other thing to be aware of is that sometimes even a "non-dangerous" plug-in can crash software. Since the plug-ins are sending non-standard data, there is always the risk, albeit rare, that a new undiscovered DOS will be stumbled upon. Therefore anytime systems are being scanned one should be aware that system crashes, although unlikely, are possible even with "non-dangerous" plug-ins. Figure 4, below, shows plug-in selection using the Unix GUI. Similarly, Figures 5 and 6 show plug-in selection using NessusWX for Windows: Figure 4: Enabling all but dangerous plugins with the Unix Nessus GUI Figure 4: Enabling all but dangerous plugins with the Unix Nessus GUI Figure 5: Selecting plug-ins with the Windows NessusWX Client Figure 5: Selecting plug-ins with the Windows NessusWX Client Figure 6: Enabling non-dangerous plug-ins with the Windows NessusWX Client Figure 6: Enabling non-dangerous plug-ins with the Windows NessusWX Client 4.2 Safe-checks This is a good place to mention the related concept of safe-checks. Safe-checks disables the dangerous parts of safe-check compatible plug-ins and causes them to check just through passive methods such as version numbers in banners. Since a patch or workaround may be installed, safe-checks are not as reliable as actually exploiting the vulnerability. They might cause false positives or false negatives. The valuable trade off is that they should not crash a machine. The safe-check option is on the scan options tab (the options tab in NessusWX). Figure 7 shows the safe-check option in the NessusWX interface: Figure 7: Choosing Safe Checks Figure 7: Choosing Safe Checks The second method of plug-in organization is in families such as Windows, FTP SNMP, SMB, Cisco, etc. I find this to be a rather unhelpful grouping due to the arbitrary categorizing process. Should an FTP vulnerability that only exists on a Windows box go in the Windows family or the FTP family? Since this decision is left up to the plug-in writer with little guidance, there are examples of both. The filtering/search mechanism is very helpful to isolate certain vulnerabilities. A filter can be initiated on name, plug-in number, etc. Clicking on the family and then the plug-in will give details of what the plug-in tests for. If intricate details are needed, the actual NASL code at cgi.nessus.org/plugins/ can be referenced. Note the DOS family is not the same as the dangerous/DOS category of plug-ins. A dangerous/DOS category plug-in actually exploits the vulnerability while a plug-in in the DOS family may just check for the vulnerability by checking software versions, for example. To perform a simple noisy scan on a non-production system, enabling all plug-ins is the best choice. If stealth or a production system is involved, choices can get complicated. We will talk in-depth about plug-ins selection in a future article. 4.3 Port scanning The other critical part of the scanning process is port scanning. Port scanning is the process by which the active ports for an IP address are identified. Each port is tied to a specific application. Nessus is a smart scanner and only runs a test if the specific program for that test is available. For example, only Web server plug-ins are run if a Web server is found. Since often ports are changed from their default to hide them, Nessus has a plug-in called services. The services plug-in attempts to identify the program running on each port. Once the program is identified, only the user-selected and pertinent plug-ins are run against it. Nessus has several options to scan for ports. There is the built-in wrapper for NMAP, widely acknowledged as the best port scanner around. There is also an internal scanner and a custom ping scan. As with plug-in selection, port scanning is very dependent on the situation. For a simple scan, the internal "sync" scan using default parameters with pings enabled, found on the "Perf" tab of the Unix GUI and the Port scan tab of NessusWX, is sufficient. Figures 8 and 9, below, show the internal SYN scan option using NessusWX and the Unix GUI client, respectively: Figure 8: Configuring the internal SYN scan for a simple port scan on NessusWX Figure 8: Configuring the internal SYN scan for a simple port scan on NessusWX Figure 9: Configuring the internal SYN scan for a simple port scan on the Unix Client Figure 9: Configuring the internal SYN scan for a simple port scan on the Unix Client 4.3 Identify targets The final task is to identify your targets. This is done on the targets tab. Targets can be specified as a single IP Address, as a subnet or as a range of IP addresses. I normally try to break them down into logical groups. It is typically easier to deal with smaller groups at one time. Figures 10 and 11 show how to select targets in both client environments: Figure 10: Specifying Targets in the Unix GUI Figure 10: Specifying Targets in the Unix GUI Figure 11: Target Selection in NessusWX Figure 11: Target Selection in NessusWX 4.4 Start a scan With your Nessus client and server in hand you are ready to scan systems. To start a scan in the Unix GUI just click "Start Scan" at the bottom of the window. In NessusWX, right click the desired session and select Execute. Properly used, Nessus can and will pinpoint problems and provide solutions. However, misused it can and will crash systems, cause the loss of data, and possibly cost you your job. As with anything powerful, there comes risk and responsibilities. Scanned systems sometimes will crash. Don't scan any system without permission. I suggest your first scan be against your own isolated test system. Future articles will lead you thorough a scan, sort out false positives and talk about stealth and firewall scanning. Figures 12, 13 and 14 show a scan using NessusWX. Figure 12: Starting a scan in NessusWX Figure 12: Starting a scan in NessusWX Figure 13: Starting a scan in NessusWX Figure 13: Starting a scan in NessusWX Figure 14: NessusWX scan in Progress Figure 14: NessusWX scan in Progress 5.0 Conclusion Nessus is an excellent tool that will greatly aid your ability to test and discover known security problems. As has been mentioned several times in this article, the power that Nessus gives you should be used wisely as it can render production systems unavailable with some of the more dangerous plus-ins. For more information on Nessus, visit the official Nessus site at www.nessus.org. Happy Scanning! 1.0 Introduction Nessus is a great tool designed to automate the testing and discovery of known security problems. Typically someone, a hacker group, a security company, or a researcher discovers a specific way to violate the security of a software product. The discovery may be accidental or through directed research; the vulnerability, in various levels of detail, is then released to the security community. Nessus is designed to help identify and solve these known problems, before a hacker takes advantage of them. Nessus is a great tool with lots of capabilities. However it is fairly complex and few articles exist to direct the new user through the intricacies of how to install and use it. Thus, this article shall endeavor to cover the basics of Nessus setup and configuration. The features of the current versions of Nessus (Nessus 2.0.8a and NessusWX 1.4.4) will be discussed. Future articles will cover Nessus in more depth. Nessus is a free program released under the GPL. Historically, many in the corporate world have ridiculed such public domain software as being a waste of time, instead choosing "supported" products developed by established companies. Typically these packages cost hundreds or thousands of dollars, and are often purchased using the logic that you get what you pay for. Some people are starting to realize that public domain software, such as Nessus, isn't always inferior and sometimes it is actually superior. Paid technical support for Nessus is even available from www.tenablesecurity.com. Nessus also has a great community of developers anchored by the primary author, Renaud Deraison. When allowed to fairly compete in reviews against other vulnerability scanners, Nessus has equaled or outshined products costing thousands of dollars. [ref: Information Security, Network Computing] One of the very powerful features of Nessus is its client server technology. Servers can be placed at various strategic points on a network allowing tests to be conducted from various points of view. A central client or multiple distributed clients can control all the servers. The server portion will run on most any flavor of Unix. It even runs on MAC OS X and IBM/AIX, but Linux tends to make the installation simpler. These features provide a great deal of flexibility for the penetration tester. Clients are available for both Windows and Unix. The Nessus server performs the actual testing while the client provides configuration and reporting functionality. 2.0 Installation Nessus server installation is fairly simple even for a Windows jockey like me. First an installed version of Unix is required. Secondly, prior installation of several external programs is recommended: NMAP is the industry standard for port scanners, Hydra is a weak password tester and Nikto is a cgi/.script checker. While not required, these external programs greatly enhance Nessus' scanning ability. They are included because they are the best applications in their class. If installed in the PATH$ before Nessus installation, they will automatically be available. The simplest installation method is using the Lynx automatic install. Lynx is included on many of the linux versions. The Lynx command is (logged in as a user, and not root) : lynx -source http://install.nessus.org | sh This should install the server on most platforms with no other steps necessary. Note that the latest install script can also be downloaded and run locally. Whether you install directly off the Website or using the same install script offline, either way the script will setup a temporary suid and ask for your root password when required -- if you don't like this feature you can download, compile and install the four required tarballs individually. The above command should also be used periodically to upgrade Nessus as new versions are regularly released. You will be questioned about proxy servers, a download method (www or CVS), and the branch of the development tree to use; most of the time the defaults are the best choice. This is the simplest method of installation however; you are effectively giving the install.nessus.org server temporary root privileges. Thus there is a security risk with this method albeit a low one. So if you are paranoid, and paranoid is not always a bad thing in the security field, installation can be done the old-fashioned way by downloading and compiling the source. For information on performing an install from scratch see: www.nessus.org/nessus_2_0.html. 3.0 Setup Once the server is installed, some basic setup steps are required. The first task to complete in the new install is to add a user. A new user can be added by the "nessus-adduser" command. The script will question you for the authentication method. Authentication can be performed by several means, however a password is the simplest and is recommended. The next question queries about rules to restrict the user account. When used across an enterprise, a user can be restricted and only allowed to scan specified IP addresses. However, for most uses this will be left blank, allowing the user to scan anything. A certificate also needs to be generated as well to be used to encrypt the traffic between the client and server. The nessus-mkcert command accomplishes this. 3.1 Update plug-ins Before a scan is done, the plug-ins should be updated. Nessus plug-ins are very much like virus signatures in a common virus scanner application. Each plug-in is written to test for a specific vulnerability. These can be written to actually exploit the vulnerability or just test for known vulnerable software versions. Plug-ins can be written in most any language but usually are written in the Nessus Attack Scripting Language (NASL). NASL is Nessus' own language, specifically designed for vulnerability test writing. Each plug-in is written to test for a specific known vulnerability and/or industry best practices. NASL plug-ins typically test by sending very specific code to the target and comparing the results against stored vulnerable values. There are a few built-in plug-ins that do not use NASL. These are C and Perl scripts to perform special purposes that can not easily be done in NASL. Among these is the Services plug-in which identifies port-to-program mappings. Plug-in updates should be done frequently. New vulnerabilities are being discovered and disseminated all the time. Typically after a new vulnerability is released to the public, someone in the Nessus community writes a NASL plug-in, releases it to the public and submits it to www.nessus.org. It is then reviewed by the developers and added to the approved plug-in list. For high risk, high profile vulnerabilities a plug-in is often released the same day the vulnerability information is publicly released. Updating plug-ins from the maintained list is fairly simple involving a simple command: nessus-update-plugins. This command must be done as root. By no means however, are you limited to the list of plug-ins from www.nessus.org. New and special purpose plug-ins can be written relatively easily using NASL, so you can write your own custom plug-ins as well. 3.2 Launch the daemon Nessus is now installed, updated and ready to go. The simplest way to get the server running is (as root) issue the nessusd -D command. In order to use it, one must use a client. There are three primary Nessus clients. The native Unix GUI version is installed at server install time. Alternatively, Nessus can be controlled from the command line. A third option, a Windows version also exists called NessusWX. The binaries for NessusWX can be found here . The NessusWX install is a straightforward Windows install. All three clients work well. Personally I prefer NessusWX. It is better organized, allows for easier reporting, and has a better facility for managing different sessions (groups of hosts to scan) then its Unix counterparts. To run the native Unix GUI client, run the nessus command or for NessusWX click the eye icon after installation. 3.3 Client connection Since Nessus is a client server technology, once running the client a connection must be made to the server. In the native client, enter the server IP, username and password (created with the nessus-adduser command) and hit login. The process in NessusWX is similar but uses the communications | connect menus. The client is connected to the server thru an SSL connection and a list of the currently installed plug-ins is downloaded. On the first run the SSL certificate is also downloaded and verification is requested. This verification ensures that in the future you are actually communicating with the server intended. Figures 1 and 2 shows the connection using the Unix and Windows GUI tools, respectively. Figure 3 shows user authentication using the NessusWX client. Figure 1: Starting the Nessus server and connecting with the Unix GUI Figure 1: Starting the Nessus server and connecting with the Unix GUI Figure 2: Connecting to the Nessus server with NessusWX (Windows Client) Figure 2: Connecting to the Nessus server with NessusWX (Windows Client) Figure 3: Enter in the server IP and the login and password setup with nessus_adduser Figure 3: Enter in the server IP and the login and password setup with nessus_adduser 4.0 Using Nessus Now that we have installed and connected to Nessus lets explore some of the options available. The most obvious and powerful aspect of Nessus is its plug-in feature. The choice of plug-ins is critical to the success of a scan. Most plug-ins are written very well and rarely trigger false positives or negatives; however, a few are not. One example of a poorly written plug-in is the test for the classic Windows IIS hack RFP's MSDAC /RDS vulnerability. Rain Forest Puppy (RFP) publicized this vulnerability in 1999. The vulnerability makes use of the %system%/msadc/msadcs.dll file and leads to total system compromise on un-patched IIS 4.0 servers. The problem is that the Nessus plug-in only tests for the existence of the /msadc/msadcs.dll file. It does not take into account patches, windows versions etc. Thus with this plug-in enabled, a false positive will show up on many IIS servers and must be sorted out manually. Before anyone yells out, "see the problems of public domain software," one should note that the same types of problems exist with the high priced "supported" vulnerability scanners as well. This problem is a result of the current state of the technology. The difference is that typically in purchased products you cannot easily examine the exact "proprietary" testing methodology as can be done with Nessus, thus making resolution of the false positive difficult. 4.1 Choosing dangerous/non-dangerous plug-ins Plug-ins are categorized in several different and sometimes confusing ways. One method of plug-in grouping is by category. Most importantly, some plug-ins are categorized as Dangerous/Denial of Service (DOS). These plug-ins will actually perform a DOS attack and crash systems that have these particular problems. Needless to say these should not be blindly run on production systems. They won't cause long term damage, but at least a reboot will be required. In both clients, there are buttons to "Enable all plug-ins" or just "Enable all but dangerous plug-ins" (called "Enable Non-DOS" in NessusWX). Note that the author of the plug-in decides if it is dangerous or not. Most of the time, this has been very well chosen. However there are instances (Exmple: the rpc_endpoint mapper plug-in), where the plug-in causes a DOS but it is not listed as dangerous. The native client denotes dangerous plug-ins with a caution triangle. NessusWX has no special way to notate a dangerous plug-in other then using the enable "Enable Non-DOS" button. One other thing to be aware of is that sometimes even a "non-dangerous" plug-in can crash software. Since the plug-ins are sending non-standard data, there is always the risk, albeit rare, that a new undiscovered DOS will be stumbled upon. Therefore anytime systems are being scanned one should be aware that system crashes, although unlikely, are possible even with "non-dangerous" plug-ins. Figure 4, below, shows plug-in selection using the Unix GUI. Similarly, Figures 5 and 6 show plug-in selection using NessusWX for Windows: Figure 4: Enabling all but dangerous plugins with the Unix Nessus GUI Figure 4: Enabling all but dangerous plugins with the Unix Nessus GUI Figure 5: Selecting plug-ins with the Windows NessusWX Client Figure 5: Selecting plug-ins with the Windows NessusWX Client Figure 6: Enabling non-dangerous plug-ins with the Windows NessusWX Client Figure 6: Enabling non-dangerous plug-ins with the Windows NessusWX Client 4.2 Safe-checks This is a good place to mention the related concept of safe-checks. Safe-checks disables the dangerous parts of safe-check compatible plug-ins and causes them to check just through passive methods such as version numbers in banners. Since a patch or workaround may be installed, safe-checks are not as reliable as actually exploiting the vulnerability. They might cause false positives or false negatives. The valuable trade off is that they should not crash a machine. The safe-check option is on the scan options tab (the options tab in NessusWX). Figure 7 shows the safe-check option in the NessusWX interface: Figure 7: Choosing Safe Checks Figure 7: Choosing Safe Checks The second method of plug-in organization is in families such as Windows, FTP SNMP, SMB, Cisco, etc. I find this to be a rather unhelpful grouping due to the arbitrary categorizing process. Should an FTP vulnerability that only exists on a Windows box go in the Windows family or the FTP family? Since this decision is left up to the plug-in writer with little guidance, there are examples of both. The filtering/search mechanism is very helpful to isolate certain vulnerabilities. A filter can be initiated on name, plug-in number, etc. Clicking on the family and then the plug-in will give details of what the plug-in tests for. If intricate details are needed, the actual NASL code at cgi.nessus.org/plugins/ can be referenced. Note the DOS family is not the same as the dangerous/DOS category of plug-ins. A dangerous/DOS category plug-in actually exploits the vulnerability while a plug-in in the DOS family may just check for the vulnerability by checking software versions, for example. To perform a simple noisy scan on a non-production system, enabling all plug-ins is the best choice. If stealth or a production system is involved, choices can get complicated. We will talk in-depth about plug-ins selection in a future article. 4.3 Port scanning The other critical part of the scanning process is port scanning. Port scanning is the process by which the active ports for an IP address are identified. Each port is tied to a specific application. Nessus is a smart scanner and only runs a test if the specific program for that test is available. For example, only Web server plug-ins are run if a Web server is found. Since often ports are changed from their default to hide them, Nessus has a plug-in called services. The services plug-in attempts to identify the program running on each port. Once the program is identified, only the user-selected and pertinent plug-ins are run against it. Nessus has several options to scan for ports. There is the built-in wrapper for NMAP, widely acknowledged as the best port scanner around. There is also an internal scanner and a custom ping scan. As with plug-in selection, port scanning is very dependent on the situation. For a simple scan, the internal "sync" scan using default parameters with pings enabled, found on the "Perf" tab of the Unix GUI and the Port scan tab of NessusWX, is sufficient. Figures 8 and 9, below, show the internal SYN scan option using NessusWX and the Unix GUI client, respectively: Figure 8: Configuring the internal SYN scan for a simple port scan on NessusWX Figure 8: Configuring the internal SYN scan for a simple port scan on NessusWX Figure 9: Configuring the internal SYN scan for a simple port scan on the Unix Client Figure 9: Configuring the internal SYN scan for a simple port scan on the Unix Client 4.3 Identify targets The final task is to identify your targets. This is done on the targets tab. Targets can be specified as a single IP Address, as a subnet or as a range of IP addresses. I normally try to break them down into logical groups. It is typically easier to deal with smaller groups at one time. Figures 10 and 11 show how to select targets in both client environments: Figure 10: Specifying Targets in the Unix GUI Figure 10: Specifying Targets in the Unix GUI Figure 11: Target Selection in NessusWX Figure 11: Target Selection in NessusWX 4.4 Start a scan With your Nessus client and server in hand you are ready to scan systems. To start a scan in the Unix GUI just click "Start Scan" at the bottom of the window. In NessusWX, right click the desired session and select Execute. Properly used, Nessus can and will pinpoint problems and provide solutions. However, misused it can and will crash systems, cause the loss of data, and possibly cost you your job. As with anything powerful, there comes risk and responsibilities. Scanned systems sometimes will crash. Don't scan any system without permission. I suggest your first scan be against your own isolated test system. Future articles will lead you thorough a scan, sort out false positives and talk about stealth and firewall scanning. Figures 12, 13 and 14 show a scan using NessusWX. Figure 12: Starting a scan in NessusWX Figure 12: Starting a scan in NessusWX Figure 13: Starting a scan in NessusWX Figure 13: Starting a scan in NessusWX Figure 14: NessusWX scan in Progress Figure 14: NessusWX scan in Progress 5.0 Conclusion Nessus is an excellent tool that will greatly aid your ability to test and discover known security problems. As has been mentioned several times in this article, the power that Nessus gives you should be used wisely as it can render production systems unavailable with some of the more dangerous plus-ins. For more information on Nessus, visit the official Nessus site at www.nessus.org. Happy Scanning! *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Thu May 22 05:13:46 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Thu May 22 05:13:51 2008 Subject: libz.so no found In-Reply-To: References: <001001c8bb66$c4713170$4d539450$@net> Message-ID: kv_bsd#cd /usr/ports/distfiles kv_bsd#mv /usr/home/kayve/Nessus-3.2.0-fbsd7.tbz . kv_bsd#pkg_add Nessus-3.2.0-fbsd7.tbz pkg_add: package VisualOS-1.0.5_3 has no origin recorded /libexec/ld-elf.so.1: Shared object "libz.so.4" not found, required by "nessusd" - Please run /usr/local/nessus/sbin/nessus-adduser to add an admin user - Register your Nessus scanner at http://www.nessus.org/register/ to obtain all the newest plugins - You can start nessusd by typing /usr/local/etc/rc.d/nessusd.sh start kv_bsd#/usr/local/etc/rc.d/nessusd.sh start Nessus/libexec/ld-elf.so.1: Shared object "libz.so.4" not found, required by "nessusd" kv_bsd# *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Thu May 22 05:36:42 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Thu May 22 05:36:47 2008 Subject: Many Nessus startup errors In-Reply-To: References: <001001c8bb66$c4713170$4d539450$@net> Message-ID: I am not generating reports http://www.monkeyview.net/id/965/fsck/nessus/nessus.vhtml During startup, 20K plugins try to load. A lot of them fail or something: http://www.monkeyview.net/id/965/fsck/nessus/p5210017.vhtml http://www.monkeyview.net/id/965/fsck/nessus/p5210018.vhtml *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From koitsu at FreeBSD.org Thu May 22 06:23:37 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Thu May 22 06:23:40 2008 Subject: libz.so no found In-Reply-To: References: <001001c8bb66$c4713170$4d539450$@net> Message-ID: <20080522062336.GA30379@eos.sc1.parodius.com> On Wed, May 21, 2008 at 10:13:43PM -0700, KAYVEN RIESE wrote: > kv_bsd#cd /usr/ports/distfiles > kv_bsd#mv /usr/home/kayve/Nessus-3.2.0-fbsd7.tbz . > kv_bsd#pkg_add Nessus-3.2.0-fbsd7.tbz > pkg_add: package VisualOS-1.0.5_3 has no origin recorded > /libexec/ld-elf.so.1: Shared object "libz.so.4" not found, required by > "nessusd" First and foremost, you hijacked an existing thread by replying to it with regards to a completely different issue. Please don't do this, it confuses mail clients which follow thread references. Please don't hit reply on unrelated messages and start a new/unrelated discussion. Secondly, the missing library error shown above would happen on machines running FreeBSD 6.x or earlier. /lib/libz.so.4 exists on RELENG_7. Another possibility is that something completely destroyed ld.so's shared library cache path. Of course, you'd be seeing all sorts of programs reporting missing libraries, and not just nexxus. If the startup script for nessus calls ldconfig or uses $LD_LIBRARY_PATH, that could explain the missing library. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From alex.wilkinson at dsto.defence.gov.au Thu May 22 06:41:23 2008 From: alex.wilkinson at dsto.defence.gov.au (Wilkinson, Alex) Date: Thu May 22 06:41:30 2008 Subject: Hifn 7955 doesn't work with Freebsd 7.0-release In-Reply-To: <4834E63E.10307@freebsd.org> References: <001001c8bb66$c4713170$4d539450$@net> <4834E63E.10307@freebsd.org> Message-ID: <20080522051320.GE97948@stlux503.dsto.defence.gov.au> 0n Wed, May 21, 2008 at 08:19:26PM -0700, Sam Leffler wrote: >Unfortunately openssl doesn't use the accelerator by default. This means >all apps that use openssl likewise are not automatically accelerated. I >suggested a patch but it was not accepted. I can't recall how you force >openssl and/or consumers to use the device. How annoying is that. Why wasn't the patch accepted ? -aW IMPORTANT: This email remains the property of the Australian Defence Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT 1914. If you have received this email in error, you are requested to contact the sender and delete the email. From alex.wilkinson at dsto.defence.gov.au Thu May 22 07:29:24 2008 From: alex.wilkinson at dsto.defence.gov.au (Wilkinson, Alex) Date: Thu May 22 07:29:28 2008 Subject: A Tale of Four Kernels... Message-ID: <20080522064319.GB98797@stlux503.dsto.defence.gov.au> Some may be interested in this. http://www.spinellis.gr/pubs/conf/2008-ICSE-4kernel/html/Spi08b.pdf -aW IMPORTANT: This email remains the property of the Australian Defence Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT 1914. If you have received this email in error, you are requested to contact the sender and delete the email. From frase at frase.id.au Thu May 22 10:00:04 2008 From: frase at frase.id.au (Fraser Tweedale) Date: Thu May 22 10:00:08 2008 Subject: A Tale of Four Kernels... In-Reply-To: <20080522064319.GB98797@stlux503.dsto.defence.gov.au> References: <20080522064319.GB98797@stlux503.dsto.defence.gov.au> Message-ID: <48354169.9040905@frase.id.au> Wilkinson, Alex wrote: > Some may be interested in this. > http://www.spinellis.gr/pubs/conf/2008-ICSE-4kernel/html/Spi08b.pdf > > -aW Interesting study; thanks for posting. frase -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080522/71a2e421/signature.pgp From hartmut.brandt at dlr.de Thu May 22 11:25:12 2008 From: hartmut.brandt at dlr.de (Hartmut Brandt) Date: Thu May 22 11:25:16 2008 Subject: Kerberized CIFS client? In-Reply-To: <20080521182722.GC40818@psu.edu> References: <20080521182722.GC40818@psu.edu> Message-ID: <483554FC.9040908@dlr.de> Derek Taylor wrote: > This question was previously posed of the freebsd-questions list, but > with no response for a week, I'd like to try my luck here. If there's > any more information I should include, please speak up: I would be glad > to oblige. > > I would like to use smb/cifs with kerberos auth, but mount_smbfs doesn't > seem to support this. > > Is anyone aware of an alternate means of performing a mount via smb/cifs > or any patches to provide such functionality? > > I already have smbclient working with -k, but I am also interested in a > mount. Try smbnetfs from ports. It's fuse based and seems to work very nice. If you have a large amount of shares floating in your network you want to restrict it to mount only the needed shares via the config file. Otherwise it will mount what it can find... It plays nicely with kerberors. When your ticket expires you immediately loose access; when you renew it you gain access again. All without the need to unmount/mount. Just call smbnetfs once you have your ticket. You may even do this from your .profile. harti From adrian at thearle.com.au Thu May 22 13:27:27 2008 From: adrian at thearle.com.au (Adrian Thearle) Date: Thu May 22 13:27:31 2008 Subject: Maximum memory allocation per process Message-ID: <48356E75.1050105@thearle.com.au> Hi Guys I have a problem with a perl script running out of memory. From my googling I have found that perl itself does not seem to impose any memory limits, and I have check ulimit and login.conf for any userclass limitations but found nothing that seems to be limiting my memory. I have 128MBytes of RAM and a 2Gbyte swap partition. I am currently running FreeBSD albert 6.2-STABLE FreeBSD 6.2-STABLE #11: Sun Sep 2 00:45:05 EST 2007 which I guess isn't exactly the latest... but the same thing happens on my REL7.0 Box The process (imapsync in this case) runs out of ram at pretty much 512MB. I read on a forum that BSD 6 imposes such a limit of 512MB per process, but i have found no where to tune this, or even see what it is. I have also read that there are two sysctl namely, kern.maxdsiz and kern.maxssiz, that can tune memory allocation but what happend to them in Freebsd 6. Your help is appreciated Adrian From koitsu at FreeBSD.org Thu May 22 13:38:19 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Thu May 22 13:38:22 2008 Subject: Maximum memory allocation per process In-Reply-To: <48356E75.1050105@thearle.com.au> References: <48356E75.1050105@thearle.com.au> Message-ID: <20080522133819.GA45578@eos.sc1.parodius.com> On Thu, May 22, 2008 at 11:00:37PM +1000, Adrian Thearle wrote: > I have a problem with a perl script running out of memory. From my googling > I have found that perl itself does not seem to impose any memory limits, > and I have check ulimit and login.conf for any userclass limitations but > found nothing that seems to be limiting my memory. > > I have 128MBytes of RAM and a 2Gbyte swap partition. > > I am currently running > FreeBSD albert 6.2-STABLE FreeBSD 6.2-STABLE #11: Sun Sep 2 00:45:05 EST > 2007 > which I guess isn't exactly the latest... but the same thing happens on my > REL7.0 Box > > The process (imapsync in this case) runs out of ram at pretty much 512MB. I > read on a forum that BSD 6 imposes such a limit of 512MB per process, but i > have found no where to tune this, or even see what it is. You need to modify some kernel settings via /boot/loader.conf and reboot. Here's what we use on our production RELENG_6 and RELENG_7 boxes: # Increase maximum allocatable memory on a process to 2GB. # (We don't choose 3GB (our max RAM) since that would # exhaust all memory, and result in a kernel panic.) # Set default memory size as 768MB. # Maximum stack size is 256MB. # kern.maxdsiz="2048M" kern.dfldsiz="768M" kern.maxssiz="256MB" > I have also read that there are two sysctl namely, kern.maxdsiz and > kern.maxssiz, that can tune memory allocation but what happend to them in > Freebsd 6. These are not sysctls, they are kernel settings. They exist on both RELENG_6 and RELENG_7. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From fabonacci at yahoo.com Thu May 22 15:54:18 2008 From: fabonacci at yahoo.com (John Timony) Date: Thu May 22 16:06:45 2008 Subject: How can I translate IP to hostname in C Message-ID: <839372.78392.qm@web46311.mail.sp1.yahoo.com> Hi,guys I am writing a c program in FreeBSD,and I can not translate a ip to hostname ,i wonder if there is a function to take this job... s.o.s -Fabonacc From bzeeb-lists at lists.zabbadoz.net Thu May 22 16:35:00 2008 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Thu May 22 16:35:04 2008 Subject: How can I translate IP to hostname in C In-Reply-To: <839372.78392.qm@web46311.mail.sp1.yahoo.com> References: <839372.78392.qm@web46311.mail.sp1.yahoo.com> Message-ID: <20080522161214.Y47338@maildrop.int.zabbadoz.net> On Thu, 22 May 2008, John Timony wrote: Hi, > I am writing a c program in FreeBSD,and I can not > translate a ip to hostname > ,i wonder if there is a function to take this job... You mean like gethostbyaddr()? See also http://www.unixguide.net/network/socketfaq/2.24.shtml for further inspiration on this but slightly different topic. -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From sbruno at miralink.com Thu May 22 17:17:43 2008 From: sbruno at miralink.com (Sean Bruno) Date: Thu May 22 17:17:50 2008 Subject: ATA patch for RELENG_6 Message-ID: <4835AAB4.60005@miralink.com> I got this patch a while ago and I don't see it appearing in RELENG_6 yet. Can someone "sheperd" this along or point out why it's not acceptable? This patch was generated by a failure to boot correctly off of a compact flash IDE module from Transcend. Index: dev/ata/ata-chipset.c =================================================================== --- dev/ata/ata-chipset.c (.../FreeBSD_RELENG_6_13APR07/src/sys) (revision 5436) +++ dev/ata/ata-chipset.c (.../miralink.FreeBSD.6/src/sys) (revision 5436) @@ -2059,7 +2059,8 @@ atadev->mode = ATA_SA150; } else { - mode = ata_limit_mode(dev, mode, ATA_UDMA5); + /*mode = ata_limit_mode(dev, mode, ATA_UDMA5);*/ + mode = ata_check_80pin(dev, ATA_UDMA5); if (!ata_controlcmd(dev, ATA_SETFEATURES, ATA_SF_SETXFER, 0, mode)) atadev->mode = mode; } -- Sean Bruno MiraLink Corporation 6015 NE 80th Ave, Ste 100 Portland, OR 97218 Phone 503-621-5143 Fax 503-621-5199 From kayve at sfsu.edu Thu May 22 17:19:06 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Thu May 22 17:19:10 2008 Subject: libz.so no found In-Reply-To: <20080522062336.GA30379@eos.sc1.parodius.com> References: <001001c8bb66$c4713170$4d539450$@net> <20080522062336.GA30379@eos.sc1.parodius.com> Message-ID: On Wed, 21 May 2008, Jeremy Chadwick wrote: > On Wed, May 21, 2008 at 10:13:43PM -0700, KAYVEN RIESE wrote: >> kv_bsd#cd /usr/ports/distfiles >> kv_bsd#mv /usr/home/kayve/Nessus-3.2.0-fbsd7.tbz . >> kv_bsd#pkg_add Nessus-3.2.0-fbsd7.tbz >> pkg_add: package VisualOS-1.0.5_3 has no origin recorded >> /libexec/ld-elf.so.1: Shared object "libz.so.4" not found, required by >> "nessusd" > > First and foremost, you hijacked an existing thread by replying to it > with regards to a completely different issue. Please don't do this, it > confuses mail clients which follow thread references. Please don't hit > reply on unrelated messages and start a new/unrelated discussion. i don't know wtf you are talking about these are all my threads. > > Secondly, the missing library error shown above would happen on machines > running FreeBSD 6.x or earlier. /lib/libz.so.4 exists on RELENG_7. > i am still on freeBSD 6.3 is this a serious problem? > Another possibility is that something completely destroyed ld.so's > shared library cache path. Of course, you'd be seeing all sorts of > programs reporting missing libraries, and not just nexxus. > so running freeBSD 6.3 is a fatal problem, or just extraneously irrelevant? > If the startup script for nessus calls ldconfig or uses > $LD_LIBRARY_PATH, that could explain the missing library. > > -- > | Jeremy Chadwick jdc at parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | > > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From xistence at 0x58.com Thu May 22 18:19:55 2008 From: xistence at 0x58.com (Bert JW Regeer) Date: Thu May 22 18:20:07 2008 Subject: libz.so no found In-Reply-To: References: <001001c8bb66$c4713170$4d539450$@net> <20080522062336.GA30379@eos.sc1.parodius.com> Message-ID: <26302C14-3599-41FD-8EA0-4BA5B4C96462@0x58.com> On May 22, 2008, at 10:19 , KAYVEN RIESE wrote: > On Wed, 21 May 2008, Jeremy Chadwick wrote: > >> On Wed, May 21, 2008 at 10:13:43PM -0700, KAYVEN RIESE wrote: >>> kv_bsd#cd /usr/ports/distfiles >>> kv_bsd#mv /usr/home/kayve/Nessus-3.2.0-fbsd7.tbz . >>> kv_bsd#pkg_add Nessus-3.2.0-fbsd7.tbz You are running on FreeBSD 6.3 and you are trying to add a package that clearly has fbsd7 in the name. Off course it is not going to work. >>> >>> pkg_add: package VisualOS-1.0.5_3 has no origin recorded >>> /libexec/ld-elf.so.1: Shared object "libz.so.4" not found, >>> required by >>> "nessusd" >> >> First and foremost, you hijacked an existing thread by replying to it >> with regards to a completely different issue. Please don't do >> this, it >> confuses mail clients which follow thread references. Please don't >> hit >> reply on unrelated messages and start a new/unrelated discussion. > > i don't know wtf you are talking about these are all my threads. You clicked reply, or something along those lines on a previous message sent to the mailling list, thereby copying over some crucial information into the header of said email: In-Reply-To: Message-ID: References: <001001c8bb66$c4713170$4d539450$@net> For mail clients that do threading, and show the conversation more as one would expect on a message board, your emails now look as if they belong underneath the topic named "Hifn 7955 doesn't work with FreeBSD 7.0-RELEASE", which is annoying to most of us who would want your messages to show up in a new thread context. Next time, please just email hackers@ directly, and do not "hi-jack" the thread. Also, it is generally considered rude to suggest that you don't know what someone is talking about using the "wtf". We point the hi-jacking out so that the mistake does not re-occur. Please refrain from such rudeness when people are attempting to help you. > > >> >> Secondly, the missing library error shown above would happen on >> machines >> running FreeBSD 6.x or earlier. /lib/libz.so.4 exists on RELENG_7. >> > > i am still on freeBSD 6.3 is this a serious problem? If you want to use the package you are using, yes. Since that package is clearly for FreeBSD 7. > > >> Another possibility is that something completely destroyed ld.so's >> shared library cache path. Of course, you'd be seeing all sorts of >> programs reporting missing libraries, and not just nexxus. >> > > so running freeBSD 6.3 is a fatal problem, or just extraneously > irrelevant? Fatal, sure, especially if you want to use that package. > > >> If the startup script for nessus calls ldconfig or uses >> $LD_LIBRARY_PATH, that could explain the missing library. >> >> -- >> | Jeremy Chadwick jdc at >> parodius.com | >> | Parodius Networking http:// >> www.parodius.com/ | >> | UNIX Systems Administrator Mountain View, CA, >> USA | >> | Making life hard for others since 1977. PGP: >> 4BD6C0CB | >> >> > > *----------------------------------------------------------* > Kayven Riese, BSCS, MS (Physiology and Biophysics) > (415) 902 5513 cellular > http://kayve.net > Webmaster http://ChessYoga.org > *----------------------------------------------------------* > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org > " From bruce at cran.org.uk Thu May 22 18:31:25 2008 From: bruce at cran.org.uk (Bruce Cran) Date: Thu May 22 18:31:27 2008 Subject: How can I translate IP to hostname in C In-Reply-To: <839372.78392.qm@web46311.mail.sp1.yahoo.com> References: <839372.78392.qm@web46311.mail.sp1.yahoo.com> Message-ID: <4835BBD5.1050003@cran.org.uk> John Timony wrote: > Hi,guys > > I am writing a c program in FreeBSD,and I can not > translate a ip to hostname > ,i wonder if there is a function to take this job... > You could use gethostbyaddr(3), but those traditional functions have been replaced with more flexible versions such as getnameinfo(3) on newer systems. There's a good introduction to modern sockets programming at http://people.redhat.com/drepper/userapi-ipv6.html -- Bruce Cran From ticso at cicely7.cicely.de Thu May 22 18:47:50 2008 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Thu May 22 18:47:53 2008 Subject: How can I translate IP to hostname in C In-Reply-To: <20080522161214.Y47338@maildrop.int.zabbadoz.net> References: <839372.78392.qm@web46311.mail.sp1.yahoo.com> <20080522161214.Y47338@maildrop.int.zabbadoz.net> Message-ID: <20080522182410.GB55002@cicely7.cicely.de> On Thu, May 22, 2008 at 04:14:46PM +0000, Bjoern A. Zeeb wrote: > On Thu, 22 May 2008, John Timony wrote: > > Hi, > > >I am writing a c program in FreeBSD,and I can not > >translate a ip to hostname > >,i wonder if there is a function to take this job... > > You mean like gethostbyaddr()? > > See also http://www.unixguide.net/network/socketfaq/2.24.shtml for > further inspiration on this but slightly different topic. You can also use the newer getaddrinfo(3)/freeaddrinfo(3). I think it is bit easier to use and can it transparently handle inet6 addresses as well. The only downside is that some rare old systems don't support it. On FreeBSD it is suppoorted since FreeBSD-4, but some commerical OS implemented it later. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From ticso at cicely7.cicely.de Thu May 22 21:16:55 2008 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Thu May 22 21:17:02 2008 Subject: How can I translate IP to hostname in C In-Reply-To: <4835BBD5.1050003@cran.org.uk> References: <839372.78392.qm@web46311.mail.sp1.yahoo.com> <4835BBD5.1050003@cran.org.uk> Message-ID: <20080522211639.GA55771@cicely7.cicely.de> On Thu, May 22, 2008 at 07:30:45PM +0100, Bruce Cran wrote: > John Timony wrote: > >Hi,guys > > > >I am writing a c program in FreeBSD,and I can not > >translate a ip to hostname > >,i wonder if there is a function to take this job... > > > > You could use gethostbyaddr(3), but those traditional functions have > been replaced with more flexible versions such as getnameinfo(3) on > newer systems. There's a good introduction to modern sockets > programming at http://people.redhat.com/drepper/userapi-ipv6.html Ups - yes that's what I ment in my mail. I wrote getaddrinfo, which is the other direction... -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From kayve at sfsu.edu Thu May 22 21:48:47 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Thu May 22 21:48:51 2008 Subject: libz.so no found In-Reply-To: <26302C14-3599-41FD-8EA0-4BA5B4C96462@0x58.com> References: <001001c8bb66$c4713170$4d539450$@net> <20080522062336.GA30379@eos.sc1.parodius.com> <26302C14-3599-41FD-8EA0-4BA5B4C96462@0x58.com> Message-ID: On Thu, 22 May 2008, Bert JW Regeer wrote: > On May 22, 2008, at 10:19 , KAYVEN RIESE wrote: >> On Wed, 21 May 2008, Jeremy Chadwick wrote: >>> On Wed, May 21, 2008 at 10:13:43PM -0700, KAYVEN RIESE wrote: >> >> i don't know wtf you are talking about these are all my threads. > > You clicked reply, or something along those lines on a previous message sent > to the mailling list, thereby copying over some crucial information into the > header of said email: > > In-Reply-To: > Message-ID: > References: <001001c8bb66$c4713170$4d539450$@net> > > > > For mail clients that do threading, and show the conversation more as one > would expect on a message board, your emails now look as if they belong > underneath the topic named "Hifn 7955 doesn't work with FreeBSD 7.0-RELEASE", > which is annoying to most of us who would want your messages to show up in a > new thread context. Next time, please just email hackers@ directly, and do > not "hi-jack" the thread. okay, so I don't have a shortcut key for freeBSD-hacker.. I went into my freeBSD saved box, grabbed the first email, replied to all, deleted everything including the subject, and despite having revised the subject, your sooperphreekiness found out I was muddling around in the deally bobber? Is that what you are talking about then? So in the future, I should know that editing the subject line will not suffice to make a new thread? If so, sorry. I get it now. >>> >>> -- >>> | Jeremy Chadwick jdc at parodius.com | >>> | Parodius Networking http://www.parodius.com/ | >>> | UNIX Systems Administrator Mountain View, CA, USA | >>> | Making life hard for others since 1977. PGP: 4BD6C0CB | >>> >>> >> >> *----------------------------------------------------------* >> Kayven Riese, BSCS, MS (Physiology and Biophysics) >> (415) 902 5513 cellular >> http://kayve.net >> Webmaster http://ChessYoga.org >> *----------------------------------------------------------* >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From olli at lurza.secnetix.de Fri May 23 06:58:59 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri May 23 06:59:07 2008 Subject: How can I translate IP to hostname in C In-Reply-To: <20080522161214.Y47338@maildrop.int.zabbadoz.net> Message-ID: <200805230658.m4N6wtm5005346@lurza.secnetix.de> Bjoern A. Zeeb wrote: > John Timony wrote: > > I am writing a c program in FreeBSD,and I can not > > translate a ip to hostname > > ,i wonder if there is a function to take this job... > > You mean like gethostbyaddr()? gethostbyaddr() is considered obsolete, I think. You should use getaddrinfo() instead, which is more flexible and easier to use, and it enables you to easily write code that is independent and agnostic of the address family (IPv4 vs. IPv6 vs. others). The manual page contains detailed example code. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "The most important decision in [programming] language design concerns what is to be left out." -- Niklaus Wirth From doconnor at gsoft.com.au Fri May 23 08:51:34 2008 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Fri May 23 08:51:39 2008 Subject: libz.so no found In-Reply-To: References: <001001c8bb66$c4713170$4d539450$@net> <26302C14-3599-41FD-8EA0-4BA5B4C96462@0x58.com> Message-ID: <200805231821.08105.doconnor@gsoft.com.au> On Fri, 23 May 2008, KAYVEN RIESE wrote: > okay, so I don't have a shortcut key for freeBSD-hacker.. I went into > my freeBSD saved box, grabbed the first email, replied to all, > deleted everything including the subject, and despite having revised > the subject, your sooperphreekiness found out I was muddling around > in the deally bobber? Is that what you are talking about then? So > in the future, I should know that editing the subject line will not > suffice to make a new thread? > > If so, sorry. I get it now. Your mail client sets the In-Reply-To header to the message ID of the email you pulled out of your saved folder. Mail clients use this header to track what thread a message is in (even if someone changed the subject). -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080523/062712b6/attachment.pgp From kayve at sfsu.edu Fri May 23 12:49:50 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Fri May 23 12:49:56 2008 Subject: /usr/src/Makefile instructions Message-ID: My professor told me about instructions being in /usr/src/Makefile for rebuilding my world. I feel better about following them because they are close to the command line to me and can't be out of date, right? I am looking at this list of makes: # check-old - List obsolete directories/files/libraries. # check-old-dirs - List obsolete directories. # check-old-files - List obsolete files. # check-old-libs - List obsolete libraries. # delete-old - Delete obsolete directories/files/libraries. # delete-old-dirs - Delete obsolete directories. # delete-old-files - Delete obsolete files. # delete-old-libs - Delete obsolete libraries. # I am wondering if I should try these out, or will it just be taken care of with the "cannonical" methods. I seem to have lots of big problems with my configuration.. I don't know. Things work, but dmesg has errors, and many ports fail and their makes, even if they succeed have errors and warnings. If I "delete-old-.." will I be messing things up? *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Fri May 23 12:52:14 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Fri May 23 12:52:18 2008 Subject: libz.so no found In-Reply-To: <200805231821.08105.doconnor@gsoft.com.au> References: <001001c8bb66$c4713170$4d539450$@net> <26302C14-3599-41FD-8EA0-4BA5B4C96462@0x58.com> <200805231821.08105.doconnor@gsoft.com.au> Message-ID: On Fri, 23 May 2008, Daniel O'Connor wrote: > On Fri, 23 May 2008, KAYVEN RIESE wrote: >> okay, so I don't have a shortcut key for freeBSD-hacker.. I went into >> my freeBSD saved box, grabbed the first email, replied to all, >> deleted everything including the subject, and despite having revised >> the subject, your sooperphreekiness found out I was muddling around >> in the deally bobber? Is that what you are talking about then? So >> in the future, I should know that editing the subject line will not >> suffice to make a new thread? >> >> If so, sorry. I get it now. > > Your mail client sets the In-Reply-To header to the message ID of the > email you pulled out of your saved folder. > > Mail clients use this header to track what thread a message is in (even > if someone changed the subject). Thanks. This makes things clear. I'm sorry that I felt "scolded" by the confusing barrage (I guess at least a couple of you thought this should have been obvious to me). > > -- > Daniel O'Connor software and network engineer > for Genesis Software - http://www.gsoft.com.au > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Fri May 23 13:15:18 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Fri May 23 13:15:25 2008 Subject: Warning during make buildworld Message-ID: There was one about "mktemp()" ./gengtype warning: structure `VEC_cp_token_position_heap' used but not defined warning: structure `c_arg_info' used but not defined warning: structure `c_switch' used but not defined warning: structure `et_node' used but not defined warning: structure `loop' used but not defined warning: structure `ipa_reference_vars_info_d' used but not defined warning: structure `reg_info_def' used but not defined warning: structure `value_set' used but not defined warning: structure `VEC_cp_token_position_heap' used but not defined warning: structure `c_arg_info' used but not defined warning: structure `c_switch' used but not defined warning: structure `et_node' used but not defined warning: structure `loop' used but not defined warning: structure `ipa_reference_vars_info_d' used but not defined warning: structure `reg_info_def' used but not defined warning: structure `value_set' used but not defined touch gtype-desc.h *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From tevans.uk at googlemail.com Fri May 23 13:39:49 2008 From: tevans.uk at googlemail.com (Tom Evans) Date: Fri May 23 13:39:54 2008 Subject: /usr/src/Makefile instructions In-Reply-To: References: Message-ID: <1211548239.10665.4.camel@localhost> On Fri, 2008-05-23 at 05:49 -0700, KAYVEN RIESE wrote: > My professor told me about instructions being in /usr/src/Makefile > for rebuilding my world. I feel better about following them because > they are close to the command line to me and can't be out of date, right? > > I am looking at this list of makes: > > # check-old - List obsolete directories/files/libraries. > # check-old-dirs - List obsolete directories. > # check-old-files - List obsolete files. > # check-old-libs - List obsolete libraries. > # delete-old - Delete obsolete directories/files/libraries. > # delete-old-dirs - Delete obsolete directories. > # delete-old-files - Delete obsolete files. > # delete-old-libs - Delete obsolete libraries. > # > > > I am wondering if I should try these out, or will it just be > taken care of with the "cannonical" methods. I seem to have lots > of big problems with my configuration.. I don't know. Things > work, but dmesg has errors, and many ports fail and their makes, > even if they succeed have errors and warnings. > > If I "delete-old-.." will I be messing things up? > No, it will "Delete obsolete directories/files/libraries", with the usual meaning of obsolete. You should only need to do this when moving to a new (major?) release. I've redirected this to questions@, as this seems more like a 'User question/technical support' rather than 'General technical discussion'. Please try to keep the mailing lists on topic. Cheers Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080523/174f61af/attachment.pgp From david at catwhisker.org Fri May 23 13:45:50 2008 From: david at catwhisker.org (David Wolfskill) Date: Fri May 23 13:45:54 2008 Subject: /usr/src/Makefile instructions In-Reply-To: References: Message-ID: <20080523132649.GB69430@bunrab.catwhisker.org> On Fri, May 23, 2008 at 05:49:48AM -0700, KAYVEN RIESE wrote: > > My professor told me about instructions being in /usr/src/Makefile > for rebuilding my world. I feel better about following them because > they are close to the command line to me and can't be out of date, right? No. Any comments or documentation *can* be "out of date" or otherwise misleading or incorrect. > I am looking at this list of makes: Err.. that would be "make targets," yeah? > # check-old - List obsolete directories/files/libraries. > # check-old-dirs - List obsolete directories. > # check-old-files - List obsolete files. > # check-old-libs - List obsolete libraries. > # delete-old - Delete obsolete directories/files/libraries. > # delete-old-dirs - Delete obsolete directories. > # delete-old-files - Delete obsolete files. > # delete-old-libs - Delete obsolete libraries. > # > > > I am wondering if I should try these out, or will it just be > taken care of with the "cannonical" methods. I expect that depends a great deal on what your objectives are. If you are merely trying to keep a system up-to-date, you are (IMO) better off reading, then paying attention to changes in, /usr/src/UPDATING. If, on the other hand, you are curious about just what make(1) will do when told to make a given target, by all means experiment to your heart's content (on your own system(s)). But I encourage you to consider the utility of the "-n" flag to make(1). Well, that, as well as the value of good backup & restore procedures. > I seem to have lots > of big problems with my configuration.. I don't know. Things > work, but dmesg has errors, and many ports fail and their makes, > even if they succeed have errors and warnings. > > If I "delete-old-.." will I be messing things up? Hard to say without more information about your current configuration than is likely to fit in a message to the mailing list. If the complaints are sufficiently severe (note that the mere quantity of them isn't a relaible measure of this), you may be better off recording salient configuration information (e.g., what ports you tried to install), make a good backup (assuming there's data worth recovering), wiping the system clean, and starting over. It is certainly possible to mess a system up enough that recovery is problematic, at best. Peace, david -- David H. Wolfskill david@catwhisker.org I submit that "conspiracy" would be an appropriate collective noun for cats. See http://www.catwhisker.org/~david/publickey.gpg for my public key. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080523/3149fdde/attachment.pgp From kayve at sfsu.edu Fri May 23 13:50:02 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Fri May 23 13:50:08 2008 Subject: /usr/src/Makefile instructions In-Reply-To: <1211548239.10665.4.camel@localhost> References: <1211548239.10665.4.camel@localhost> Message-ID: On Fri, 23 May 2008, Tom Evans wrote: > On Fri, 2008-05-23 at 05:49 -0700, KAYVEN RIESE wrote: > > I've redirected this to questions@, as this seems more like a 'User > question/technical support' rather than 'General technical discussion'. > Please try to keep the mailing lists on topic. That list is too busy. I find I don't have to unsubscribe to "hackers," and it doesn't seem as hard core to misinterpret what "hackers" are, than say "ports" or "acpi" I realized that "make delete-old" and "make delete-old-libs" are both part of the "cannonical," I guess because I am going RELENG_6_3 to RELENG_7 On that note, was I given misinformation when I was advised that it would be impossible to upgrade RELENG_6_2 directly to RELENG_7 ? > > Cheers > > Tom > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From soralx at cydem.org Fri May 23 14:24:55 2008 From: soralx at cydem.org (soralx@cydem.org) Date: Fri May 23 14:25:02 2008 Subject: /usr/src/Makefile instructions In-Reply-To: References: <1211548239.10665.4.camel@localhost> Message-ID: <20080523072426.7de88f38@soralx> > On that note, was I given misinformation when I was advised > that it would be impossible to upgrade RELENG_6_2 directly to > RELENG_7 ? Close to implausible, perhaps? That would indeed be the case, unless you truly are longing for a major workout, either with mergemaster et al, or both with mergemaster and the ports. The former case, which assumes you don't have many ports installed, is often a no-brainer: install a fresh system. The latter case may be somewhat more complicated: install a fresh system for the least effort on your side, or go the update route if you need to keep your system up and usable during the process. I should note that I always took the update trail, and never regretted it afterwards (well, if only so slightly). For instance, my workstation lived through 5.2.1-R, 6.2-R, RELENG_6, and finally RELENG_7, all with the aid of cvsup. The process is straightforward, well-designed and easily executed (thanks to the developers), but problems often pop-up with ports (especially such messy ones as Gnome, etc) which take lots of time to correct. So, in summary, a sane person should probably go with clean system update. P.S.: whoever replies next, it's safe to drop hackers@ from CC: anytime now > Kayven Riese, BSCS, MS (Physiology and Biophysics) [SorAlx] ridin' VS1400 From olli at lurza.secnetix.de Fri May 23 14:43:04 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri May 23 14:43:12 2008 Subject: /usr/src/Makefile instructions In-Reply-To: Message-ID: <200805231442.m4NEgxnm022251@lurza.secnetix.de> KAYVEN RIESE wrote: > Tom Evans wrote: > > I've redirected this to questions@, as this seems more like a 'User > > question/technical support' rather than 'General technical discussion'. > > Please try to keep the mailing lists on topic. > > That list is too busy. I find I don't have to unsubscribe to > "hackers," and it doesn't seem as hard core to misinterpret > what "hackers" are, than say "ports" or "acpi" Well, "hackers" usually means developers, i.e. people hacking on the FreeBSD code. Therefore I'm afraid I have to agree with Tom: Your questions should better go to the questions list. > I realized that "make delete-old" and "make delete-old-libs" > are both part of the "cannonical," I guess because I am going > RELENG_6_3 to RELENG_7 I always use "make delete-old", as instructed in the /usr/src/UPDATING file, and it has never bitten me. Please have a look at that file; the important part starts at the section titled "To rebuild everything". > On that note, was I given misinformation when I was advised > that it would be impossible to upgrade RELENG_6_2 directly to > RELENG_7 ? "Nothing is impossible!", as Dr. Farnsworth from the Futurama series used to say. :-) But seriously ... I think going from 6.2 to 7.0 should work fine. However, the official notion is that updates across major versions have to be supported only for the latest stable release. Any other configurations might work, but it's not guaranteed. If it fails, you're not expected to complain or ask for help, but instead try the officially supported way (i.e. first update to the latest stable on your existing branch, then update across the major version boundary). If that still fails, you may complain and ask for help. Note that it is IMPORTANT to rebuild *all* of your ports when you update from 6.x to 7.x. (This holds true for any major version update.) If you don't do this, you will get library dependency collisions, i.e. port A uses libc.so.7 and depends on port B, but port B still links against the older libc.so.6. Things will break sooner or later. That's why you should rebuild *all* ports after updating to 7.x. (You can keep older ports only if you are absolutely sure that they are not part of any dependencies, and never will be.) In your previous mail you mentioned: > Things work, but dmesg has errors, Would you please tell us what those errors are? We might be able to help you, but only if you tell us. > and many ports fail and their makes Again: Please post messages and everything relevant to the problems. There are really people on these lists that are willing to help, but we need as much information as possible in order to be able to help. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' From sfourman at gmail.com Fri May 23 17:18:28 2008 From: sfourman at gmail.com (Sam Fourman Jr.) Date: Fri May 23 17:18:32 2008 Subject: patch for Intel G33 on FreeBSD 6.3 Message-ID: <11167f520805230952t67ade6bboca089bf7752b107d@mail.gmail.com> hello, I have a New motherboard that has a integrated Intel G33 based video card. I see there is support for this in FreeBSD 7 kernel. (Disabled by Default) http://lists.freebsd.org/pipermail/cvs-src/2007-July/080677.html I was wondering if anyone had a patch laying around that would apply on FreeBSD 6.3 I would be more than willing to test this and even provide back traces. Thank you Sam Fourman Jr. From kayve at sfsu.edu Sat May 24 01:57:29 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sat May 24 01:57:40 2008 Subject: /usr/src/Makefile instructions In-Reply-To: <20080523072426.7de88f38@soralx> References: <1211548239.10665.4.camel@localhost> <20080523072426.7de88f38@soralx> Message-ID: On Fri, 23 May 2008, soralx@cydem.org wrote: > >> On that note, was I given misinformation when I was advised >> that it would be impossible to upgrade RELENG_6_2 directly to >> RELENG_7 ? > > Close to implausible, perhaps? That would indeed be the case, unless you > truly are longing for a major workout, either with mergemaster et al, or > both with mergemaster and the ports. The former case, which assumes you > don't have many ports installed, is often a no-brainer: install a fresh > system. The latter case may be somewhat more complicated: install a fresh > system for the least effort on your side, or go the update route if you need > to keep your system up and usable during the process. I didn't really understand that and I don't understand why I am a bad person for spamming my idiotic thoughts on the matter, but in any case this is moot because I am up an runing RELENG_6_3 and making kernel after editing the stable-supfiles RELENG_6_3 to RELENG_7 let's all cross our fingers that communication has just happened. > I should note that I always took the update trail, and never regretted it > afterwards (well, if only so slightly). For instance, my workstation lived > through 5.2.1-R, 6.2-R, RELENG_6, and finally RELENG_7, all with the aid of > cvsup. The process is straightforward, well-designed and easily executed > (thanks to the developers), but problems often pop-up with ports > (especially such messy ones as Gnome, etc) which take lots of time to > correct. I am feeling better about cvsup and even mergemaster nowadays. Thank you very much for your support. > > So, in summary, a sane person should probably go with clean system update. > Is that what I am doing? Umm.. maybe not. I have all these errors that I don't understand and that people ignore but I have a browser and a terminal, so I feel like a functioning pile of carbon compounds. > P.S.: whoever replies next, it's safe to drop hackers@ from CC: anytime now > Naaaaah.. hackers needs the publicity! >> Kayven Riese, BSCS, MS (Physiology and Biophysics) > > [SorAlx] ridin' VS1400 > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Sat May 24 02:00:51 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sat May 24 02:01:01 2008 Subject: /usr/src/Makefile instructions In-Reply-To: <20080523072426.7de88f38@soralx> References: <1211548239.10665.4.camel@localhost> <20080523072426.7de88f38@soralx> Message-ID: On Fri, 23 May 2008, soralx@cydem.org wrote: > >> On that note, was I given misinformation when I was advised >> that it would be impossible to upgrade RELENG_6_2 directly to >> RELENG_7 ? pcm/mixer.c /usr/src/sys/modules/sound/sound/../../../dev/sound/pcm/sndstat.c /usr/src/sys/modules/sound/sound/../../../dev/sound/pcm/sound.c /usr/src/sys/modules/sound/sound/../../../dev/sound/unit.c /usr/src/sys/module > > Close to implausible, perhaps? That would indeed be the case, unless you > truly are longing for a major workout, either with mergemaster et al, or > both with mergemaster and the ports. The former case, which assumes you > don't have many ports installed, is often a no-brainer: install a fresh > system. The latter case may be somewhat more complicated: install a fresh > system for the least effort on your side, or go the update route if you need > to keep your system up and usable during the process. > awk -f @/tools/makeobjops.awk @/kern/bus_if.m -h awk -f @/tools/makeobjops.awk @/dev/pci/pci_if.m -h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/sym/../../dev/sym/sym_hipd.c ===> syscons (depend) ===> syscons/apm (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/apm/../../../dev/syscons/apm/apm_saver.c ===> syscons/blank (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/blank/../../../dev/syscons/blank/blank_saver.c ===> syscons/daemon (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/daemon/../../../dev/syscons/daemon/daemon_saver.c ===> syscons/dragon (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/dragon/../../../dev/syscons/dragon/dragon_saver.c ===> syscons/fade (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/fade/../../../dev/syscons/fade/fade_saver.c ===> syscons/fire (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/fire/../../../dev/syscons/fire/fire_saver.c ===> syscons/green (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/green/../../../dev/syscons/green/green_saver.c ===> syscons/logo (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/logo/../../../dev/syscons/logo/logo_saver.c /usr/src/sys/modules/syscons/logo/../../../dev/syscons/logo/logo.c > I should note that I always took the update trail, and never regretted it > afterwards (well, if only so slightly). For instance, my workstation lived > through 5.2.1-R, 6.2-R, RELENG_6, and finally RELENG_7, all with the aid of > cvsup. The process is straightforward, well-designed and easily executed > (thanks to the developers), but problems often pop-up with ports > (especially such messy ones as Gnome, etc) which take lots of time to > correct. rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/sysvipc/sysvshm/../../../kern/sysv_shm.c ===> ti (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include awk -f @/tools/makeobjops.awk @/kern/device_if.m -h awk -f @/tools/makeobjops.awk @/kern/bus_if.m -h awk -f @/tools/makeobjops.awk @/dev/pci/pci_if.m -h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_ti.h opt_ti.h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_zero.h opt_zero.h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/ti/../../dev/ti/if_ti.c ^C*** .depend removed kv_bsd# oops? mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/rain/../../../dev/syscons/rain/rain_saver.c ===> syscons/snake (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/snake/../../../dev/syscons/snake/snake_saver.c ===> syscons/star (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/star/../../../dev/syscons/star/star_saver.c ===> syscons/warp (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/syscons/warp/../../../dev/syscons/warp/warp_saver.c ===> sysvipc (depend) ===> sysvipc/sysvmsg (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include ln -sf /usr/obj/usr/src/sys/GENERIC/opt_mac.h opt_mac.h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_sysvipc.h opt_sysvipc.h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/sysvipc/sysvmsg/../../../kern/sysv_msg.c ===> sysvipc/sysvsem (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include ln -sf /usr/obj/usr/src/sys/GENERIC/opt_mac.h opt_mac.h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_sysvipc.h opt_sysvipc.h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/sysvipc/sysvsem/../../../kern/sysv_sem.c ===> sysvipc/sysvshm (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include ln -sf /usr/obj/usr/src/sys/GENERIC/opt_mac.h opt_mac.h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_sysvipc.h opt_sysvipc.h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_compat.h opt_compat.h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/sysvipc/sysvshm/../../../kern/sysv_shm.c ===> ti (depend) @ -> /usr/src/sys machine -> /usr/src/sys/i386/include awk -f @/tools/makeobjops.awk @/kern/device_if.m -h awk -f @/tools/makeobjops.awk @/kern/bus_if.m -h awk -f @/tools/makeobjops.awk @/dev/pci/pci_if.m -h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_ti.h opt_ti.h ln -sf /usr/obj/usr/src/sys/GENERIC/opt_zero.h opt_zero.h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -I. -I@ -I@/contrib/altq -I/usr/obj/usr/src/sys/GENERIC /usr/src/sys/modules/ti/../../dev/ti/if_ti.c ^C*** .depend removed kv_bsd#make kernel -------------------------------------------------------------- >>> Kernel build for GENERIC started on Fri May 23 18:59:22 PDT 2008 -------------------------------------------------------------- ===> GENERIC mkdir -p /usr/obj/usr/src/sys -------------------------------------------------------------- >>> stage 1: configuring the kernel -------------------------------------------------------------- cd /usr/src/sys/i386/conf; PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/usr/obj/usr/src/tmp/usr/games:/sbin:/bin:/usr/sbin:/usr/bin config -d /usr/obj/usr/src/sys/GENERIC /usr/src/sys/i386/conf/GENERIC Kernel build directory is /usr/obj/usr/src/sys/GENERIC Don't forget to do ``make cleandepend && make depend'' -------------------------------------------------------------- >>> stage 2.1: cleaning up the object tree -------------------------------------------------------------- cd /usr/obj/usr/src/sys/GENERIC; MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=i386 MACHINE=i386 CPUTYPE= GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac _SHLIBDIRPREFIX=/usr/obj/usr/src/tmp INSTALL="sh /usr/src/tools/install.sh" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/usr/obj/usr/src/tmp/usr/games:/sbin:/bin:/usr/sbin:/usr/bin make KERNEL=kernel cleandir rm -f *.o *.so *.So *.ko *.s eddep errs kernel.debug kernel kernel.symbols linterrs makelinks tags vers.c vnode_if.c vnode_if.h vnode_if_newproto.h vnode_if_typedef.h ata_if.c eisa_if.c miibus_if.c mmcbr_if.c mmcbus_if.c card_if.c power_if.c pci_if.c pcib_if.c ppbus_if.c uart_if.c usb_if.c g_part_if.c isa_if.c bus_if.c cpufreq_if.c device_if.c linker_if.c serdev_if.c agp_if.c acpi_if.c ata_if.h eisa_if.h miibus_if.h mmcbr_if.h mmcbus_if.h card_if.h power_if.h pci_if.h pcib_if.h ppbus_if.h uart_if.h usb_if.h g_part_if.h isa_if.h bus_if.h cpufreq_if.h device_if.h linker_if.h serdev_if.h agp_if.h acpi_if.h aicasm* y.tab.h aic7xxx_seq.h aic7xxx_reg.h aic7xxx_reg_print.c aic79xx_seq.h aic79xx_reg.h aic79xx_reg_print.c miidevs.h pccarddevs.h usbdevs.h usbdevs_data.h opt_ah.h rm -f .depend machine cd /usr/src/sys/modules; MAKEOBJDIRPREFIX=/usr/obj/usr/src/sys/GENERIC/modules KMODDIR=/boot/kernel DEBUG_FLAGS="-g" MACHINE=i386 KERNBUILDDIR="/usr/obj/usr/src/sys/GENERIC" make cleandir ===> 3dfx (cleandir) ===> 3dfx_linux (cleandir) ===> aac (cleandir) ===> aac/aac_linux (cleandir) rm -f export_syms aac_linux.ko aac_linux.kld aac_linux.o aac_linux.ko.debug aac_linux.ko.symbols rm -f @ machine rm -f .depend GPATH GRTAGS GSYMS GTAGS ===> accf_data (cleandir) > > So, in summary, a sane person should probably go with clean system update. ==> netgraph/l2tp (cleandir) ==> netgraph/lmi (cleandir) ==> netgraph/mppc (cleandir) ==> netgraph/nat (cleandir) ==> netgraph/netflow (cleandir) ==> netgraph/netgraph (cleandir) ==> netgraph/one2many (cleandir) ==> netgraph/ppp (cleandir) ==> netgraph/pppoe (cleandir) ==> netgraph/pptpgre (cleandir) ==> netgraph/pred1 (cleandir) ==> netgraph/rfc1490 (cleandir) ==> netgraph/socket (cleandir) ==> netgraph/source (cleandir) ==> netgraph/split (cleandir) ==> netgraph/sppp (cleandir) ==> netgraph/sync_ar (cleandir) ==> netgraph/sync_sr (cleandir) ==> netgraph/tag (cleandir) ==> netgraph/tcpmss (cleandir) ==> netgraph/tee (cleandir) > > P.S.: whoever replies next, it's safe to drop hackers@ from CC: anytime now sr/src/sys/modules/cd9660_iconv => cdce (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cdce created for /usr/src ys/modules/cdce => ce (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/ce created for /usr/src/s /modules/ce => ciss (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/ciss created for /usr/src ys/modules/ciss => cm (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cm created for /usr/src/s /modules/cm => cmx (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cmx created for /usr/src/sys/modules/cmx => coda (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/coda created for /usr/src/sys/modules/coda => coda5 (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/coda5 created for /usr/src/sys/modules/coda5 => coff (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/coff created for /usr/src/sys/modules/coff => coretemp (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/coretemp created for /usr/src/sys/modules/coretemp => cp (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cp created for /usr/src/sys/modules/cp => cpufreq (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cpufreq created for /usr/src/sys/modules/cpufreq => crypto (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/crypto created for /usr/src/sys/modules/crypto => cryptodev (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cryptodev created for /usr/src/sys/modules/cryptodev => cs (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cs created for /usr/src/sys/modules/cs => ctau (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/ctau created for /usr/src/sys/modules/ctau => cue (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cue created for /usr/src/sys/modules/cue => cxgb (obj) => cxgb/cxgb (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cxgb/cxgb created for /usr/src/sys/modules/cxgb/cxgb => cxgb/cxgb_t3fw (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cxgb/cxgb_t3fw created for /usr/src/sys/modules/cxgb/cxgb_t3fw => cx (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/cx created for /usr/src/sys/modules/cx => dc (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/dc created for /usr/src/sys/modules/dc => dcons (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/dcons created for /usr/src/sys/modules/dcons => dcons_crom (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/dcons_crom created for /usr/src/sys/modules/dcons_crom => de (obj) sr/obj/usr/src/sys/GENERIC/modules/usr/src/sys/modules/de created for /usr/src/sys/modules/de => digi (obj) => digi/digi (obj) > >> Kayven Riese, BSCS, MS (Physiology and Biophysics) > > [SorAlx] ridin' VS1400 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Sat May 24 02:09:14 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sat May 24 02:09:26 2008 Subject: /usr/src/Makefile instructions In-Reply-To: <200805231442.m4NEgxnm022251@lurza.secnetix.de> References: <200805231442.m4NEgxnm022251@lurza.secnetix.de> Message-ID: On Fri, 23 May 2008, Oliver Fromme wrote: > KAYVEN RIESE wrote: > > Tom Evans wrote: > > > I've redirected this to questions@, as this seems more like a 'User > > > question/technical support' rather than 'General technical discussion'. > > > Please try to keep the mailing lists on topic. > > > > That list is too busy. I find I don't have to unsubscribe to > > "hackers," and it doesn't seem as hard core to misinterpret > > what "hackers" are, than say "ports" or "acpi" > > Well, "hackers" usually means developers, i.e. people > hacking on the FreeBSD code. Therefore I'm afraid I > have to agree with Tom: Your questions should better > go to the questions list. ergo: because obviously I am a flumming idiot. I thought hacker was something you took eucalyptus fro. > > > I realized that "make delete-old" and "make delete-old-libs" > > are both part of the "cannonical," I guess because I am going > > RELENG_6_3 to RELENG_7 > > I always use "make delete-old", as instructed in the > /usr/src/UPDATING file, and it has never bitten me. > Please have a look at that file; the important part > starts at the section titled "To rebuild everything". > Actually, after composing this I kicked myself, because /usr/src/Makefile has clearly instructed me to do make delete-old after make installworld and I think I will throw caution to the wind and append -U to my subsequent mergemaster followed by make delete-old-libs > > On that note, was I given misinformation when I was advised > > that it would be impossible to upgrade RELENG_6_2 directly to > > RELENG_7 ? > > "Nothing is impossible!", as Dr. Farnsworth from the > Futurama series used to say. :-) > Oh well. Water under the bridge. I expect to someday edit this to RELENG_7_1 or the like when freebsd.org says so and following the instructions in /usr/src/Makefile again. > But seriously ... I think going from 6.2 to 7.0 should > work fine. However, the official notion is that updates > across major versions have to be supported only for the > latest stable release. Any other configurations might > work, but it's not guaranteed. If it fails, you're not > expected to complain or ask for help, but instead try > the officially supported way (i.e. first update to the > latest stable on your existing branch, then update > across the major version boundary). If that still fails, > you may complain and ask for help. > Interesting. No. Fascinitating. Captian Kirk, I believe this star will supernova in approximately 12 days, 13 hours, 34 minutes and 23.3425 seconds. > Note that it is IMPORTANT to rebuild *all* of your ports > when you update from 6.x to 7.x. (This holds true for > any major version update.) If you don't do this, you > will get library dependency collisions, i.e. port A uses > libc.so.7 and depends on port B, but port B still links > against the older libc.so.6. Things will break sooner > or later. That's why you should rebuild *all* ports > after updating to 7.x. (You can keep older ports only > if you are absolutely sure that they are not part of any > dependencies, and never will be.) > My habit is to run cvsup standard-supfile followed by cvsup ports-supfile. IS that a dumb thing to do? > In your previous mail you mentioned: > > Things work, but dmesg has errors, > > Would you please tell us what those errors are? We might > be able to help you, but only if you tell us. > I told the ACPI folks and they told me nicely that my appropriate post was too much of a hassle to bother with. Some ding dong was attaching after the fact of the wing ding and the fact that the errors occured between the wing and the ding was irrelevant since the dong ding subsequent to the ding ding recalibrated the whosits in an adequate fashion before reaching single user mode. > > and many ports fail and their makes > > Again: Please post messages and everything relevant to > the problems. There are really people on these lists > that are willing to help, but we need as much information > as possible in order to be able to help. > > Best regards > Oliver > Well.. I reckon I mights a git up thah gumpshun whenis I's gonna get tootin' on sumptin that gits mah goat subsequently. > -- > Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. > Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: > secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- > chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart > > FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd > > 'Instead of asking why a piece of software is using "1970s technology," > start asking why software is ignoring 30 years of accumulated wisdom.' > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Sat May 24 03:38:18 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sat May 24 03:38:21 2008 Subject: kldxref oh oh Message-ID: kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kv_bsd# *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From kayve at sfsu.edu Sat May 24 03:39:12 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sat May 24 03:39:17 2008 Subject: kldxref oh oh In-Reply-To: References: Message-ID: On Fri, 23 May 2008, KAYVEN RIESE wrote: > > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked ref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kldxref: file isn't dynamically-linked kv_bsd#make kernel -------------------------------------------------------------- >>> Kernel build for GENERIC started on Fri May 23 20:38:21 PDT 2008 -------------------------------------------------------------- ===> GENERIC mkdir -p /usr/obj/usr/src/sys -------------------------------------------------------------- >>> stage 1: configuring the kernel -------------------------------------------------------------- cd /usr/src/sys/i386/conf; PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/usr/obj/usr/src/tmp/usr/games:/sbin:/bin:/usr/sbin:/usr/bin config -d /usr/obj/usr/src/sys/GENERIC /usr/src/sys/i386/conf/GENERIC Kernel build directory is /usr/obj/usr/src/sys/GENERIC Don't forget to do ``make cleandepend && make depend'' -------------------------------------------------------------- >>> stage 2.1: cleaning up the object tree > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kv_bsd# > > > *----------------------------------------------------------* > Kayven Riese, BSCS, MS (Physiology and Biophysics) > (415) 902 5513 cellular > http://kayve.net > Webmaster http://ChessYoga.org > *----------------------------------------------------------* > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From frase at frase.id.au Sat May 24 03:50:04 2008 From: frase at frase.id.au (Fraser Tweedale) Date: Sat May 24 03:50:09 2008 Subject: temporary freezes when pressing capslock / numlock Message-ID: <48378DA0.8040506@frase.id.au> Skipped content of type multipart/mixed-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080524/d4154f18/signature.pgp From niclas.zeising at gmail.com Sat May 24 13:43:41 2008 From: niclas.zeising at gmail.com (Niclas Zeising) Date: Sat May 24 13:43:52 2008 Subject: kldxref oh oh In-Reply-To: References: Message-ID: <4838172B.9050008@gmail.com> KAYVEN RIESE wrote: > On Fri, 23 May 2008, KAYVEN RIESE wrote: > >> >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked > > ref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kv_bsd#make kernel > > -------------------------------------------------------------- >>>> Kernel build for GENERIC started on Fri May 23 20:38:21 PDT 2008 > -------------------------------------------------------------- > ===> GENERIC > mkdir -p /usr/obj/usr/src/sys > > -------------------------------------------------------------- >>>> stage 1: configuring the kernel > -------------------------------------------------------------- > cd /usr/src/sys/i386/conf; > PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/usr/obj/usr/src/tmp/usr/games:/sbin:/bin:/usr/sbin:/usr/bin > config -d /usr/obj/usr/src/sys/GENERIC /usr/src/sys/i386/conf/GENERIC > Kernel build directory is /usr/obj/usr/src/sys/GENERIC > Don't forget to do ``make cleandepend && make depend'' > > -------------------------------------------------------------- >>>> stage 2.1: cleaning up the object tree > > > >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kv_bsd# >> Hi! Please send error reports to freebsd-current@freebsd.org (for current, i. e. 8.0-CURRENT) or freebsd-stable@freebsd.org (for stable, i. e. 5-STABLE, 6-STABLE, 7-STABLE and the various releases) and questions to questions@freebsd.org. Apart from that, that error message shows when upgrading from an old version of FreeBSD (6.x i think) to a newer version (7 or 8). It's as far as I can tell completely harmless. I've seen it before, and my computers haven't malfunctioned yet. Regards! //Niclas From xi at borderworlds.dk Sat May 24 19:32:28 2008 From: xi at borderworlds.dk (Christian Laursen) Date: Sat May 24 19:32:35 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <48378DA0.8040506@frase.id.au> (Fraser Tweedale's message of "Sat\, 24 May 2008 13\:38\:08 +1000") References: <48378DA0.8040506@frase.id.au> Message-ID: Fraser Tweedale writes: > Since upgrading to RELENG_7_0 I was experiencing momentary freezes (of > about .5 seconds) whenever the capslock or numlock buttons were > pressed. I have experienced this too - also on RELENG_7_0. > I would probably never have noticed it except for the strange noises > produced when music is playing, and of course that is when it is the > most annoying ;) Indeed. > The issue occurs both in console and in X, and for both ULE and > 4BSD. The problem was reproducible with USB keyboards only (ukbd); > atkbd seems fine. It also occurs when numlockx is used to set numlock > on or off without keyboard interaction. For me the issue went away when I removed atkbd support from my kernel. I still have ukbd and kbdmux. -- Christian Laursen From koitsu at FreeBSD.org Sat May 24 20:16:34 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Sat May 24 20:16:40 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <48378DA0.8040506@frase.id.au> References: <48378DA0.8040506@frase.id.au> Message-ID: <20080524201633.GA81364@eos.sc1.parodius.com> On Sat, May 24, 2008 at 01:38:08PM +1000, Fraser Tweedale wrote: > Since upgrading to RELENG_7_0 I was experiencing momentary freezes (of > about .5 seconds) whenever the capslock or numlock buttons were pressed. Let me guess -- you're using a USB keyboard. > The issue occurs both in console and in X, and for both ULE and 4BSD. The > problem was reproducible with USB keyboards only (ukbd); atkbd seems fine. > It also occurs when numlockx is used to set numlock on or off without > keyboard interaction. I figured as much. I hadn't checked numlock/capslock myself, but vcons switching also has that problem. Yes, it's semi-documented; see section "USB" below: http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues For what it's worth, the systems I've confirmed the problem on are Intel and nForce 4. Meaning, the issue is not specific to a single chipset. > Interestingly, if you add enough keyboards, the problem vanishes, which led > me to kbdmux. Sure enough, removing device kbdmux from the kernel makes > the problem go away (at the expensive of some functionality of course, but > this is my current workaround). An interesting find! I'll add this to the details in the above wiki. I've also CC'd kbdmux's maintainer. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From rwatson at FreeBSD.org Sat May 24 20:37:52 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Sat May 24 20:37:57 2008 Subject: kldxref oh oh In-Reply-To: References: Message-ID: <20080524213732.J9809@fledge.watson.org> On Fri, 23 May 2008, KAYVEN RIESE wrote: > On Fri, 23 May 2008, KAYVEN RIESE wrote: > >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked > > ref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kldxref: file isn't dynamically-linked > kv_bsd#make kernel >From the 7.0 errata notes: [20080307] Source upgrades from FreeBSD 6.X to FreeBSD 7.0-RELEASE will generate warnings from kldxref(8) during the installkernel step. These warnings are harmless and can be ignored. Robert N M Watson Computer Laboratory University of Cambridge From kayve at sfsu.edu Sat May 24 22:01:55 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sat May 24 22:02:00 2008 Subject: kldxref oh oh In-Reply-To: <20080524213732.J9809@fledge.watson.org> References: <20080524213732.J9809@fledge.watson.org> Message-ID: On Sat, 24 May 2008, Robert Watson wrote: > On Fri, 23 May 2008, KAYVEN RIESE wrote: >> On Fri, 23 May 2008, KAYVEN RIESE wrote: >> >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kldxref: file isn't dynamically-linked >> kv_bsd#make kernel > >> From the 7.0 errata notes: > > [20080307] Source upgrades from FreeBSD 6.X to FreeBSD 7.0-RELEASE will > generate warnings from kldxref(8) during the installkernel step. These > warnings are harmless and can be ignored. Thank you very much for showing me I should have RTFM in a very nice way. > > Robert N M Watson > Computer Laboratory > University of Cambridge > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From rwatson at FreeBSD.org Sat May 24 22:36:25 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Sat May 24 22:36:29 2008 Subject: kldxref oh oh In-Reply-To: References: <20080524213732.J9809@fledge.watson.org> Message-ID: <20080524233543.R9809@fledge.watson.org> On Sat, 24 May 2008, KAYVEN RIESE wrote: > On Sat, 24 May 2008, Robert Watson wrote: >> On Fri, 23 May 2008, KAYVEN RIESE wrote: >>> On Fri, 23 May 2008, KAYVEN RIESE wrote: >>> >>> kldxref: file isn't dynamically-linked >>> kldxref: file isn't dynamically-linked >>> kldxref: file isn't dynamically-linked >>> kv_bsd#make kernel >> >>> From the 7.0 errata notes: >> >> [20080307] Source upgrades from FreeBSD 6.X to FreeBSD 7.0-RELEASE will >> generate warnings from kldxref(8) during the installkernel step. These >> warnings are harmless and can be ignored. > > Thank you very much for showing me I should have RTFM in a very nice way. No problem :-). This is arguably a bug, and you're not the first person to ask about it, hence its being in the errata. I'd like to see this fixed for 7.1 so people sliding forwards from 6.x to 7.1 don't have the same obvious question. Robert N M Watson Computer Laboratory University of Cambridge From stefan at fafoe.narf.at Sun May 25 15:46:02 2008 From: stefan at fafoe.narf.at (Stefan Farfeleder) Date: Sun May 25 15:46:11 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <18489.32903.477434.465037@gromit.timing.com> References: <20080309152712.42752293@bhuda.mired.org> <20080516074433.GA1032@lizard.fafoe.narf.at> <20080516101143.3545937c@bhuda.mired.org> <20080516141307.GB1032@lizard.fafoe.narf.at> <18489.32903.477434.465037@gromit.timing.com> Message-ID: <20080525154537.GB1026@lizard.fafoe.narf.at> On Sun, May 25, 2008 at 09:06:47AM -0600, John E Hein wrote: > FWIW, it seems bash and sh report line number differently. > > # grep -n ^ ~/tmp/ln > 1:#!/bin/sh > 2:echo f line: $LINENO > 3:f() > 4:{ > 5:echo f line: $LINENO > 6:} > 7: > 8:f > 9:echo main line: $LINENO > 10:f > > > # /bin/sh ~/tmp/ln > f line: 2 > f line: 3 > main line: 9 > f line: 3 > > > # bash ~/tmp/ln > f line: 2 > f line: 5 > main line: 9 > f line: 5 Yes, I know. I think it is a bug in bash as SUSv3 states: "Set by the shell to a decimal number representing the current sequential line number (numbered starting with 1) within a script or function before it executes each command." Stefan From jhein at timing.com Sun May 25 15:58:39 2008 From: jhein at timing.com (John E Hein) Date: Sun May 25 15:58:45 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080525154537.GB1026@lizard.fafoe.narf.at> References: <20080309152712.42752293@bhuda.mired.org> <20080516074433.GA1032@lizard.fafoe.narf.at> <20080516101143.3545937c@bhuda.mired.org> <20080516141307.GB1032@lizard.fafoe.narf.at> <18489.32903.477434.465037@gromit.timing.com> <20080525154537.GB1026@lizard.fafoe.narf.at> Message-ID: <18489.36010.220739.891583@gromit.timing.com> Stefan Farfeleder wrote at 17:45 +0200 on May 25, 2008: > On Sun, May 25, 2008 at 09:06:47AM -0600, John E Hein wrote: > > FWIW, it seems bash and sh report line number differently. > > > > # grep -n ^ ~/tmp/ln > > 1:#!/bin/sh > > 2:echo f line: $LINENO > > 3:f() > > 4:{ > > 5:echo f line: $LINENO > > 6:} > > 7: > > 8:f > > 9:echo main line: $LINENO > > 10:f > > > > > > # /bin/sh ~/tmp/ln > > f line: 2 > > f line: 3 > > main line: 9 > > f line: 3 > > > > > > # bash ~/tmp/ln > > f line: 2 > > f line: 5 > > main line: 9 > > f line: 5 > > Yes, I know. I think it is a bug in bash as SUSv3 states: > > "Set by the shell to a decimal number representing the current > sequential line number (numbered starting with 1) within a script or > function before it executes each command." Okay. Not knowing more context, I guess the 'or' (script or function) leaves things somewhat subject to interpretation. From jhein at timing.com Sun May 25 16:09:20 2008 From: jhein at timing.com (John E Hein) Date: Sun May 25 16:09:25 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080516141307.GB1032@lizard.fafoe.narf.at> References: <20080309152712.42752293@bhuda.mired.org> <20080516074433.GA1032@lizard.fafoe.narf.at> <20080516101143.3545937c@bhuda.mired.org> <20080516141307.GB1032@lizard.fafoe.narf.at> Message-ID: <18489.32903.477434.465037@gromit.timing.com> Stefan Farfeleder wrote at 16:13 +0200 on May 16, 2008: > On Fri, May 16, 2008 at 10:11:43AM -0400, Mike Meyer wrote: > > On Fri, 16 May 2008 09:44:33 +0200 > > Stefan Farfeleder wrote: > > > > > On Sun, Mar 09, 2008 at 03:27:12PM -0400, Mike Meyer wrote: > > > > I've stumbled on to an obscure problem with autoconf 2.61, and I'm not > > > > sure quite what to do with it. I've already sent mail to the autoconf > > > > folks, but I'd like to understand what's going on. > > > > > > > > The problem is that, on a FreeBSD system with only /bin/sh and the > > > > ports zsh as installed shells, if you have SHELL set to zsh when > > > > invoking the autoconf-generated configure script, the script produces > > > > a broken Makefile. It doesn't generate an error, it just complains > > > > that: > > > > > > Can you please retry? /bin/sh now supports expanding $LINENO which was > > > often the reason for configure not liking it. > > > > Which branch, and how recently? > > HEAD and yesterday. FWIW, it seems bash and sh report line number differently. # grep -n ^ ~/tmp/ln 1:#!/bin/sh 2:echo f line: $LINENO 3:f() 4:{ 5:echo f line: $LINENO 6:} 7: 8:f 9:echo main line: $LINENO 10:f # /bin/sh ~/tmp/ln f line: 2 f line: 3 main line: 9 f line: 3 # bash ~/tmp/ln f line: 2 f line: 5 main line: 9 f line: 5 From kayve at sfsu.edu Sun May 25 22:15:07 2008 From: kayve at sfsu.edu (KAYVEN RIESE) Date: Sun May 25 22:15:11 2008 Subject: "Hackers" Documentation Message-ID: A couple of people on this list came to an agreement that "hackers" is appropriately understood as being for "developers." Well. I made my decision to subscribe based on the "documentation" here: http://tinyurl.com/623qw Namely: freebsd-hackers General technical discussion Googling "hackers" gives me 42.4 million hits. Wikipedia gives: * Hacker (computing) has a spectrum of meanings: o Hacker (computer security), someone who breaks computer and network security o Hacker (free and open source software), a programmer in the free software and open source movement o Hacker (hobbyist), an enthusiastic home computer hobbyist. Hmm.. I think I at very least belong to that third category. Personally, I take issue with the first defintion, but the facts are that most "laypersons" understand hackers in that way. I would say it is a "cracker" who breaks into a computer and not a "hacker". Somebody instilled this idea in me, I forget who. Anyway. I won't belabor this point further here. Thank you for your time. *----------------------------------------------------------* Kayven Riese, BSCS, MS (Physiology and Biophysics) (415) 902 5513 cellular http://kayve.net Webmaster http://ChessYoga.org *----------------------------------------------------------* From mathieu.prevot at gmail.com Mon May 26 14:52:06 2008 From: mathieu.prevot at gmail.com (Mathieu Prevot) Date: Mon May 26 14:52:09 2008 Subject: network bitrate of a poll of processes Message-ID: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> Hi, I would like to know the bitrate of a pool of child processes that use a network connection, how can I have something like netstat -w1 provide but at the process level ? Thanks Mathieu From dnelson at allantgroup.com Mon May 26 15:16:16 2008 From: dnelson at allantgroup.com (Dan Nelson) Date: Mon May 26 15:16:23 2008 Subject: network bitrate of a poll of processes In-Reply-To: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> References: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> Message-ID: <20080526151614.GA5582@dan.emsphone.com> In the last episode (May 26), Mathieu Prevot said: > Hi, > > I would like to know the bitrate of a pool of child processes that use > a network connection, how can I have something like netstat -w1 > provide but at the process level ? If you can segregate them to their own UID, you can use an ipfw "count ip from any to any uid 6666" rule to count the packets. Another option would be to jail them to a dedicated IP address and count traffic on that IP. -- Dan Nelson dnelson@allantgroup.com From mathieu.prevot at gmail.com Mon May 26 15:54:22 2008 From: mathieu.prevot at gmail.com (Mathieu Prevot) Date: Mon May 26 15:54:26 2008 Subject: flush file descriptor in a sh script while passing through a pipe Message-ID: <3e473cc60805260626k2469143ay9b0d52ea4044938b@mail.gmail.com> Hi, Let's be an example. With 1) I have a progress bar that is updated regularly but with 2) I have to wait for the end of the download (the next \n character ?) ... 1) wget http://tinyurl.com/5ztnb2 2) wget http://tinyurl.com/5ztnb2 --progress=bar:force 2>&1 | sed '/^Location/d;/^HTTP/d;/^--/d' I would like the progress bar to be updated through sed ... how can I flush the file descriptor from sh or with a tiny command/signal ? Thanks Mathieu From mathieu.prevot at gmail.com Mon May 26 16:14:19 2008 From: mathieu.prevot at gmail.com (Mathieu Prevot) Date: Mon May 26 16:14:22 2008 Subject: network bitrate of a poll of processes In-Reply-To: <20080526151614.GA5582@dan.emsphone.com> References: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> <20080526151614.GA5582@dan.emsphone.com> Message-ID: <3e473cc60805260914r537f113eh8d840a94c0a5d958@mail.gmail.com> 2008/5/26 Dan Nelson : > In the last episode (May 26), Mathieu Prevot said: >> Hi, >> >> I would like to know the bitrate of a pool of child processes that use >> a network connection, how can I have something like netstat -w1 >> provide but at the process level ? > > If you can segregate them to their own UID, you can use an ipfw "count > ip from any to any uid 6666" rule to count the packets. Another option > would be to jail them to a dedicated IP address and count traffic on > that IP. Both are nice... thank you. And don't you have something more portable ie. that could run on "any" posix os - like a sh script that use standard tools - and that doesn't need root user intervention ? Mathieu From olli at lurza.secnetix.de Mon May 26 17:59:51 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Mon May 26 17:59:54 2008 Subject: flush file descriptor in a sh script while passing through a pipe In-Reply-To: <3e473cc60805260626k2469143ay9b0d52ea4044938b@mail.gmail.com> Message-ID: <200805261759.m4QHxmpi091704@lurza.secnetix.de> Mathieu Prevot wrote: > Let's be an example. With 1) I have a progress bar that is updated > regularly but with 2) I have to wait for the end of the download (the > next \n character ?) ... > > 1) > wget http://tinyurl.com/5ztnb2 > > 2) > wget http://tinyurl.com/5ztnb2 --progress=bar:force 2>&1 | sed > '/^Location/d;/^HTTP/d;/^--/d' > > I would like the progress bar to be updated through sed ... how can I > flush the file descriptor from sh or with a tiny command/signal ? There are two problems with that. The first problem is the fact that wget's output is fully buffered when output is a pipe (not a tty). Some programs have an option for unbuffered or line-buffered output (e.g. tcpdump -l), but unfortunately wget does not. In fact most programs don't have such an option. There's a little trick that "emulates" a tty environment for a process so it thinks that its standard output is a tty, so output will be unbuffered: Simply run it inside script(1). I've needed that for myself occasionally so I've made a small alias for this: alias intty='script -qt0 /dev/null References: Message-ID: <20080526220157.J35550@mp2.macomnet.net> On Sun, 18 May 2008, 20:56+0100, Steven Hartland wrote: > sendmsg is not documented as ever returning EINVAL but yet when > using the following code to send credentials to a remote host > results in EINVAL from sendmsg. > > I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX > sockets and not PF_INET sockets and hence the code in dbus > is actually invalid. > > Can anyone confirm this is the case or not? > [...] confirmed :-) -- Maxim Konovalov From kris at FreeBSD.org Mon May 26 18:47:57 2008 From: kris at FreeBSD.org (Kris Kennaway) Date: Mon May 26 18:48:00 2008 Subject: Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket In-Reply-To: References: Message-ID: <483B05DB.3000309@FreeBSD.org> Steven Hartland wrote: > sendmsg is not documented as ever returning EINVAL but yet when > using the following code to send credentials to a remote host > results in EINVAL from sendmsg. > > I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX > sockets and not PF_INET sockets and hence the code in dbus > is actually invalid. > > Can anyone confirm this is the case or not? What would it even mean to send credentials to a remote host over an AF_INET socket? Kris From dillon at apollo.backplane.com Mon May 26 19:49:19 2008 From: dillon at apollo.backplane.com (Matthew Dillon) Date: Mon May 26 19:49:22 2008 Subject: Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket References: <483B05DB.3000309@FreeBSD.org> Message-ID: <200805261936.m4QJaS7Q074045@apollo.backplane.com> :> using the following code to send credentials to a remote host :> results in EINVAL from sendmsg. :> :> I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX :> sockets and not PF_INET sockets and hence the code in dbus :> is actually invalid. :> :> Can anyone confirm this is the case or not? : :What would it even mean to send credentials to a remote host over an :AF_INET socket? : :Kris The only way to do it is to do a connect-back to a secure port. In otherwords, to use the identd protocol. Control messages like SCM_CREDS (and descriptor passing for that matter) only work over local protocol sockets. -Matt Matthew Dillon From mspitzer at gmail.com Mon May 26 20:03:25 2008 From: mspitzer at gmail.com (Marc Spitzer) Date: Mon May 26 20:03:29 2008 Subject: network bitrate of a poll of processes In-Reply-To: <3e473cc60805260914r537f113eh8d840a94c0a5d958@mail.gmail.com> References: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> <20080526151614.GA5582@dan.emsphone.com> <3e473cc60805260914r537f113eh8d840a94c0a5d958@mail.gmail.com> Message-ID: <8c50a3c30805261237y6573a2c9ra4af56477d93af78@mail.gmail.com> On Mon, May 26, 2008 at 12:14 PM, Mathieu Prevot wrote: > 2008/5/26 Dan Nelson : >> In the last episode (May 26), Mathieu Prevot said: >>> Hi, >>> >>> I would like to know the bitrate of a pool of child processes that use >>> a network connection, how can I have something like netstat -w1 >>> provide but at the process level ? >> >> If you can segregate them to their own UID, you can use an ipfw "count >> ip from any to any uid 6666" rule to count the packets. Another option >> would be to jail them to a dedicated IP address and count traffic on >> that IP. > > Both are nice... thank you. And don't you have something more portable > ie. that could run on "any" posix os - like a sh script that use > standard tools - and that doesn't need root user intervention ? > > Mathieu I do not think so, firewall for example is controlled by root. And users do not generally have detailed access to other users information. You can look into the BASM framework to track what your apps are doing and if you know the behavior to expect you can use netflow/argus/tcpdump to track that and post process the logs in each case into something useful. What are you trying to do, details do help? Thanks, marc -- Freedom is nothing but a chance to be better. Albert Camus From joel at FreeBSD.org Mon May 26 20:11:20 2008 From: joel at FreeBSD.org (Joel Dahl) Date: Mon May 26 20:11:47 2008 Subject: Please welcome our Summer of Code Students In-Reply-To: References: <20080422075642.GA72273@freebsdmall.com> Message-ID: <483B1598.2010006@FreeBSD.org> (Catching up on old mail) Nick Barkas skrev: > Hi everyone > I'm Nick Barkas, and will be working on making memory usage for > dirhash dynamic. I'm a master's student studying scientific computing > at Kungliga Tekniska h?gskolan (Royal Institute of Technology) in > Stockholm. I have been using FreeBSD for several years and I am very > happy to get this chance to spend a lot of time hacking on it, > learning more about what goes on in the kernel, and hopefully making > UFS2 faster. V?lkommen! :-) (Which means "Welcome" for those of you not familiar with the Swedish language) -- Joel From killing at multiplay.co.uk Mon May 26 20:19:33 2008 From: killing at multiplay.co.uk (Steven Hartland) Date: Mon May 26 20:19:37 2008 Subject: Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket References: <483B05DB.3000309@FreeBSD.org> <200805261936.m4QJaS7Q074045@apollo.backplane.com> Message-ID: <7EB9398C5B46415B96D2C20A2E261E13@multiplay.co.uk> ----- Original Message ----- From: "Matthew Dillon" >>> >>>[...] >>>confirmed :-) >>> >:> using the following code to send credentials to a remote host > :> results in EINVAL from sendmsg. > :> > :> I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX > :> sockets and not PF_INET sockets and hence the code in dbus > :> is actually invalid. > :> > :> Can anyone confirm this is the case or not? > : > :What would it even mean to send credentials to a remote host over an > :AF_INET socket? > : > :Kris > > The only way to do it is to do a connect-back to a secure port. In > otherwords, to use the identd protocol. > > Control messages like SCM_CREDS (and descriptor passing for that matter) > only work over local protocol sockets. Thank you for the confirmation guys. I'll look to create a patch I can submit to the dbus project to disable the use of SCM_CREDS when using PF_UNIX sockets. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From avg at icyb.net.ua Mon May 26 20:26:26 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon May 26 20:26:30 2008 Subject: make universe: only kernels, no worlds Message-ID: <483B1CEB.90405@icyb.net.ua> Is there a built-in way to do an equivalent of building universe but only for kernels (and kernel tool-chains)? Or some straightforward to emulate that? Thanks! -- Andriy Gapon From mspitzer at gmail.com Mon May 26 21:02:37 2008 From: mspitzer at gmail.com (Marc Spitzer) Date: Mon May 26 21:02:41 2008 Subject: network bitrate of a poll of processes In-Reply-To: <3e473cc60805261327g38349857ga87dcfa0b855c1f6@mail.gmail.com> References: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> <20080526151614.GA5582@dan.emsphone.com> <3e473cc60805260914r537f113eh8d840a94c0a5d958@mail.gmail.com> <8c50a3c30805261237y6573a2c9ra4af56477d93af78@mail.gmail.com> <3e473cc60805261327g38349857ga87dcfa0b855c1f6@mail.gmail.com> Message-ID: <8c50a3c30805261402y596a37a2m61a8993b37d13d7b@mail.gmail.com> On Mon, May 26, 2008 at 4:27 PM, Mathieu Prevot wrote: > 2008/5/26 Marc Spitzer : >> On Mon, May 26, 2008 at 12:14 PM, Mathieu Prevot >> wrote: >>> 2008/5/26 Dan Nelson : >>>> In the last episode (May 26), Mathieu Prevot said: >>>>> Hi, >>>>> >>>>> I would like to know the bitrate of a pool of child processes that use >>>>> a network connection, how can I have something like netstat -w1 >>>>> provide but at the process level ? >>>> >>>> If you can segregate them to their own UID, you can use an ipfw "count >>>> ip from any to any uid 6666" rule to count the packets. Another option >>>> would be to jail them to a dedicated IP address and count traffic on >>>> that IP. >>> >>> Both are nice... thank you. And don't you have something more portable >>> ie. that could run on "any" posix os - like a sh script that use >>> standard tools - and that doesn't need root user intervention ? >>> >>> Mathieu >> >> I do not think so, firewall for example is controlled by root. And >> users do not generally have detailed access to other users >> information. You can look into the BASM framework to track what your >> apps are doing and if you know the behavior to expect you can use >> netflow/argus/tcpdump to track that and post process the logs in each >> case into something useful. >> >> What are you trying to do, details do help? >> >> marc > > I have a script that generates many child processes that download > videos (~10MB) in this case, or do heavy data processing/analysing. I > can tune the number of workers as I want, but It's a shared cluster so > I would like to > > 1) measure 2) optimize/control (by number of workers ? at the script > level ! so I can update thing very dynamically) > a) CPU load b) IO on drive(s) c) network load > > and I prefer solutions that don't need root access/intervention. I can > measure indirectly network load using `date +%s` and `du -b |sed > 's/\t.*//' ` regularly... but I may have to deal with several 10GB > later - and actually the number of files is more critical than their > size. > Maybe writing a (very) quick and dirty python app that listen/attach > to a group of processes (for the network case) ? > > Mathieu > for the CPU stuff take a look at ps: bob% ps aO%mem,%cpu,ppid PID %MEM %CPU PPID TT STAT TIME COMMAND 768 0.2 0.0 1 v0 Is+ 0:00.01 /usr/libexec/getty Pc ttyv0 769 0.2 0.0 1 v1 Is+ 0:00.01 /usr/libexec/getty Pc ttyv1 770 0.2 0.0 1 v2 Is+ 0:00.01 /usr/libexec/getty Pc ttyv2 771 0.2 0.0 1 v3 Is+ 0:00.01 /usr/libexec/getty Pc ttyv3 772 0.2 0.0 1 v4 Is+ 0:00.01 /usr/libexec/getty Pc ttyv4 773 0.2 0.0 1 v5 Is+ 0:00.01 /usr/libexec/getty Pc ttyv5 774 0.2 0.0 1 v6 Is+ 0:00.01 /usr/libexec/getty Pc ttyv6 775 0.2 0.0 1 v7 Is+ 0:00.01 /usr/libexec/getty Pc ttyv7 272 0.2 0.0 1 con- I 0:00.04 dhclient: le0 [priv] (dhclient) 951 0.6 0.0 949 p0 Rs 0:00.28 -zsh (zsh) 1056 0.2 0.0 951 p0 R+ 0:00.01 ps aO%mem,%cpu,ppid add what you need and you have your cpu and such stats, it might do your IO as well also look for thing in (s)bin that end in stat for things that might be a better fit. Network I do not know how to get you the level of access you want without root/trusted access at least on collection. You will not need it on processing. as far as managing thing after you have the information I have nothing for you there. good luck, marc -- Freedom is nothing but a chance to be better. Albert Camus From mathieu.prevot at gmail.com Mon May 26 22:45:21 2008 From: mathieu.prevot at gmail.com (Mathieu Prevot) Date: Mon May 26 22:45:25 2008 Subject: network bitrate of a poll of processes In-Reply-To: <8c50a3c30805261237y6573a2c9ra4af56477d93af78@mail.gmail.com> References: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> <20080526151614.GA5582@dan.emsphone.com> <3e473cc60805260914r537f113eh8d840a94c0a5d958@mail.gmail.com> <8c50a3c30805261237y6573a2c9ra4af56477d93af78@mail.gmail.com> Message-ID: <3e473cc60805261327g38349857ga87dcfa0b855c1f6@mail.gmail.com> 2008/5/26 Marc Spitzer : > On Mon, May 26, 2008 at 12:14 PM, Mathieu Prevot > wrote: >> 2008/5/26 Dan Nelson : >>> In the last episode (May 26), Mathieu Prevot said: >>>> Hi, >>>> >>>> I would like to know the bitrate of a pool of child processes that use >>>> a network connection, how can I have something like netstat -w1 >>>> provide but at the process level ? >>> >>> If you can segregate them to their own UID, you can use an ipfw "count >>> ip from any to any uid 6666" rule to count the packets. Another option >>> would be to jail them to a dedicated IP address and count traffic on >>> that IP. >> >> Both are nice... thank you. And don't you have something more portable >> ie. that could run on "any" posix os - like a sh script that use >> standard tools - and that doesn't need root user intervention ? >> >> Mathieu > > I do not think so, firewall for example is controlled by root. And > users do not generally have detailed access to other users > information. You can look into the BASM framework to track what your > apps are doing and if you know the behavior to expect you can use > netflow/argus/tcpdump to track that and post process the logs in each > case into something useful. > > What are you trying to do, details do help? > > marc I have a script that generates many child processes that download videos (~10MB) in this case, or do heavy data processing/analysing. I can tune the number of workers as I want, but It's a shared cluster so I would like to 1) measure 2) optimize/control (by number of workers ? at the script level ! so I can update thing very dynamically) a) CPU load b) IO on drive(s) c) network load and I prefer solutions that don't need root access/intervention. I can measure indirectly network load using `date +%s` and `du -b |sed 's/\t.*//' ` regularly... but I may have to deal with several 10GB later - and actually the number of files is more critical than their size. Maybe writing a (very) quick and dirty python app that listen/attach to a group of processes (for the network case) ? Mathieu From steve at Watt.COM Tue May 27 01:30:55 2008 From: steve at Watt.COM (Steve Watt) Date: Tue May 27 01:31:02 2008 Subject: Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket In-Reply-To: <7EB9398C5B46415B96D2C20A2E261E13@multiplay.co.uk> References: <483B05DB.3000309@FreeBSD.org> <200805261936.m4QJaS7Q074045@apollo.backplane.com> Message-ID: <200805270101.m4R11fEq075003@wattres.watt.com> In <7EB9398C5B46415B96D2C20A2E261E13@multiplay.co.uk>, Steven Hartland wrote: [ Attributions fatally damaged, removing ] >> :> using the following code to send credentials to a remote host >> :> results in EINVAL from sendmsg. >> :> >> :> I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX >> :> sockets and not PF_INET sockets and hence the code in dbus >> :> is actually invalid. >> :> >> :> Can anyone confirm this is the case or not? >> : >> :What would it even mean to send credentials to a remote host over an >> :AF_INET socket? >> >> The only way to do it is to do a connect-back to a secure port. In >> otherwords, to use the identd protocol. >> >> Control messages like SCM_CREDS (and descriptor passing for that matter) >> only work over local protocol sockets. > >Thank you for the confirmation guys. I'll look to create a patch >I can submit to the dbus project to disable the use of SCM_CREDS >when using PF_UNIX sockets. I think you got the sense backwards here; SCM_CREDS is OK when using PF_UNIX / PF_LOCAL. It's PF_INET that shouldn't be going there. Steve -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.5" / 37N 20' 15.3" Internet: steve @ Watt.COM Whois: SW32-ARIN Free time? There's no such thing. It just comes in varying prices... From keramida at ceid.upatras.gr Tue May 27 02:44:48 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Tue May 27 02:44:53 2008 Subject: make universe: only kernels, no worlds In-Reply-To: <483B1CEB.90405@icyb.net.ua> (Andriy Gapon's message of "Mon, 26 May 2008 23:26:19 +0300") References: <483B1CEB.90405@icyb.net.ua> Message-ID: <87k5hg1n5b.fsf@kobe.laptop> On Mon, 26 May 2008 23:26:19 +0300, Andriy Gapon wrote: > Is there a built-in way to do an equivalent of building universe but > only for kernels (and kernel tool-chains)? Or some straightforward to > emulate that? Yes, there is :) You can set MAKE_JUST_KERNELS: env MAKE_JUST_KERNELS='yes' make universe The tunable is a bit `hidden' in the src/Makefile code, near line 292: 292 universe_${target}: 293 .if !defined(MAKE_JUST_KERNELS) 294 @echo ">> ${target} started on `LC_ALL=C date`" 295 -cd ${.CURDIR} && ${MAKE} ${JFLAG} buildworld \ 296 TARGET=${target} \ 297 __MAKE_CONF=/dev/null \ 298 > _.${target}.buildworld 2>&1 299 @echo ">> ${target} buildworld completed on `LC_ALL=C date`" 300 .endif but it's there already. From rick-freebsd at kiwi-computer.com Tue May 27 04:39:18 2008 From: rick-freebsd at kiwi-computer.com (Rick C. Petty) Date: Tue May 27 04:39:21 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080524201633.GA81364@eos.sc1.parodius.com> References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> Message-ID: <20080527041236.GB68298@keira.kiwi-computer.com> On Sat, May 24, 2008 at 01:16:33PM -0700, Jeremy Chadwick wrote: > On Sat, May 24, 2008 at 01:38:08PM +1000, Fraser Tweedale wrote: > > Since upgrading to RELENG_7_0 I was experiencing momentary freezes (of > > about .5 seconds) whenever the capslock or numlock buttons were pressed. > > Let me guess -- you're using a USB keyboard. > > > The issue occurs both in console and in X, and for both ULE and 4BSD. The > > problem was reproducible with USB keyboards only (ukbd); atkbd seems fine. > > It also occurs when numlockx is used to set numlock on or off without > > keyboard interaction. > > I figured as much. I hadn't checked numlock/capslock myself, but vcons > switching also has that problem. Yes, it's semi-documented; see section > "USB" below: > > http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues > > For what it's worth, the systems I've confirmed the problem on are Intel > and nForce 4. Meaning, the issue is not specific to a single chipset. I've seen this on certain chipsets (nForce4, Intel) and not on others (nForce5+). I originally reported this problem over two years ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-March/015724.html This problem started when the kbdmux was added to the kernel and has not been fixed since (although the "delay" has been reduced from around two seconds to about 0.4 seconds). I verified the problem still persists with 7-stable as of last week. This "pause" actually blocks the entire kernel, not just things under GIANT. One test involved having a gmirror sync or fsck run while I switched virtal consoles and watched the drive activity LEDs turn off for the entire duration of the pause. > > Interestingly, if you add enough keyboards, the problem vanishes, which led > > me to kbdmux. Sure enough, removing device kbdmux from the kernel makes > > the problem go away (at the expensive of some functionality of course, but > > this is my current workaround). > > An interesting find! I'll add this to the details in the above wiki. > I've also CC'd kbdmux's maintainer. I've noticed that even plugging in a PS/2 keyboard fixes the problem. However unplugging the PS/2 keyboard restores the problematic behavior. I never found a solution to this problem except for buying different hardware or keeping a PS/2 keyboard plugged in. Being that the issue is on your wiki page, does that mean this problem will be fixed some day? -- Rick C. Petty From koitsu at FreeBSD.org Tue May 27 05:11:10 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Tue May 27 05:11:13 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080527041236.GB68298@keira.kiwi-computer.com> References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> Message-ID: <20080527051109.GA26502@eos.sc1.parodius.com> On Mon, May 26, 2008 at 11:12:36PM -0500, Rick C. Petty wrote: > On Sat, May 24, 2008 at 01:16:33PM -0700, Jeremy Chadwick wrote: > > On Sat, May 24, 2008 at 01:38:08PM +1000, Fraser Tweedale wrote: > > > Since upgrading to RELENG_7_0 I was experiencing momentary freezes (of > > > about .5 seconds) whenever the capslock or numlock buttons were pressed. > > > > Let me guess -- you're using a USB keyboard. > > > > > The issue occurs both in console and in X, and for both ULE and 4BSD. The > > > problem was reproducible with USB keyboards only (ukbd); atkbd seems fine. > > > It also occurs when numlockx is used to set numlock on or off without > > > keyboard interaction. > > > > I figured as much. I hadn't checked numlock/capslock myself, but vcons > > switching also has that problem. Yes, it's semi-documented; see section > > "USB" below: > > > > http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues > > > > For what it's worth, the systems I've confirmed the problem on are Intel > > and nForce 4. Meaning, the issue is not specific to a single chipset. > > I've seen this on certain chipsets (nForce4, Intel) and not on others > (nForce5+). I originally reported this problem over two years ago: > > http://lists.freebsd.org/pipermail/freebsd-hackers/2006-March/015724.html > > This problem started when the kbdmux was added to the kernel and has not > been fixed since (although the "delay" has been reduced from around two > seconds to about 0.4 seconds). I verified the problem still persists with > 7-stable as of last week. > > This "pause" actually blocks the entire kernel, not just things under > GIANT. One test involved having a gmirror sync or fsck run while I > switched virtal consoles and watched the drive activity LEDs turn off for > the entire duration of the pause. > > > > Interestingly, if you add enough keyboards, the problem vanishes, which led > > > me to kbdmux. Sure enough, removing device kbdmux from the kernel makes > > > the problem go away (at the expensive of some functionality of course, but > > > this is my current workaround). > > > > An interesting find! I'll add this to the details in the above wiki. > > I've also CC'd kbdmux's maintainer. > > I've noticed that even plugging in a PS/2 keyboard fixes the problem. > However unplugging the PS/2 keyboard restores the problematic behavior. > I never found a solution to this problem except for buying different > hardware or keeping a PS/2 keyboard plugged in. > > Being that the issue is on your wiki page, does that mean this problem will > be fixed some day? Hopefully. :-) I'd like to point fingers at kbdmux being the responsible piece, but the USB stack has a history of being flakey in many regards, hence my reluctance. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From rick-freebsd at kiwi-computer.com Tue May 27 05:48:43 2008 From: rick-freebsd at kiwi-computer.com (Rick C. Petty) Date: Tue May 27 05:48:47 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080527051109.GA26502@eos.sc1.parodius.com> References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> Message-ID: <20080527054842.GA69670@keira.kiwi-computer.com> On Mon, May 26, 2008 at 10:11:09PM -0700, Jeremy Chadwick wrote: > > > > Being that the issue is on your wiki page, does that mean this problem will > > be fixed some day? > > Hopefully. :-) I'd like to point fingers at kbdmux being the > responsible piece, but the USB stack has a history of being flakey in > many regards, hence my reluctance. Agreed. I was never able to pinpoint it myself, but it certainly didn't show up before kbdmux. -- Rick C. Petty From avg at icyb.net.ua Tue May 27 08:21:28 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 27 08:21:36 2008 Subject: make universe: only kernels, no worlds In-Reply-To: <87k5hg1n5b.fsf@kobe.laptop> References: <483B1CEB.90405@icyb.net.ua> <87k5hg1n5b.fsf@kobe.laptop> Message-ID: <483BC480.2030302@icyb.net.ua> on 27/05/2008 05:44 Giorgos Keramidas said the following: > On Mon, 26 May 2008 23:26:19 +0300, Andriy Gapon wrote: >> Is there a built-in way to do an equivalent of building universe but >> only for kernels (and kernel tool-chains)? Or some straightforward to >> emulate that? > > Yes, there is :) > > You can set MAKE_JUST_KERNELS: Thank you! The name is very intuitive, I should have grep-ed for something like this :-) > env MAKE_JUST_KERNELS='yes' make universe > > The tunable is a bit `hidden' in the src/Makefile code, near line 292: > > 292 universe_${target}: > 293 .if !defined(MAKE_JUST_KERNELS) > 294 @echo ">> ${target} started on `LC_ALL=C date`" > 295 -cd ${.CURDIR} && ${MAKE} ${JFLAG} buildworld \ > 296 TARGET=${target} \ > 297 __MAKE_CONF=/dev/null \ > 298 > _.${target}.buildworld 2>&1 > 299 @echo ">> ${target} buildworld completed on `LC_ALL=C date`" > 300 .endif > > but it's there already. > -- Andriy Gapon From killing at multiplay.co.uk Tue May 27 09:19:59 2008 From: killing at multiplay.co.uk (Steven Hartland) Date: Tue May 27 09:20:02 2008 Subject: Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket References: <483B05DB.3000309@FreeBSD.org><200805261936.m4QJaS7Q074045@apollo.backplane.com> <200805270101.m4R11fEq075003@wattres.watt.com> Message-ID: <39E6E2BA17E34472BC3E05749632E766@multiplay.co.uk> ----- Original Message ----- From: "Steve Watt" >>Thank you for the confirmation guys. I'll look to create a patch >>I can submit to the dbus project to disable the use of SCM_CREDS >>when using PF_UNIX sockets. > > I think you got the sense backwards here; SCM_CREDS is OK when using > PF_UNIX / PF_LOCAL. It's PF_INET that shouldn't be going there. Yer just posted while tired ;-) Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From maksim.yevmenkin at gmail.com Tue May 27 18:00:13 2008 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Tue May 27 18:00:20 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080527054842.GA69670@keira.kiwi-computer.com> References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> Message-ID: On 5/26/08, Rick C. Petty wrote: > On Mon, May 26, 2008 at 10:11:09PM -0700, Jeremy Chadwick wrote: > > > > > > Being that the issue is on your wiki page, does that mean this problem will > > > be fixed some day? > > > > Hopefully. :-) I'd like to point fingers at kbdmux being the > > responsible piece, but the USB stack has a history of being flakey in > > many regards, hence my reluctance. > > > Agreed. I was never able to pinpoint it myself, but it certainly didn't > show up before kbdmux. 1) what keyboards are used? usb only, ps2 only or mix? 2) what mice are used? usb only, ps2 only or mix? if ps2 devices (i.e. keyboard and mouse) are not used at all, does disabling atkbd(4) and/or psm(4) help? thanks, max From rick-freebsd at kiwi-computer.com Tue May 27 19:45:55 2008 From: rick-freebsd at kiwi-computer.com (Rick C. Petty) Date: Tue May 27 19:45:58 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> Message-ID: <20080527194554.GA79741@keira.kiwi-computer.com> On Tue, May 27, 2008 at 10:34:47AM -0700, Maksim Yevmenkin wrote: > > > > Agreed. I was never able to pinpoint it myself, but it certainly didn't > > show up before kbdmux. > > 1) what keyboards are used? usb only, ps2 only or mix? > > 2) what mice are used? usb only, ps2 only or mix? > > if ps2 devices (i.e. keyboard and mouse) are not used at all, does > disabling atkbd(4) and/or psm(4) help? I think I stated this in my original thread but let me restate. I've only noticed the problem when both usbkbd and atkbd are both enabled (as in the GENERIC kernel) and a USB keyboard (only) is attached. Once I plug in the atkbd, the problem goes away. The problem also does not exist if just the atkbd is plugged in with no USB keyboard. I haven't noticed a problem with the mice, but I've not used a PS/2 mouse in almost a decade. I have noticed a problem which has hit me rarely on a 6.x system; perhaps because the problem is not easily reproducable have I never seen it in 7.x. Basically I've seen the kernel get "stuck" and the only way to unstick it was to move the USB mouse. Moving the mouse allowed the interrupt to trigger and things would happen elsewhere in the kernel, but as soon as I stopped moving the mouse everything else stopped (ata reads/writes, graphics updates, etc.). Unplugging the mouse was a bad idea because the plugin event never triggered. I've only witnessed this on post-kbdmux systems but since it wasn't reproducable in any reliable fashion, I couldn't pin the problem down. -- Rick C. Petty From maksim.yevmenkin at gmail.com Tue May 27 20:32:14 2008 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Tue May 27 20:32:17 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080527194554.GA79741@keira.kiwi-computer.com> References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> <20080527194554.GA79741@keira.kiwi-computer.com> Message-ID: On 5/27/08, Rick C. Petty wrote: > On Tue, May 27, 2008 at 10:34:47AM -0700, Maksim Yevmenkin wrote: > > > > > > Agreed. I was never able to pinpoint it myself, but it certainly didn't > > > show up before kbdmux. > > > > 1) what keyboards are used? usb only, ps2 only or mix? > > > > 2) what mice are used? usb only, ps2 only or mix? > > > > if ps2 devices (i.e. keyboard and mouse) are not used at all, does > > disabling atkbd(4) and/or psm(4) help? > > > I think I stated this in my original thread but let me restate. I've only > noticed the problem when both usbkbd and atkbd are both enabled (as in the > GENERIC kernel) and a USB keyboard (only) is attached. Once I plug in the > atkbd, the problem goes away. The problem also does not exist if just the > atkbd is plugged in with no USB keyboard. right, which (to me) seems to indicate that something is going wrong when kbdmux is trying to work on two keyboards (i.e. ukbd and atkbd) and one of them (i.e. atkbd) is "not present". the problem, as i see it, is that atkbd keyboard is always present even if no physical ps2 keyboard is connected to the ps2 port. i understand that people are doing this in order to be able to "hot plug" ps2 keyboard (which is a bad idea as ps2 port was never designed to be "hot pluggable", imo). now, kbdmux simply switches all slave keyboards into raw mode and pretends to have one keyboard with lots of duplicated keys. it also tries to preserve consistent state on all slave keyboards. so, for example, when num/scroll/capslock etc. key is pressed, ioctl is called to light up corresponding led on the keyboard. this ioctl calls driver specific routine which will talk to the hardware. because kbdmux tries to keep consistent state, it will call this ioctl for every slave keyboard. i suspect that because physical ps2 keyboard is not actually present, the atkbd driver will have to timeout while talking to non-present hardware. obviously the things "work" without kbdmux because without kbdmux it is not possible to have a keyboard without physical keyboard present. well, technically it is possible with atkbd, but then one could not press any keys to trigger atkbd driver to talk back to the non-present hardware. i suppose it is possible to write a simple program that uses console ioctl to flash keyboard leds and try to run it on a machine without kbdmux and no physical keyboard attached to the ps2 port to see if this triggers the same problem. so, as a suggestion, please try to specify 0x1 flag to atkbd, i.e. from atkbd(4) Driver Flags The atkbd driver accepts the following driver flags. They can be set either in /boot/device.hints, or else from within the boot loader (see loader(8)). bit 0 (FAIL_IF_NO_KBD) By default the atkbd driver will install even if a keyboard is not actually connected to the system. This option prevents the driver from being installed in this situation. ... and see if this helps. you wont be able to "hot plug" ps2 keyboards, but i suspect you probably can live without it. > I haven't noticed a problem with the mice, but I've not used a PS/2 mouse > in almost a decade. I have noticed a problem which has hit me rarely on a > 6.x system; perhaps because the problem is not easily reproducable have I > never seen it in 7.x. Basically I've seen the kernel get "stuck" and the > only way to unstick it was to move the USB mouse. Moving the mouse allowed > the interrupt to trigger and things would happen elsewhere in the kernel, > but as soon as I stopped moving the mouse everything else stopped (ata > reads/writes, graphics updates, etc.). Unplugging the mouse was a bad idea > because the plugin event never triggered. I've only witnessed this on > post-kbdmux systems but since it wasn't reproducable in any reliable > fashion, I couldn't pin the problem down. well, i'm not sure it is the same problem, but if you have a way to reproduce it reliably and believe that it is a kbdmux related problem please let me know. thanks, max From rick-freebsd at kiwi-computer.com Tue May 27 21:01:59 2008 From: rick-freebsd at kiwi-computer.com (Rick C. Petty) Date: Tue May 27 21:02:02 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> <20080527194554.GA79741@keira.kiwi-computer.com> Message-ID: <20080527210158.GB81555@keira.kiwi-computer.com> On Tue, May 27, 2008 at 01:32:10PM -0700, Maksim Yevmenkin wrote: > > right, which (to me) seems to indicate that something is going wrong > when kbdmux is trying to work on two keyboards (i.e. ukbd and atkbd) > and one of them (i.e. atkbd) is "not present". the problem, as i see > it, is that atkbd keyboard is always present even if no physical ps2 > keyboard is connected to the ps2 port. That's precisely how I see it. > i understand that people are > doing this in order to be able to "hot plug" ps2 keyboard (which is a > bad idea as ps2 port was never designed to be "hot pluggable", imo). Although not designed as hot pluggable, there's nothing in the technical specs which should prevent it. In fact, the specs imply that this is plausible. The only issue I know about is when older hardware didn't have TVS or line driver protection which could blow the serial driver chips (on either side) for example when both the keyboard and host try to drive the clock at the same time, but I doubt is a problem with modern hardware. Also, I believe the spec specifies the proper use of pullup resistors, so this shouldn't be a problem. A common complaint I've heard is that some (cheap) keyboards don't respond with a power-on self test completed (0xAA) when they are initially plugged in until they receive an initial command from the host. I've never seen such a keyboard; most keyboards send 0xAA immediately after power on, IIRC. > i suspect that because physical ps2 keyboard is not actually > present, the atkbd driver will have to timeout while talking to > non-present hardware. Sure, but this timeout shouldn't be ~0.4 seconds. I believe the spec says the maximum wait period to tell if the device is nonresponsive is around or less than 20ms. Certainly, blocking the entire kernel for the timeout is a bad thing, especially since the driver should make full use of the asynchronous onboard keyboard controller. > so, as a suggestion, please try to specify 0x1 flag to atkbd, i.e. from atkbd(4) > ... > > and see if this helps. you wont be able to "hot plug" ps2 keyboards, > but i suspect you probably can live without it. I'm almost certain this will help, but I believe this shouldn't be the answer to this problem. There's no reason atkbdc(4) should block the entire kernel for any appreciable amount of time. -- Rick C. Petty From gabor at kovesdan.org Tue May 27 21:01:07 2008 From: gabor at kovesdan.org (Gabor Kovesdan) Date: Tue May 27 21:26:20 2008 Subject: Progress of the BSD-licensed grep, sort and diff Message-ID: <483C72D4.7030806@kovesdan.org> Hello all, as you might have already known, I'm working on the BSD-licensed versions of these utilities, porting them from OpenBSD and complete the functionality. Still, I have a lot to do, but here's a little bit of status report: grep: - Implemented --label - Implemented --null - Implemented -D / --devices - Implemented -H / --with-filename - Implemented --version - Eliminate warnings - style(9) cleanup sort: - Added support for long command line options - Implemented -S / --buffer-size - Implemented --version - Eliminate warnings - style(9) cleanup diff: - Implemented --version - Eliminate warnings - style(9) cleanup Apart from the missing features, they should work fine, you can try it via the bsdgrep, bsdsort and bsddiff ports in the textproc category. These ports create bsd* and gnu* symlinks for the easier testing. If any of you could use these ports and report the possible problems, that would be cool. Cheers, G?bor K?vesd?n From maksim.yevmenkin at gmail.com Tue May 27 21:28:28 2008 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Tue May 27 21:28:35 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080527210158.GB81555@keira.kiwi-computer.com> References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> <20080527194554.GA79741@keira.kiwi-computer.com> <20080527210158.GB81555@keira.kiwi-computer.com> Message-ID: On 5/27/08, Rick C. Petty wrote: > On Tue, May 27, 2008 at 01:32:10PM -0700, Maksim Yevmenkin wrote: > > [...] > > i suspect that because physical ps2 keyboard is not actually > > present, the atkbd driver will have to timeout while talking to > > non-present hardware. > > Sure, but this timeout shouldn't be ~0.4 seconds. I believe the spec says > the maximum wait period to tell if the device is nonresponsive is around or > less than 20ms. Certainly, blocking the entire kernel for the timeout is a > bad thing, especially since the driver should make full use of the > asynchronous onboard keyboard controller. well, i just took a brief look at atkbd(4). specifically one function - wait_while_controller_busy(). this function polls status every KBDC_DELAYTIME (20) usec with retry count of 5000. so, just this function alone can give up to 100 msec delay. keep in mind that wait_while_controller_busy() is apparently called every time driver need to talk to the hardware. i can see how we could delay kernel for 400 msec or even more. > > so, as a suggestion, please try to specify 0x1 flag to atkbd, i.e. from atkbd(4) > > ... > > > > and see if this helps. you wont be able to "hot plug" ps2 keyboards, > > but i suspect you probably can live without it. > > I'm almost certain this will help, but I believe this shouldn't be the > answer to this problem. There's no reason atkbdc(4) should block the > entire kernel for any appreciable amount of time. agree. all i was trying to say is that kbdmux is only as good as underlying low level keyboard drivers. if a low level keyboard driver uses completely synchronous approach to poll the hardware, there is not much kbdmux can do about it. thanks, max From gabor at FreeBSD.org Tue May 27 21:37:06 2008 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Tue May 27 21:37:11 2008 Subject: Progress of the BSD-licensed grep, sort and diff Message-ID: <483C7F00.4060007@FreeBSD.org> Hello all, as you might have already known, I'm working on the BSD-licensed versions of these utilities, porting them from OpenBSD and complete the functionality. Still, I have a lot to do, but here's a little bit of status report: grep: - Implemented --label - Implemented --null - Implemented -D / --devices - Implemented -H / --with-filename - Implemented --version - Eliminate warnings - style(9) cleanup sort: - Added support for long command line options - Implemented -S / --buffer-size - Implemented --version - Eliminate warnings - style(9) cleanup diff: - Implemented --version - Eliminate warnings - style(9) cleanup Apart from the missing features, they should work fine, you can try it via the bsdgrep, bsdsort and bsddiff ports in the textproc category. These ports create bsd* and gnu* symlinks for the easier testing. If any of you could use these ports and report the possible problems, that would be cool. Cheers, G?bor K?vesd?n From gabor at FreeBSD.org Tue May 27 21:37:10 2008 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Tue May 27 21:37:19 2008 Subject: Progress of the BSD-licensed grep, sort and diff Message-ID: <483C7F04.4030003@FreeBSD.org> Hello all, as you might have already known, I'm working on the BSD-licensed versions of these utilities, porting them from OpenBSD and complete the functionality. Still, I have a lot to do, but here's a little bit of status report: grep: - Implemented --label - Implemented --null - Implemented -D / --devices - Implemented -H / --with-filename - Implemented --version - Eliminate warnings - style(9) cleanup sort: - Added support for long command line options - Implemented -S / --buffer-size - Implemented --version - Eliminate warnings - style(9) cleanup diff: - Implemented --version - Eliminate warnings - style(9) cleanup Apart from the missing features, they should work fine, you can try it via the bsdgrep, bsdsort and bsddiff ports in the textproc category. These ports create bsd* and gnu* symlinks for the easier testing. If any of you could use these ports and report the possible problems, that would be cool. Cheers, G?bor K?vesd?n From rick-freebsd at kiwi-computer.com Tue May 27 22:42:58 2008 From: rick-freebsd at kiwi-computer.com (Rick C. Petty) Date: Tue May 27 22:43:01 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: References: <48378DA0.8040506@frase.id.au> <20080524201633.GA81364@eos.sc1.parodius.com> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> <20080527194554.GA79741@keira.kiwi-computer.com> <20080527210158.GB81555@keira.kiwi-computer.com> Message-ID: <20080527224256.GA83303@keira.kiwi-computer.com> On Tue, May 27, 2008 at 02:28:26PM -0700, Maksim Yevmenkin wrote: > > well, i just took a brief look at atkbd(4). specifically one function > - wait_while_controller_busy(). this function polls status every > KBDC_DELAYTIME (20) usec with retry count of 5000. so, just this > function alone can give up to 100 msec delay. keep in mind that > wait_while_controller_busy() is apparently called every time driver > need to talk to the hardware. i can see how we could delay kernel for > 400 msec or even more. I'm not sure why we retry 5000 times. 100ms seems like a long time to block the entire kernel. Is there any reason we can't spawn a kernel thread to deal with the waits? I recommend that we also reduce the timeouts to at most twice what the spec states. How come this doesn't happen when other keys are pressed? Just when the console is flipped. Perhaps because it tries to set the LEDs first? -- Rick C. Petty From maksim.yevmenkin at gmail.com Tue May 27 23:05:14 2008 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Tue May 27 23:05:20 2008 Subject: temporary freezes when pressing capslock / numlock In-Reply-To: <20080527224256.GA83303@keira.kiwi-computer.com> References: <48378DA0.8040506@frase.id.au> <20080527041236.GB68298@keira.kiwi-computer.com> <20080527051109.GA26502@eos.sc1.parodius.com> <20080527054842.GA69670@keira.kiwi-computer.com> <20080527194554.GA79741@keira.kiwi-computer.com> <20080527210158.GB81555@keira.kiwi-computer.com> <20080527224256.GA83303@keira.kiwi-computer.com> Message-ID: On 5/27/08, Rick C. Petty wrote: > On Tue, May 27, 2008 at 02:28:26PM -0700, Maksim Yevmenkin wrote: > > > > well, i just took a brief look at atkbd(4). specifically one function > > - wait_while_controller_busy(). this function polls status every > > KBDC_DELAYTIME (20) usec with retry count of 5000. so, just this > > function alone can give up to 100 msec delay. keep in mind that > > wait_while_controller_busy() is apparently called every time driver > > need to talk to the hardware. i can see how we could delay kernel for > > 400 msec or even more. > > I'm not sure why we retry 5000 times. 100ms seems like a long time to > block the entire kernel. Is there any reason we can't spawn a kernel > thread to deal with the waits? I recommend that we also reduce the > timeouts to at most twice what the spec states. all great ideas. please send in the patches and i will be happy to review and commit them for you. > How come this doesn't happen when other keys are pressed? Just when the > console is flipped. Perhaps because it tries to set the LEDs first? like i said, i suspect delay happens only when driver needs to send command to the keyboard. that is what happens when you press num/scroll/capslock. thanks, max > > > -- Rick C. Petty > From brooks at freebsd.org Wed May 28 01:33:29 2008 From: brooks at freebsd.org (Brooks Davis) Date: Wed May 28 01:33:31 2008 Subject: FreeBSD and LLVM In-Reply-To: <20080521172352.GJ1181@hoeg.nl> References: <20080520170639.GE1181@hoeg.nl> <48330CDA.2080802@FreeBSD.org> <20080521172352.GJ1181@hoeg.nl> Message-ID: <20080528013333.GA821@lor.one-eyed-alien.net> On Wed, May 21, 2008 at 07:23:52PM +0200, Ed Schouten wrote: > Hello Stefan, > > (CC'ing this back to the list) > > * Stefan Esser wrote: > > this is great news! Can you provide your patches to upgrade the port > > to 2.3-pre? > > I could, but I patched the ports rather poorly. The plists are > incorrect. I just incremented the version numbers, updated the distinfo > and made some changes to the llvm-gcc4 port to use the proper wrksrc. I've got WIP updates of both ports, but probably won't get them committed until next week. > > If 2.3-pre allows the kernel to boot and run, then many user-land > > programs should work as well. Testing a make world (and preparing > > the infrastructure to support llvm in addition to gcc and icc might > > be worthwhile ...). > > We should really try that. Unfortunately I'm quite busy, hacking on the > TTY layer. ;-) > > > You probably know about the clang project, which tries to completely > > replace the gcc parts needed for llvm-gcc ... > > Yes. I haven't looked at it yet. It doesn't seem to be in Ports yet. Any > takers? ;-) Unless you make an evil port that does a checkout of last nights sources from svn, there isn't much point today. Chris Latner said it would probably be ready to start publishing snapshots in 3-6 months (i.e. llvm 2.4 or 2.5). I'd like to whip one up, but haven't yet. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080528/7f2b9c20/attachment.pgp From xcllnt at mac.com Wed May 28 01:58:04 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Wed May 28 01:58:08 2008 Subject: FreeBSD and LLVM In-Reply-To: <20080528013333.GA821@lor.one-eyed-alien.net> References: <20080520170639.GE1181@hoeg.nl> <48330CDA.2080802@FreeBSD.org> <20080521172352.GJ1181@hoeg.nl> <20080528013333.GA821@lor.one-eyed-alien.net> Message-ID: On May 27, 2008, at 6:33 PM, Brooks Davis wrote: >>> You probably know about the clang project, which tries to completely >>> replace the gcc parts needed for llvm-gcc ... >> >> Yes. I haven't looked at it yet. It doesn't seem to be in Ports >> yet. Any >> takers? ;-) > > Unless you make an evil port that does a checkout of last nights > sources > from svn, there isn't much point today. Chris Latner said it would > probably be ready to start publishing snapshots in 3-6 months (i.e. > llvm > 2.4 or 2.5). I'd like to whip one up, but haven't yet. BTW: I'm working on LLVM directly. I'm keeping track of FreeBSD support here: http://wiki.freebsd.org/LowLevelVirtualMachine What I'd like to have is some nightly automation that does the testing across all platforms. We could create nightly snapshots as part of the automation and create a port that installs the latest snapshot? -- Marcel Moolenaar xcllnt@mac.com From ivoras at freebsd.org Wed May 28 13:05:15 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Wed May 28 13:05:22 2008 Subject: Impact of having a large number of open file descriptors Message-ID: Hi, Im thinking again of the old idea of implementing poor man's file replication system using kqueue to monitor changes on files. This would require opening every file that needs to be monitored and using EVFILT_VNODE to monitor them. I suppose this would work for a small-ish number of files like for a user's home directory but what about 100,000 files or millions of files? One other question: do kqueue events "coalesce" in the sense that if N operations happen (like write()s), there can be < N events passed to the kqueue (NOTE_WRITE)? While at it, will EVFILT_VNODE and NOTE_WRITE catch "additional" ways the file can be modified, meaning mmap()? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080528/a8926c06/signature.pgp From olli at lurza.secnetix.de Wed May 28 14:46:53 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Wed May 28 14:46:55 2008 Subject: Impact of having a large number of open file descriptors In-Reply-To: Message-ID: <200805281446.m4SEkojn099133@lurza.secnetix.de> Ivan Voras wrote: > Im thinking again of the old idea of implementing poor man's file > replication system using kqueue to monitor changes on files. It would be cool to have a kernel interface so you could attach to a mountpoint and receive a log of all activity on that file system. That's similar to what DragonFly's journaling feature does. Unfortunately the kqueue interface isn't capable of doing something like that ... So this is not an answer to your question, I'm afraid. > One other question: do kqueue events "coalesce" in the sense that if N > operations happen (like write()s), there can be < N events passed to the > kqueue (NOTE_WRITE)? The manpage says: "Multiple events which trigger the filter do not result in multiple kevents being placed on the kqueue; instead, the filter will aggregate the events into a single struct kevent." > While at it, will EVFILT_VNODE and NOTE_WRITE catch "additional" ways > the file can be modified, meaning mmap()? A quick grep for NOTE_WRITE on the sys tree indicates that it doesn't. I'm not 100% sure though. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Documentation is like sex; when it's good, it's very, very good, and when it's bad, it's better than nothing." -- Dick Brandon From ivoras at freebsd.org Wed May 28 16:26:06 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Wed May 28 16:26:10 2008 Subject: Impact of having a large number of open file descriptors In-Reply-To: <200805281446.m4SEkojn099133@lurza.secnetix.de> References: <200805281446.m4SEkojn099133@lurza.secnetix.de> Message-ID: <9bbcef730805280900i1195b5f4rb879d22e9e0319d9@mail.gmail.com> 2008/5/28 Oliver Fromme : > Ivan Voras wrote: > > Im thinking again of the old idea of implementing poor man's file > > replication system using kqueue to monitor changes on files. > > It would be cool to have a kernel interface so you could > attach to a mountpoint and receive a log of all activity > on that file system. That's similar to what DragonFly's > journaling feature does. /me agrees. > Unfortunately the kqueue interface isn't capable of doing > something like that ... So this is not an answer to your > question, I'm afraid. > > > One other question: do kqueue events "coalesce" in the sense that if N > > operations happen (like write()s), there can be < N events passed to the > > kqueue (NOTE_WRITE)? > > The manpage says: "Multiple events which trigger the filter > do not result in multiple kevents being placed on the kqueue; > instead, the filter will aggregate the events into a single > struct kevent." That's mildly unfortunate but I think you're right - it agrees with the statement from the kqueue paper: "Events will normally considered to be "level-triggered", as opposed to "edge-triggered"." > > While at it, will EVFILT_VNODE and NOTE_WRITE catch "additional" ways > > the file can be modified, meaning mmap()? > > A quick grep for NOTE_WRITE on the sys tree indicates that > it doesn't. I'm not 100% sure though. Also bad for some users. From israsilva at gmail.com Wed May 28 22:11:08 2008 From: israsilva at gmail.com (Israel Lehnen Silva) Date: Wed May 28 22:11:12 2008 Subject: FreeBSD + LDAP + SAMBA + WINDOWS Message-ID: <5ce468b90805281511h2729be73l65dccdcfe13ad4db@mail.gmail.com> Friends, I have the following scenario: Server FreeBSD 7.0 Stable authenticating in one basis LDAP through of the PAM (pam_ldap and nss_ldap) In same server, have running the SAMBA 3.0.28 authenticating too in basis LDAP and using the scripts smbldap-tools. Tool LDAPAdmin for administration of basis LDAP. THE PROBLEM: When chang the pass of user in basis LDAP trhough of LDAPAdmin, select th cryptograpy "MD5 Crypt" for the atribuct userPassword This way, I achieve log in the Windows and FreeBSD by terminal, ssh... but when chang pass of user by Windows, the cryptograpy of password in atribuct userPassword is chanded for SSHA and so not conect in FreeBSD, also just conect in windows. FreeBSD and SAMBA authenticating in LDAP, and changing the password by own user, not interfering in auth of ssh in FreeBSD... Someone implemented??? The configuration of Samba: # Samba config file created using SWAT # from 0.0.0.0 (0.0.0.0) # Date: 2008/05/05 16:13:37 [global] dos charset = CP850 unix charset = ISO8859-1 workgroup = NOVOARQ netbios name = NARQ server string = LDAP Teste # update encrypted = Yes # unix password sync = Yes passwd program = /usr/local/sbin/smbldap-passwd -u "%u" encrypt passwords = Yes # obey pam restrictions = Yes socket options = TCP_NODELAY IPTOS_LOWDELAY IPTOS_THROUGHPUT SO_KEEPALIVE SO_RCVBUF=8192 SO_SNDBUF=8192 log level = 1 log file = /var/log/samba/samba.log max log size = 0 time server = Yes machine password timeout = 0 logon script = %G.bat logon drive = H: logon home = \\NARQ\%U os level = 255 preferred master = Yes domain master = yes domain logons = yes local master = yes passdb backend = ldapsam:ldap://ldap.dominio.com.br ldap passwd sync = Yes ldap delete dn = Yes ldap ssl = no ldap admin dn = cn=admin,dc=unilasalle,dc=edu,dc=br ldap suffix = dc=unilasalle,dc=edu,dc=br ldap machine suffix = ou=computadores ldap user suffix = ou=usuarios ldap group suffix = ou=grupos ldap idmap suffix = sambaDomainName=NOVOARQ idmap backend = ldap:ldap://ldap.dominio.com.br idmap uid = 10000-65000 idmap gid = 10000-65000 enable privileges = yes add user script = /usr/local/sbin/smbldap-useradd -m "%u" # delete user script = /usr/local/sbin/smbldap-userdel "%u" add group script = /usr/local/sbin/smbldap-groupadd -p "%g" # delete group script = /usr/local/sbin/smbldap-groupdel "%g" add user to group script = /usr/local/sbin/smbldap-groupmod -m "%u" "%g" delete user from group script = /usr/local/sbin/smbldap-groupmod -x "%u" "%g" set primary group script = /usr/local/sbin/smbldap-usermod -g "%g" "%u" add machine script = /usr/local/sbin/smbldap-useradd -w "%u" utmp = Yes smb ports = 445 139 name resolve order = wins bcast hosts time server = Yes template shell = /bin/false winbind use default domain = no map acl inherit = Yes strict locking = Yes wins support = Yes interfaces = bce0 bind interfaces only = Yes dns proxy = No create mask = 0770 force create mode = 0770 directory mask = 0770 force directory mode = 0770 Best regards, Israel Lehnen Silva. From jhb at freebsd.org Thu May 29 04:15:26 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu May 29 04:15:30 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080525154537.GB1026@lizard.fafoe.narf.at> References: <20080309152712.42752293@bhuda.mired.org> <18489.32903.477434.465037@gromit.timing.com> <20080525154537.GB1026@lizard.fafoe.narf.at> Message-ID: <200805290004.41653.jhb@freebsd.org> On Sunday 25 May 2008 11:45:37 am Stefan Farfeleder wrote: > On Sun, May 25, 2008 at 09:06:47AM -0600, John E Hein wrote: > > FWIW, it seems bash and sh report line number differently. > > > > # grep -n ^ ~/tmp/ln > > 1:#!/bin/sh > > 2:echo f line: $LINENO > > 3:f() > > 4:{ > > 5:echo f line: $LINENO > > 6:} > > 7: > > 8:f > > 9:echo main line: $LINENO > > 10:f > > > > > > # /bin/sh ~/tmp/ln > > f line: 2 > > f line: 3 > > main line: 9 > > f line: 3 > > > > > > # bash ~/tmp/ln > > f line: 2 > > f line: 5 > > main line: 9 > > f line: 5 > > Yes, I know. I think it is a bug in bash as SUSv3 states: > > "Set by the shell to a decimal number representing the current > sequential line number (numbered starting with 1) within a script or > function before it executes each command." Actually, the bash way seems more intuitive. And it does say "the current sequentional line number within a ... function before it executes each command" The "within a function" implies that this property goes inside of functions instead of forcing all commands in a function to use the starting line of the function which is what you are saying? -- John Baldwin From jhb at freebsd.org Thu May 29 04:30:56 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu May 29 04:31:01 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <20080525154537.GB1026@lizard.fafoe.narf.at> References: <20080309152712.42752293@bhuda.mired.org> <18489.32903.477434.465037@gromit.timing.com> <20080525154537.GB1026@lizard.fafoe.narf.at> Message-ID: <200805290004.41653.jhb@freebsd.org> On Sunday 25 May 2008 11:45:37 am Stefan Farfeleder wrote: > On Sun, May 25, 2008 at 09:06:47AM -0600, John E Hein wrote: > > FWIW, it seems bash and sh report line number differently. > > > > # grep -n ^ ~/tmp/ln > > 1:#!/bin/sh > > 2:echo f line: $LINENO > > 3:f() > > 4:{ > > 5:echo f line: $LINENO > > 6:} > > 7: > > 8:f > > 9:echo main line: $LINENO > > 10:f > > > > > > # /bin/sh ~/tmp/ln > > f line: 2 > > f line: 3 > > main line: 9 > > f line: 3 > > > > > > # bash ~/tmp/ln > > f line: 2 > > f line: 5 > > main line: 9 > > f line: 5 > > Yes, I know. I think it is a bug in bash as SUSv3 states: > > "Set by the shell to a decimal number representing the current > sequential line number (numbered starting with 1) within a script or > function before it executes each command." Actually, the bash way seems more intuitive. And it does say "the current sequentional line number within a ... function before it executes each command" The "within a function" implies that this property goes inside of functions instead of forcing all commands in a function to use the starting line of the function which is what you are saying? -- John Baldwin From ssouhlal at FreeBSD.org Thu May 29 16:48:35 2008 From: ssouhlal at FreeBSD.org (Suleiman Souhlal) Date: Thu May 29 16:48:41 2008 Subject: Impact of having a large number of open file descriptors In-Reply-To: <200805281446.m4SEkojn099133@lurza.secnetix.de> References: <200805281446.m4SEkojn099133@lurza.secnetix.de> Message-ID: <64200F15-4444-44FE-B904-673543441F35@FreeBSD.org> On May 28, 2008, at 7:46 AM, Oliver Fromme wrote: > Ivan Voras wrote: >> Im thinking again of the old idea of implementing poor man's file >> replication system using kqueue to monitor changes on files. > > It would be cool to have a kernel interface so you could > attach to a mountpoint and receive a log of all activity > on that file system. That's similar to what DragonFly's > journaling feature does. > > Unfortunately the kqueue interface isn't capable of doing > something like that ... So this is not an answer to your > question, I'm afraid. I have an old patch that makes kqueue monitor every file write on the system and return the inode number in the knote's data field: http:// people.freebsd.org/~ssouhlal/testing/kqueue-anyvnode-20050503.diff . I'd think it shouldn't be too hard to make it per-mountpoint.. >> While at it, will EVFILT_VNODE and NOTE_WRITE catch "additional" ways >> the file can be modified, meaning mmap()? I think it does, although it will get delayed until the mmapped pages get flushed to disk. -- Suleiman From yuri at rawbw.com Thu May 29 17:56:05 2008 From: yuri at rawbw.com (Yuri) Date: Thu May 29 17:56:10 2008 Subject: airodump-ng loses all stations after 30 mins Message-ID: <483EEA59.7010104@rawbw.com> Playing with wireless I tried to use airodump-ng to see what one can get out of it. Command /usr/local/sbin/airodump-ng -w xx ral0 works fine for ~30 minutes and then loses all stations. Restarting this command gets it to the state with no stations right away. But after I do 'ifconfig ral0 down ; ifconfig ral0 up' command works again. Until it loses stations in ~30 minutes again and everything repeats. It seems that ral0 gets into some internal state that prevents airodump-ng from picking up packets. And down/up clears it. But the output of 'ifconfig ral0' doesn't change after down/up: ral0: flags=2a9c3 metric 0 mtu 1500 ether 00:18:f8:2e:40:25 inet 192.168.168.133 netmask 0xffffff00 broadcast 192.168.168.255 media: IEEE 802.11 Wireless Ethernet autoselect (autoselect ) status: associated ssid "" channel 5 (2432 Mhz 11g) bssid 00:18:f8:2e:40:25 authmode OPEN privacy OFF txpower 50 scanvalid 60 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi11g 7 roam:rate11g 5 protmode CTS My card is recognized as: ral0: mem 0xcffe8000-0xcffeffff irq 17 at device 10.0 on pci0 And I use 7.0-STABLE. This seems like a bug in the driver. But I am asking here hoping someone will have an answer off the top of their head. Yuri From dinesh at alphaque.com Fri May 30 07:45:13 2008 From: dinesh at alphaque.com (Dinesh Nair) Date: Fri May 30 07:45:19 2008 Subject: Maximum memory allocation per process In-Reply-To: <20080522133819.GA45578@eos.sc1.parodius.com> References: <48356E75.1050105@thearle.com.au> <20080522133819.GA45578@eos.sc1.parodius.com> Message-ID: <20080530135814.390321ee@prophet.alphaque.com> On Thu, 22 May 2008 06:38:19 -0700, Jeremy Chadwick wrote: > You need to modify some kernel settings via /boot/loader.conf and > reboot. Here's what we use on our production RELENG_6 and RELENG_7 > boxes: > > # Increase maximum allocatable memory on a process to 2GB. > # (We don't choose 3GB (our max RAM) since that would > # exhaust all memory, and result in a kernel panic.) > # Set default memory size as 768MB. > # Maximum stack size is 256MB. > # > kern.maxdsiz="2048M" > kern.dfldsiz="768M" > kern.maxssiz="256MB" for those of us who're booting off a stripped down freebsd and are not using the 4th routines, are the above to be set before 'load /kernel' or after 'load /kernel' ? currently, loader.rc contains, --- cut here --- set console=comconsole set hw.ata.atapi_dma="0" set hw.ata.ata_dma="0" set hw.ata.wc="0" load /kernel load -t mfs_root /mfsroot boot --- cut here --- -- Regards, /\_/\ "All dogs go to heaven." dinesh@alphaque.com (0 0) http://www.openmalaysiablog.com/ +==========================----oOO--(_)--OOo----==========================+ | for a in past present future; do | | for b in clients employers associates relatives neighbours pets; do | | echo "The opinions here in no way reflect the opinions of my $a $b." | | done; done | +=========================================================================+ From koitsu at FreeBSD.org Fri May 30 07:46:57 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Fri May 30 07:47:22 2008 Subject: Maximum memory allocation per process In-Reply-To: <20080530135814.390321ee@prophet.alphaque.com> References: <48356E75.1050105@thearle.com.au> <20080522133819.GA45578@eos.sc1.parodius.com> <20080530135814.390321ee@prophet.alphaque.com> Message-ID: <20080530074656.GA27066@eos.sc1.parodius.com> On Fri, May 30, 2008 at 01:58:14PM +0800, Dinesh Nair wrote: > On Thu, 22 May 2008 06:38:19 -0700, Jeremy Chadwick wrote: > > You need to modify some kernel settings via /boot/loader.conf and > > reboot. Here's what we use on our production RELENG_6 and RELENG_7 > > boxes: > > > > # Increase maximum allocatable memory on a process to 2GB. > > # (We don't choose 3GB (our max RAM) since that would > > # exhaust all memory, and result in a kernel panic.) > > # Set default memory size as 768MB. > > # Maximum stack size is 256MB. > > # > > kern.maxdsiz="2048M" > > kern.dfldsiz="768M" > > kern.maxssiz="256MB" > > for those of us who're booting off a stripped down freebsd and are not > using the 4th routines, are the above to be set before 'load /kernel' or > after 'load /kernel' ? > > currently, loader.rc contains, > > --- cut here --- > set console=comconsole > set hw.ata.atapi_dma="0" > set hw.ata.ata_dma="0" > set hw.ata.wc="0" > load /kernel > load -t mfs_root /mfsroot > boot > --- cut here --- I believe these would either need to be set *before* loading the kernel, or, possibly it doesn't matter at all (they might be passed to the kernel itself during "boot"). -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From vasanth.raonaik at gmail.com Fri May 30 08:51:31 2008 From: vasanth.raonaik at gmail.com (vasanth raonaik) Date: Fri May 30 08:52:01 2008 Subject: Pahole on freebsd Message-ID: Hello Hackers, I have searched online to find the pahole utility for freebsd. Do we have a pahole utility for freebsd? If yes, can some one point me to the correct location from where i can download. Thanks, Vasanth From olli at lurza.secnetix.de Fri May 30 14:37:11 2008 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Fri May 30 14:37:16 2008 Subject: Maximum memory allocation per process In-Reply-To: <20080530135814.390321ee@prophet.alphaque.com> Message-ID: <200805301437.m4UEb0J8010462@lurza.secnetix.de> Dinesh Nair wrote: > for those of us who're booting off a stripped down freebsd and are not > using the 4th routines, are the above to be set before 'load /kernel' or > after 'load /kernel' ? It doesn't matter. The tunables are passed to the kernel when it is booted. In fact, the standard beastie.4th stuff loads the kernel before displaying the menu, so the settings happen after the kernel is loaded. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Being really good at C++ is like being really good at using rocks to sharpen sticks." -- Thant Tessman From flz at xbsd.org Fri May 30 15:39:05 2008 From: flz at xbsd.org (Florent Thoumie) Date: Fri May 30 16:09:01 2008 Subject: CFT: adding configuration file support to pkg_install Message-ID: This adds support for /etc/pkg.conf configuration file. Also, this adds support for naive multi-site package fetching. Any comment welcome (and appreciated). Patch is here: http://people.freebsd.org/~flz/local/ports/pkg-install-config.diff Tarball is here: http://people.freebsd.org/~flz/local/ports/pkg-install-0a553aac.tar.bz2 -- Florent Thoumie flz@FreeBSD.org FreeBSD Committer From stef at memberwebs.com Fri May 30 21:21:39 2008 From: stef at memberwebs.com (Stef Walter) Date: Fri May 30 21:37:48 2008 Subject: RAID status issues with Intel MatrixRAID ataraid Message-ID: <20080530210347.5E1A6F1808E@mx.npubs.com> This isn't a bug report, just a heads up to anyone using the same setup. This occurred in production, and I don't currently have a non-production hardware on which to investigate this bug and/or create a patch. In certain conditions, even though the Firmware RAID BIOS, says that a RAID is degraded, the FreeBSD driver thinks it's business as usual. In my case this resulted in file system corruption. Relevant output below. FreeBSD version: 6.3-RELEASE-p2 Cheers, Stef Walter BIOS OUTPUT: Copyright(C) 2003-06 Intel Corporation. All Rights Reserved. RAID Volumes: ID Name Level Strip Size Status Bootable 0 RAID Set 1 RAID1(Mirror) N/A 465.8GB Degraded Yes 1 RAID1(Mirror) N/A 465.8GB Normal Yes Physical Disks: Port Drive Model Serial # Size Type/Status(Vol ID) 0 WDC WD5000ABYS-0 WD-WCAPW5637184 465.8GB Member Disk(0) 1 ST3500320NS 5QM09E6F 465.8GB Error Occurred(0) 2 WDC WD5000ABYS-0 WD-WCAPW5548822 465.8GB Member Disk(1) 3 ST3500320NS 5QM0991D 465.8GB Member Disk(1) Press to enter Configuration Utility........... RELEVANT SNIPPET FROM DMESG: ar0: 476937MB status: READY ar0: disk0 READY (master) using ad4 at ata2-master ar0: disk1 READY (mirror) using ad6 at ata3-master From koitsu at FreeBSD.org Fri May 30 23:02:58 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Fri May 30 23:03:02 2008 Subject: RAID status issues with Intel MatrixRAID ataraid In-Reply-To: <20080530210347.5E1A6F1808E@mx.npubs.com> References: <20080530210347.5E1A6F1808E@mx.npubs.com> Message-ID: <20080530230258.GA61615@eos.sc1.parodius.com> On Fri, May 30, 2008 at 09:03:48PM +0000, Stef Walter wrote: > This isn't a bug report, just a heads up to anyone using the same setup. > This occurred in production, and I don't currently have a non-production > hardware on which to investigate this bug and/or create a patch. > > In certain conditions, even though the Firmware RAID BIOS, says that a > RAID is degraded, the FreeBSD driver thinks it's business as usual. In > my case this resulted in file system corruption. Relevant output below. See the "ATA (SATA, PATA, ATAPI)" section, specifically the Intel MatrixRAID items: http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From jhb at freebsd.org Sat May 31 01:49:27 2008 From: jhb at freebsd.org (John Baldwin) Date: Sat May 31 01:49:31 2008 Subject: RAID status issues with Intel MatrixRAID ataraid In-Reply-To: <20080530210347.5E1A6F1808E@mx.npubs.com> References: <20080530210347.5E1A6F1808E@mx.npubs.com> Message-ID: <200805302143.11920.jhb@freebsd.org> On Friday 30 May 2008 05:03:48 pm Stef Walter wrote: > This isn't a bug report, just a heads up to anyone using the same setup. > This occurred in production, and I don't currently have a non-production > hardware on which to investigate this bug and/or create a patch. > > In certain conditions, even though the Firmware RAID BIOS, says that a > RAID is degraded, the FreeBSD driver thinks it's business as usual. In > my case this resulted in file system corruption. Relevant output below. > > FreeBSD version: 6.3-RELEASE-p2 > > Cheers, > Stef Walter > > > > BIOS OUTPUT: > > Copyright(C) 2003-06 Intel Corporation. All Rights Reserved. > > RAID Volumes: > ID Name Level Strip Size Status Bootable > 0 RAID Set 1 RAID1(Mirror) N/A 465.8GB Degraded Yes > 1 RAID1(Mirror) N/A 465.8GB Normal Yes > > Physical Disks: > Port Drive Model Serial # Size Type/Status(Vol ID) > 0 WDC WD5000ABYS-0 WD-WCAPW5637184 465.8GB Member Disk(0) > 1 ST3500320NS 5QM09E6F 465.8GB Error Occurred(0) > 2 WDC WD5000ABYS-0 WD-WCAPW5548822 465.8GB Member Disk(1) > 3 ST3500320NS 5QM0991D 465.8GB Member Disk(1) > Press to enter Configuration Utility........... > > > > RELEVANT SNIPPET FROM DMESG: > > ar0: 476937MB status: READY > ar0: disk0 READY (master) using ad4 at ata2-master > ar0: disk1 READY (mirror) using ad6 at ata3-master Err, you have two RAID volumes, one degraded, and one ok. ar0 is apparently the one that is ok (drives 2 and 3). What is more odd is that it isn't seeing a RAID config at all for ad0 and ad2 (drives 0 and 1). -- John Baldwin From stef at memberwebs.com Sat May 31 00:38:53 2008 From: stef at memberwebs.com (Stef Walter) Date: Sat May 31 02:26:22 2008 Subject: RAID status issues with Intel MatrixRAID ataraid References: <20080530210347.5E1A6F1808E@mx.npubs.com> <20080530230258.GA61615@eos.sc1.parodius.com> Message-ID: <20080531003848.D33C7F18286@mx.npubs.com> Jeremy Chadwick wrote: >> In certain conditions, even though the Firmware RAID BIOS, says that a >> RAID is degraded, the FreeBSD driver thinks it's business as usual. In >> my case this resulted in file system corruption. Relevant output below. > > See the "ATA (SATA, PATA, ATAPI)" section, specifically the Intel > MatrixRAID items: > > http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues Yes, I reported several of those as well as provided patches for them. That was back when I had relevant hardware in my lab. Cheers, Stef Walter From tzhuan at csie.org Sat May 31 06:19:07 2008 From: tzhuan at csie.org (Tz-Huan Huang) Date: Sat May 31 06:19:11 2008 Subject: Is there any way to increase the KVM? Message-ID: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> Hi, Our nfs server is running 7-stable/amd64 with 8G ram, the size of zfs pool is 12T. We have set vm.kmem_size and vm.kmem_size_max to 1.5G, but the kernel still panics by "kmem_map too small" often. According to [1], the limitation is not only by the loader (is it fixed now?) but also by the default layout of KVM. [2] points a way to increase the KVM, but we get the similar linking error. Is there any standard way to modify the layout of KVM? For example, we may want to set KVM to 6G and leave the 2G for user space usage. Thanks, Tz-Huan [1] http://lists.freebsd.org/pipermail/freebsd-current/2007-October/077964.html [2] http://lists.freebsd.org/pipermail/freebsd-current/2008-March/084325.html From ivoras at freebsd.org Sat May 31 10:05:13 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Sat May 31 10:05:18 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> Message-ID: Tz-Huan Huang wrote: > Is there any standard way to modify the layout of KVM? For example, we > may want to set KVM to 6G and leave the 2G for user space usage. > [2] http://lists.freebsd.org/pipermail/freebsd-current/2008-March/084325.html Isn't this a limitation of gcc and the kernel/userland memory model inherited from Linux? ref: http://www.network-theory.co.uk/docs/gccintro/gccintro_65.html http://gcc.gnu.org/onlinedocs/gcc-3.4.4/gcc/i386-and-x86_002d64-Options.html (see -mcmodel=kernel) If it is, I don't think it will be "fixed" soon. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080531/64d91258/signature.pgp From stefan at fafoe.narf.at Sat May 31 10:11:38 2008 From: stefan at fafoe.narf.at (Stefan Farfeleder) Date: Sat May 31 10:11:43 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <200805290004.41653.jhb@freebsd.org> References: <20080309152712.42752293@bhuda.mired.org> <18489.32903.477434.465037@gromit.timing.com> <20080525154537.GB1026@lizard.fafoe.narf.at> <200805290004.41653.jhb@freebsd.org> Message-ID: <20080531101101.GA1077@lizard.fafoe.narf.at> On Thu, May 29, 2008 at 12:04:41AM -0400, John Baldwin wrote: > On Sunday 25 May 2008 11:45:37 am Stefan Farfeleder wrote: > > On Sun, May 25, 2008 at 09:06:47AM -0600, John E Hein wrote: > > > FWIW, it seems bash and sh report line number differently. > > > > > > # grep -n ^ ~/tmp/ln > > > 1:#!/bin/sh > > > 2:echo f line: $LINENO > > > 3:f() > > > 4:{ > > > 5:echo f line: $LINENO > > > 6:} > > > 7: > > > 8:f > > > 9:echo main line: $LINENO > > > 10:f > > > > > > > > > # /bin/sh ~/tmp/ln > > > f line: 2 > > > f line: 3 > > > main line: 9 > > > f line: 3 > > > > > > > > > # bash ~/tmp/ln > > > f line: 2 > > > f line: 5 > > > main line: 9 > > > f line: 5 > > > > Yes, I know. I think it is a bug in bash as SUSv3 states: > > > > "Set by the shell to a decimal number representing the current > > sequential line number (numbered starting with 1) within a script or > > function before it executes each command." > > Actually, the bash way seems more intuitive. And it does say "the current > sequentional line number within a ... function before it executes each > command" > > The "within a function" implies that this property goes inside of functions > instead of forcing all commands in a function to use the starting line of the > function which is what you are saying? I've started a thread about that on standards@. From stefan at fafoe.narf.at Sat May 31 10:26:59 2008 From: stefan at fafoe.narf.at (Stefan Farfeleder) Date: Sat May 31 10:27:03 2008 Subject: Why doesn't autoconf like our /bin/sh? In-Reply-To: <200805290004.41653.jhb@freebsd.org> References: <20080309152712.42752293@bhuda.mired.org> <18489.32903.477434.465037@gromit.timing.com> <20080525154537.GB1026@lizard.fafoe.narf.at> <200805290004.41653.jhb@freebsd.org> Message-ID: <20080531101101.GA1077@lizard.fafoe.narf.at> On Thu, May 29, 2008 at 12:04:41AM -0400, John Baldwin wrote: > On Sunday 25 May 2008 11:45:37 am Stefan Farfeleder wrote: > > On Sun, May 25, 2008 at 09:06:47AM -0600, John E Hein wrote: > > > FWIW, it seems bash and sh report line number differently. > > > > > > # grep -n ^ ~/tmp/ln > > > 1:#!/bin/sh > > > 2:echo f line: $LINENO > > > 3:f() > > > 4:{ > > > 5:echo f line: $LINENO > > > 6:} > > > 7: > > > 8:f > > > 9:echo main line: $LINENO > > > 10:f > > > > > > > > > # /bin/sh ~/tmp/ln > > > f line: 2 > > > f line: 3 > > > main line: 9 > > > f line: 3 > > > > > > > > > # bash ~/tmp/ln > > > f line: 2 > > > f line: 5 > > > main line: 9 > > > f line: 5 > > > > Yes, I know. I think it is a bug in bash as SUSv3 states: > > > > "Set by the shell to a decimal number representing the current > > sequential line number (numbered starting with 1) within a script or > > function before it executes each command." > > Actually, the bash way seems more intuitive. And it does say "the current > sequentional line number within a ... function before it executes each > command" > > The "within a function" implies that this property goes inside of functions > instead of forcing all commands in a function to use the starting line of the > function which is what you are saying? I've started a thread about that on standards@. From lulf at stud.ntnu.no Sat May 31 14:36:52 2008 From: lulf at stud.ntnu.no (Ulf Lilleengen) Date: Sat May 31 14:36:57 2008 Subject: CFT: CVSMode for csup with MD5 check Message-ID: <20080531143655.GA14020@nobby.studby.ntnu.no> Hello, As a followup to my previous patch on csup, I've tried to do some fixes to RCS-files. However, instead of doing major workarounds in csup to handle files which does not behave correctly to RCS, I implemented MD5 check of RCS content. This means that the MD5 sum from cvsupd is checked against the actual RCS content (which is different from a normal MD5 check, because only the content is compared), and if it's not correct, a fixup of the file will be sent, thus making sure that the contents are correct. I hope some of you are able to test this. There are still a few issues with cvsmode: - Not correct entries in status file. - I get unnatural high memory usage. Patches here: http://people.freebsd.org/~lulf/patches/csup/csup_2008-05-31_CURRENT.diff and http://people.freebsd.org/~lulf/patches/csup/csup_2008-05-31_RELENG_7.diff -- Ulf Lilleengen From imp at bsdimp.com Sat May 31 16:04:15 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Sat May 31 16:04:19 2008 Subject: openbsd solution to mounted umass removal In-Reply-To: <48285372.9060103@icyb.net.ua> References: <48285372.9060103@icyb.net.ua> Message-ID: <20080531.100353.451147141.imp@bsdimp.com> In message: <48285372.9060103@icyb.net.ua> Andriy Gapon writes: : : I've just came back from a good 2 week vacation and catching up on news. : In release notes for OpenBSD 4.3 I see the following: : http://openbsd.org/43.html : Filesystems on USB devices are automatically dismounted if the device is : disconnected. : : Does anybody have more [technical] details on this? effectively, that's what -current winds up doing... except for the actual umount() call... Warner From jhb at freebsd.org Sat May 31 16:52:51 2008 From: jhb at freebsd.org (John Baldwin) Date: Sat May 31 16:52:55 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> Message-ID: <200805311249.53986.jhb@freebsd.org> On Saturday 31 May 2008 01:52:56 am Tz-Huan Huang wrote: > Hi, > > Our nfs server is running 7-stable/amd64 with 8G ram, the size of zfs > pool is 12T. We have set vm.kmem_size and vm.kmem_size_max to > 1.5G, but the kernel still panics by "kmem_map too small" often. > According to [1], the limitation is not only by the loader (is it fixed > now?) but also by the default layout of KVM. [2] points a way to increase > the KVM, but we get the similar linking error. > > Is there any standard way to modify the layout of KVM? For example, we > may want to set KVM to 6G and leave the 2G for user space usage. On i386 you only have 4GB of virtual address space period. For amd64 you can jack up KVM just fine AFAIK. The mcmodel=kernel stuff should only affect global variables (so .data and .bss) and not malloc'd stuff. Have you tried increasing the KVM size and seeing what happens? -- John Baldwin From tzhuan at csie.org Sat May 31 17:58:10 2008 From: tzhuan at csie.org (Tz-Huan Huang) Date: Sat May 31 17:58:14 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <200805311249.53986.jhb@freebsd.org> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> <200805311249.53986.jhb@freebsd.org> Message-ID: <6a7033710805311058t41ef1ffah97648975f842846d@mail.gmail.com> On Sun, Jun 1, 2008 at 12:49 AM, John Baldwin wrote: > On Saturday 31 May 2008 01:52:56 am Tz-Huan Huang wrote: >> Hi, >> >> Our nfs server is running 7-stable/amd64 with 8G ram, the size of zfs >> pool is 12T. We have set vm.kmem_size and vm.kmem_size_max to >> 1.5G, but the kernel still panics by "kmem_map too small" often. >> According to [1], the limitation is not only by the loader (is it fixed >> now?) but also by the default layout of KVM. [2] points a way to increase >> the KVM, but we get the similar linking error. >> >> Is there any standard way to modify the layout of KVM? For example, we >> may want to set KVM to 6G and leave the 2G for user space usage. > > On i386 you only have 4GB of virtual address space period. For amd64 you can > jack up KVM just fine AFAIK. The mcmodel=kernel stuff should only affect > global variables (so .data and .bss) and not malloc'd stuff. Have you tried > increasing the KVM size and seeing what happens? Actaully I have no idea how to increase the KVM size ... Should I change some knobs on kernel conifg or some codes? When setting the vm.kmem_size to almost or more than 2G, the kernel panics immediately after booting. When I change the KPDPI like [1], I have linking error. Thanks, Tz-Huan [1] http://lists.freebsd.org/pipermail/freebsd-current/2008-March/084325.html From kostikbel at gmail.com Sat May 31 18:18:10 2008 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat May 31 18:18:14 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <200805311249.53986.jhb@freebsd.org> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> <200805311249.53986.jhb@freebsd.org> Message-ID: <20080531175608.GH21317@deviant.kiev.zoral.com.ua> On Sat, May 31, 2008 at 12:49:53PM -0400, John Baldwin wrote: > On Saturday 31 May 2008 01:52:56 am Tz-Huan Huang wrote: > > Hi, > > > > Our nfs server is running 7-stable/amd64 with 8G ram, the size of zfs > > pool is 12T. We have set vm.kmem_size and vm.kmem_size_max to > > 1.5G, but the kernel still panics by "kmem_map too small" often. > > According to [1], the limitation is not only by the loader (is it fixed > > now?) but also by the default layout of KVM. [2] points a way to increase > > the KVM, but we get the similar linking error. > > > > Is there any standard way to modify the layout of KVM? For example, we > > may want to set KVM to 6G and leave the 2G for user space usage. > > On i386 you only have 4GB of virtual address space period. For amd64 you can > jack up KVM just fine AFAIK. The mcmodel=kernel stuff should only affect > global variables (so .data and .bss) and not malloc'd stuff. Have you tried > increasing the KVM size and seeing what happens? I believe the module memory is malloc'ed, isn't it ? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20080531/a0596f70/attachment.pgp From stef-list at memberwebs.com Sat May 31 18:28:18 2008 From: stef-list at memberwebs.com (Stef Walter) Date: Sat May 31 18:28:22 2008 Subject: RAID status issues with Intel MatrixRAID ataraid References: <20080530210347.5E1A6F1808E@mx.npubs.com> <200805302143.11920.jhb@freebsd.org> Message-ID: <20080531180759.EAEBAF180A0@mx.npubs.com> John Baldwin wrote: >> Physical Disks: >> Port Drive Model Serial # Size Type/Status(Vol ID) >> 0 WDC WD5000ABYS-0 WD-WCAPW5637184 465.8GB Member Disk(0) >> 1 ST3500320NS 5QM09E6F 465.8GB Error Occurred(0) >> 2 WDC WD5000ABYS-0 WD-WCAPW5548822 465.8GB Member Disk(1) >> 3 ST3500320NS 5QM0991D 465.8GB Member Disk(1) >> Press to enter Configuration Utility........... >> >> >> >> RELEVANT SNIPPET FROM DMESG: >> >> ar0: 476937MB status: READY >> ar0: disk0 READY (master) using ad4 at ata2-master >> ar0: disk1 READY (mirror) using ad6 at ata3-master > > Err, you have two RAID volumes, one degraded, and one ok. ar0 is apparently > the one that is ok (drives 2 and 3). What is more odd is that it isn't > seeing a RAID config at all for ad0 and ad2 (drives 0 and 1). Nope, ar1 (with ad8 and ad10) is the second RAID, both are marked as READY even though one is degraded in the BIOS. ar0: 476937MB status: READY ar0: disk0 READY (master) using ad4 at ata2-master ar0: disk1 READY (mirror) using ad6 at ata3-master ar1: 476937MB status: READY ar1: disk0 READY (master) using ad8 at ata4-master ar1: disk1 READY (mirror) using ad10 at ata5-master In any case, this is water under the bridge for me now. Cheers, Stef Walter From koitsu at FreeBSD.org Sat May 31 18:47:33 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Sat May 31 18:47:36 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <200805311249.53986.jhb@freebsd.org> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> <200805311249.53986.jhb@freebsd.org> Message-ID: <20080531184733.GA8470@eos.sc1.parodius.com> On Sat, May 31, 2008 at 12:49:53PM -0400, John Baldwin wrote: > On Saturday 31 May 2008 01:52:56 am Tz-Huan Huang wrote: > > Hi, > > > > Our nfs server is running 7-stable/amd64 with 8G ram, the size of zfs > > pool is 12T. We have set vm.kmem_size and vm.kmem_size_max to > > 1.5G, but the kernel still panics by "kmem_map too small" often. > > According to [1], the limitation is not only by the loader (is it fixed > > now?) but also by the default layout of KVM. [2] points a way to increase > > the KVM, but we get the similar linking error. > > > > Is there any standard way to modify the layout of KVM? For example, we > > may want to set KVM to 6G and leave the 2G for user space usage. > > On i386 you only have 4GB of virtual address space period. For amd64 you can > jack up KVM just fine AFAIK. The mcmodel=kernel stuff should only affect > global variables (so .data and .bss) and not malloc'd stuff. Have you tried > increasing the KVM size and seeing what happens? On amd64 there's still a 2GB limit, re: vm.kmem_size. Increasing it to or past 2048M results in the message "kmem_suballoc(): bad status return of 3". http://lists.freebsd.org/pipermail/freebsd-stable/2008-May/042764.html -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From tzhuan at csie.org Sat May 31 19:01:57 2008 From: tzhuan at csie.org (Tz-Huan Huang) Date: Sat May 31 19:02:01 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <319cceca0805311154w5f705b2cp771392cd86ba888@mail.gmail.com> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> <319cceca0805311154w5f705b2cp771392cd86ba888@mail.gmail.com> Message-ID: <6a7033710805311201t4085edcpc2f1ac9fa554c000@mail.gmail.com> On Sun, Jun 1, 2008 at 2:54 AM, Maslan wrote: > Hi, > > is PAE enabled in your kernel config ? Our server runs on amd64. I might be wrong, but I think PAE is for i386 only, right? Thanks, Tz-Huan From maslanbsd at gmail.com Sat May 31 19:18:15 2008 From: maslanbsd at gmail.com (Maslan) Date: Sat May 31 19:18:19 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> Message-ID: <319cceca0805311154w5f705b2cp771392cd86ba888@mail.gmail.com> Hi, is PAE enabled in your kernel config ? Thanks On Sat, May 31, 2008 at 5:52 AM, Tz-Huan Huang wrote: > Hi, > > Our nfs server is running 7-stable/amd64 with 8G ram, the size of zfs > pool is 12T. We have set vm.kmem_size and vm.kmem_size_max to > 1.5G, but the kernel still panics by "kmem_map too small" often. > According to [1], the limitation is not only by the loader (is it fixed now?) > but also by the default layout of KVM. [2] points a way to increase the > KVM, but we get the similar linking error. > > Is there any standard way to modify the layout of KVM? For example, we > may want to set KVM to 6G and leave the 2G for user space usage. > > Thanks, > Tz-Huan > > [1] http://lists.freebsd.org/pipermail/freebsd-current/2007-October/077964.html > [2] http://lists.freebsd.org/pipermail/freebsd-current/2008-March/084325.html > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- System Programmer -- I'm Searching For Perfection, So Even If U Need Portability U've To Use Assembly ;-) -- http://libosdk.berlios.de From maslanbsd at gmail.com Sat May 31 20:03:52 2008 From: maslanbsd at gmail.com (Maslan) Date: Sat May 31 20:03:55 2008 Subject: Is there any way to increase the KVM? In-Reply-To: <6a7033710805311201t4085edcpc2f1ac9fa554c000@mail.gmail.com> References: <6a7033710805302252v43a7b240x66ca3f5e3dd5fda4@mail.gmail.com> <319cceca0805311154w5f705b2cp771392cd86ba888@mail.gmail.com> <6a7033710805311201t4085edcpc2f1ac9fa554c000@mail.gmail.com> Message-ID: <319cceca0805311303o61729e0dta615c442b6bd03ce@mail.gmail.com> Your are right PAE is for i386, i mean try running i386 freebsd with PAE enabled rather than amd64. PAE will let you access 64GB which is far than you got. On Sat, May 31, 2008 at 7:01 PM, Tz-Huan Huang wrote: > On Sun, Jun 1, 2008 at 2:54 AM, Maslan wrote: >> Hi, >> >> is PAE enabled in your kernel config ? > > Our server runs on amd64. I might be wrong, but I think PAE is for i386 > only, right? > > Thanks, > Tz-Huan > -- System Programmer -- I'm Searching For Perfection, So Even If U Need Portability U've To Use Assembly ;-) -- http://libosdk.berlios.de From pgollucci at p6m7g8.com Sat May 31 21:15:50 2008 From: pgollucci at p6m7g8.com (Philip M. Gollucci) Date: Sat May 31 21:15:55 2008 Subject: CFT: adding configuration file support to pkg_install In-Reply-To: References: Message-ID: <4841BDA9.5090007@p6m7g8.com> Florent Thoumie wrote: > This adds support for /etc/pkg.conf configuration file. > Also, this adds support for naive multi-site package fetching. > > Any comment welcome (and appreciated). > > Patch is here: http://people.freebsd.org/~flz/local/ports/pkg-install-config.diff > Tarball is here: > http://people.freebsd.org/~flz/local/ports/pkg-install-0a553aac.tar.bz2 Hi flz, I don't quite get what the end goal is. It looks like /etc/pkg.conf is duplicating a lot of things already in /usr/ports/Mk/bsd.port.mk. Would not it be better to just have the pkg_install tools read that file instead ? I probably missed all the back story here, so feel free to put me in my place. The multi-site package fetching is definitely something I'm interested it, but I also figured it would just iterate over the values in PACKAGESITE PACKAGESITE=ftp://foo/stdpath/base/Latest/ ftp://foo/stdpath/www/Latest where base would have things like sudo, bash, vim, etc... and could be used on multiple computers. www would have things like apache22 mod_X and would be used on 'www' class machines. -- ------------------------------------------------------------------------ Philip M. Gollucci (philip@ridecharge.com) o:703.549.2050x206 Senior System Admin - Riderway, Inc. http://riderway.com / http://ridecharge.com 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching.