From imp at bsdimp.com Fri May 1 00:38:48 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri May 1 00:38:54 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090430233648.GA95360@keira.kiwi-computer.com> References: <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <20090430233648.GA95360@keira.kiwi-computer.com> Message-ID: <20090430.183727.803597558.imp@bsdimp.com> In message: <20090430233648.GA95360@keira.kiwi-computer.com> "Rick C. Petty" writes: : On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: : > : > This is the biggest one, and I think it may be too soon. Also, we : > need to be careful on the initialization side of things because we : > currently have a lot of code that looks like: : > : > : > struct foo *fp; : > struct bar *bp; : > : > fp = get_foo(); : > if (!fp) return; : > bp = fp->bp; : > : > this can't easily be translated to the more natural: : > : > struct foo *fp = get_foo(); : > struct bar *bp = fp->bp; : > : > since really you'd want to write: : > : > struct foo *fp = get_foo(); : > if (!fp) return; : > struct bar *bp = fp->bp; : > : > which isn't legal in 'C'. : : I thought we were talking about C99, in which case this is perfectly legal. : I certainly use it all the time in my C99 code. Hmmm, looks like that was added. That's ugly as C++... : And I thought this was the point of this discussion, to be able to declare : variables when you first use them. That's one of the proposed changes, which I think is a mistake and would cause the most code churn. And it isn't one of the items that's being discussed: only moving variables into inner scopes is on the table... Warner From christoph.mallon at gmx.de Fri May 1 05:49:42 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 05:49:50 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090430.090226.1569754707.imp@bsdimp.com> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> Message-ID: <49FA8D73.6040207@gmx.de> M. Warner Losh schrieb: > In message: <20090428114754.GB89235@server.vk2pj.dyndns.org> > Peter Jeremy writes: > : >Maybe using all of these changes is a bit too big change at once, but > : >I'd like some of them applied to style(9). So, please consider the > : >proposed changes individually and not a as a all-or-nothing package. > : > : One area you do not address is code consistency. Currently, the > : FreeBSD kernel (and, to a lesser extent, userland) are mostly style(9) > : compliant - ie the entire codebase is mostly written in the same > : style. Whilst you may not like it (and I don't know that anyone > : completely agrees with everything in style(9)), it does mean that > : the code is consistent. > : > : Changing style(9) in a way that is not consistent with current code > : means that either existing code must be brought into line with the > : new standard - which incurs a one-off cost - or the code base becomes > : split into "old" and "new" style - incurring an ongoing maintenance > : cost as maintainers switch between styles. Both approaches incur a > : risk of introducing new bugs. > : > : Note that I'm not saying that style(9) can never be changed, I'm just > : saying that any change _will_ incur a cost and the cost as well as > : the potential benefits need to be considered. > > This is my biggest worry about changing style(9) quickly. We don't > want needless churn in the tree for just style changes since it makes > it harder to track bug fixes into the past. I'm not saying that all the code has to be changed at once, but no new code of the "old" style should be added. E.g. you should never use K&R declarations when adding a new function. And if you are going to make a big change somewhere, then it is sensible to use the "new" style. > : [Reduce declaration scope as far as possible, initialise variables where > : they are declared, sort by name] > : > : Whilst my personal preference is for the style suggested by Christoph > : (and I generally agree with the points he makes), this is also the > : change that incurs the biggest stylistic change. This is not a change > : that can be practically retrofitted to existing code and therefore its > : implementation would result in a mixture of code styles - increasing > : the risk of bugs due to confusion as to which style was being used. I > : am not yet sure whether the benefits outweigh the costs, > > This is the biggest one, and I think it may be too soon. Also, we This is the reason, why I explicitly mentioned, that the proposed changes should be examined independently. > need to be careful on the initialization side of things because we > currently have a lot of code that looks like: > > > struct foo *fp; > struct bar *bp; > > fp = get_foo(); > if (!fp) return; > bp = fp->bp; > > this can't easily be translated to the more natural: > > struct foo *fp = get_foo(); > struct bar *bp = fp->bp; > > since really you'd want to write: > > struct foo *fp = get_foo(); > if (!fp) return; > struct bar *bp = fp->bp; > > which isn't legal in 'C'. However, we have enough where this isn't You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 (E) ?6.8.2:1. In short: you can mix statements and declarations. > the case that it should be allowed. We should separate the > initialization part of this from the scope part of this too. The > initialization part is already leaking into the code, while instances > of the scope restriction is rarer. Sorry, I do not understand these sentences. > : [Don't parenthesize return values] > : >Removed, because it does not improve maintainability in any way. > : > : This change could be made and tested mechanically. But there is > : no justification for making it - stating that the existing rule > : is no better than the proposed rule is no reason to change. > > I'm fine with this. > > : [No K&R declarations] > : >Removed, because there is no need to use them anymore. > : > : Whilst this is a change that could be performed mechanically, there > : are some gotchas relating to type promotion (as you point out). > : The kernel already contains a mixture of ANSI & K&R declarations and > : style(9) recommends using ANSI. IMHO, there is no need to make this > : change until all K&R code is removed from FreeBSD. > > I'm fine with this change. > > : [ Don't insert an empty line if the function has no local variables.] > : > : This change could be made and tested mechanically. IMHO, this change > : has neglible risk and could be safely implemented. > > I'm fine with this change. Would you commit these three proposals? > : >> +.Sh LOCAL VARIABLES > : > : >Last, but definitely not least, I added this paragraph about the use of > : >local variables. This is to clarify, how today's compilers handle > : >unaliased local variables. > : > : Every version of gcc that FreeBSD has ever used would do this for > : primitive types when optimisation was enabled. This approach can > : become expensive in stack (and this is an issue for kernel code) when > : using non-primitive types or when optimisation is not enabled (though > : I'm not sure if this is supported). Assuming that gcc (and icc and > : clang) behaves as stated in all supported optimisation modes, this > : change would appear to be quite safe to make. > > Agreed, in general. We don't want to optimize our code style based on > what one compiler does, perhaps on x86. I'm not sure whether I understand this - in particular the last three words. Christoph From christoph.mallon at gmx.de Fri May 1 05:54:22 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 05:54:28 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090430.183727.803597558.imp@bsdimp.com> References: <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <20090430233648.GA95360@keira.kiwi-computer.com> <20090430.183727.803597558.imp@bsdimp.com> Message-ID: <49FA8E88.1040905@gmx.de> M. Warner Losh schrieb: > In message: <20090430233648.GA95360@keira.kiwi-computer.com> > "Rick C. Petty" writes: > : On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: > : > > : > This is the biggest one, and I think it may be too soon. Also, we > : > need to be careful on the initialization side of things because we > : > currently have a lot of code that looks like: > : > > : > > : > struct foo *fp; > : > struct bar *bp; > : > > : > fp = get_foo(); > : > if (!fp) return; > : > bp = fp->bp; > : > > : > this can't easily be translated to the more natural: > : > > : > struct foo *fp = get_foo(); > : > struct bar *bp = fp->bp; > : > > : > since really you'd want to write: > : > > : > struct foo *fp = get_foo(); > : > if (!fp) return; > : > struct bar *bp = fp->bp; > : > > : > which isn't legal in 'C'. > : > : I thought we were talking about C99, in which case this is perfectly legal. > : I certainly use it all the time in my C99 code. > > Hmmm, looks like that was added. That's ugly as C++... I do not think, this is ugly. On the contrary, it aids maintainability, because it reduces the scope of variables. Also quite some variables are only initialised and not changed afterwards, so it's nice to have the declaration and the only assignment in a single place. IMO this is a quite natural style, because you don't have anything in the code, before it is needed: Get the first pointer; if something is wrong, bail out; get the second pointer - the second pointer does not (textually) exist before it is needed. > : And I thought this was the point of this discussion, to be able to declare > : variables when you first use them. > > That's one of the proposed changes, which I think is a mistake and > would cause the most code churn. And it isn't one of the items that's > being discussed: only moving variables into inner scopes is on the > table... No, this is not what I intended. The idea is to limit the scope of local variables as much as is sensible. Maybe I should have been more explicit. On the other hand, I also did not mention that it is just about moving to the start of inner block statements. Christoph From christoph.mallon at gmx.de Fri May 1 05:57:04 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 06:07:52 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090428121327.GA41168@freebsd.org> References: <49F4070C.2000108@gmx.de> <20090428121327.GA41168@freebsd.org> Message-ID: <49FA8F2D.5090708@gmx.de> Roman Divacky schrieb: > I like the part about using as many variables as possible because > of documentation and performance enhancements. I tend to like > the other changes as well.. This is not about using as many variables as possible. The goal is to use as many variables as you have logically distinct entities in the function. I suppose, this is what you mean, but I want to clarify this point. Christoph From julian at elischer.org Fri May 1 08:30:27 2009 From: julian at elischer.org (Julian Elischer) Date: Fri May 1 08:30:34 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FA8D73.6040207@gmx.de> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> Message-ID: <49FAB322.9030103@elischer.org> As an old-fart I have found many cases where what I thought was a silly style rule, turned out to save my work in some way. Christoph Mallon wrote: >> >> >> struct foo *fp; >> struct bar *bp; >> >> fp = get_foo(); >> if (!fp) return; >> bp = fp->bp; >> >> this can't easily be translated to the more natural: >> >> struct foo *fp = get_foo(); >> struct bar *bp = fp->bp; Well more natural for you, but not necessarily for everyone, and NOT the same as what is there now, as you noticed. >> >> since really you'd want to write: >> >> struct foo *fp = get_foo(); >> if (!fp) return; >> struct bar *bp = fp->bp; >> >> which isn't legal in 'C'. However, we have enough where this isn't > > You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 (E) > ?6.8.2:1. In short: you can mix statements and declarations. now, but not all C compilers are C99 and a lot of FreeBSD code is taken and run in other situations. There is FreeBSD code in all sorts of environments, not all of which have new compilers. Besides which I'd vote against that just on principle that it breaks current style too much, and it makes it hard to find where things are declared. > >> : [Don't parenthesize return values] >> : >Removed, because it does not improve maintainability in any way. >> : : This change could be made and tested mechanically. But there is >> : no justification for making it - stating that the existing rule >> : is no better than the proposed rule is no reason to change. >> >> I'm fine with this. I find return values being parenthesised is easier to for me to read. I'd vote against his too >> >> : [No K&R declarations] >> : >Removed, because there is no need to use them anymore. >> : : Whilst this is a change that could be performed mechanically, there >> : are some gotchas relating to type promotion (as you point out). >> : The kernel already contains a mixture of ANSI & K&R declarations and >> : style(9) recommends using ANSI. IMHO, there is no need to make this >> : change until all K&R code is removed from FreeBSD. This is not new. It's already policy to remove them whenever you are working in the area. it improves code error checking.. >> >> : [ Don't insert an empty line if the function has no local variables.] >> : : This change could be made and tested mechanically. IMHO, this change >> : has neglible risk and could be safely implemented. >> >> I'm fine with this change. > > Would you commit these three proposals? > >> : >> +.Sh LOCAL VARIABLES >> : : >Last, but definitely not least, I added this paragraph about the >> use of : >local variables. This is to clarify, how today's compilers >> handle : >unaliased local variables. >> : : Every version of gcc that FreeBSD has ever used would do this for >> : primitive types when optimisation was enabled. This approach can >> : become expensive in stack (and this is an issue for kernel code) when >> : using non-primitive types or when optimisation is not enabled (though >> : I'm not sure if this is supported). Assuming that gcc (and icc and >> : clang) behaves as stated in all supported optimisation modes, this >> : change would appear to be quite safe to make. >> >> Agreed, in general. We don't want to optimize our code style based on >> what one compiler does, perhaps on x86. it does however make it harder to find stuff in gdb As a general rule, one of the things that is good about BSD code is that it all looks about the same. This makes it easier to read, once you have got used to it. changing it for no reason except "I felt like it" will and I think should, meet stiff opposition. From julian at elischer.org Fri May 1 08:33:29 2009 From: julian at elischer.org (Julian Elischer) Date: Fri May 1 08:33:40 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FA8E88.1040905@gmx.de> References: <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <20090430233648.GA95360@keira.kiwi-computer.com> <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> Message-ID: <49FAB3D8.90607@elischer.org> Christoph Mallon wrote: > M. Warner Losh schrieb: >> In message: <20090430233648.GA95360@keira.kiwi-computer.com> >> "Rick C. Petty" writes: >> : On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: >> : > : > This is the biggest one, and I think it may be too soon. >> Also, we >> : > need to be careful on the initialization side of things because we >> : > currently have a lot of code that looks like: >> : > : > : > struct foo *fp; >> : > struct bar *bp; >> : > : > fp = get_foo(); >> : > if (!fp) return; >> : > bp = fp->bp; >> : > : > this can't easily be translated to the more natural: >> : > : > struct foo *fp = get_foo(); >> : > struct bar *bp = fp->bp; >> : > : > since really you'd want to write: >> : > : > struct foo *fp = get_foo(); >> : > if (!fp) return; >> : > struct bar *bp = fp->bp; >> : > : > which isn't legal in 'C'. >> : : I thought we were talking about C99, in which case this is >> perfectly legal. >> : I certainly use it all the time in my C99 code. >> >> Hmmm, looks like that was added. That's ugly as C++... > > I do not think, this is ugly. On the contrary, it aids maintainability, > because it reduces the scope of variables. Also quite some variables are > only initialised and not changed afterwards, so it's nice to have the > declaration and the only assignment in a single place. IMO this is a > quite natural style, because you don't have anything in the code, before > it is needed: Get the first pointer; if something is wrong, bail out; > get the second pointer - the second pointer does not (textually) exist > before it is needed. > >> : And I thought this was the point of this discussion, to be able to >> declare >> : variables when you first use them. >> >> That's one of the proposed changes, which I think is a mistake and >> would cause the most code churn. And it isn't one of the items that's >> being discussed: only moving variables into inner scopes is on the >> table... > > No, this is not what I intended. The idea is to limit the scope of local > variables as much as is sensible. Maybe I should have been more > explicit. On the other hand, I also did not mention that it is just > about moving to the start of inner block statements. I can see moving declarations to an inner scope {} in some cases but I think allowing us to declare them mixed in with the code, (even though some compilers allow it) will be a mistake. I think this was done to allow macros to declare vars they needed. I'd hate to see it in our code.. > > Christoph > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From bruce at cran.org.uk Fri May 1 08:41:40 2009 From: bruce at cran.org.uk (Bruce Cran) Date: Fri May 1 08:41:47 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FAB322.9030103@elischer.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> Message-ID: <20090501094134.77b6de04@gluon.draftnet> On Fri, 01 May 2009 01:30:26 -0700 Julian Elischer wrote: > Christoph Mallon wrote: > >> > >> since really you'd want to write: > >> > >> struct foo *fp = get_foo(); > >> if (!fp) return; > >> struct bar *bp = fp->bp; > >> > >> which isn't legal in 'C'. However, we have enough where this isn't > > > > You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 > > (E) ?6.8.2:1. In short: you can mix statements and declarations. > > now, but not all C compilers are C99 and a lot of FreeBSD code > is taken and run in other situations. There is FreeBSD code > in all sorts of environments, not all of which have new compilers. > Doesn't FreeBSD already use C99 features such as stdint and named initializers? I don't think sys/cam/scsi/scsi_ses.c would compile with a C89 compiler for example. -- Bruce Cran From christoph.mallon at gmx.de Fri May 1 11:30:20 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 11:30:28 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FAB322.9030103@elischer.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> Message-ID: <49FADD47.7060507@gmx.de> Julian Elischer schrieb: > As an old-fart I have found many cases where what I thought was > a silly style rule, turned out to save my work in some way. > > Christoph Mallon wrote: > > >>> >>> >>> struct foo *fp; >>> struct bar *bp; >>> >>> fp = get_foo(); >>> if (!fp) return; >>> bp = fp->bp; >>> >>> this can't easily be translated to the more natural: >>> >>> struct foo *fp = get_foo(); >>> struct bar *bp = fp->bp; > > Well more natural for you, but not necessarily for everyone, > and NOT the same as what is there now, as you noticed. You partially misquoted this (only my name is seen above). I did not write this, Warner did. But I agree with Warner, that it is more natural. Also Warner had the wrong impression that declarations and statements cannot be mixed. But this is not true and so you can put the if inbetween, which is seen below. >>> >>> since really you'd want to write: >>> >>> struct foo *fp = get_foo(); >>> if (!fp) return; >>> struct bar *bp = fp->bp; >>> >>> which isn't legal in 'C'. However, we have enough where this isn't >> >> You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 (E) >> ?6.8.2:1. In short: you can mix statements and declarations. > > now, but not all C compilers are C99 and a lot of FreeBSD code > is taken and run in other situations. There is FreeBSD code > in all sorts of environments, not all of which have new compilers. There are already several C99 features in use (like designated initialisers). So this is obviously not a problem. > Besides which I'd vote against that just on principle that it breaks > current style too much, and it makes it hard to find where things are > declared. "Changes cannot be done because something would be different" is - of course - the killer pseudo-argument against any kind of change ever. ): >>> >>> : [No K&R declarations] >>> : >Removed, because there is no need to use them anymore. >>> : : Whilst this is a change that could be performed mechanically, there >>> : are some gotchas relating to type promotion (as you point out). >>> : The kernel already contains a mixture of ANSI & K&R declarations and >>> : style(9) recommends using ANSI. IMHO, there is no need to make this >>> : change until all K&R code is removed from FreeBSD. > > This is not new. It's already policy to remove them whenever you are > working in the area. it improves code error checking.. So clearly style(9) should reflect this by removing the paragraph. >>> >>> : [ Don't insert an empty line if the function has no local variables.] >>> : : This change could be made and tested mechanically. IMHO, this >>> change >>> : has neglible risk and could be safely implemented. >>> >>> I'm fine with this change. >> >> Would you commit these three proposals? >> >>> : >> +.Sh LOCAL VARIABLES >>> : : >Last, but definitely not least, I added this paragraph about the >>> use of : >local variables. This is to clarify, how today's compilers >>> handle : >unaliased local variables. >>> : : Every version of gcc that FreeBSD has ever used would do this for >>> : primitive types when optimisation was enabled. This approach can >>> : become expensive in stack (and this is an issue for kernel code) when >>> : using non-primitive types or when optimisation is not enabled (though >>> : I'm not sure if this is supported). Assuming that gcc (and icc and >>> : clang) behaves as stated in all supported optimisation modes, this >>> : change would appear to be quite safe to make. >>> >>> Agreed, in general. We don't want to optimize our code style based on >>> what one compiler does, perhaps on x86. > > it does however make it harder to find stuff in gdb Can you elaborate on this? Because I cannot follow what you mean by this. "p i" will still print the value of "i". > As a general rule, one of the things that is good about BSD code is that > it all looks about the same. This makes it easier to read, once you have > got used to it. Again this is the same "any change cannot happen, because it changes something which not already is this way" circular argument. ): If you don't dare to take the first step at some point, nothing can ever change and no improvement can take place ever. It's sad that changes (like designated initialisers) have to creep in via the backdoor and then at some point you can say "See? The world did not end.", because for any proposed change always somebody objects with "this is bad, because it is different than before". ): > changing it for no reason except "I felt like it" will and I think > should, meet stiff opposition. I want to make absolutely clear, that I in no way said or implied "I felt like it". I explained the reason for any of the changes I propsed and none of the reasons is even close to "I felt like it". Christoph From christoph.mallon at gmx.de Fri May 1 11:37:26 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 11:37:33 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501112239.GA23199@alchemy.franken.de> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> Message-ID: <49FADEF3.5010106@gmx.de> Marius Strobl schrieb: > On Sun, Apr 26, 2009 at 09:02:36AM +0200, Christoph Mallon wrote: >> return with parentheses: >> Removed, because it does not improve maintainability in any way. There >> is no source for confusion here, so the rule even contradicts the rule, >> which states not to use redundant parentheses. Maybe, decades ago it was >> just a workaround for a broken compiler, which does not exist anymore. > > FYI, the idea behind this rule is said to be to able to use > a macro return(), f.e. for debugging you then can do: > #define return(x) do { \ > printf("returning from %s with %d\n", __func__, (x)); \ > return (x); \ > } while (0) > > Given the this is a nifty feature and parentheses around the > return value don't hurt maintainability in any way IMO this > rule should stay. This is mentioned nowhere in style(9) (in general it is lacking reasons why something is some way or the other). Also I consider this as gross abuse: Macro names shall be in all uppercase, so it is clear that there is a macro at work. Therefore "return" is not a candidate. So this would violate yet another rule in style(9) (the original return already violates the no-redundant parentheses rule). Also I would not mention __func__: there were objections against using it in the past (though I, logically, prefer its use). Christoph From marius at alchemy.franken.de Fri May 1 11:45:01 2009 From: marius at alchemy.franken.de (Marius Strobl) Date: Fri May 1 11:45:08 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49F4070C.2000108@gmx.de> References: <49F4070C.2000108@gmx.de> Message-ID: <20090501112239.GA23199@alchemy.franken.de> On Sun, Apr 26, 2009 at 09:02:36AM +0200, Christoph Mallon wrote: > > return with parentheses: > Removed, because it does not improve maintainability in any way. There > is no source for confusion here, so the rule even contradicts the rule, > which states not to use redundant parentheses. Maybe, decades ago it was > just a workaround for a broken compiler, which does not exist anymore. FYI, the idea behind this rule is said to be to able to use a macro return(), f.e. for debugging you then can do: #define return(x) do { \ printf("returning from %s with %d\n", __func__, (x)); \ return (x); \ } while (0) Given the this is a nifty feature and parentheses around the return value don't hurt maintainability in any way IMO this rule should stay. Marius From christoph.mallon at gmx.de Fri May 1 11:46:50 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 11:46:56 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FAB3D8.90607@elischer.org> References: <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <20090430233648.GA95360@keira.kiwi-computer.com> <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> <49FAB3D8.90607@elischer.org> Message-ID: <49FAE126.5050903@gmx.de> Julian Elischer schrieb: > Christoph Mallon wrote: >> No, this is not what I intended. The idea is to limit the scope of >> local variables as much as is sensible. Maybe I should have been more >> explicit. On the other hand, I also did not mention that it is just >> about moving to the start of inner block statements. > > I can see moving declarations to an inner scope {} in some cases but > I think allowing us to declare them mixed in with the code, > (even though some compilers allow it) will be a mistake. Some compilers? According to my information every compiler, which is even remotely relevant, supports it. Even PCC claims it does! The only compiler, which I am aware of and which has a relevant distribution, which doesn't support it, is MSVC - but I highly doubt, that it is relevant in any way for FreeBSD. > I think this was done to allow macros to declare vars they needed. > I'd hate to see it in our code.. You are accusing me for proposing changes because "I felt like it", but all you give is "I'd hate [...] it" and "[it] will be a mistake" without any further justification. It seems to me, that you're applying double standards. /: Christoph From sobomax at FreeBSD.org Fri May 1 11:56:58 2009 From: sobomax at FreeBSD.org (Maxim Sobolev) Date: Fri May 1 11:57:05 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FA8F2D.5090708@gmx.de> References: <49F4070C.2000108@gmx.de> <20090428121327.GA41168@freebsd.org> <49FA8F2D.5090708@gmx.de> Message-ID: <49FAE36D.9030109@FreeBSD.org> Christoph Mallon wrote: > Roman Divacky schrieb: >> I like the part about using as many variables as possible because >> of documentation and performance enhancements. I tend to like >> the other changes as well.. > > This is not about using as many variables as possible. The goal is to > use as many variables as you have logically distinct entities in the > function. I suppose, this is what you mean, but I want to clarify this > point. Why don't just put "logically distinct entities" into separate functions on their own? It's a good indicator that the re-factoring is due when you reach this point. -Maxim From christoph.mallon at gmx.de Fri May 1 12:02:55 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 12:03:10 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090428114754.GB89235@server.vk2pj.dyndns.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> Message-ID: <49FAE4EA.1010205@gmx.de> Peter Jeremy schrieb: > On 2009-Apr-26 09:02:36 +0200, Christoph Mallon wrote: >> as some of you may have noticed, several years ago a new millenium >> started and a decade ago there was a new C standard. > > Your implication that FreeBSD is therefore a decade behind the times > is unfair. Whilst the C99 standard was published a decade ago, > compilers implementing that standard are still not ubiquitous. > >> HEAD recently >> switched to C99 as default (actually gnu99, but that's rather close). > > Note that gcc 4.2 (the FreeBSD base compiler) states that it is not > C99 compliant. It's good enough. Only one proposed change (mixing declarations and statements) actually needs C99. GCC supports this for many years. >> Maybe using all of these changes is a bit too big change at once, but >> I'd like some of them applied to style(9). So, please consider the >> proposed changes individually and not a as a all-or-nothing package. > > One area you do not address is code consistency. Currently, the > FreeBSD kernel (and, to a lesser extent, userland) are mostly style(9) > compliant - ie the entire codebase is mostly written in the same > style. Whilst you may not like it (and I don't know that anyone > completely agrees with everything in style(9)), it does mean that > the code is consistent. > > Changing style(9) in a way that is not consistent with current code > means that either existing code must be brought into line with the > new standard - which incurs a one-off cost - or the code base becomes > split into "old" and "new" style - incurring an ongoing maintenance > cost as maintainers switch between styles. Both approaches incur a > risk of introducing new bugs. > > Note that I'm not saying that style(9) can never be changed, I'm just > saying that any change _will_ incur a cost and the cost as well as > the potential benefits need to be considered. I see no problem with removing/changing the existing rules to make sure new code is not forced to use them just because it always has been done this way. Nobody will change all the source code at once, because its insane to check and change millions of lines of code at once. Considering this, you pretty much said, that style(9) cannot be changed. I see no problem in improving the rules and then gradually changing the code when it is convenient. Especially new code will profit from the changes, because it is not condemned to use the old rules for eternity. There are several C99 features used already, e.g. designated initializers: bla bli = { .blub = "foo", .arr[0] = 42 }; Do you suggest that this should not be used, because it is inconsistent with all the other existing compound initialisations? > [Reduce declaration scope as far as possible, initialise variables where > they are declared, sort by name] > > Whilst my personal preference is for the style suggested by Christoph > (and I generally agree with the points he makes), this is also the > change that incurs the biggest stylistic change. This is not a change > that can be practically retrofitted to existing code and therefore its > implementation would result in a mixture of code styles - increasing > the risk of bugs due to confusion as to which style was being used. I > am not yet sure whether the benefits outweigh the costs, There is no need to change all the existing code at once. But new code should be enabled to use the improved style. > [Don't parenthesize return values] >> Removed, because it does not improve maintainability in any way. > > This change could be made and tested mechanically. But there is > no justification for making it - stating that the existing rule > is no better than the proposed rule is no reason to change. Just remove the rule. There's no need to add more redundant parentheses. Again: There is no need to change all existing code at once, but the rule is removed so that not anymore redundant parentheses are added. > [No K&R declarations] >> Removed, because there is no need to use them anymore. > > Whilst this is a change that could be performed mechanically, there > are some gotchas relating to type promotion (as you point out). > The kernel already contains a mixture of ANSI & K&R declarations and > style(9) recommends using ANSI. IMHO, there is no need to make this > change until all K&R code is removed from FreeBSD. Again: No need to change all the code at once, but the rule is removed, so not any more old style declarations are added. The current attempts to compile FreeBSD using LLVM/clang revealed several existing bugs in functions with old style declarations. > [ Don't insert an empty line if the function has no local variables.] > > This change could be made and tested mechanically. IMHO, this change > has neglible risk and could be safely implemented. Again: No need for immediate global change, but just do not add anymore of those. There are already quite some functions, which do not have the unnecessary empty line. >>> +.Sh LOCAL VARIABLES > >> Last, but definitely not least, I added this paragraph about the use of >> local variables. This is to clarify, how today's compilers handle >> unaliased local variables. > > Every version of gcc that FreeBSD has ever used would do this for > primitive types when optimisation was enabled. This approach can > become expensive in stack (and this is an issue for kernel code) when > using non-primitive types or when optimisation is not enabled (though > I'm not sure if this is supported). Assuming that gcc (and icc and > clang) behaves as stated in all supported optimisation modes, this > change would appear to be quite safe to make. Most local variables are scalars (pointers, ints, ...). A struct or array as local variable is rare and then usually used in one context. So IMO this is fine. Even considereing the extremly rare case of multiple non-scalar declarations in a function, then a compiler can coalesce them if their life ranges are disjoint. It is easier to do so for a compiler to do so, if they are declared with smaller life ranges, example: struct Foo { int x[16]; }; struct Bar { int* y[16]; }; void f(struct Foo*); void g(struct Bar*); void e(int x) { struct Foo a; struct Bar b; if (x) { f(&a); } else { g(&b); } } Stack usage: subl $140, %esp moving the declarations into the branches: subl $76, %esp So, apart from improved readability, it also can lead to better code. But I consider the latter way less important the benefits observed by a reader of the code. Christoph From kabaev at gmail.com Fri May 1 12:05:29 2009 From: kabaev at gmail.com (Alexander Kabaev) Date: Fri May 1 12:05:38 2009 Subject: NetBSD 5.0 statistics In-Reply-To: References: <6101e8c40904300750i3e86fc0cnef09b0d4533627f7@mail.gmail.com> Message-ID: <20090501073224.518cb833@kan.dnsalias.net> On Thu, 30 Apr 2009 20:32:00 +0100 (BST) Robert Watson wrote: > > On Thu, 30 Apr 2009, Oliver Pinter wrote: > > > Is the FreeBSD's FS management so slow? > > > > http://www.netbsd.org/~ad/50/img15.html > > > > Or so big is the difference between the two cpu scheduler? > > Also, there's a known and serious performance regression in CAM > relating to tgged queueing, and the generic disk sort routine, > introduced 7.1, which will be fixed in 7.2. I can't speak more > generally to the benchmarks -- we'll need to run them in a controlled > environment and see if we can reproduce the results. > Well, there is also r185801 of vfs_cache.c which all but destroys the usefulness of negative name caching and surely can account for a large percentage of reported performance drop in 7.1, especially with build.sh benchmarks. The bug was fixed in stable/7 soon after 7.1 was released. -- Alexander Kabaev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 188 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090501/130c9d3f/signature.pgp From marius at alchemy.franken.de Fri May 1 12:10:21 2009 From: marius at alchemy.franken.de (Marius Strobl) Date: Fri May 1 12:10:28 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FADEF3.5010106@gmx.de> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> <49FADEF3.5010106@gmx.de> Message-ID: <20090501121018.GC2943@alchemy.franken.de> On Fri, May 01, 2009 at 01:37:23PM +0200, Christoph Mallon wrote: > Marius Strobl schrieb: > >On Sun, Apr 26, 2009 at 09:02:36AM +0200, Christoph Mallon wrote: > >>return with parentheses: > >>Removed, because it does not improve maintainability in any way. There > >>is no source for confusion here, so the rule even contradicts the rule, > >>which states not to use redundant parentheses. Maybe, decades ago it was > >>just a workaround for a broken compiler, which does not exist anymore. > > > >FYI, the idea behind this rule is said to be to able to use > >a macro return(), f.e. for debugging you then can do: > >#define return(x) do { \ > > printf("returning from %s with %d\n", __func__, (x)); \ > > return (x); \ > >} while (0) > > > >Given the this is a nifty feature and parentheses around the > >return value don't hurt maintainability in any way IMO this > >rule should stay. > > This is mentioned nowhere in style(9) (in general it is lacking reasons > why something is some way or the other). I agree that style(9) lacks explanations, however this doesn't necessarily mean that non-obvious rules are "silly" and should be removed. > Also I consider this as gross abuse: Macro names shall be in all > uppercase, so it is clear that there is a macro at work. Therefore > "return" is not a candidate. So this would violate yet another rule in > style(9) (the original return already violates the no-redundant > parentheses rule). > Also I would not mention __func__: there were objections against using > it in the past (though I, logically, prefer its use). In general this is correct, a return() macro (in which case the parentheses are not redundant) would be for local debugging only and not ever hit the tree just like any other printf()- debugging should not, so neither is relevant here. Besides, a macro without side effects, which a return() macro typically shouldn't have, is fine to be named all lowercase according to style(9). Marius From danny at cs.huji.ac.il Fri May 1 12:31:37 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri May 1 12:31:45 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501112239.GA23199@alchemy.franken.de> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> Message-ID: > On Sun, Apr 26, 2009 at 09:02:36AM +0200, Christoph Mallon wrote: > > > > return with parentheses: > > Removed, because it does not improve maintainability in any way. There > > is no source for confusion here, so the rule even contradicts the rule, > > which states not to use redundant parentheses. Maybe, decades ago it was > > just a workaround for a broken compiler, which does not exist anymore. > > FYI, the idea behind this rule is said to be to able to use > a macro return(), f.e. for debugging you then can do: > #define return(x) do { \ > printf("returning from %s with %d\n", __func__, (x)); \ > return (x); \ > } while (0) > > Given the this is a nifty feature and parentheses around the > return value don't hurt maintainability in any way IMO this > rule should stay. short version: not nifty, dirty yes! long version: it's already quiet difficult to read the sources with so many MaCrOs roaming around, but if you change if, return, then, else, switch etc, etc to a macro invocation, there will be a slight discrepancy between the undertsanding of the code and its running effect. btw, what if x is a pointer?, or a quad? or a complex ... my .02$ danny From xorquewasp at googlemail.com Fri May 1 13:05:40 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 1 13:05:47 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) Message-ID: <20090501130536.GA95601@logik.internal.network> Hello. After extensive hardware testing, new thermal compound, new case and a lot of work improving airflow, I'm now confident my new machine is OK from a hardware point of view (a weeks worth of memory testing, days of running prime95, temperature monitoring, extensive sessions with sysutils/stress running from a liveCD). About 20 minutes ago, I booted into the version of FreeBSD I had built previously to try to debug a DRI problem (which turned out not to exist, evidently). The build has every debugging option I could find enabled (WITNESS, INVARIANTS, etc). Upon trying to build the 'lmmon' port, I was greeted with a panic. I was dropped into ddb and thinking that the relevent crash info would be saved, I rebooted. It wasn't saved. The panic is reproducable: it always occurs in vfs_cache.c:345. How can I get all the info I need saved in order to file a PR? thanks, xw From anchie at fer.hr Fri May 1 14:01:54 2009 From: anchie at fer.hr (Ana Kukec) Date: Fri May 1 14:02:14 2009 Subject: IPsec in GENERIC kernel config In-Reply-To: <49F5B6F8.4040808@melen.org> References: <49F5B6F8.4040808@melen.org> Message-ID: <49FB00CB.5080402@fer.hr> Hi Jan, Jan Melen wrote: > Hi, > > Again when I compiled a custom kernel just to enable IPsec in the > FreeBSD kernel it came to my mind why is it so that the IPsec is not > enabled by default in the GENERIC kernel configuration file? At least > for me the GENERIC kernel configuration would do just fine if the > IPsec would be enabled in it by default. Now I have to build a custom > kernel just for IPsec btw IPsec is even mandatory for a host > supporting IPv6. > > IETF just says that IPsec support is mandatory in IPv6, but IPsec use is not. Most of current IPv6 implementations do not include IPsec, and there is nothing unusual with that. It is mainly about the performance, but there are also other issues, mainly security ones, e.g. it actually cannot defend against DoS attacks and cannot strictly eliminate spoofing, it is only a network-level security tool.. and there are still lots of incompatibility issues between different vendors' implementations of IPsec.. etc.. Ana From imp at bsdimp.com Fri May 1 14:14:17 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri May 1 14:14:23 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FA8D73.6040207@gmx.de> References: <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> Message-ID: <20090501.080927.-1923761682.imp@bsdimp.com> In message: <49FA8D73.6040207@gmx.de> Christoph Mallon writes: : M. Warner Losh schrieb: : > In message: <20090428114754.GB89235@server.vk2pj.dyndns.org> : > Peter Jeremy writes: : > : >Maybe using all of these changes is a bit too big change at once, but : > : >I'd like some of them applied to style(9). So, please consider the : > : >proposed changes individually and not a as a all-or-nothing package. : > : : > : One area you do not address is code consistency. Currently, the : > : FreeBSD kernel (and, to a lesser extent, userland) are mostly style(9) : > : compliant - ie the entire codebase is mostly written in the same : > : style. Whilst you may not like it (and I don't know that anyone : > : completely agrees with everything in style(9)), it does mean that : > : the code is consistent. : > : : > : Changing style(9) in a way that is not consistent with current code : > : means that either existing code must be brought into line with the : > : new standard - which incurs a one-off cost - or the code base becomes : > : split into "old" and "new" style - incurring an ongoing maintenance : > : cost as maintainers switch between styles. Both approaches incur a : > : risk of introducing new bugs. : > : : > : Note that I'm not saying that style(9) can never be changed, I'm just : > : saying that any change _will_ incur a cost and the cost as well as : > : the potential benefits need to be considered. : > : > This is my biggest worry about changing style(9) quickly. We don't : > want needless churn in the tree for just style changes since it makes : > it harder to track bug fixes into the past. : : I'm not saying that all the code has to be changed at once, but no new : code of the "old" style should be added. E.g. you should never use K&R : declarations when adding a new function. And if you are going to make a : big change somewhere, then it is sensible to use the "new" style. Mixing and matching style has proven to be bad when it has been practiced in the past. At least when practiced on a sub-file level. : > : [Reduce declaration scope as far as possible, initialise variables where : > : they are declared, sort by name] : > : : > : Whilst my personal preference is for the style suggested by Christoph : > : (and I generally agree with the points he makes), this is also the : > : change that incurs the biggest stylistic change. This is not a change : > : that can be practically retrofitted to existing code and therefore its : > : implementation would result in a mixture of code styles - increasing : > : the risk of bugs due to confusion as to which style was being used. I : > : am not yet sure whether the benefits outweigh the costs, : > : > This is the biggest one, and I think it may be too soon. Also, we : : This is the reason, why I explicitly mentioned, that the proposed : changes should be examined independently. : : > need to be careful on the initialization side of things because we : > currently have a lot of code that looks like: : > : > : > struct foo *fp; : > struct bar *bp; : > : > fp = get_foo(); : > if (!fp) return; : > bp = fp->bp; : > : > this can't easily be translated to the more natural: : > : > struct foo *fp = get_foo(); : > struct bar *bp = fp->bp; : > : > since really you'd want to write: : > : > struct foo *fp = get_foo(); : > if (!fp) return; : > struct bar *bp = fp->bp; : > : > which isn't legal in 'C'. However, we have enough where this isn't : : You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 (E) : ?6.8.2:1. In short: you can mix statements and declarations. : : > the case that it should be allowed. We should separate the : > initialization part of this from the scope part of this too. The : > initialization part is already leaking into the code, while instances : > of the scope restriction is rarer. : : Sorry, I do not understand these sentences. It's a stupid idea and I don't think we should do it. It is a bad practice to intermix statements and declarations, and I think a too radical departure from the current style. : > : [Don't parenthesize return values] : > : >Removed, because it does not improve maintainability in any way. : > : : > : This change could be made and tested mechanically. But there is : > : no justification for making it - stating that the existing rule : > : is no better than the proposed rule is no reason to change. : > : > I'm fine with this. : > : > : [No K&R declarations] : > : >Removed, because there is no need to use them anymore. : > : : > : Whilst this is a change that could be performed mechanically, there : > : are some gotchas relating to type promotion (as you point out). : > : The kernel already contains a mixture of ANSI & K&R declarations and : > : style(9) recommends using ANSI. IMHO, there is no need to make this : > : change until all K&R code is removed from FreeBSD. : > : > I'm fine with this change. : > : > : [ Don't insert an empty line if the function has no local variables.] : > : : > : This change could be made and tested mechanically. IMHO, this change : > : has neglible risk and could be safely implemented. : > : > I'm fine with this change. : : Would you commit these three proposals? Yes. : > : >> +.Sh LOCAL VARIABLES : > : : > : >Last, but definitely not least, I added this paragraph about the use of : > : >local variables. This is to clarify, how today's compilers handle : > : >unaliased local variables. : > : : > : Every version of gcc that FreeBSD has ever used would do this for : > : primitive types when optimisation was enabled. This approach can : > : become expensive in stack (and this is an issue for kernel code) when : > : using non-primitive types or when optimisation is not enabled (though : > : I'm not sure if this is supported). Assuming that gcc (and icc and : > : clang) behaves as stated in all supported optimisation modes, this : > : change would appear to be quite safe to make. : > : > Agreed, in general. We don't want to optimize our code style based on : > what one compiler does, perhaps on x86. : : I'm not sure whether I understand this - in particular the last three words. I'm saying we shouldn't tune our coding standard to the optimizations that the compiler of the hour gives, especially if those optimizations to the style are tuned to one architecture. Since there's little evidence presented on how these style changes will help any architecture, it is hard to judge if this is the case or not. Warner From imp at bsdimp.com Fri May 1 14:15:18 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri May 1 14:15:25 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FA8E88.1040905@gmx.de> References: <20090430233648.GA95360@keira.kiwi-computer.com> <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> Message-ID: <20090501.081229.1359784281.imp@bsdimp.com> In message: <49FA8E88.1040905@gmx.de> Christoph Mallon writes: : M. Warner Losh schrieb: : > In message: <20090430233648.GA95360@keira.kiwi-computer.com> : > "Rick C. Petty" writes: : > : On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: : > : > : > : > This is the biggest one, and I think it may be too soon. Also, we : > : > need to be careful on the initialization side of things because we : > : > currently have a lot of code that looks like: : > : > : > : > : > : > struct foo *fp; : > : > struct bar *bp; : > : > : > : > fp = get_foo(); : > : > if (!fp) return; : > : > bp = fp->bp; : > : > : > : > this can't easily be translated to the more natural: : > : > : > : > struct foo *fp = get_foo(); : > : > struct bar *bp = fp->bp; : > : > : > : > since really you'd want to write: : > : > : > : > struct foo *fp = get_foo(); : > : > if (!fp) return; : > : > struct bar *bp = fp->bp; : > : > : > : > which isn't legal in 'C'. : > : : > : I thought we were talking about C99, in which case this is perfectly legal. : > : I certainly use it all the time in my C99 code. : > : > Hmmm, looks like that was added. That's ugly as C++... : : I do not think, this is ugly. On the contrary, it aids maintainability, : because it reduces the scope of variables. Also quite some variables are : only initialised and not changed afterwards, so it's nice to have the : declaration and the only assignment in a single place. IMO this is a : quite natural style, because you don't have anything in the code, before : it is needed: Get the first pointer; if something is wrong, bail out; : get the second pointer - the second pointer does not (textually) exist : before it is needed. It is ugly. Hunting for declarations sucks, and it is one of the things I really like about BSD code is that I don't have to. This is a religious point, and we're dangerously close to saying my religion is better than your religion. I don't like this part of the proposal at all. I can see the value in relaxing it for when you have a series of variables that are initialized, but relaxing it to the point where you intermix code and declarations goes way too far. It is bad enough to have to deal with inner scopes, but tolerable. It is intolerable to have to look for it anywhere in a big function. It tends to encourage spaghetti code, which is one of the things that style(9) tries to discourage in many subtle ways. Warner From imp at bsdimp.com Fri May 1 14:21:12 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri May 1 14:21:19 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FADEF3.5010106@gmx.de> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> <49FADEF3.5010106@gmx.de> Message-ID: <20090501.082020.698246310.imp@bsdimp.com> In message: <49FADEF3.5010106@gmx.de> Christoph Mallon writes: : Marius Strobl schrieb: : > On Sun, Apr 26, 2009 at 09:02:36AM +0200, Christoph Mallon wrote: : >> return with parentheses: : >> Removed, because it does not improve maintainability in any way. There : >> is no source for confusion here, so the rule even contradicts the rule, : >> which states not to use redundant parentheses. Maybe, decades ago it was : >> just a workaround for a broken compiler, which does not exist anymore. : > : > FYI, the idea behind this rule is said to be to able to use : > a macro return(), f.e. for debugging you then can do: : > #define return(x) do { \ : > printf("returning from %s with %d\n", __func__, (x)); \ : > return (x); \ : > } while (0) : > : > Given the this is a nifty feature and parentheses around the : > return value don't hurt maintainability in any way IMO this : > rule should stay. : : This is mentioned nowhere in style(9) (in general it is lacking reasons : why something is some way or the other). It has been an example used for the past 15 years at least as to why to do this... I don't know how many people have actually used the ability to do this in code. : Also I consider this as gross abuse: Macro names shall be in all : uppercase, so it is clear that there is a macro at work. Therefore : "return" is not a candidate. So this would violate yet another rule in : style(9) (the original return already violates the no-redundant : parentheses rule). : Also I would not mention __func__: there were objections against using : it in the past (though I, logically, prefer its use). It is a debugging aid, but one of dubious value for a far more fundamental reason: return; will break any macro. Warner From imp at bsdimp.com Fri May 1 14:40:07 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri May 1 14:40:14 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.081229.1359784281.imp@bsdimp.com> References: <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> <20090501.081229.1359784281.imp@bsdimp.com> Message-ID: <20090501.083712.396385864.imp@bsdimp.com> In message: <20090501.081229.1359784281.imp@bsdimp.com> M. Warner Losh writes: : In message: <49FA8E88.1040905@gmx.de> : Christoph Mallon writes: : : M. Warner Losh schrieb: : : > In message: <20090430233648.GA95360@keira.kiwi-computer.com> : : > "Rick C. Petty" writes: : : > : On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: : : > : > : : > : > This is the biggest one, and I think it may be too soon. Also, we : : > : > need to be careful on the initialization side of things because we : : > : > currently have a lot of code that looks like: : : > : > : : > : > : : > : > struct foo *fp; : : > : > struct bar *bp; : : > : > : : > : > fp = get_foo(); : : > : > if (!fp) return; : : > : > bp = fp->bp; : : > : > : : > : > this can't easily be translated to the more natural: : : > : > : : > : > struct foo *fp = get_foo(); : : > : > struct bar *bp = fp->bp; : : > : > : : > : > since really you'd want to write: : : > : > : : > : > struct foo *fp = get_foo(); : : > : > if (!fp) return; : : > : > struct bar *bp = fp->bp; : : > : > : : > : > which isn't legal in 'C'. : : > : : : > : I thought we were talking about C99, in which case this is perfectly legal. : : > : I certainly use it all the time in my C99 code. : : > : : > Hmmm, looks like that was added. That's ugly as C++... : : : : I do not think, this is ugly. On the contrary, it aids maintainability, : : because it reduces the scope of variables. Also quite some variables are : : only initialised and not changed afterwards, so it's nice to have the : : declaration and the only assignment in a single place. IMO this is a : : quite natural style, because you don't have anything in the code, before : : it is needed: Get the first pointer; if something is wrong, bail out; : : get the second pointer - the second pointer does not (textually) exist : : before it is needed. : : It is ugly. Hunting for declarations sucks, and it is one of the : things I really like about BSD code is that I don't have to. : : This is a religious point, and we're dangerously close to saying my : religion is better than your religion. I don't like this part of the : proposal at all. I can see the value in relaxing it for when you have : a series of variables that are initialized, but relaxing it to the : point where you intermix code and declarations goes way too far. It : is bad enough to have to deal with inner scopes, but tolerable. It is : intolerable to have to look for it anywhere in a big function. It : tends to encourage spaghetti code, which is one of the things that : style(9) tries to discourage in many subtle ways. This came off a little more absolute than I wanted. I should have added "in the absence of hard data..." Warner From matthew.fleming at isilon.com Fri May 1 16:09:13 2009 From: matthew.fleming at isilon.com (Matthew Fleming) Date: Fri May 1 16:09:20 2009 Subject: C++ incompatability, was C99: Suggestions for style(9) In-Reply-To: <49FAE4EA.1010205@gmx.de> References: <49F4070C.2000108@gmx.de><20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> Message-ID: <06D5F9F6F655AD4C92E28B662F7F853E02ACA843@seaxch09.desktop.isilon.com> [snip exciting discussion on style] > There are several C99 features used already, e.g. designated initializers: > bla bli = { .blub = "foo", .arr[0] = 42 }; > Do you suggest that this should not be used, because it is inconsistent > with all the other existing compound initialisations? Regarding this great feature of C99, sadly, it's not C++ compatible. So while designated initializers in a C source file are great, in a header file they will give a compile error if included in e.g. a C++ kernel module (which otherwise would work fine). Actually, as a further digression, I was wondering if/when FreeBSD would add #ifdef __cplusplus extern "C" { #endif to sys/sys/*.h and other headers that can be included by a kernel module. Thanks, matthew From justin at neutroni.net Fri May 1 16:16:24 2009 From: justin at neutroni.net (Justin G.) Date: Fri May 1 16:22:45 2009 Subject: Cobalt Raq 550 Message-ID: <5da021490905010848q441781den54eefba85a69f342@mail.gmail.com> Hello Hackers, We came into a Cobalt Raq 550 the other day and were wondering if we could put it to use. I've googled and googled and found only guides for Linux installs. Much of it is quite similar, but my issue is with the loader on the device being able to boot into FreeBSD. The device searches for linux kernel images when booting. Does anyone have any experience with installing FreeBSD on a Raq 550? I wasn't able to find a website with much detail and would appreciate any hints or leads :-) I hope everyone has an excellent weekend. From bakul at bitblocks.com Fri May 1 16:50:52 2009 From: bakul at bitblocks.com (Bakul Shah) Date: Fri May 1 16:50:59 2009 Subject: C++ incompatability, was C99: Suggestions for style(9) In-Reply-To: Your message of "Fri, 01 May 2009 08:57:34 PDT." <06D5F9F6F655AD4C92E28B662F7F853E02ACA843@seaxch09.desktop.isilon.com> References: <49F4070C.2000108@gmx.de><20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> <06D5F9F6F655AD4C92E28B662F7F853E02ACA843@seaxch09.desktop.isilon.com> Message-ID: <20090501163108.576465B3B@mail.bitblocks.com> On Fri, 01 May 2009 08:57:34 PDT "Matthew Fleming" wrote: > [snip exciting discussion on style] > > > There are several C99 features used already, e.g. designated initializers: > > bla bli = { .blub = "foo", .arr[0] = 42 }; > > Do you suggest that this should not be used, because it is inconsistent > > with all the other existing compound initialisations? > > Regarding this great feature of C99, sadly, it's not C++ compatible. So > while designated initializers in a C source file are great, in a header > file they will give a compile error if included in e.g. a C++ kernel > module (which otherwise would work fine). Why would you put initializers in a header file? If included in more than one file, the linker will complain that the initialized variable is multiply defined. If creating header files that get included in in only one file *and* you want to use initializers, why not use the right language for include file code. From matthew.fleming at isilon.com Fri May 1 17:00:09 2009 From: matthew.fleming at isilon.com (Matthew Fleming) Date: Fri May 1 17:00:16 2009 Subject: C++ incompatability, was C99: Suggestions for style(9) In-Reply-To: <20090501163108.576465B3B@mail.bitblocks.com> References: <49F4070C.2000108@gmx.de><20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> <06D5F9F6F655AD4C92E28B662F7F853E02ACA843@seaxch09.desktop.isilon.com> <20090501163108.576465B3B@mail.bitblocks.com> Message-ID: <06D5F9F6F655AD4C92E28B662F7F853E02ACA852@seaxch09.desktop.isilon.com> > > [snip exciting discussion on style] > > > > > There are several C99 features used already, e.g. designated initializers: > > > bla bli = { .blub = "foo", .arr[0] = 42 }; > > > Do you suggest that this should not be used, because it is inconsistent > > > with all the other existing compound initialisations? > > > > Regarding this great feature of C99, sadly, it's not C++ compatible. So > > while designated initializers in a C source file are great, in a header > > file they will give a compile error if included in e.g. a C++ kernel > > module (which otherwise would work fine). > > Why would you put initializers in a header file? If included > in more than one file, the linker will complain that the > initialized variable is multiply defined. If creating header > files that get included in in only one file *and* you want to > use initializers, why not use the right language for include > file code. Macros, like MALLOC_DEFINE, DB_COMMAND, etc., define initialized variables when used. These can't be changed to use named initializers without breaking C++. Thanks, matthew From lgj at usenix.org Fri May 1 17:03:33 2009 From: lgj at usenix.org (Lionel Garth Jones) Date: Fri May 1 17:10:43 2009 Subject: USENIX WebApps '10 Call for Papers Now Available Message-ID: On behalf of the Program Committee, I would like to invite you to submit your work to the USENIX Conference on Web Application Development (WebApps '10). WebApps '10 is a new technical conference designed to bring together experts in all aspects of developing and deploying Web applications. Suggested topics related to Web application development include but are not limited to: * Computing substrates and deployment technologies ("cloud computing") * Frameworks for developing Web applications * Client-side toolkits, libraries, and plug-ins * Storage systems * Security issues for Web applications * Management techniques for large-scale Web applications * Languages for Web applications * Scalability issues and techniques * Techniques for creating highly interactive Web applications * Software as a service * Applications that illustrate interesting new features or implementation techniques * Performance measurements of Web applications * Real-time data delivery over the Web * Web services Paper submissions are due by January 11, 2010. The Call for Papers, with submission guidelines, can be found at http://www.usenix.org/webapps10/cfpa/ The USENIX Conference on Web Application Development (WebApps '10) will take place June 20?25, 2010, in Boston, MA. The technical sessions will take place on June 23?25. I look forward to receiving your submissions! Sincerely, John Ousterhout, Stanford University WebApps '10 Program Chair webapps10chair@usenix.org ------------------------------------------------------------------------ Call for Papers: USENIX Conference on Web Application Development (WebApps '10) June 20?25, 2010 Boston, MA http://www.usenix.org/webapps10/cfpa Paper submissions deadline: January 11, 2010 ------------------------------------------------------------------------ From xorquewasp at googlemail.com Fri May 1 17:35:17 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 1 17:35:23 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <20090501130536.GA95601@logik.internal.network> References: <20090501130536.GA95601@logik.internal.network> Message-ID: <20090501173513.GA27141@logik.internal.network> Filed under: http://www.freebsd.org/cgi/query-pr.cgi?pr=134142 Would be incredibly grateful if somebody in the know could take a look at this. xw From xorquewasp at googlemail.com Fri May 1 17:42:12 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 1 17:42:19 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <3bbf2fe10905011039o7af18b98n70202ecf78ec7904@mail.gmail.com> References: <20090501130536.GA95601@logik.internal.network> <20090501173513.GA27141@logik.internal.network> <3bbf2fe10905011039o7af18b98n70202ecf78ec7904@mail.gmail.com> Message-ID: <20090501174208.GA42682@logik.internal.network> On 2009-05-01 19:39:43, Attilio Rao wrote: > 2009/5/1 : > > Filed under: > > > > http://www.freebsd.org/cgi/query-pr.cgi?pr=134142 > > > > Would be incredibly grateful if somebody in the know could take > > a look at this. > > But, what's the panic message? > It's at the bottom: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x1a0 fault code = supervisor read data, page not present instruction pointer = 0x8:0xffffffff80509409 stack pointer = 0x10:0xffffffff7a3d46b0 frame pointer = 0x10:0xffffffff7a3d46e0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 75381 (sh) exclusive sleep mutex Name Cache r = 0 (0xffffffff80be9940) locked @ /usr/src/sys/kern/vfs_cache.c:345 exclusive sleep mutex Name Cache r = 0 (0xffffffff80be9940) locked @ /usr/src/sys/kern/vfs_cache.c:345 exclusive sx so_rcv_sx r = 0 (0xffffff0005b42970) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 The web interface mangled the whitespace somewhat. xw From julian at elischer.org Fri May 1 17:58:04 2009 From: julian at elischer.org (Julian Elischer) Date: Fri May 1 17:58:11 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.082020.698246310.imp@bsdimp.com> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> <49FADEF3.5010106@gmx.de> <20090501.082020.698246310.imp@bsdimp.com> Message-ID: <49FB382B.6030406@elischer.org> M. Warner Losh wrote: [...] (about return ();) > > It has been an example used for the past 15 years at least as to why > to do this... I don't know how many people have actually used the > ability to do this in code. I have done so.. > > : Also I consider this as gross abuse: Macro names shall be in all > : uppercase, so it is clear that there is a macro at work. Therefore > : "return" is not a candidate. So this would violate yet another rule in > : style(9) (the original return already violates the no-redundant > : parentheses rule). > : Also I would not mention __func__: there were objections against using > : it in the past (though I, logically, prefer its use). > > It is a debugging aid, but one of dubious value for a far more > fundamental reason: > > return; > > will break any macro. > > Warner > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From attilio at freebsd.org Fri May 1 18:05:55 2009 From: attilio at freebsd.org (Attilio Rao) Date: Fri May 1 18:06:58 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <20090501173513.GA27141@logik.internal.network> References: <20090501130536.GA95601@logik.internal.network> <20090501173513.GA27141@logik.internal.network> Message-ID: <3bbf2fe10905011039o7af18b98n70202ecf78ec7904@mail.gmail.com> 2009/5/1 : > Filed under: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=134142 > > Would be incredibly grateful if somebody in the know could take > a look at this. But, what's the panic message? Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From yuri at rawbw.com Fri May 1 18:28:06 2009 From: yuri at rawbw.com (Yuri) Date: Fri May 1 18:28:13 2009 Subject: Why top never shows ~100% CPU usage with heavy PCU load? Message-ID: <49FB3F32.2010800@rawbw.com> When I run cycle process: main() {for (;;) {}} I never see that it consumes ~100% CPU. Instead 'top -C' shows something like this, with numbers fluctuating around the shown numbers: CPU: 96.2% user, 0.0% nice, 20.0% system, 0.0% interrupt, 0.0% idle Mem: 653M Active, 995M Inact, 241M Wired, 90M Cache, 112M Buf, 11M Free Swap: 16G Total, 204M Used, 16G Free, 1% Inuse, 16K In PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 85422 yuri 1 99 0 2520K 980K RUN 0:21 57.47% cycle Total of CPU column for all processes is never what top shows in "CPU:" field. At the same time kde system guard app shows 100% overall CPU load and same cycle app shows as consuming ~85% CPU. So system guard always shows consistently higher number for the same process than top -C shows. On Linux process the cycle app shows as consuming <~100% CPU and total is also <~100% that makes sense. Why top's "CPU:" field doesn't equal total of all CPUs for all processes that it shows? Why top doesn't show ~100% CPU consumption for the cycle process? Why sysguard in kde consistently shows higher CPU consumption numbers than top -C? Yuri From yuri at rawbw.com Fri May 1 18:50:57 2009 From: yuri at rawbw.com (Yuri) Date: Fri May 1 18:51:04 2009 Subject: Why top never shows ~100% CPU usage with heavy PCU load? In-Reply-To: <20090501183847.GB47845@wep4035.physik.uni-wuerzburg.de> References: <49FB3F32.2010800@rawbw.com> <20090501183847.GB47845@wep4035.physik.uni-wuerzburg.de> Message-ID: <49FB448A.9050607@rawbw.com> Alexey Shuvaev wrote: > Strange is 20% system load. The summary line is about all cpus/cores/... > Correction: instead of 20% should be 3.8%. Also if this matters I have FreeBSD 7.2, single CPU AMD3200 @ 2GHz. Yuri From shuvaev at physik.uni-wuerzburg.de Fri May 1 19:00:46 2009 From: shuvaev at physik.uni-wuerzburg.de (Alexey Shuvaev) Date: Fri May 1 19:00:53 2009 Subject: Why top never shows ~100% CPU usage with heavy PCU load? In-Reply-To: <49FB3F32.2010800@rawbw.com> References: <49FB3F32.2010800@rawbw.com> Message-ID: <20090501183847.GB47845@wep4035.physik.uni-wuerzburg.de> On Fri, May 01, 2009 at 11:28:02AM -0700, Yuri wrote: > When I run cycle process: main() {for (;;) {}} I never see that it > consumes ~100% CPU. > Instead 'top -C' shows something like this, with numbers fluctuating > around the shown numbers: > > > CPU: 96.2% user, 0.0% nice, 20.0% system, 0.0% interrupt, 0.0% idle > Mem: 653M Active, 995M Inact, 241M Wired, 90M Cache, 112M Buf, 11M Free > Swap: 16G Total, 204M Used, 16G Free, 1% Inuse, 16K In > > PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND > 85422 yuri 1 99 0 2520K 980K RUN 0:21 57.47% cycle > > > [snip] > Strange is 20% system load. The summary line is about all cpus/cores/... Here I have: top: last pid: 48056; load averages: 0.91, 0.38, 0.15 up 12+23:14:39 20:34:58 49 processes: 2 running, 47 sleeping CPU: 50.0% user, 0.0% nice, 0.2% system, 0.0% interrupt, 49.8% idle Mem: 259M Active, 1630M Inact, 537M Wired, 16M Cache, 417M Buf, 1495M Free Swap: 4096M Total, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 48032 lexx 1 118 0 2624K 604K CPU1 1 2:31 100.00% bbb ... ~>uname -a FreeBSD wep4035 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Sat Apr 18 20:38:14 CEST 2009 root@wep4035:/usr/obj/usr/src/sys/GENERIC amd64 ~> cat bbb.c int main(void) { for (;;); return; } My $0.02, Alexey. From kabaev at gmail.com Fri May 1 19:13:17 2009 From: kabaev at gmail.com (Alexander Kabaev) Date: Fri May 1 19:13:24 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <20090501130536.GA95601@logik.internal.network> References: <20090501130536.GA95601@logik.internal.network> Message-ID: <20090501151308.5c9109da@kan.dnsalias.net> How recent are your sources? There were a number of bugs introduced and then fixed in releng/7.2 and stable/7 and line number you post does not match anything interesting in either. Please make sure you have latest vfs_cache.c file at the least. -- Alexander Kabaev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 188 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090501/d7b14746/signature.pgp From xorquewasp at googlemail.com Fri May 1 19:21:17 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 1 19:21:24 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <20090501151308.5c9109da@kan.dnsalias.net> References: <20090501130536.GA95601@logik.internal.network> <20090501151308.5c9109da@kan.dnsalias.net> Message-ID: <20090501192113.GA15572@logik.internal.network> On 2009-05-01 15:13:08, Alexander Kabaev wrote: > How recent are your sources? There were a number of bugs introduced and > then fixed in releng/7.2 and stable/7 and line number you post does not > match anything interesting in either. > > Please make sure you have latest vfs_cache.c file at the least. Hello. Checking back through my sent mail from the DRI thread, this version of -STABLE was checked out and compiled on Fri, 10 Apr 2009 16:42:51 +0100. The machine was so new that I hadn't even set the clock, so the kernel timestamps appear to be Jan 1. I'll make an attempt to check out today's -STABLE and compile it but I'm not optimistic that the machine will actually be able to build world without reverting to a release first. xw From kabaev at gmail.com Fri May 1 19:50:47 2009 From: kabaev at gmail.com (Alexander Kabaev) Date: Fri May 1 19:50:54 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <20090501192113.GA15572@logik.internal.network> References: <20090501130536.GA95601@logik.internal.network> <20090501151308.5c9109da@kan.dnsalias.net> <20090501192113.GA15572@logik.internal.network> Message-ID: <20090501155038.692eb891@kan.dnsalias.net> On Fri, 1 May 2009 20:21:13 +0100 xorquewasp@googlemail.com wrote: > Hello. > > Checking back through my sent mail from the DRI thread, this version > of -STABLE was checked out and compiled on Fri, 10 Apr 2009 16:42:51 > +0100. > > The machine was so new that I hadn't even set the clock, so the kernel > timestamps appear to be Jan 1. > > I'll make an attempt to check out today's -STABLE and compile it but > I'm not optimistic that the machine will actually be able to build > world without reverting to a release first. You can always try to bypass namecache altogether while you are building the new kernel: do sysctl debug.vfscache=0 -- Alexander Kabaev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 188 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090501/923f6f5c/signature.pgp From marta at freebsd.org Fri May 1 20:22:34 2009 From: marta at freebsd.org (Marta Carbone) Date: Fri May 1 20:23:15 2009 Subject: SoC2009: Ipfw and dummyent improvements Message-ID: <20090501200931.GA55379@onelab2.iet.unipi.it> Hello, my name is Marta Carbone, I am at the first year of my PhD program in Information Engineering at the University of Pisa. As part of the Google SoC I will work on FreeBSD ipfw and dummynet. My mentor is Luigi Rizzo. The main goal of the project is to revise and improve the ipfw and dummynet subsystems. The project will mainly involve: - the kernel source code, making it more manageable, splitting large functions, improving the microinstruction compiler; - the userland kernel-interface, identifying locking issues or performance problems. A more detailed version of the proposal can be found on the FreeBSD Wiki page: http://wiki.freebsd.org/SOC2009MartaCarbone marta From christoph.mallon at gmx.de Fri May 1 20:32:27 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 20:32:34 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> Message-ID: <49FB5C57.6050407@gmx.de> Zaphod Beeblebrox schrieb: > On Fri, May 1, 2009 at 4:30 AM, Julian Elischer wrote: > >> As an old-fart I have found many cases where what I thought was >> a silly style rule, turned out to save my work in some way. >> >> Christoph Mallon wrote: >> >> >> >>>> struct foo *fp; >>>> struct bar *bp; >>>> >>>> fp = get_foo(); >>>> if (!fp) return; >>>> bp = fp->bp; >>>> >>>> this can't easily be translated to the more natural: >>>> >>>> struct foo *fp = get_foo(); >>>> struct bar *bp = fp->bp; >>>> >> Well more natural for you, but not necessarily for everyone, >> and NOT the same as what is there now, as you noticed. >> >> >> >>>> since really you'd want to write: >>>> >>>> struct foo *fp = get_foo(); >>>> if (!fp) return; >>>> struct bar *bp = fp->bp; >>>> >>>> which isn't legal in 'C'. However, we have enough where this isn't >>>> >>> You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 (E) >>> ?6.8.2:1. In short: you can mix statements and declarations. >>> > Sure, but it's still very bad: If I'm not mistaken, this would mean that > "bp" would only be valid within the "if" clause --- which isn't very useful. You are mistaken. Re-read the "if": It already contains a "return;" as then-part. The declaration of "bp" has no relation to the "if". In fact this is very good: "bp" can only be used after the "if", because it is declared after it. Further, it most probably is only assigned a value once, so declaration and the signle assignment are in the same place, which aids readability and makes the code more concise. Christoph From julian at elischer.org Fri May 1 20:38:14 2009 From: julian at elischer.org (Julian Elischer) Date: Fri May 1 20:38:21 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FB5C57.6050407@gmx.de> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> <49FB5C57.6050407@gmx.de> Message-ID: <49FB5DB3.9030200@elischer.org> Christoph Mallon wrote: > > You are mistaken. Re-read the "if": It already contains a "return;" as > then-part. The declaration of "bp" has no relation to the "if". > In fact this is very good: "bp" can only be used after the "if", because > it is declared after it. Further, it most probably is only assigned a > value once, so declaration and the signle assignment are in the same > place, which aids readability and makes the code more concise. the fact that people misread it allows me to say "the defense rests m'lord" > > Christoph > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From christoph.mallon at gmx.de Fri May 1 20:42:04 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 20:42:11 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FB5DB3.9030200@elischer.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> <49FB5C57.6050407@gmx.de> <49FB5DB3.9030200@elischer.org> Message-ID: <49FB5E99.5070004@gmx.de> Julian Elischer schrieb: > Christoph Mallon wrote: > >> >> You are mistaken. Re-read the "if": It already contains a "return;" as >> then-part. The declaration of "bp" has no relation to the "if". >> In fact this is very good: "bp" can only be used after the "if", >> because it is declared after it. Further, it most probably is only >> assigned a value once, so declaration and the signle assignment are in >> the same place, which aids readability and makes the code more concise. > > the fact that people misread it allows me to say > > "the defense rests m'lord" Non sequitur. Warner wrote the "return;" in the same line as the if, which easily hides it. If the "return;" wasn't there, the original statement would be almost correct - actually it would be a compile error, because if (x) int i; is not allowed[1]. Christoph [1] if (x) { int i; } is allowed, of course. From zbeeble at gmail.com Fri May 1 20:48:48 2009 From: zbeeble at gmail.com (Zaphod Beeblebrox) Date: Fri May 1 20:48:55 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FAB322.9030103@elischer.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> Message-ID: <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> On Fri, May 1, 2009 at 4:30 AM, Julian Elischer wrote: > As an old-fart I have found many cases where what I thought was > a silly style rule, turned out to save my work in some way. > > Christoph Mallon wrote: > > > >>> >>> struct foo *fp; >>> struct bar *bp; >>> >>> fp = get_foo(); >>> if (!fp) return; >>> bp = fp->bp; >>> >>> this can't easily be translated to the more natural: >>> >>> struct foo *fp = get_foo(); >>> struct bar *bp = fp->bp; >>> >> > Well more natural for you, but not necessarily for everyone, > and NOT the same as what is there now, as you noticed. > > > >>> since really you'd want to write: >>> >>> struct foo *fp = get_foo(); >>> if (!fp) return; >>> struct bar *bp = fp->bp; >>> >>> which isn't legal in 'C'. However, we have enough where this isn't >>> >> >> You're mistaken, this is perfectly legal C. See ISO/IEC 9899:1999 (E) >> ?6.8.2:1. In short: you can mix statements and declarations. >> > Sure, but it's still very bad: If I'm not mistaken, this would mean that "bp" would only be valid within the "if" clause --- which isn't very useful. From christoph.mallon at gmx.de Fri May 1 20:55:19 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 20:55:26 2009 Subject: C99: Suggestions for style(9) In-Reply-To: References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> <49FB5C57.6050407@gmx.de> <49FB5DB3.9030200@elischer.org> Message-ID: <49FB61B3.4090604@gmx.de> Daniel Eischen schrieb: > +1 for leaving style(9) alone. Have you looked at all the proposed changes? I ask to consider them individually. Christoph From christoph.mallon at gmx.de Fri May 1 20:55:42 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 20:55:51 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501121018.GC2943@alchemy.franken.de> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> <49FADEF3.5010106@gmx.de> <20090501121018.GC2943@alchemy.franken.de> Message-ID: <49FB61CB.80906@gmx.de> Marius Strobl schrieb: > On Fri, May 01, 2009 at 01:37:23PM +0200, Christoph Mallon wrote: >> Marius Strobl schrieb: >>> On Sun, Apr 26, 2009 at 09:02:36AM +0200, Christoph Mallon wrote: >>>> return with parentheses: >>>> Removed, because it does not improve maintainability in any way. There >>>> is no source for confusion here, so the rule even contradicts the rule, >>>> which states not to use redundant parentheses. Maybe, decades ago it was >>>> just a workaround for a broken compiler, which does not exist anymore. >>> FYI, the idea behind this rule is said to be to able to use >>> a macro return(), f.e. for debugging you then can do: >>> #define return(x) do { \ >>> printf("returning from %s with %d\n", __func__, (x)); \ >>> return (x); \ >>> } while (0) >>> >>> Given the this is a nifty feature and parentheses around the >>> return value don't hurt maintainability in any way IMO this >>> rule should stay. >> This is mentioned nowhere in style(9) (in general it is lacking reasons >> why something is some way or the other). > > I agree that style(9) lacks explanations, however this doesn't > necessarily mean that non-obvious rules are "silly" and should > be removed. I neither said not implied this. I said, that it is lacking in justification, which clearly is bad. >> Also I consider this as gross abuse: Macro names shall be in all >> uppercase, so it is clear that there is a macro at work. Therefore >> "return" is not a candidate. So this would violate yet another rule in >> style(9) (the original return already violates the no-redundant >> parentheses rule). >> Also I would not mention __func__: there were objections against using >> it in the past (though I, logically, prefer its use). > > In general this is correct, a return() macro (in which case > the parentheses are not redundant) would be for local debugging > only and not ever hit the tree just like any other printf()- > debugging should not, so neither is relevant here. Besides, > a macro without side effects, which a return() macro typically > shouldn't have, is fine to be named all lowercase according > to style(9). So everybody has to pay with a small amount of obfuscation per return for an obscure trick, which compared to the number of returns is never used. Therefore I don't consider the return-macro convincing at all. Christoph From deischen at freebsd.org Fri May 1 21:01:52 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Fri May 1 21:01:59 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FB61B3.4090604@gmx.de> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <49FAB322.9030103@elischer.org> <5f67a8c40905011324s2ad5e02dy47c73ae950845b54@mail.gmail.com> <49FB5C57.6050407@gmx.de> <49FB5DB3.9030200@elischer.org> <49FB61B3.4090604@gmx.de> Message-ID: On Fri, 1 May 2009, Christoph Mallon wrote: > Daniel Eischen schrieb: >> +1 for leaving style(9) alone. > > Have you looked at all the proposed changes? I ask to consider them > individually. Point taken, my previous comment will only be for the part about inline declarations. I'll go back and look at the other proposed changes. -- DE From christoph.mallon at gmx.de Fri May 1 21:03:29 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri May 1 21:03:36 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.080927.-1923761682.imp@bsdimp.com> References: <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> <49FA8D73.6040207@gmx.de> <20090501.080927.-1923761682.imp@bsdimp.com> Message-ID: <49FB639E.9020401@gmx.de> M. Warner Losh schrieb: > In message: <49FA8D73.6040207@gmx.de> > Christoph Mallon writes: > : M. Warner Losh schrieb: > : > In message: <20090428114754.GB89235@server.vk2pj.dyndns.org> > : > Peter Jeremy writes: > : > : >> +.Sh LOCAL VARIABLES > : > : > : > : >Last, but definitely not least, I added this paragraph about the use of > : > : >local variables. This is to clarify, how today's compilers handle > : > : >unaliased local variables. > : > : > : > : Every version of gcc that FreeBSD has ever used would do this for > : > : primitive types when optimisation was enabled. This approach can > : > : become expensive in stack (and this is an issue for kernel code) when > : > : using non-primitive types or when optimisation is not enabled (though > : > : I'm not sure if this is supported). Assuming that gcc (and icc and > : > : clang) behaves as stated in all supported optimisation modes, this > : > : change would appear to be quite safe to make. > : > > : > Agreed, in general. We don't want to optimize our code style based on > : > what one compiler does, perhaps on x86. > : > : I'm not sure whether I understand this - in particular the last three words. > > I'm saying we shouldn't tune our coding standard to the optimizations > that the compiler of the hour gives, especially if those optimizations > to the style are tuned to one architecture. Since there's little > evidence presented on how these style changes will help any > architecture, it is hard to judge if this is the case or not. The main goal of the proposed change is not about optimisation, but about clarity of the code: It is better to declare multiple variables with meaningful names instead of having one generic "int k;" used in several different contexts. I just also mentioned, that re-using variables in different contexts when taking its address is involved, leads to worse machine code, but this is a minor point. It's just that clarity for a reader and quality of the generated code nicely correlate here. Christoph From fabio at freebsd.org Fri May 1 21:04:17 2009 From: fabio at freebsd.org (Fabio Checconi) Date: Fri May 1 21:04:24 2009 Subject: SoC2009: Geom-based Disk Schedulers Message-ID: <20090501201045.GA8738@gandalf.sssup.it> Hi all, I'm a PhD student, this summer I'll work on a SoC project on disk scheduling. I will extend the work we started with luigi, that we already presented in [1, 2]. Two of the main areas that still need improvement, and that will be considered during the project, are doing proper request classification and performance tuning for the specific scheduling algorithms. More info available on the wiki page of the project: http://wiki.freebsd.org/SOC2009FabioChecconi [1] http://lists.freebsd.org/pipermail/freebsd-stable/2009-January/047597.html [2] http://lists.freebsd.org/pipermail/freebsd-stable/2009-March/048704.html From xorquewasp at googlemail.com Fri May 1 21:39:50 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 1 21:39:57 2009 Subject: vfs_cache panic, 7.2-prerelease (stable) In-Reply-To: <20090501155038.692eb891@kan.dnsalias.net> References: <20090501130536.GA95601@logik.internal.network> <20090501151308.5c9109da@kan.dnsalias.net> <20090501192113.GA15572@logik.internal.network> <20090501155038.692eb891@kan.dnsalias.net> Message-ID: <20090501213945.GA87334@logik.internal.network> On 2009-05-01 15:50:38, Alexander Kabaev wrote: > On Fri, 1 May 2009 20:21:13 +0100 > xorquewasp@googlemail.com wrote: > > > Hello. > > > > Checking back through my sent mail from the DRI thread, this version > > of -STABLE was checked out and compiled on Fri, 10 Apr 2009 16:42:51 > > +0100. > > > > The machine was so new that I hadn't even set the clock, so the kernel > > timestamps appear to be Jan 1. > > > > I'll make an attempt to check out today's -STABLE and compile it but > > I'm not optimistic that the machine will actually be able to build > > world without reverting to a release first. > > You can always try to bypass namecache altogether while you are > building the new kernel: do sysctl debug.vfscache=0 Hi. Updating to today's -STABLE appears to have fixed the problem (or at least it doesn't occur for the usage described in the PR!). Thanks! xw From deeptech71 at gmail.com Sat May 2 02:32:44 2009 From: deeptech71 at gmail.com (deeptech71@gmail.com) Date: Sat May 2 02:32:51 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.081229.1359784281.imp@bsdimp.com> References: <20090430233648.GA95360@keira.kiwi-computer.com> <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> <20090501.081229.1359784281.imp@bsdimp.com> Message-ID: <49FBB147.4060305@gmail.com> M. Warner Losh wrote: > Hunting for declarations sucks I'd rather hunt a bit for its declaration and find uses of it on the way, rather than find the declaration..and then what? > This is a religious point, and we're dangerously close to saying my > religion is better than your religion. I don't like this part of the > proposal at all. I can see the value in relaxing it for when you have > a series of variables that are initialized, but relaxing it to the > point where you intermix code and declarations goes way too far. It > is bad enough to have to deal with inner scopes, but tolerable. It is > intolerable to have to look for it anywhere in a big function. It > tends to encourage spaghetti code, which is one of the things that > style(9) tries to discourage in many subtle ways. If there is a function so big that it overwhelms you at first look, do not make micro changes to parts unless you know what you are doing, that is, you basically looked through the function. I think the thinking required for a linear skim is decreased with localized code. And instead of making large functions, isn't it recommended to divide it into subfunctions, which self-documents the code? So you should be able to see the declaration "just" above the use. But let's consider large functions. If, and after looking at the top of the function, you can't see the declatation there either, the only thing you have to do is run a search down the function, and the declaration is found [epsilon time wasted?]. From deeptech71 at gmail.com Sat May 2 02:38:17 2009 From: deeptech71 at gmail.com (deeptech71@gmail.com) Date: Sat May 2 02:38:25 2009 Subject: C99: Suggestions for style(9) Message-ID: <49FBB295.5030301@gmail.com> Well I agree with the proposed changes. What about allowing // comments? From rick-freebsd2008 at kiwi-computer.com Sat May 2 04:03:30 2009 From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty) Date: Sat May 2 04:03:37 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090430.090226.1569754707.imp@bsdimp.com> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <20090430.090226.1569754707.imp@bsdimp.com> Message-ID: <20090430233648.GA95360@keira.kiwi-computer.com> On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: > > This is the biggest one, and I think it may be too soon. Also, we > need to be careful on the initialization side of things because we > currently have a lot of code that looks like: > > > struct foo *fp; > struct bar *bp; > > fp = get_foo(); > if (!fp) return; > bp = fp->bp; > > this can't easily be translated to the more natural: > > struct foo *fp = get_foo(); > struct bar *bp = fp->bp; > > since really you'd want to write: > > struct foo *fp = get_foo(); > if (!fp) return; > struct bar *bp = fp->bp; > > which isn't legal in 'C'. I thought we were talking about C99, in which case this is perfectly legal. I certainly use it all the time in my C99 code. And I thought this was the point of this discussion, to be able to declare variables when you first use them. -- Rick C. Petty From christoph.mallon at gmx.de Sat May 2 07:27:56 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 07:28:04 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.083712.396385864.imp@bsdimp.com> References: <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> <20090501.081229.1359784281.imp@bsdimp.com> <20090501.083712.396385864.imp@bsdimp.com> Message-ID: <49FBF5F7.7000600@gmx.de> M. Warner Losh schrieb: > In message: <20090501.081229.1359784281.imp@bsdimp.com> > M. Warner Losh writes: > : In message: <49FA8E88.1040905@gmx.de> > : Christoph Mallon writes: > : : M. Warner Losh schrieb: > : : > In message: <20090430233648.GA95360@keira.kiwi-computer.com> > : : > "Rick C. Petty" writes: > : : > : On Thu, Apr 30, 2009 at 09:02:26AM -0600, M. Warner Losh wrote: > : : > : > > : : > : > This is the biggest one, and I think it may be too soon. Also, we > : : > : > need to be careful on the initialization side of things because we > : : > : > currently have a lot of code that looks like: > : : > : > > : : > : > > : : > : > struct foo *fp; > : : > : > struct bar *bp; > : : > : > > : : > : > fp = get_foo(); > : : > : > if (!fp) return; > : : > : > bp = fp->bp; > : : > : > > : : > : > this can't easily be translated to the more natural: > : : > : > > : : > : > struct foo *fp = get_foo(); > : : > : > struct bar *bp = fp->bp; > : : > : > > : : > : > since really you'd want to write: > : : > : > > : : > : > struct foo *fp = get_foo(); > : : > : > if (!fp) return; > : : > : > struct bar *bp = fp->bp; > : : > : > > : : > : > which isn't legal in 'C'. > : : > : > : : > : I thought we were talking about C99, in which case this is perfectly legal. > : : > : I certainly use it all the time in my C99 code. > : : > > : : > Hmmm, looks like that was added. That's ugly as C++... > : : > : : I do not think, this is ugly. On the contrary, it aids maintainability, > : : because it reduces the scope of variables. Also quite some variables are > : : only initialised and not changed afterwards, so it's nice to have the > : : declaration and the only assignment in a single place. IMO this is a > : : quite natural style, because you don't have anything in the code, before > : : it is needed: Get the first pointer; if something is wrong, bail out; > : : get the second pointer - the second pointer does not (textually) exist > : : before it is needed. > : > : It is ugly. Hunting for declarations sucks, and it is one of the > : things I really like about BSD code is that I don't have to. > : > : This is a religious point, and we're dangerously close to saying my > : religion is better than your religion. I don't like this part of the > : proposal at all. I can see the value in relaxing it for when you have > : a series of variables that are initialized, but relaxing it to the > : point where you intermix code and declarations goes way too far. It > : is bad enough to have to deal with inner scopes, but tolerable. It is > : intolerable to have to look for it anywhere in a big function. It > : tends to encourage spaghetti code, which is one of the things that > : style(9) tries to discourage in many subtle ways. > > This came off a little more absolute than I wanted. I should have > added "in the absence of hard data..." There is hard data: For example look at r190098 and the following discussion, which you took part in. Obviously style(9) is too subtle at best or counterproductive at worst to achieve the desired goal. It is worrisome that somebody is inclined to obfuscate the code (in this case replacing multiple variables with descriptive names by a single variable with a generic name) because it is less hassle to conform to style(9) this way. I also have to object, that it leads to hunting for declarations. Actually it usually reduces scrolling around in the code: Many variables are only assigned once. A typical example is getting the softc of a device; especially there the type is important, because device_get_softc() returns void*. So it is very convenient to have this single assignment and its declaration at the same place. Not only the type of a variable is important, but also its value, so this assignment needs to be looked up, too. Doing declaration and initialisation at once you only have to look at one place instead of two. If you use vim as editor, then the current way makes it hard to find this assignment: "gd" jumps only to the declaration, the assignment is somewhere else. But if the declaration and the only assignment are the same, you get both at the same place and time. Even if there are multiple assignments, the first one often is a point of interest, e.g. int found = 0; followed by a search loop. Of course, if you abuse a variable in multiple contexts, then there are multiple "first" assignments and this coupling of declaration and first assignment gets lost. Therefore I am against re-using a variable in multiple contexts. In conclusion, I argue that declaring a variable right at its first (and often only) assignment does not lead to "spaghetti code" or "sloppy code", but in contrary makes the code more concise and easier to navigate. Christoph From christoph.mallon at gmx.de Sat May 2 07:28:37 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 07:28:44 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FBB295.5030301@gmail.com> References: <49FBB295.5030301@gmail.com> Message-ID: <49FBF622.2050208@gmx.de> deeptech71@gmail.com schrieb: > Well I agree with the proposed changes. > > What about allowing // comments? These are already widely used. grep shows hundreds of hits in sys/. Probably a clause should be added to style(9) to allow them "officially", but not right now. Christoph From dwmalone at maths.tcd.ie Sat May 2 07:37:02 2009 From: dwmalone at maths.tcd.ie (David Malone) Date: Sat May 2 07:37:08 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49F4070C.2000108@gmx.de> References: <49F4070C.2000108@gmx.de> Message-ID: <20090502073658.GA65633@walton.maths.tcd.ie> FWIW, I'd rarely support changing style(9), unless it is actually causing people to write bad code. It's designed to produce consistent code, and changing it does not encourage consistency. > >-Do not put declarations > >-inside blocks unless the routine is unusually complicated. > >+Prefer declaring loop iterators in the for-statement to limit their scope. I usually don't like this style - I like being able to review the variables used in a function at the top. > >-When declaring variables in functions declare them sorted by size, > >-then in alphabetical order; multiple ones per line are okay. > >+When declaring variables in functions declare them sorted in alphabetical > >order; > >+prefer one declaration per line, especially when pointer declarators or > >+initializations are involved. The change to prefer one declaration per line seems like an unnecessary, except to better accomodate the declare-at-initialisation below. > >+Prefer initializing variables right at their declaration. While I buy your argument about scope, as I said, I prefer to see everything declared at the top. I don't think it outweighs the advantage of being able to look through the declarations. > >+When the value is not supposed to change in the function, make the > >variable > >+const. Using const seems sensible, and a good example of a time where declaring at initialisation makes sense. > >+This makes it easier for a reader to identify the where the value of a > >variable > >+originates. I'm not sure I buy this - the initialisation is unlikely to move in a piece of code, so it's as hard to find now as it was before. Editors supporting finding declarations should be able to find initialisations just as easily. (I'm old fashioned and do it via regexps.) Note that I also compile code from time to time to versions of FreeBSD that don't support C99. I understand that this will get more difficult over time, but mixing declarations and code is one of the things that I trip over that are just an annoying unnecessary change. For me it makes my life more difficult for a small gain. > >+Do not reuse the same variable in a different context, delare a new > >variable. I buy this largely - though it should be applied with common sense as usual. > >-Values in > >-.Ic return > >-statements should be enclosed in parentheses. > >-.Pp This is another change that isn't worth making - style(9) has had this rule for years and changing it because you don't think it adds anything isn't a good reason. > >-Use ANSI function declarations unless you explicitly need K&R > >compatibility. > >+Use ANSI function declarations. Seems reasonable for modern code - I guess it might be worth noting that when converting from K&R to ANSI changes should be made with care to avoid the sort problems you mention. > (It's a bug in GCC that it does not complain about the former.) I thought gcc had some magic glue to make functions with ANSI declarations and K&R definitions work. Bruce would know the details. > Empty line when there are no local variables: > Removed, because it does not improve readability and it actually hinders > readability for very short functions, e.g. static inline functions, > which just consist of one or two lines of code without any local > variables except for parameters. Further, it makes even less sense with > the changed recommendations for declarations. FWIW, I buy the empty line rule as a matter of consistency. A better reason for getting rid of it (IMHO) would be that it is one of the rules that is not so well followed by our existing code base. On balance I'd leave this rule here, because I don't think the changes to decalarations are good and because changing for no good reason produces less consistent code in the long run. (The example of very short static inline functions is probably a good one where no one would complain anyway if the code was missing the blank line. As always, style needs to be applied with good sense.) > Last, but definitely not least, I added this paragraph about the use of > local variables. This is to clarify, how today's compilers handle > unaliased local variables. There is absolutely no need to reuse them in > different contexts. Trying to optimise stack usage in this way is > misguided and is a source of bugs, when a reused variable is not as dead > as thought. For more reasons, please read the quoted diff. > Maxim, you requested this paragraph: Does this addition suit you? This paragraph looks useful. If I were you I'd hook it in to the recommendation to not reuse the same variable in a different context. The only thing is that it reads a bit more like programming advice rather than style advice. I'd also suggest you run any changes by Bruce. David. From yanefbsd at gmail.com Sat May 2 12:46:54 2009 From: yanefbsd at gmail.com (Garrett Cooper) Date: Sat May 2 12:47:02 2009 Subject: SoC2009: libpkg, pkg tools rewrite In-Reply-To: <20090430214520.GA37974@flint.openpave.org> References: <20090430214520.GA37974@flint.openpave.org> Message-ID: <7d6fde3d0905020546s697ad5f6r2098ff6c80a80e1e@mail.gmail.com> On Thu, Apr 30, 2009 at 2:45 PM, Jeremy Lea wrote: > Hi, > > On Sat, Apr 25, 2009 at 03:20:59PM -0400, David Forsythe wrote: >> This summer I'll be working on creating a package library and using >> that library to rewrite the pkg tools. ?A package library has been >> discussed and even started before, but FreeBSD still does not have >> one. ?This summer I'd like to get enough of the library done to atle >> ast have a new set of pkg tools completed with the current features, >> but ideally I'd like to get far enough to splice in some of the ideas >> I have for new features. > > Since I've already done most of the work on this already, please, > please, please, don't ignore what I have done. ?I know that it is not > perfect, and that it is now five plus years out of date. ?But I have > covered half of the bullet points on your to-do list already. > > The code is at http://sourceforge.net/projects/fpkg and some README > stuff is at http://fpkg.sourceforge.net/ > > Some things which I think are critical: > > 1. The library needs a global "package manager". ?This needs to perform > all of the tasks, and it should ideally do this through a task queue > (which I didn't implement). ?See the lib/lib.h header in FreePKG. > > 2. The package manager must be able to separate out a structure of > variables which can be determined without opening the +CONTENTS file. > This must also include a package state, and the package manager must > move these package headers from state to state. > > 3. There needs to be proper depends handling in the packages, so that we > can maintain the correct dependency graph. ?This is vital for upgrading. > This means adding @libdep and @filedep types to the package file. > > 4. bsd.port.mk should do everything through the tools. ?It should have > no knowledge of the contents of /var/db/pkg. > > These are all done, to some degree in the code. ?The mistakes I made: > > 1. I made the file->pkg database to sensitive. ?If there is a miss it > rebuilds the database for scratch - it should do a search through the > +CONTENTS files and only rebuild it if there was a hit (meaning the > database was wrong). > > 2. There needs to be a pkgname and origin database, which can be loaded > at startup to prime the package manager. ?The dependency graph should > also be stored in a database. ?These should be rebuilt if any directory > in /var/db/pkg has a mtime later than the database (so could the file > database). > > 3. There needs to be a set of flags which indicate how a package got > installed (as a dependency or by the user), and if it has been upgraded > in-place and might have old leftover libraries. ?These could go in > +CONTENTS. > > In addition I also began the design of a new on disk package format. > This was back when there was a group called openpackages which was > trying to standardize ports/packages between the BSDs. ?In particular it > has some ideas on package signing, which is an often requested feature. > > http://people.freebsd.org/~reg/opdesign.html > > Regards, > ?-Jeremy There's also http://libpkg.berlios.de/ that you could use as a starting point, and please, PLEASE, don't neglect the SoC work that was done last year by Anders Nore. It was committed to p4, but hasn't made it upstream to CVS yet... Also, don't fall into the trap I fell into 2 years ago by trying to reinvent the wheel. Plenty of people have been down this road already, and can help you in finding your way if you ask. Best of luck, -Garrett From brampton+freebsd-hackers at gmail.com Sat May 2 15:59:06 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Sat May 2 15:59:13 2009 Subject: Definition of NULL Message-ID: I'm writing a C++ Kernel Module, and one thing that has been bugging me is the kernel's definition of NULL. sys/sys/_null.h (in CURRENT): #if defined(_KERNEL) || !defined(__cplusplus) #define NULL ((void *)0) #else #if defined(__GNUG__) && defined(__GNUC__) && __GNUC__ >= 4 #define NULL __null #else #if defined(__LP64__) #define NULL (0L) #else #define NULL 0 #endif /* __LP64__ */ #endif /* __GNUG__ */ #endif /* _KERNEL || !__cplusplus */ >From what I've read online the definition of NULL in C is (void *)0, whereas in C++ it should be 0, or 0L (on 64bit machines). Now, my C++ kernel module is built with _KERNEL definited, like any other C kernel module. This leads to NULL being defined incorrectly. So I have a question and two suggestions. Firstly, why is the #if defined(_KERNEL) in _null.h? Is it to stop userland application applications picking up this definition? Or for another reason? and two, how about we change the first line of _null.h so that we use a && instead of a || like so: #if defined(_KERNEL) && !defined(__cplusplus) That should ensure the definition is correct. Or, a more radical approach, we could remove the check for _KERNEL, since I can't figure out why it is needed and do something like: #if defined(__GNUG__) && defined(__GNUC__) && __GNUC__ >= 4 # define NULL __null #elif !defined(__cplusplus) # define NULL ((void *)0) #elif defined(__LP64__) # define NULL (0L) #else # define NULL 0 #endif That way, if we are using GCC 4+ we use their __null definition, otherwise if we are not c++ we use the standard (void *)0, and then if we are 64bit we use 0L, and finally anything else uses 0. A quick amd64 kernel compile seems to allow my new definition I hope this makes sense, and I welcome all feedback. Andrew From christoph.mallon at gmx.de Sat May 2 16:15:18 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 16:15:26 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090502073658.GA65633@walton.maths.tcd.ie> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> Message-ID: <49FC7193.40200@gmx.de> David Malone schrieb: >>> +When the value is not supposed to change in the function, make the >>> variable >>> +const. > > Using const seems sensible, and a good example of a time where > declaring at initialisation makes sense. > >>> +This makes it easier for a reader to identify the where the value of a >>> variable >>> +originates. > > I'm not sure I buy this - the initialisation is unlikely to move in > a piece of code, so it's as hard to find now as it was before. Editors > supporting finding declarations should be able to find initialisations > just as easily. (I'm old fashioned and do it via regexps.) But why not combine the benefits and find both at once in one place instead of of having an arbitrary amount of code between them? For example vim used "gd" to go to a local declaration. > Note that I also compile code from time to time to versions of > FreeBSD that don't support C99. I understand that this will get > more difficult over time, but mixing declarations and code is one > of the things that I trip over that are just an annoying unnecessary > change. For me it makes my life more difficult for a small gain. This is one thing, which I don't understand from the point of view of compiler construction: It is really simple to handle declarations, which are not at the beginning of a block. Other C99 features are way harder to implement. E.g. designator initialisers (bla bli = { .blub[23].gna = { 42, "narf" } };) require *way* more effort. In the C99 standard initialisation covers 3 pages followed by another 3 pages of examples - which are quite some explanations and a *lot* of examples compared to the other parts of the standard. >>> +Do not reuse the same variable in a different context, delare a new >>> variable. > > I buy this largely - though it should be applied with common sense > as usual. Of course, every rule has to be applied with common sense. >>> -Use ANSI function declarations unless you explicitly need K&R >>> compatibility. >>> +Use ANSI function declarations. > > Seems reasonable for modern code - I guess it might be worth noting > that when converting from K&R to ANSI changes should be made with > care to avoid the sort problems you mention. Removing the rule is in the first place about not adding any more old-style declarations. Removing the existing ones is a separate matter. rdivacky@ already did a good job at removing the defective declarations. >> (It's a bug in GCC that it does not complain about the former.) > > I thought gcc had some magic glue to make functions with ANSI > declarations and K&R definitions work. Bruce would know the details. The standard explains in detail, what are compatible declarations. ISO/IEC 9899:1999 (E) ?6.7.5.3:15 handles compatible function types. GCC plain violates this. >> Empty line when there are no local variables: >> Removed, because it does not improve readability and it actually hinders >> readability for very short functions, e.g. static inline functions, >> which just consist of one or two lines of code without any local >> variables except for parameters. Further, it makes even less sense with >> the changed recommendations for declarations. > > FWIW, I buy the empty line rule as a matter of consistency. A better > reason for getting rid of it (IMHO) would be that it is one of the > rules that is not so well followed by our existing code base. On > balance I'd leave this rule here, because I don't think the changes > to decalarations are good and because changing for no good reason > produces less consistent code in the long run. Why leave it as it is? Just remove it, so no more of these empty lines are added. There is no need to remove the existing ones at once. Just slowly get rid of them. > (The example of very short static inline functions is probably a > good one where no one would complain anyway if the code was missing > the blank line. As always, style needs to be applied with good sense.) > >> Last, but definitely not least, I added this paragraph about the use of >> local variables. This is to clarify, how today's compilers handle >> unaliased local variables. There is absolutely no need to reuse them in >> different contexts. Trying to optimise stack usage in this way is >> misguided and is a source of bugs, when a reused variable is not as dead >> as thought. For more reasons, please read the quoted diff. >> Maxim, you requested this paragraph: Does this addition suit you? > > This paragraph looks useful. If I were you I'd hook it in to the > recommendation to not reuse the same variable in a different context. It is rather long, so it should be kept separate. > The only thing is that it reads a bit more like programming advice > rather than style advice. It has two sides: On the one side it is easier for a reader to understand the data flow. On the other side if aliasing comes into play (taking the address of a variable), then it is also beneficial from the point of view of code generation to keep different contexts separate. My main concern is the reader of a piece of code. Better generated code is a nice side effect. It also shows that clarity for a reader and quality of the generated code nicely correlate here. Christoph From julian at elischer.org Sat May 2 16:30:38 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 2 16:30:46 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090502073658.GA65633@walton.maths.tcd.ie> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> Message-ID: <49FC752D.5080807@elischer.org> David Malone wrote: > FWIW, I'd rarely support changing style(9), unless it is actually > causing people to write bad code. It's designed to produce consistent > code, and changing it does not encourage consistency. > >>> -Do not put declarations >>> -inside blocks unless the routine is unusually complicated. >>> +Prefer declaring loop iterators in the for-statement to limit their scope. > > I usually don't like this style - I like being able to review the > variables used in a function at the top. me too though I've softenned a bit regarding puting variables in blocks, I do NOT want to see variable declaration "not at the top of a block". > >>> -When declaring variables in functions declare them sorted by size, >>> -then in alphabetical order; multiple ones per line are okay. >>> +When declaring variables in functions declare them sorted in alphabetical >>> order; >>> +prefer one declaration per line, especially when pointer declarators or >>> +initializations are involved. > > The change to prefer one declaration per line seems like an unnecessary, > except to better accomodate the declare-at-initialisation below. the 'size' bit used to be to save space.. all the chars would be grouped together, etc. > >>> +Prefer initializing variables right at their declaration. > I have gone the other way on this. Unless 'const' is being used I find I prefer to see them separately initialized. > While I buy your argument about scope, as I said, I prefer to see > everything declared at the top. I don't think it outweighs the > advantage of being able to look through the declarations. > >>> +When the value is not supposed to change in the function, make the >>> variable >>> +const. > > Using const seems sensible, and a good example of a time where > declaring at initialisation makes sense. > >>> +This makes it easier for a reader to identify the where the value of a >>> variable >>> +originates. yeah.. at the top of the function.. > > I'm not sure I buy this - the initialisation is unlikely to move in > a piece of code, so it's as hard to find now as it was before. Editors > supporting finding declarations should be able to find initialisations > just as easily. (I'm old fashioned and do it via regexps.) > > Note that I also compile code from time to time to versions of > FreeBSD that don't support C99. I understand that this will get > more difficult over time, but mixing declarations and code is one > of the things that I trip over that are just an annoying unnecessary > change. For me it makes my life more difficult for a small gain. > >>> +Do not reuse the same variable in a different context, delare a new >>> variable. > > I buy this largely - though it should be applied with common sense > as usual. hmm. I think an exception might be made for our old friends i,j,k everyone knows them and what they are doing.. otherwise,yes this makes sense. > >>> -Values in >>> -.Ic return >>> -statements should be enclosed in parentheses. >>> -.Pp > > This is another change that isn't worth making - style(9) has had > this rule for years and changing it because you don't think it adds > anything isn't a good reason. > amen >>> -Use ANSI function declarations unless you explicitly need K&R >>> compatibility. >>> +Use ANSI function declarations. > Everyone shuld be doing that already. In new code, and in old code that they touch. > Seems reasonable for modern code - I guess it might be worth noting > that when converting from K&R to ANSI changes should be made with > care to avoid the sort problems you mention. > >> (It's a bug in GCC that it does not complain about the former.) > > I thought gcc had some magic glue to make functions with ANSI > declarations and K&R definitions work. Bruce would know the details. > >> Empty line when there are no local variables: >> Removed, because it does not improve readability and it actually hinders >> readability for very short functions, e.g. static inline functions, >> which just consist of one or two lines of code without any local >> variables except for parameters. Further, it makes even less sense with >> the changed recommendations for declarations. > > FWIW, I buy the empty line rule as a matter of consistency. A better > reason for getting rid of it (IMHO) would be that it is one of the > rules that is not so well followed by our existing code base. On > balance I'd leave this rule here, because I don't think the changes > to decalarations are good and because changing for no good reason > produces less consistent code in the long run. > > (The example of very short static inline functions is probably a > good one where no one would complain anyway if the code was missing > the blank line. As always, style needs to be applied with good sense.) > >> Last, but definitely not least, I added this paragraph about the use of >> local variables. This is to clarify, how today's compilers handle >> unaliased local variables. There is absolutely no need to reuse them in >> different contexts. Trying to optimise stack usage in this way is >> misguided and is a source of bugs, when a reused variable is not as dead >> as thought. For more reasons, please read the quoted diff. >> Maxim, you requested this paragraph: Does this addition suit you? > > This paragraph looks useful. If I were you I'd hook it in to the > recommendation to not reuse the same variable in a different context. > The only thing is that it reads a bit more like programming advice > rather than style advice. > > I'd also suggest you run any changes by Bruce. > > David. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From ertr1013 at student.uu.se Sat May 2 16:50:53 2009 From: ertr1013 at student.uu.se (Erik Trulsson) Date: Sat May 2 16:50:59 2009 Subject: Definition of NULL In-Reply-To: References: Message-ID: <20090502163535.GA17027@owl.midgard.homeip.net> On Sat, May 02, 2009 at 04:59:03PM +0100, Andrew Brampton wrote: > I'm writing a C++ Kernel Module, and one thing that has been bugging > me is the kernel's definition of NULL. Is the use of C++ inside the kernel really supported? I don't think so, but I could be wrong. > > sys/sys/_null.h (in CURRENT): > > #if defined(_KERNEL) || !defined(__cplusplus) > #define NULL ((void *)0) > #else > #if defined(__GNUG__) && defined(__GNUC__) && __GNUC__ >= 4 > #define NULL __null > #else > #if defined(__LP64__) > #define NULL (0L) > #else > #define NULL 0 > #endif /* __LP64__ */ > #endif /* __GNUG__ */ > #endif /* _KERNEL || !__cplusplus */ > > >From what I've read online the definition of NULL in C is (void *)0, > whereas in C++ it should be 0, or 0L (on 64bit machines). Not quite. Any of those (as well as a whole bunch more) are legal definitions of NULL in C. NULL is defined (in the C standard) to be a null pointer constant. A null pointer constant is defined as a constant integer expression with value zero, or such an expression cast to (void*). (In C++ it the cast to (void*) is not allowed.) This means that it would be perfectly legal (but of dubious utility) to have NULL defined as (5*5L+('1'-'0')-26) for example. The decision to define NULL as 0 or 0L or ((void*)0) is pretty much just a question of which buggy programs one wishes to break, or hide the bugs in. A correct C program should work regardless of which of those is used. > > Now, my C++ kernel module is built with _KERNEL definited, like any > other C kernel module. This leads to NULL being defined incorrectly. > > So I have a question and two suggestions. Firstly, why is the #if > defined(_KERNEL) in _null.h? Is it to stop userland application > applications picking up this definition? Or for another reason? Perhaps to stop people from mistakenly using C++ inside the kernel? > > and two, how about we change the first line of _null.h so that we use > a && instead of a || like so: > #if defined(_KERNEL) && !defined(__cplusplus) > > That should ensure the definition is correct. Or, a more radical > approach, we could remove the check for _KERNEL, since I can't figure > out why it is needed and do something like: > > #if defined(__GNUG__) && defined(__GNUC__) && __GNUC__ >= 4 > # define NULL __null > #elif !defined(__cplusplus) > # define NULL ((void *)0) > #elif defined(__LP64__) > # define NULL (0L) > #else > # define NULL 0 > #endif > > That way, if we are using GCC 4+ we use their __null definition, > otherwise if we are not c++ we use the standard (void *)0, and then if > we are 64bit we use 0L, and finally anything else uses 0. A quick > amd64 kernel compile seems to allow my new definition If you want to keep things simple you could just define NULL as 0 everywhere, and see what bugs are exposed that way. > > I hope this makes sense, and I welcome all feedback. > Andrew -- Erik Trulsson ertr1013@student.uu.se From brampton+freebsd-hackers at gmail.com Sat May 2 16:52:41 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Sat May 2 16:52:48 2009 Subject: Definition of NULL In-Reply-To: <20090502163535.GA17027@owl.midgard.homeip.net> References: <20090502163535.GA17027@owl.midgard.homeip.net> Message-ID: 2009/5/2 Erik Trulsson : > On Sat, May 02, 2009 at 04:59:03PM +0100, Andrew Brampton wrote: >> I'm writing a C++ Kernel Module, and one thing that has been bugging >> me is the kernel's definition of NULL. > > Is the use of C++ inside the kernel really supported? ?I don't think so, > but I could be wrong. There are a few projects (other than mine) which uses C++ inside the kernel, the biggest being the Click modular router. So, with minimal effort and no kernel patches it is easy to build a C++ kernel module, thus I must assume it is supported even if it is unofficial. The question of if C++ is sensible inside the kernel is left for another day, and perhaps has been answered in numerous other freebsd-hackers@ threads. >> >From what I've read online the definition of NULL in C is (void *)0, >> whereas in C++ it should be 0, or 0L (on 64bit machines). > > Not quite. ?Any of those (as well as a whole bunch more) are legal > definitions of NULL in C. ?NULL is defined (in the C standard) to be a null > pointer constant. ?A null pointer constant is defined as a constant integer > expression with value zero, or such an expression cast to (void*). (In C++ > it the cast to (void*) is not allowed.) > This means that it would be perfectly legal (but of dubious utility) to have > NULL defined as (5*5L+('1'-'0')-26) for example. > > The decision to define NULL as 0 or 0L or ((void*)0) is pretty much just a > question of which buggy programs one wishes to break, or hide the bugs in. > A correct C program should work regardless of which of those is used. Thanks for the additional information. > > >> >> Now, my C++ kernel module is built with _KERNEL definited, like any >> other C kernel module. This leads to NULL being defined incorrectly. >> >> So I have a question and two suggestions. Firstly, why is the #if >> defined(_KERNEL) in _null.h? Is it to stop userland application >> applications picking up this definition? Or for another reason? > > Perhaps to stop people from mistakenly using C++ inside the kernel? The definition doesn't stop C++, it just generates additional warnings and sometimes errors. I have avoided those warning by defining NULL myself, however, I thought changing it in _null.h might help others, even if you think they are mistakenly using C++. > > ... > > Erik Trulsson > ertr1013@student.uu.se > Thanks for your input. Andrew From dwmalone at maths.tcd.ie Sat May 2 17:08:36 2009 From: dwmalone at maths.tcd.ie (David Malone) Date: Sat May 2 17:08:43 2009 Subject: C99: Suggestions for style(9) In-Reply-To: Your message of "Sat, 02 May 2009 18:15:15 +0200." <49FC7193.40200@gmx.de> Message-ID: <200905021738.aa71693@walton.maths.tcd.ie> > > I'm not sure I buy this - the initialisation is unlikely to move in > > a piece of code, so it's as hard to find now as it was before. Editors > > supporting finding declarations should be able to find initialisations > > just as easily. (I'm old fashioned and do it via regexps.) > But why not combine the benefits and find both at once in one place > instead of of having an arbitrary amount of code between them? For > example vim used "gd" to go to a local declaration. As I said, I like having all the declarations in one place so I can review all of them as a unit. I find it helps me understand the function. You can never have all the initialisations in one place, because they may depend on complicated code, so I accept that I have to search for initialisations. > > Note that I also compile code from time to time to versions of > > FreeBSD that don't support C99. I understand that this will get > > more difficult over time, but mixing declarations and code is one > > of the things that I trip over that are just an annoying unnecessary > > change. For me it makes my life more difficult for a small gain. > This is one thing, which I don't understand from the point of view of > compiler construction: It is really simple to handle declarations, which > are not at the beginning of a block. Other C99 features are way harder > to implement. E.g. designator initialisers (bla bli = { .blub[23].gna = > { 42, "narf" } };) require *way* more effort. In the C99 standard > initialisation covers 3 pages followed by another 3 pages of examples - > which are quite some explanations and a *lot* of examples compared to > the other parts of the standard. True - but that's the compilers that I have to work with. I've tripped over other C99 features considerably less often for some reason. > > Seems reasonable for modern code - I guess it might be worth noting > > that when converting from K&R to ANSI changes should be made with > > care to avoid the sort problems you mention. > Removing the rule is in the first place about not adding any more > old-style declarations. Removing the existing ones is a separate matter. > rdivacky@ already did a good job at removing the defective declarations. Indeed! > > FWIW, I buy the empty line rule as a matter of consistency. A better > > reason for getting rid of it (IMHO) would be that it is one of the > > rules that is not so well followed by our existing code base. On > > balance I'd leave this rule here, because I don't think the changes > > to decalarations are good and because changing for no good reason > > produces less consistent code in the long run. > Why leave it as it is? Just remove it, so no more of these empty lines > are added. There is no need to remove the existing ones at once. Just > slowly get rid of them. As I said, the point of a style guide is consistency. Changing it doesn't promote consistency, so there needs to be a good reason for the change, rather than a good reason not to change. In my opinion, there isn't a strong reason. Similarly for parens around return values - there's nothing (substantial) wrong with either rule, so they should probably be left as-is. > > This paragraph looks useful. If I were you I'd hook it in to the > > recommendation to not reuse the same variable in a different context. > It is rather long, so it should be kept separate. Maybe include a forward ref then, to say the rational is explained in a later section? David. From christoph.mallon at gmx.de Sat May 2 17:40:08 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 17:40:23 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC752D.5080807@elischer.org> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> Message-ID: <49FC8574.5060205@gmx.de> Julian Elischer schrieb: > David Malone wrote: >>>> -Do not put declarations >>>> -inside blocks unless the routine is unusually complicated. >>>> +Prefer declaring loop iterators in the for-statement to limit their >>>> scope. >> >> I usually don't like this style - I like being able to review the >> variables used in a function at the top. > > me too > though I've softenned a bit regarding puting variables in blocks, > I do NOT want to see variable declaration "not at the top of a block". I find this kind of strange, because this in-beetwen makes least sense to me: Neither are all declarations grouped together nor are declarations nicely connected to their initialisation and their scope is minimal. >>>> -When declaring variables in functions declare them sorted by size, >>>> -then in alphabetical order; multiple ones per line are okay. >>>> +When declaring variables in functions declare them sorted in >>>> alphabetical order; >>>> +prefer one declaration per line, especially when pointer >>>> declarators or +initializations are involved. >> >> The change to prefer one declaration per line seems like an unnecessary, >> except to better accomodate the declare-at-initialisation below. > > the 'size' bit used to be to save space.. all the chars > would be grouped together, etc. Today's compilers plain do not care in which order you declare variables. Sorting them in a beneficial way for space efficiency is better left to them (and it is a rather trivial thing to do). Also you cannot control if more spill slots have to be inserted or some values do not live in memory at all, so only the compiler can determine, which order is best. But a compiler can do more: It could coalesce the space of variables with disjoint life ranges. But this is much harder, because you have to properly determine the life ranges. Finding an arbitrary overestimation is easy, but the fun starts when looking at called functions, which get passed addresses of local variables. Also finding an optimal coalescence for known life ranges is NP-complete in the general case. But that's another story. (: >>>> +Prefer initializing variables right at their declaration. >> > > I have gone the other way on this. Unless 'const' is being used I find > I prefer to see them separately initialized. So you like hunting in multiple places instead of having both type and value conveniently in one place? >>>> +This makes it easier for a reader to identify the where the value >>>> of a variable >>>> +originates. > > yeah.. at the top of the function.. Currently only the *declaration* is at the top of the function. This bears absolutely no relation to where the *value* originates. >>>> +Do not reuse the same variable in a different context, delare a new >>>> variable. >> >> I buy this largely - though it should be applied with common sense >> as usual. > > hmm. I think an exception might be made for our old friends i,j,k Especially they should be declared right where they are needed. C99 even conveniently allows the iterator variable to be declared in the first part of a for-loop: for (int i = 0; i != n; ++i) { ... }. By limiting their scope, this prevents accidently re-using stale values in a different place or other mistakes like the following: int i, j; for (i = 0;; ++i) { for (i = 0;; ++i) { /* oops, but compiler is happy */ } } for (j = 0;; ++j) { /* more */ } vs. for (int i = 0;; ++i) { for (int i = 0;; ++i) { /* warning about shadowed variable */ } } for (int j = 0;; ++j) { /* more */ } > everyone knows them and what they are doing.. otherwise,yes this makes > sense. > >> >>>> -Values in >>>> -.Ic return >>>> -statements should be enclosed in parentheses. >>>> -.Pp >> >> This is another change that isn't worth making - style(9) has had >> this rule for years and changing it because you don't think it adds >> anything isn't a good reason. >> > > amen This is no church. I presented reasons, I do not want beliefs in return. Also stating that it should not be changed because it was always this way is a very slippery slope. >>>> -Use ANSI function declarations unless you explicitly need K&R >>>> compatibility. >>>> +Use ANSI function declarations. >> > > Everyone shuld be doing that already. > In new code, and in old code that they touch. So, is this pro or contra removing the K&R-clause? Christoph From julian at elischer.org Sat May 2 17:40:44 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 2 17:40:50 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <200905021738.aa71693@walton.maths.tcd.ie> References: <200905021738.aa71693@walton.maths.tcd.ie> Message-ID: <49FC859B.20906@elischer.org> David Malone wrote: > > As I said, the point of a style guide is consistency. Changing it > doesn't promote consistency, so there needs to be a good reason for > the change, rather than a good reason not to change. In my opinion, > there isn't a strong reason. Similarly for parens around return > values - there's nothing (substantial) wrong with either rule, so > they should probably be left as-is. Many of the previous changes to style(9) came from cases where the existing rules led to programmers increasing their chances of making errors, or people found themselves often misinterpreting code in the same way, over and over again. Changing them to bring them closer to someone's personal style has never been an acceptable reason. There is a barrier to entry that any change must overcome which is "It will decrease code consistency (by definition) so it it worth it?" . "parens on return statements is an example.. there are some posibilities as to what one can do with this, but I really do find that the parens are part of the style that I'd notice as inconsistent. > From julian at elischer.org Sat May 2 17:55:43 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 2 17:55:51 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC8574.5060205@gmx.de> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> Message-ID: <49FC8920.6050408@elischer.org> Christoph Mallon wrote: > Julian Elischer schrieb: >> David Malone wrote: >>>>> -Do not put declarations >>>>> -inside blocks unless the routine is unusually complicated. >>>>> +Prefer declaring loop iterators in the for-statement to limit >>>>> their scope. >>> >>> I usually don't like this style - I like being able to review the >>> variables used in a function at the top. >> >> me too >> though I've softenned a bit regarding puting variables in blocks, >> I do NOT want to see variable declaration "not at the top of a block". > > I find this kind of strange, because this in-beetwen makes least sense > to me: Neither are all declarations grouped together nor are > declarations nicely connected to their initialisation and their scope is > minimal. I only have to look just after the enclosing "{" I actually prefer to see them all at the top but as I said I have seen cases where having them elsewhere makes sense. > >>>>> -When declaring variables in functions declare them sorted by size, >>>>> -then in alphabetical order; multiple ones per line are okay. >>>>> +When declaring variables in functions declare them sorted in >>>>> alphabetical order; >>>>> +prefer one declaration per line, especially when pointer >>>>> declarators or +initializations are involved. >>> >>> The change to prefer one declaration per line seems like an unnecessary, >>> except to better accomodate the declare-at-initialisation below. >> >> the 'size' bit used to be to save space.. all the chars >> would be grouped together, etc. > > Today's compilers plain do not care in which order you declare I said "used to" > variables. Sorting them in a beneficial way for space efficiency is > better left to them (and it is a rather trivial thing to do). Also you > cannot control if more spill slots have to be inserted or some values do > not live in memory at all, so only the compiler can determine, which > order is best. > But a compiler can do more: It could coalesce the space of variables > with disjoint life ranges. But this is much harder, because you have to > properly determine the life ranges. Finding an arbitrary overestimation > is easy, but the fun starts when looking at called functions, which get > passed addresses of local variables. Also finding an optimal coalescence > for known life ranges is NP-complete in the general case. But that's > another story. (: re-using space or having block-local variables means that when you are in the debugger, you can not look at the final value that a variable had when you left (unexpectedly) the block. This can be a pain in the neck. (but only a debugging feature) > >>>>> +Prefer initializing variables right at their declaration. >>> >> >> I have gone the other way on this. Unless 'const' is being used I >> find I prefer to see them separately initialized. > > So you like hunting in multiple places instead of having both type and > value conveniently in one place? Actually.. yes unconditional initialization is right at the top, AFTER the blank line. Conditional initialization must of course come after the condition, and can not be in the same line as the declaration anyhow. > >>>>> +This makes it easier for a reader to identify the where the value >>>>> of a variable >>>>> +originates. >> >> yeah.. at the top of the function.. > > Currently only the *declaration* is at the top of the function. This > bears absolutely no relation to where the *value* originates. As I said.. unconditional values originate just after that, at the top. :-) > >>>>> +Do not reuse the same variable in a different context, delare a >>>>> new variable. >>> >>> I buy this largely - though it should be applied with common sense >>> as usual. >> >> hmm. I think an exception might be made for our old friends i,j,k > > Especially they should be declared right where they are needed. C99 even > conveniently allows the iterator variable to be declared in the first > part of a for-loop: for (int i = 0; i != n; ++i) { ... }. By limiting > their scope, this prevents accidently re-using stale values in a > different place or other mistakes like the following: > int i, j; > for (i = 0;; ++i) { > for (i = 0;; ++i) { > /* oops, but compiler is happy */ > } > } > for (j = 0;; ++j) { > /* more */ > } > vs. > for (int i = 0;; ++i) { > for (int i = 0;; ++i) { > /* warning about shadowed variable */ > } > } > for (int j = 0;; ++j) { > /* more */ > } while tempting, I can't see that hapenning.. it stops teh following: for (;;) { blah if (foo) break; } if (i == end_condition) { then we didn't break out; } > >> everyone knows them and what they are doing.. otherwise,yes this makes >> sense. >> >>> >>>>> -Values in >>>>> -.Ic return >>>>> -statements should be enclosed in parentheses. >>>>> -.Pp >>> >>> This is another change that isn't worth making - style(9) has had >>> this rule for years and changing it because you don't think it adds >>> anything isn't a good reason. >>> >> >> amen > > This is no church. I presented reasons, I do not want beliefs in return. > Also stating that it should not be changed because it was always this > way is a very slippery slope. > >>>>> -Use ANSI function declarations unless you explicitly need K&R >>>>> compatibility. >>>>> +Use ANSI function declarations. >>> >> >> Everyone shuld be doing that already. >> In new code, and in old code that they touch. > > So, is this pro or contra removing the K&R-clause? K&R code should be changed as part of related changes if possible. A sweep to change a whole file is probably also ok. changing them one at a time is probably not ok. > > Christoph From christoph.mallon at gmx.de Sat May 2 18:19:03 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 18:19:10 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC8920.6050408@elischer.org> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> Message-ID: <49FC8E92.1090207@gmx.de> Julian Elischer schrieb: > Christoph Mallon wrote: >> variables. Sorting them in a beneficial way for space efficiency is >> better left to them (and it is a rather trivial thing to do). Also you >> cannot control if more spill slots have to be inserted or some values >> do not live in memory at all, so only the compiler can determine, >> which order is best. >> But a compiler can do more: It could coalesce the space of variables >> with disjoint life ranges. But this is much harder, because you have >> to properly determine the life ranges. Finding an arbitrary >> overestimation is easy, but the fun starts when looking at called >> functions, which get passed addresses of local variables. Also finding >> an optimal coalescence for known life ranges is NP-complete in the >> general case. But that's another story. (: > > re-using space or having block-local variables means that when you > are in the debugger, you can not look at the final value that > a variable had when you left (unexpectedly) the block. This can > be a pain in the neck. (but only a debugging feature) I'm talking about an optimized build - no matter what the style of the original source was, you will have a hard time debugging it. >>>>>> +Prefer initializing variables right at their declaration. >>>> >>> >>> I have gone the other way on this. Unless 'const' is being used I >>> find I prefer to see them separately initialized. >> >> So you like hunting in multiple places instead of having both type and >> value conveniently in one place? > > Actually.. yes So at the one hand you argue that hunting things is bad, but at the same time you prefer it? I am confused. > unconditional initialization is right at the top, AFTER the blank line. > Conditional initialization must of course come after the condition, and > can not be in the same line as the declaration anyhow. It is perfectly valid to do so. Warner presented a simple example. You just have to limit the scope of the variable, which is a good for other reasons, which were explained, too. >>>>>> +Do not reuse the same variable in a different context, delare a >>>>>> new variable. >>>> >>>> I buy this largely - though it should be applied with common sense >>>> as usual. >>> >>> hmm. I think an exception might be made for our old friends i,j,k >> >> Especially they should be declared right where they are needed. C99 >> even conveniently allows the iterator variable to be declared in the >> first part of a for-loop: for (int i = 0; i != n; ++i) { ... }. By >> limiting their scope, this prevents accidently re-using stale values >> in a different place or other mistakes like the following: >> int i, j; >> for (i = 0;; ++i) { >> for (i = 0;; ++i) { >> /* oops, but compiler is happy */ >> } >> } >> for (j = 0;; ++j) { >> /* more */ >> } >> vs. >> for (int i = 0;; ++i) { >> for (int i = 0;; ++i) { >> /* warning about shadowed variable */ >> } >> } >> for (int j = 0;; ++j) { >> /* more */ >> } > > while tempting, I can't see that hapenning.. > > it stops teh following: > > > for (;;) { > blah > if (foo) > break; > } > if (i == end_condition) { > then we didn't break out; > } You reject a common case because of one special case? This is sad. (Also there are multiple ways to resolve this nicely, e.g. split the loop into a function or invert the condition and restructure the code slightly) >>>>>> -Use ANSI function declarations unless you explicitly need K&R >>>>>> compatibility. >>>>>> +Use ANSI function declarations. >>>> >>> >>> Everyone shuld be doing that already. >>> In new code, and in old code that they touch. >> >> So, is this pro or contra removing the K&R-clause? > > K&R code should be changed as part of related changes if possible. > A sweep to change a whole file is probably also ok. > changing them one at a time is probably not ok. But this is what actually is practiced. You still did not answer my question: Do you agree to remove the clause so no new old style declarations may be added? Christoph From christoph.mallon at gmx.de Sat May 2 18:30:57 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 18:31:04 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC859B.20906@elischer.org> References: <200905021738.aa71693@walton.maths.tcd.ie> <49FC859B.20906@elischer.org> Message-ID: <49FC915E.4050006@gmx.de> Julian Elischer schrieb: > David Malone wrote: >> As I said, the point of a style guide is consistency. Changing it >> doesn't promote consistency, so there needs to be a good reason for >> the change, rather than a good reason not to change. In my opinion, >> there isn't a strong reason. Similarly for parens around return >> values - there's nothing (substantial) wrong with either rule, so >> they should probably be left as-is. > > Many of the previous changes to style(9) came from cases where > the existing rules led to programmers increasing their chances > of making errors, or people found themselves often misinterpreting > code in the same way, over and over again. Changing them to bring > them closer to someone's personal style has never been an acceptable > reason. There is a barrier to entry that any change must overcome > which is "It will decrease code consistency (by definition) so > it it worth it?" . For every proposed changed, I presented comprehensible reasons why I propose it. I never used "because I like it" as a pseudo-reason. Yes, these changes reflect my personal preferences, but they are my preferences for a reason and I try really hard to convey these reasons. > "parens on return statements is an example.. there are some posibilities > as to what one can do with this, but I really do find that the parens > are part of the style that I'd notice as inconsistent. What does this mean in one simple sentence? Should the clause be removed so they may not be added anymore? Christoph From julian at elischer.org Sat May 2 19:05:44 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 2 19:05:50 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC8E92.1090207@gmx.de> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> <49FC8E92.1090207@gmx.de> Message-ID: <49FC9988.2070306@elischer.org> Christoph Mallon wrote: > > I'm talking about an optimized build - no matter what the style of the > original source was, you will have a hard time debugging it. but by removing -Ox I can debug it and you can't From julian at elischer.org Sat May 2 19:07:46 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 2 19:07:53 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC8E92.1090207@gmx.de> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> <49FC8E92.1090207@gmx.de> Message-ID: <49FC9A02.6080901@elischer.org> Christoph Mallon wrote: > Julian Elischer schrieb: >> Christoph Mallon wrote: > > So at the one hand you argue that hunting things is bad, but at the same > time you prefer it? I am confused. well, I won't hold your problems against you.. :-) From christoph.mallon at gmx.de Sat May 2 19:38:51 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 19:38:58 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC9A02.6080901@elischer.org> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> <49FC8E92.1090207@gmx.de> <49FC9A02.6080901@elischer.org> Message-ID: <49FCA148.9060707@gmx.de> Julian Elischer schrieb: > Christoph Mallon wrote: >> Julian Elischer schrieb: >>> Christoph Mallon wrote: > >> >> So at the one hand you argue that hunting things is bad, but at the >> same time you prefer it? I am confused. > > well, I won't hold your problems against you.. :-) It is sad that you are just toying around instead of answering a simple question: Christoph Mallon wrote: >> K&R code should be changed as part of related changes if possible. >> A sweep to change a whole file is probably also ok. >> changing them one at a time is probably not ok. > > But this is what actually is practiced. > You still did not answer my question: Do you agree to remove the clause so no new old style declarations may be added? From christoph.mallon at gmx.de Sat May 2 19:39:59 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 19:40:06 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FC9988.2070306@elischer.org> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> <49FC8E92.1090207@gmx.de> <49FC9988.2070306@elischer.org> Message-ID: <49FCA18B.6060502@gmx.de> Julian Elischer schrieb: > Christoph Mallon wrote: >> I'm talking about an optimized build - no matter what the style of the >> original source was, you will have a hard time debugging it. > > but by removing -Ox I can debug it and you can't Declaring variables earlier does not magically fill meaningful values into them any earlier. Christoph From julian at elischer.org Sat May 2 20:16:27 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 2 20:16:34 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FCA148.9060707@gmx.de> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> <49FC8E92.1090207@gmx.de> <49FC9A02.6080901@elischer.org> <49FCA148.9060707@gmx.de> Message-ID: <49FCAA1D.1080208@elischer.org> Christoph Mallon wrote: > Julian Elischer schrieb: >> Christoph Mallon wrote: >>> Julian Elischer schrieb: >>>> Christoph Mallon wrote: >> >>> >>> So at the one hand you argue that hunting things is bad, but at the >>> same time you prefer it? I am confused. >> >> well, I won't hold your problems against you.. :-) > > It is sad that you are just toying around instead of answering a simple > question: > > Christoph Mallon wrote: >>> K&R code should be changed as part of related changes if possible. >>> A sweep to change a whole file is probably also ok. >>> changing them one at a time is probably not ok. >> >> But this is what actually is practiced. >> You still did not answer my question: Do you agree to remove the >> clause so no new old style declarations may be added? I think a new clause should be added specifying what should happen and replacing the old clause. From christoph.mallon at gmx.de Sat May 2 20:40:05 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sat May 2 20:40:12 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FCAA1D.1080208@elischer.org> References: <49F4070C.2000108@gmx.de> <20090502073658.GA65633@walton.maths.tcd.ie> <49FC752D.5080807@elischer.org> <49FC8574.5060205@gmx.de> <49FC8920.6050408@elischer.org> <49FC8E92.1090207@gmx.de> <49FC9A02.6080901@elischer.org> <49FCA148.9060707@gmx.de> <49FCAA1D.1080208@elischer.org> Message-ID: <49FCAFA2.60603@gmx.de> Julian Elischer schrieb: >> Christoph Mallon wrote: >>>> K&R code should be changed as part of related changes if possible. >>>> A sweep to change a whole file is probably also ok. >>>> changing them one at a time is probably not ok. >>> >>> But this is what actually is practiced. >>> You still did not answer my question: Do you agree to remove the >>> clause so no new old style declarations may be added? > > I think a new clause should be added specifying what should happen > and replacing the old clause. This is not sensible. style(9) says right at the start that it "[...] specifies the preferred style for kernel source files [...]". The preferred style would be to use ANSI function declarations - what else is there to say? There is no point in adding more when less is sufficient. From imp at bsdimp.com Sat May 2 21:21:21 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Sat May 2 21:21:27 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FCAFA2.60603@gmx.de> References: <49FCA148.9060707@gmx.de> <49FCAA1D.1080208@elischer.org> <49FCAFA2.60603@gmx.de> Message-ID: <20090502.151931.1396014860.imp@bsdimp.com> In message: <49FCAFA2.60603@gmx.de> Christoph Mallon writes: : Julian Elischer schrieb: : >> Christoph Mallon wrote: : >>>> K&R code should be changed as part of related changes if possible. : >>>> A sweep to change a whole file is probably also ok. : >>>> changing them one at a time is probably not ok. : >>> : >>> But this is what actually is practiced. : >>> You still did not answer my question: Do you agree to remove the : >>> clause so no new old style declarations may be added? : > : > I think a new clause should be added specifying what should happen : > and replacing the old clause. : : This is not sensible. style(9) says right at the start that it "[...] : specifies the preferred style for kernel source files [...]". The : preferred style would be to use ANSI function declarations - what else : is there to say? There is no point in adding more when less is sufficient. Actually, in a style guide, there is a point. Adding language that says we're actively removing K&R-style declarations and definitions reinforces this point and explains to people what's going on when they see this in the tree today. This would have been understandable by some folks if I'd left it at the first paragraph, but by adding the second it becomes clear the reasoning behind my post. Warner From christoph.mallon at gmx.de Sun May 3 07:11:17 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sun May 3 07:11:25 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090502.151931.1396014860.imp@bsdimp.com> References: <49FCA148.9060707@gmx.de> <49FCAA1D.1080208@elischer.org> <49FCAFA2.60603@gmx.de> <20090502.151931.1396014860.imp@bsdimp.com> Message-ID: <49FD4391.9070605@gmx.de> M. Warner Losh schrieb: > In message: <49FCAFA2.60603@gmx.de> > Christoph Mallon writes: > : Julian Elischer schrieb: > : >> Christoph Mallon wrote: > : >>>> K&R code should be changed as part of related changes if possible. > : >>>> A sweep to change a whole file is probably also ok. > : >>>> changing them one at a time is probably not ok. > : >>> > : >>> But this is what actually is practiced. > : >>> You still did not answer my question: Do you agree to remove the > : >>> clause so no new old style declarations may be added? > : > > : > I think a new clause should be added specifying what should happen > : > and replacing the old clause. > : > : This is not sensible. style(9) says right at the start that it "[...] > : specifies the preferred style for kernel source files [...]". The > : preferred style would be to use ANSI function declarations - what else > : is there to say? There is no point in adding more when less is sufficient. > > Actually, in a style guide, there is a point. > > Adding language that says we're actively removing K&R-style > declarations and definitions reinforces this point and explains to > people what's going on when they see this in the tree today. This just overcomplicates things. "removing old style definitions" is not the preferred style, but "using prototyped definitions" is. Old style definitions should not be added anymore, so just remove the clause, which allows it currently. Adding even more about old style definitions is counterproductive - I cannot support this. What to do, when you are seeing an old style definition is clear: Don't Panic! Christoph From imp at bsdimp.com Sun May 3 16:28:04 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Sun May 3 16:28:11 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FD4391.9070605@gmx.de> References: <49FCAFA2.60603@gmx.de> <20090502.151931.1396014860.imp@bsdimp.com> <49FD4391.9070605@gmx.de> Message-ID: <20090503.102444.1683323216.imp@bsdimp.com> In message: <49FD4391.9070605@gmx.de> Christoph Mallon writes: : M. Warner Losh schrieb: : > In message: <49FCAFA2.60603@gmx.de> : > Christoph Mallon writes: : > : Julian Elischer schrieb: : > : >> Christoph Mallon wrote: : > : >>>> K&R code should be changed as part of related changes if possible. : > : >>>> A sweep to change a whole file is probably also ok. : > : >>>> changing them one at a time is probably not ok. : > : >>> : > : >>> But this is what actually is practiced. : > : >>> You still did not answer my question: Do you agree to remove the : > : >>> clause so no new old style declarations may be added? : > : > : > : > I think a new clause should be added specifying what should happen : > : > and replacing the old clause. : > : : > : This is not sensible. style(9) says right at the start that it "[...] : > : specifies the preferred style for kernel source files [...]". The : > : preferred style would be to use ANSI function declarations - what else : > : is there to say? There is no point in adding more when less is sufficient. : > : > Actually, in a style guide, there is a point. : > : > Adding language that says we're actively removing K&R-style : > declarations and definitions reinforces this point and explains to : > people what's going on when they see this in the tree today. : : This just overcomplicates things. "removing old style definitions" is : not the preferred style, but "using prototyped definitions" is. Old : style definitions should not be added anymore, so just remove the : clause, which allows it currently. Adding even more about old style : definitions is counterproductive - I cannot support this. What to do, : when you are seeing an old style definition is clear: Don't Panic! I think you are wrong, and I think your failure to take constructive criticism is alienating a lot of people that would otherwise support at least part of what you are trying to do. I know I've had it enough. Warner From peter at vk2pj.dyndns.org Sun May 3 18:59:57 2009 From: peter at vk2pj.dyndns.org (Peter Jeremy) Date: Sun May 3 19:12:48 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <108501c9c8fe$3a30ab0e$72020a0a@desktop.isilon.com> References: <108501c9c8fe$3a30ab0e$72020a0a@desktop.isilon.com> Message-ID: <20090503185950.GA62081@server.vk2pj.dyndns.org> On 2009-Apr-29 12:10:44 -0700, John Gemignani wrote: >Are local variables allocated on-the-fly on the stack or does the >compiler preallocate the space on entry? This is compiler and optimisation dependent. As a general rule, if a compiler is not performing any optimisation, it is likely to allocate all variables on the stack. An obvious code optimisation is to keep variables in registers instead of storing them on the stack. In the specific case of older gcc versions with optimisation enabled (I'm not sure of the behaviour of gcc 4.x), gcc will immediately spill (allocate on the stack) local variables that aren't eligible to be held in registers (eg too large or the variable address is taken) and allocate remaining variables to virtual registers. During code generation, virtual registers are mapped to physical registers and any remaining virtual registers are allocated on the stack. > If I have to delve into a >crashdump, having the variables on the big entry allocation has been >very helpful in the past. OTOH, not caching variables in registers has a significant adverse impact on performance. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090503/510d1e1f/attachment.pgp From kientzle at freebsd.org Mon May 4 15:16:26 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Mon May 4 15:16:33 2009 Subject: SoC2009: libpkg, pkg tools rewrite In-Reply-To: <7d6fde3d0905020546s697ad5f6r2098ff6c80a80e1e@mail.gmail.com> References: <20090430214520.GA37974@flint.openpave.org> <7d6fde3d0905020546s697ad5f6r2098ff6c80a80e1e@mail.gmail.com> Message-ID: <49FF06C9.6070001@freebsd.org> >> On Sat, Apr 25, 2009 at 03:20:59PM -0400, David Forsythe wrote: >>> This summer I'll be working on creating a package library and using >>> that library to rewrite the pkg tools. Jeremy Lea wrote: > Since I've already done most of the work on this already, please, > please, please, don't ignore what I have done. Garrett Cooper wrote: > There's also http://libpkg.berlios.de/ that you could use as a > starting point, and please, PLEASE, don't neglect the SoC work that > was done last year by Anders Nore. It was committed to p4, but hasn't > made it upstream to CVS yet... Good architectural summaries would help here; I know David already has a lot of background reading planned before start-of-coding this summer. Can you provide pointers to design documents or email threads discussing any of these? I'm not familiar with Anders work; what is the current status of it? Why hasn't it been pushed into SVN yet? Is there a good summary of what it does so far and what remains to be done? Cheers, Tim From kientzle at freebsd.org Mon May 4 15:35:20 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Mon May 4 15:35:28 2009 Subject: SoC2009: libpkg, pkg tools rewrite In-Reply-To: <20090430214520.GA37974@flint.openpave.org> References: <20090430214520.GA37974@flint.openpave.org> Message-ID: <49FF0B36.2030805@freebsd.org> Jeremy Lea: > 1. The library needs a global "package manager". This needs to perform > all of the tasks, and it should ideally do this through a task queue > (which I didn't implement). See the lib/lib.h header in FreePKG. This sounds like a good idea to implement ... eventually. First job is to identify the tasks and a way to implement them. It's relatively straightforward to add a task queue wrapper later on. > 4. bsd.port.mk should do everything through the tools. It should have > no knowledge of the contents of /var/db/pkg. I agree with this in principle, but I think it would be better to avoid touching bsd.port.mk at all in this iteration. Certainly, it's worthwhile to read bsd.port.mk carefully to try to understand the capabilities missing from the current tools with an eye to supporting those in a future iteration, but I think it's more important to keep the scope of this summer's work as narrow as possible. > 1. I made the file->pkg database to sensitive. If there is a miss it > rebuilds the database for scratch - it should do a search through the > +CONTENTS files and only rebuild it if there was a hit (meaning the > database was wrong). I'm going to show my ignorance here: Why is this database necessary at all? On my system with just over 500 packages, it takes <1s to read all of /var/db/pkg with a warm cache (~10s cold cache), and I can only think of a couple of cases where the file->pkg database is useful at all. I fear that maintaining a file->pkg database is a lot of extra effort for very little gain. I would be more interested in experimenting with using extended attributes directly on the files to record what package they came from. In particular, that provides much more robust handling for a variety of use cases, including conflicts, stale files, and manually-updated files. > 2. There needs to be a pkgname and origin database, which can be loaded > at startup to prime the package manager. The dependency graph should > also be stored in a database. These should be rebuilt if any directory > in /var/db/pkg has a mtime later than the database (so could the file > database). Again, I'm a little skeptical of the need for separate databases here at all. In the current system, /var/db/pkg and the package archives themselves have to be authoritative, so any package system has to be able to work directly from that information. Building a separate database from that information seems like a lot of extra work that will pay off someday but can be largely avoided for the current iteration. Remember that the Single Point of Truth for any package manager is the files on disk. If someone deletes the files from disk, then the package is not installed, regardless of what any "package database" might say. In the case of the FreeBSD package system, /var/db/pkg is therefore secondary data and any additional databases you compute from /var/db/pkg are tertiary. This is getting pretty far from the SPoT and is going to lead to increasing consistency problems. If we can avoid these additional databases, we'll get a simpler, more robust system. > 3. There needs to be a set of flags which indicate how a package got > installed (as a dependency or by the user), and if it has been upgraded > in-place and might have old leftover libraries. These could go in > +CONTENTS. Yes, these are the kind of features that should be easy to add after the package tools have been rewritten around libpkg. But first we need a basic libpkg that works and a set of package tools that use it. > In addition I also began the design of a new on disk package format. I'm pretty firmly opposed to this. I think that the INDEX, /var/db/pkg, and binary format for package archives all need to be kept essentially unchanged, simply because of the growing number of third-party tools that interact with them. Eventually, such tools may all be rewritten to use libpkg, at which point it may be reasonable to reconsider, but for the foreseeable future, any changes to these would cause a lot of pain for little gain. There may be some incremental changes (such as the installation flags you mention above) that would help, but I think we're stuck with the current formats for the time being. I'll also observe that some of the concerns that drove your new package archive design are no longer relevant; libarchive allows us to use tar format without forking the tar executable. Tim From msaad at datapipe.com Mon May 4 17:22:24 2009 From: msaad at datapipe.com (Mark Saad) Date: Mon May 4 17:22:31 2009 Subject: ZFS ARC Cache: What is the lowest tested size Message-ID: <49FF21E7.4060201@datapipe.com> Hello List Worked on breaking ZFS we came across a interesting question; What is the lowest tested ZFS ARC Cache size ? Also could you reliably run with it set to 0 ? -- Mark Saad msaad@datapipe.com () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/emaildisclaimer.aspx for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. From matt at ixsystems.com Mon May 4 17:39:03 2009 From: matt at ixsystems.com (Matt Olander) Date: Mon May 4 17:39:10 2009 Subject: VirtualBox on FreeBSD Message-ID: Looks like the VirtualBox developers at Sun ported VirtualBox to FreeBSD in their spare time: http://www.freebsdnews.net/2009/05/02/sun-virtualbox-on-freebsd/ They're looking for developers/testers to checkout the source and try it out :-) -matt From gamaral at amaral.com.mx Mon May 4 18:18:30 2009 From: gamaral at amaral.com.mx (Guillermo Antonio Amaral Bastidas) Date: Mon May 4 18:18:37 2009 Subject: VirtualBox on FreeBSD In-Reply-To: References: Message-ID: <20090504175520.GA3163@DAEDALUS.localdomain> On Mon, May 04, 2009 at 10:25:58AM -0700, Matt Olander wrote: > Looks like the VirtualBox developers at Sun ported VirtualBox to > FreeBSD in their spare time: > http://www.freebsdnews.net/2009/05/02/sun-virtualbox-on-freebsd/ > > They're looking for developers/testers to checkout the source and try > it out :-) Awesome! I will test it out, if it works out that means I could change my GFs desktop to FreeBSD. ^_^ -- Guillermo Antonio Amaral Bastidas (gamaral) Free/Libre/Open-Source Software Developer : http://www.guillermoamaral.com/ KDE Official Representative MX : http://www.kde.org/ PC-BSD Official Representative MX : http://www.pcbsd.org/ GPG Fingerprint: E068 811D 4AA2 7FDA A327 38BD 640D 014C 76FE 7D5A -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090504/c3d175e4/attachment.pgp From xorquewasp at googlemail.com Mon May 4 18:27:21 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Mon May 4 18:27:27 2009 Subject: bootstrapping gnat GCC on amd64 Message-ID: <20090504182714.GA52480@logik.internal.network> Hello. I'm attempting to compile GNAT on AMD64 with an eye to extending support to the platform (the gnat-gcc43 port is ONLY_FOR_ARCH=i386). GNAT obviously requires an Ada compiler to bootstrap. What are my options here? I suspect that I need to create an i386 jail to build a cross compiler but am not sure. Any help would be appreciated. xw From deischen at freebsd.org Mon May 4 18:44:57 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon May 4 18:45:04 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090504182714.GA52480@logik.internal.network> References: <20090504182714.GA52480@logik.internal.network> Message-ID: On Mon, 4 May 2009, xorquewasp@googlemail.com wrote: > Hello. > > I'm attempting to compile GNAT on AMD64 with an eye to > extending support to the platform (the gnat-gcc43 port > is ONLY_FOR_ARCH=i386). > > GNAT obviously requires an Ada compiler to bootstrap. > > What are my options here? > > I suspect that I need to create an i386 jail to build > a cross compiler but am not sure. Is that your only system (amd64)? I originally ported GNAT to FreeBSD x86 from a solaris-sparc32 system. I built a sparc-sun-freebsd GNAT cross compiler using the native Solaris GNAT binary and its associated sources. I also (first) had to cross build binutils similarly. This made a cross compiler that ran on Solaris and built ELF binaries for FreeBSD. I then used this cross compiler to rebuild GNAT as a FreeBSD binary. So it was 2 major steps: build a cross compiler, then use the cross to build a native compiler. It's been years since I've done that. I don't know how much has changed, but you probably have to do something similar. -- DE From xorquewasp at googlemail.com Mon May 4 18:56:49 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Mon May 4 18:56:56 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090504182714.GA52480@logik.internal.network> Message-ID: <20090504185644.GA16315@logik.internal.network> On 2009-05-04 14:44:52, Daniel Eischen wrote: > Is that your only system (amd64)? I originally > ported GNAT to FreeBSD x86 from a solaris-sparc32 system. > I built a sparc-sun-freebsd GNAT cross compiler using > the native Solaris GNAT binary and its associated > sources. I also (first) had to cross build binutils > similarly. This made a cross compiler that ran on > Solaris and built ELF binaries for FreeBSD. I then > used this cross compiler to rebuild GNAT as a FreeBSD > binary. So it was 2 major steps: build a cross compiler, > then use the cross to build a native compiler. > > It's been years since I've done that. I don't know > how much has changed, but you probably have to do > something similar. > > -- > DE Hello. I have a debian system with a working x86 gnat gcc 4.3.3 and a freebsd x86 system running gnat gcc 4.3.2. I guess the first step will be an i386->amd64 cross compiler then. Thanks, xw From deischen at freebsd.org Mon May 4 19:03:36 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon May 4 19:03:43 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090504185644.GA16315@logik.internal.network> References: <20090504182714.GA52480@logik.internal.network> <20090504185644.GA16315@logik.internal.network> Message-ID: On Mon, 4 May 2009, xorquewasp@googlemail.com wrote: > On 2009-05-04 14:44:52, Daniel Eischen wrote: >> Is that your only system (amd64)? I originally >> ported GNAT to FreeBSD x86 from a solaris-sparc32 system. >> I built a sparc-sun-freebsd GNAT cross compiler using >> the native Solaris GNAT binary and its associated >> sources. I also (first) had to cross build binutils >> similarly. This made a cross compiler that ran on >> Solaris and built ELF binaries for FreeBSD. I then >> used this cross compiler to rebuild GNAT as a FreeBSD >> binary. So it was 2 major steps: build a cross compiler, >> then use the cross to build a native compiler. >> >> It's been years since I've done that. I don't know >> how much has changed, but you probably have to do >> something similar. >> >> -- >> DE > > Hello. > > I have a debian system with a working x86 gnat gcc 4.3.3 > and a freebsd x86 system running gnat gcc 4.3.2. > > I guess the first step will be an i386->amd64 cross > compiler then. Right, you should be able to do it from either of those, but perhaps the freebsd x86 may be easier. I would use a PREFIX other than /usr/local (or something different than whatever your actual PREFIX is) for the builds. I was looking around for my notes but can't find them. If I do find them, I'll post them. -- DE From jhb at freebsd.org Mon May 4 19:22:23 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon May 4 19:22:45 2009 Subject: Definition of NULL In-Reply-To: References: Message-ID: <200905041046.02920.jhb@freebsd.org> On Saturday 02 May 2009 11:59:03 am Andrew Brampton wrote: > I'm writing a C++ Kernel Module, and one thing that has been bugging > me is the kernel's definition of NULL. > > sys/sys/_null.h (in CURRENT): > > #if defined(_KERNEL) || !defined(__cplusplus) > #define NULL ((void *)0) > #else > #if defined(__GNUG__) && defined(__GNUC__) && __GNUC__ >= 4 > #define NULL __null > #else > #if defined(__LP64__) > #define NULL (0L) > #else > #define NULL 0 > #endif /* __LP64__ */ > #endif /* __GNUG__ */ > #endif /* _KERNEL || !__cplusplus */ > > >From what I've read online the definition of NULL in C is (void *)0, > whereas in C++ it should be 0, or 0L (on 64bit machines). > > Now, my C++ kernel module is built with _KERNEL definited, like any > other C kernel module. This leads to NULL being defined incorrectly. > > So I have a question and two suggestions. Firstly, why is the #if > defined(_KERNEL) in _null.h? Is it to stop userland application > applications picking up this definition? Or for another reason? Yes. NULL used to be 0. When it was changed to '(void *)0' I believe it broke several applications in ports. As a compromise, NULL was restored back to 0 in userland and only set to '(void *)0' in the kernel. > and two, how about we change the first line of _null.h so that we use > a && instead of a || like so: > #if defined(_KERNEL) && !defined(__cplusplus) I think this would be ok to let C++ work in the kernel. "Embedded" C++ (no exceptions and no dynamic_cast<>) should work fine in theory. I would not change the value of NULL that userland sees though as I think that may be too risky. -- John Baldwin From kmacy at freebsd.org Mon May 4 20:05:18 2009 From: kmacy at freebsd.org (Kip Macy) Date: Mon May 4 20:05:25 2009 Subject: ZFS ARC Cache: What is the lowest tested size In-Reply-To: <49FF21E7.4060201@datapipe.com> References: <49FF21E7.4060201@datapipe.com> Message-ID: <3c1674c90905041305g285b4609icd7435d436d3bbe0@mail.gmail.com> On Mon, May 4, 2009 at 10:12 AM, Mark Saad wrote: > > Hello List > ? Worked on breaking ZFS we came across a interesting question; > What is the lowest tested ZFS ARC Cache size ? I haven't tested it with just over 64MB, but the code indicates that that is the lowest you should try. http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c 3500 /* 3501 * Allow the tunables to override our calculations if they are 3502 * reasonable (ie. over 64MB) 3503 */ 3504 if (zfs_arc_max > 64<<20 && zfs_arc_max < physmem * PAGESIZE) 3505 arc_c_max = zfs_arc_max; 3506 if (zfs_arc_min > 64<<20 && zfs_arc_min <= arc_c_max) 3507 arc_c_min = zfs_arc_min; >Also could you reliably > run with it set to 0 ? For better and worse, ZFS isn't primarily a file system. It is a transactional object store that has a file system as one type of object. The ARC is not a file system buffer cache, it caches virtual device blocks. You might think of it as being more like a very large write-back drive cache. It might work with less than 64MB, but the ARC buffers are what ZIO sends down to disk so it certainly would not work with zero. Cheers, Kip From xorquewasp at googlemail.com Tue May 5 00:51:31 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Tue May 5 00:51:38 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090504182714.GA52480@logik.internal.network> <20090504185644.GA16315@logik.internal.network> Message-ID: <20090505005128.GA4519@logik.internal.network> On 2009-05-04 15:03:32, Daniel Eischen wrote: > Right, you should be able to do it from either of those, > but perhaps the freebsd x86 may be easier. > > I would use a PREFIX other than /usr/local (or something > different than whatever your actual PREFIX is) for the > builds. > > I was looking around for my notes but can't find them. > If I do find them, I'll post them. I've built an i386 jail and am currently building the gnat-gcc43 port to compile a GNAT cross-compiler (i386 -> amd64). There are two reasons for this whole exercise: I just moved over to amd64 and want a native toolchain and I also have a lot of software written in Ada that I'd like to submit to FreeBSD ports but can't due a lack of an Ada compiler on amd64. I noticed that the gnat-gcc43 port downloads a rather anonymous set of gcc 4.1 bootstrap binaries. I assume I'd have to provide my own in the same manner to create a port? xw From eischen at vigrid.com Tue May 5 01:19:21 2009 From: eischen at vigrid.com (Daniel Eischen) Date: Tue May 5 01:19:28 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090505005128.GA4519@logik.internal.network> References: <20090504182714.GA52480@logik.internal.network> <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> Message-ID: On Tue, 5 May 2009, xorquewasp@googlemail.com wrote: > On 2009-05-04 15:03:32, Daniel Eischen wrote: >> Right, you should be able to do it from either of those, >> but perhaps the freebsd x86 may be easier. >> >> I would use a PREFIX other than /usr/local (or something >> different than whatever your actual PREFIX is) for the >> builds. >> >> I was looking around for my notes but can't find them. >> If I do find them, I'll post them. > > I've built an i386 jail and am currently building the > gnat-gcc43 port to compile a GNAT cross-compiler (i386 -> > amd64). > > There are two reasons for this whole exercise: I just moved > over to amd64 and want a native toolchain and I also have a > lot of software written in Ada that I'd like to submit to > FreeBSD ports but can't due a lack of an Ada compiler on amd64. > > I noticed that the gnat-gcc43 port downloads a rather > anonymous set of gcc 4.1 bootstrap binaries. I assume I'd > have to provide my own in the same manner to create a port? Yes, you can look at my lang/gnat port to find its bootstrap compiler. I would recommend making a binary bootstrap compiler on the earliest version of FreeBSD amd64 as you can. If you use 8.0-current for instance, others will not be able to build the port on 7.x. -- DE From xorquewasp at googlemail.com Tue May 5 02:21:55 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Tue May 5 02:22:02 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090504182714.GA52480@logik.internal.network> <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> Message-ID: <20090505022151.GA32477@logik.internal.network> On 2009-05-04 20:54:46, Daniel Eischen wrote: > Yes, you can look at my lang/gnat port to find its > bootstrap compiler. I would recommend making a binary > bootstrap compiler on the earliest version of FreeBSD > amd64 as you can. If you use 8.0-current for instance, > others will not be able to build the port on 7.x. Earliest version of FreeBSD that fully supports this hardware is 7.2, so I personally can't go back to an earlier release than that. From deischen at freebsd.org Tue May 5 02:24:57 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Tue May 5 02:25:04 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090505022151.GA32477@logik.internal.network> References: <20090504182714.GA52480@logik.internal.network> <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> Message-ID: On Tue, 5 May 2009, xorquewasp@googlemail.com wrote: > On 2009-05-04 20:54:46, Daniel Eischen wrote: >> Yes, you can look at my lang/gnat port to find its >> bootstrap compiler. I would recommend making a binary >> bootstrap compiler on the earliest version of FreeBSD >> amd64 as you can. If you use 8.0-current for instance, >> others will not be able to build the port on 7.x. > > Earliest version of FreeBSD that fully supports this > hardware is 7.2, so I personally can't go back to an > earlier release than that. Keep good notes - maybe someone else can do it once you have the procedure down and working ;-) -- DE From joerg at britannica.bec.de Tue May 5 03:47:06 2009 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Tue May 5 03:47:13 2009 Subject: NetBSD 5.0 statistics In-Reply-To: <78367CB5-7DAD-4DB1-99DA-2618CFACF376@mac.com> References: <6101e8c40904300750i3e86fc0cnef09b0d4533627f7@mail.gmail.com> <78367CB5-7DAD-4DB1-99DA-2618CFACF376@mac.com> Message-ID: <20090505034719.GG3417@britannica.bec.de> On Thu, Apr 30, 2009 at 01:41:42PM -0700, Marcel Moolenaar wrote: > I recall that our "make -j X" actually limits the number > of make processes/jobs to X. I don't know anything about > build.sh, so I don't know if our make is at all being > involved, but it would be good to know how the load varies > per OS. build.sh is using NetBSD's make for all but the tool build and the tool build is not included in the stats. Joerg From joerg at britannica.bec.de Tue May 5 03:50:25 2009 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Tue May 5 03:50:31 2009 Subject: Definition of NULL In-Reply-To: <200905041046.02920.jhb@freebsd.org> References: <200905041046.02920.jhb@freebsd.org> Message-ID: <20090505035034.GH3417@britannica.bec.de> On Mon, May 04, 2009 at 10:46:02AM -0400, John Baldwin wrote: > I think this would be ok to let C++ work in the kernel. "Embedded" C++ (no > exceptions and no dynamic_cast<>) should work fine in theory. I would not > change the value of NULL that userland sees though as I think that may be too > risky. Well, use __null if present all the time, fallback to the alternatives as currently defined. Joerg From jt at 0xabadba.be Tue May 5 08:35:41 2009 From: jt at 0xabadba.be (jt@0xabadba.be) Date: Tue May 5 08:35:49 2009 Subject: concurrent sysctl implementation Message-ID: Hackers, I've been using FreeBSD since a boy and now its time for me to give back. I will be doing my final projects in university concerning concurrency in the FreeBSD Kernel. I've been discussing with Ed@ methods of implementing sysctl concurrently since we use sysctl quite a lot for _everything_ essentially (make, general kernel information, etc). I've been reading the sysctl man pages and some of the code in the kernel; however, I wanted to shoot this out to the public since many of you know better than I do about where I should be looking to do the required reading to get the job done correctly. I'm also open to implementing other things once this is done. I hope everyone is doing well. respectfully, =jt From dwmalone at maths.tcd.ie Tue May 5 08:58:08 2009 From: dwmalone at maths.tcd.ie (David Malone) Date: Tue May 5 08:58:15 2009 Subject: Patch for MS Hyper V (virtualization) In-Reply-To: <20090406.111755.1973602189.imp@bsdimp.com> References: <1366225354.253456.1238948619308.JavaMail.root@vms124.mailsrvcs.net> <200904061154.19601.jhb@freebsd.org> <20090406.111755.1973602189.imp@bsdimp.com> Message-ID: <20090505085804.GA1274@walton.maths.tcd.ie> I was talking about the Hyper-V problem with a guy from MS, and he followed up on it for me. It seems this is a known issue, which should be fixed in the latest version of Hyper-V (i.e. the RC of Windows Server 2008 R2 that was released on TechNet last week). David. From vasanth.raonaik at gmail.com Tue May 5 10:06:41 2009 From: vasanth.raonaik at gmail.com (vasanth raonaik) Date: Tue May 5 10:06:47 2009 Subject: concurrent sysctl implementation In-Reply-To: References: Message-ID: Hello Jt, I am a newbee in this alias. I am having a very basic question. It would be really good if you could give me some of this information. Could you please elaborate on what is the current architecture of sysctl implementation and How the concurrency would benefit us. Thanks in advance, Vasanth On Tue, May 5, 2009 at 1:37 PM, wrote: > Hackers, > > I've been using FreeBSD since a boy and now its time for me to give > back. I will be doing my final projects in university concerning > concurrency in the FreeBSD Kernel. I've been discussing with Ed@ methods > of > implementing sysctl concurrently since we use sysctl quite a lot for > _everything_ essentially (make, general kernel information, etc). I've > been > reading the sysctl man pages and some of the code in the kernel; however, I > wanted to shoot this out to the public since many of you know better than I > do about where I should be looking to do the required reading to get the > job > done correctly. I'm also open to implementing other things once this is > done. I hope everyone is doing well. > > respectfully, > > =jt > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From simond at irrelevant.org Tue May 5 10:31:14 2009 From: simond at irrelevant.org (Simon Dick) Date: Tue May 5 10:31:21 2009 Subject: Cobalt Raq 550 In-Reply-To: <5da021490905010848q441781den54eefba85a69f342@mail.gmail.com> References: <5da021490905010848q441781den54eefba85a69f342@mail.gmail.com> Message-ID: <1241518462.1533.1313800225@webmail.messagingengine.com> On Fri, 1 May 2009 08:48:09 -0700, "Justin G." said: > Hello Hackers, > > We came into a Cobalt Raq 550 the other day and were wondering if we > could put it to use. I've googled and googled and found only guides > for Linux installs. Much of it is quite similar, but my issue is with > the loader on the device being able to boot into FreeBSD. The device > searches for linux kernel images when booting. > > Does anyone have any experience with installing FreeBSD on a Raq 550? > I wasn't able to find a website with much detail and would appreciate > any hints or leads :-) Doesn't the RaQ 550 have a basic Linux kernel in it's flash? I worked with them at my last job, I don't remember hearing about anyone who'd managed to get anything except linux working on it though (though I'm sure someone must have!) -- Simon Dick simond@irrelevant.org From jruohonen at iki.fi Tue May 5 11:37:06 2009 From: jruohonen at iki.fi (Jukka Ruohonen) Date: Tue May 5 11:37:17 2009 Subject: Cobalt Raq 550 In-Reply-To: <1241518462.1533.1313800225@webmail.messagingengine.com> References: <5da021490905010848q441781den54eefba85a69f342@mail.gmail.com> <1241518462.1533.1313800225@webmail.messagingengine.com> Message-ID: <20090505110346.GA1155@marx.bitnet> On 05.05.2009, Simon Dick wrote: > with them at my last job, I don't remember hearing about anyone who'd > managed to get anything except linux working on it though (though I'm > sure someone must have!) Perhaps somewhat off-topic, but: http://www.netbsd.org/ports/cobalt/ - Jukka. From avg at icyb.net.ua Tue May 5 15:46:10 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 5 15:46:17 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FAE36D.9030109@FreeBSD.org> References: <49F4070C.2000108@gmx.de> <20090428121327.GA41168@freebsd.org> <49FA8F2D.5090708@gmx.de> <49FAE36D.9030109@FreeBSD.org> Message-ID: <4A005F3D.7010606@icyb.net.ua> on 01/05/2009 14:56 Maxim Sobolev said the following: > Christoph Mallon wrote: >> Roman Divacky schrieb: >>> I like the part about using as many variables as possible because >>> of documentation and performance enhancements. I tend to like >>> the other changes as well.. >> >> This is not about using as many variables as possible. The goal is to >> use as many variables as you have logically distinct entities in the >> function. I suppose, this is what you mean, but I want to clarify this >> point. > > Why don't just put "logically distinct entities" into separate functions > on their own? It's a good indicator that the re-factoring is due when > you reach this point. I think that you overreach. A very trivial example. An array and an index into the array are two sufficiently "logically distinct entities" [as to be different variables]. But they are sufficiently related to be used in the same function. -- Andriy Gapon From avg at icyb.net.ua Tue May 5 15:50:26 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue May 5 15:50:33 2009 Subject: Definition of NULL In-Reply-To: <20090502163535.GA17027@owl.midgard.homeip.net> References: <20090502163535.GA17027@owl.midgard.homeip.net> Message-ID: <4A00603D.8000204@icyb.net.ua> on 02/05/2009 19:35 Erik Trulsson said the following: > Is the use of C++ inside the kernel really supported? I don't think so, > but I could be wrong. First question is what did you mean by "supported"? :-) Fortunately, it almost does not need to be supported, it can simply be used (with some known limitations). -- Andriy Gapon From gabor at FreeBSD.org Tue May 5 22:07:23 2009 From: gabor at FreeBSD.org (Gabor Kovesdan) Date: Tue May 5 22:07:47 2009 Subject: SoC 2009: BSD-licensed libiconv in base system In-Reply-To: <20090428180624.GA2223@britannica.bec.de> References: <20090427183836.GA10793@zim.MIT.EDU> <49F5FE45.2090101@freebsd.org> <20090427193326.GA7654@britannica.bec.de> <20090427194904.GA11137@zim.MIT.EDU> <49F6C7A1.6070708@FreeBSD.org> <20090428122225.GA2862@britannica.bec.de> <24e9a86bf5995ba551db8f27aa204191.squirrel@webmail.kovesdan.org> <20090428180624.GA2223@britannica.bec.de> Message-ID: <4A00B897.809@FreeBSD.org> Joerg Sonnenberger escribi?: > > Unicode covers Korean. It just violates the "one logic character equals > one UCS-4 character" or however you want to put. More trivial example > can be obtained when looking at both your and my name. Diacrets have > historically been part of the character, but it is possible to use > combining characters in unicode for the cleaner description. > OK, I know that but that's other problem. You explicitly wrote that there were characters, which couldn't be represented un UCS-4, that was what I reacted on: > Everything can be represented as UCS-4 is a bad > assumption, but something Americans and Europeans naturally don't have > to care about. Cheers, -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From alexanderchuranov at gmail.com Wed May 6 10:57:35 2009 From: alexanderchuranov at gmail.com (Alexander Churanov) Date: Wed May 6 10:57:41 2009 Subject: SoC 2009: BSD-licensed libiconv in base system In-Reply-To: <4A00B897.809@FreeBSD.org> References: <20090427183836.GA10793@zim.MIT.EDU> <49F5FE45.2090101@freebsd.org> <20090427193326.GA7654@britannica.bec.de> <20090427194904.GA11137@zim.MIT.EDU> <49F6C7A1.6070708@FreeBSD.org> <20090428122225.GA2862@britannica.bec.de> <24e9a86bf5995ba551db8f27aa204191.squirrel@webmail.kovesdan.org> <20090428180624.GA2223@britannica.bec.de> <4A00B897.809@FreeBSD.org> Message-ID: <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> Gabor, Joerg, I am currently working on UTF-8 support in syscons and highly interested in making FreeBSD using UTF-8 out of box. There is my $0.02: 1) Why discuss UCS-4 at all? UTF-32 is alreay in place. SImple, standardized, fixed-width and stateless. 2) I'm against using wchar_t internally, because C language standard does not require that a wchar_t variable can hold an UTF-32 code point. 3) Please, give an example of character that does not fit into UCS-4. I'll check whether it fits into UTF-32. I expect that any character fits into a single UTF-32 code point. Sincerely, Alexander Churanov From ertr1013 at student.uu.se Wed May 6 13:13:20 2009 From: ertr1013 at student.uu.se (Erik Trulsson) Date: Wed May 6 13:13:27 2009 Subject: SoC 2009: BSD-licensed libiconv in base system In-Reply-To: <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> References: <20090427183836.GA10793@zim.MIT.EDU> <49F5FE45.2090101@freebsd.org> <20090427193326.GA7654@britannica.bec.de> <20090427194904.GA11137@zim.MIT.EDU> <49F6C7A1.6070708@FreeBSD.org> <20090428122225.GA2862@britannica.bec.de> <24e9a86bf5995ba551db8f27aa204191.squirrel@webmail.kovesdan.org> <20090428180624.GA2223@britannica.bec.de> <4A00B897.809@FreeBSD.org> <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> Message-ID: <20090506131312.GA48658@owl.midgard.homeip.net> On Wed, May 06, 2009 at 02:28:51PM +0400, Alexander Churanov wrote: > Gabor, Joerg, > > I am currently working on UTF-8 support in syscons and highly > interested in making FreeBSD using UTF-8 out of box. > > There is my $0.02: > > 1) Why discuss UCS-4 at all? UTF-32 is alreay in place. SImple, > standardized, fixed-width and stateless. UCS-4 and UTF-32 are, for most purposes, just two different names for the same encoding. > 2) I'm against using wchar_t internally, because C language standard > does not require that a wchar_t variable can hold an UTF-32 code > point. The C standard has very few requirements on wchar_t. It is up to each implementation to decide how wchar_t is defined. There is nothing which prevents the FreeBSD project from deciding that on FreeBSD wchar_t is always 32 bits wide, which can then be relied upon in FreeBSD-specific code. It is not like somebody else will change the relevant include files without warning. > 3) Please, give an example of character that does not fit into UCS-4. > I'll check whether it fits into UTF-32. I expect that any character > fits into a single UTF-32 code point. > > Sincerely, > Alexander Churanov > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Erik Trulsson ertr1013@student.uu.se From alexanderchuranov at gmail.com Wed May 6 13:20:03 2009 From: alexanderchuranov at gmail.com (Alexander Churanov) Date: Wed May 6 13:20:15 2009 Subject: SoC 2009: BSD-licensed libiconv in base system In-Reply-To: <20090506131312.GA48658@owl.midgard.homeip.net> References: <20090427183836.GA10793@zim.MIT.EDU> <20090427193326.GA7654@britannica.bec.de> <20090427194904.GA11137@zim.MIT.EDU> <49F6C7A1.6070708@FreeBSD.org> <20090428122225.GA2862@britannica.bec.de> <24e9a86bf5995ba551db8f27aa204191.squirrel@webmail.kovesdan.org> <20090428180624.GA2223@britannica.bec.de> <4A00B897.809@FreeBSD.org> <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> <20090506131312.GA48658@owl.midgard.homeip.net> Message-ID: <3cb459ed0905060620x78b98bcak5cea5570ee3120a9@mail.gmail.com> 2009/5/6 Erik Trulsson : > The C standard has very few requirements on wchar_t. ?It is up to each > implementation to decide how wchar_t is defined. > There is nothing which prevents the FreeBSD project from deciding > that on FreeBSD wchar_t is always 32 bits wide, which can then be relied > upon in FreeBSD-specific code. ?It is not like somebody else will change > the relevant include files without warning. Ah, then wchar_t is for internal usage? Then it's OK. Alexander Churanov From xorquewasp at googlemail.com Wed May 6 15:21:44 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Wed May 6 15:21:51 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090504182714.GA52480@logik.internal.network> <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> Message-ID: <20090506152139.GB69468@logik.internal.network> On 2009-05-06 10:57:25, Daniel Eischen wrote: > > Back in the day when I did it, it was with gcc-2.7.x or > gcc-2.8.x I believe. The cross build process with gnat > was a little different. I couldn't do a normal gnat build, > which did a bootstrap and then rebuilt the compiler again > using the bootstrap. I believe I had to manually build > bootstrap, gnatlib-cross, and gnattools-cross or targets > named something like that. I don't even think "install" > worked - I think I had to manually install the cross. > > This does sound vaguely familiar though. I may have > just rebuilt the target->target GNAT with the cross > GNAT, then whenever the build process stopped with > gnatmake errors, I would manually build whatever target > it stopped at with the cross gnatbind and gnatlink. > > I don't even know if gnatmake is required to rebuild > the target->target GNAT, all the dependencies should > be listed in the Makefiles, so gcc should get called > directly as opposed through gnatmake. So try going > forward with rebuilding that x86_64 native GNAT and > see what happens. This is really what your goal is, > having the i386->x86_64 cross is nice, but not really > essential. > > Hmm, I do know that building gnat as a cross > does work though, since we have and use GNAT for > sparc-solaris hosted ppc-vxworks cross compilers. > And gnatmake works just fine as a cross. Thanks for the notes. This is an example of one of the build failures when compiling the x86_64 native GNAT: mkdir -p ada/bldtools/nmake_b rm -f ada/bldtools/nmake_b/sinfo.ads ada/bldtools/nmake_b/nmake.adt ada/bldtools/nmake_b/xnmake.adb ada/bldtools/nmake_b/xutil.ads ada/bldtools/nmake_b/xutil.adb cp -p ../../gcc-4.4.0/gcc/ada/sinfo.ads ../../gcc-4.4.0/gcc/ada/nmake.adt ../../gcc-4.4.0/gcc/ada/xnmake.adb ../../gcc-4.4.0/gcc/ada/xutil.ads ../../gcc-4.4.0/gcc/ada/xutil.adb ada/bldtools/nmake_b (cd ada/bldtools/nmake_b && x86_64-unknown-freebsd7.2-gnatmake -q xnmake && ./xnmake -b ../../nmake.adb ) x86_64-unknown-freebsd7.2-gnatmake: "xnmake.ali" incompatible ALI file, please recompile x86_64-unknown-freebsd7.2-gnatmake: "xnmake.adb" compilation error gmake[2]: *** [ada/nmake.adb] Error 4 gmake[2]: Leaving directory `/root/gcc-4.4.0-obj/gcc' gmake[1]: *** [all-gcc] Error 2 gmake[1]: Leaving directory `/root/gcc-4.4.0-obj' gmake: *** [all] Error 2 The Makefiles are pretty twisted and hard to comprehend. I'll try your suggestion, stepping through each target and ideally try to get the whole process down to shell script form. More news to follow... xw From joerg at britannica.bec.de Wed May 6 16:22:33 2009 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Wed May 6 16:22:47 2009 Subject: SoC 2009: BSD-licensed libiconv in base system In-Reply-To: <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> References: <20090427183836.GA10793@zim.MIT.EDU> <49F5FE45.2090101@freebsd.org> <20090427193326.GA7654@britannica.bec.de> <20090427194904.GA11137@zim.MIT.EDU> <49F6C7A1.6070708@FreeBSD.org> <20090428122225.GA2862@britannica.bec.de> <24e9a86bf5995ba551db8f27aa204191.squirrel@webmail.kovesdan.org> <20090428180624.GA2223@britannica.bec.de> <4A00B897.809@FreeBSD.org> <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> Message-ID: <20090506162247.GA23015@britannica.bec.de> On Wed, May 06, 2009 at 02:28:51PM +0400, Alexander Churanov wrote: > 1) Why discuss UCS-4 at all? UTF-32 is alreay in place. SImple, > standardized, fixed-width and stateless. Which part of "combining characters" is stateless? Sure, you can ignore that in some/many applications, but it still exists. UCS-4 and UTF-32 are identical, so discussing one is enough. > 2) I'm against using wchar_t internally, because C language standard > does not require that a wchar_t variable can hold an UTF-32 code > point. See my original point of that locale independent wchar_t might be useful, but creates problems. If the OS supports full Unicode 3+ locales, it will have to be able to fit any UCS-4 code point into wchar_t. Joerg From simond at irrelevant.org Wed May 6 16:42:25 2009 From: simond at irrelevant.org (Simon Dick) Date: Wed May 6 16:42:32 2009 Subject: Cobalt Raq 550 In-Reply-To: <20090505110346.GA1155@marx.bitnet> References: <5da021490905010848q441781den54eefba85a69f342@mail.gmail.com> <1241518462.1533.1313800225@webmail.messagingengine.com> <20090505110346.GA1155@marx.bitnet> Message-ID: <1241628144.28461.1314123239@webmail.messagingengine.com> On Tue, 5 May 2009 14:03:46 +0300, "Jukka Ruohonen" said: > On 05.05.2009, Simon Dick wrote: > > with them at my last job, I don't remember hearing about anyone who'd > > managed to get anything except linux working on it though (though I'm > > sure someone must have!) > > Perhaps somewhat off-topic, but: > > http://www.netbsd.org/ports/cobalt/ Unfortunately that's just the older MIPS based ones, the RaQ4 and 550 were x86 based but with a very weird BIOS (I know it'd be possible to install onto them, I just don't think anyone's thought it worthwhile yet :) ) -- Simon Dick simond@irrelevant.org From ken at mthelicon.com Thu May 7 09:55:35 2009 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Thu May 7 09:55:42 2009 Subject: DAD Detected duplicate ipv6 address - 7.2 Stable Message-ID: <6CBB129E81A54104BCFD00735C71B5F4@PegaPegII> Hi Hackers, I was wondering if someone could give me some advice. I just updated one of my servers from 7.2-pre to 7.2-stable (07/05/2009) and lost my IPv6 connectivity. The machine has been given a static IPv6 address and ND is not used (I set the address in the rc.conf file). On reboot, I get messages like this in the dmesg: vr0: DAD detected duplicate IPv6 address 2001:49f0:2023::2: NS in/out=3/1, NA in=0 vr0: DAD complete for 2001:49f0:2023::2 - duplicate found vr0: manual intervention required vr0: DAD detected duplicate IPv6 address fe80:1::213:8fff:fe5c:1a5f: NS in/out=3/1, NA in=0 vr0: DAD complete for fe80:1::213:8fff:fe5c:1a5f - duplicate found vr0: manual intervention required vr0: possible hardware address duplication detected, disable IPv6 hercules$ ifconfig vr0: flags=8843 metric 0 mtu 1500 options=2808 ether 00:13:8f:5c:1a:5f inet6 fe80::213:8fff:fe5c:1a5f%vr0 prefixlen 64 duplicated scopeid 0x1 inet 66.90.118.40 netmask 0xffffff00 broadcast 66.90.118.255 inet6 2001:49f0:2023::2 prefixlen 64 duplicated media: Ethernet autoselect (100baseTX ) status: active plip0: flags=108810 metric 0 mtu 1500 lo0: flags=8049 metric 0 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 This fault has only shown up since the upgrade and I am at a loss for what to try to do to fix this or debug. Any advice would be appreciated. Peg From kientzle at freebsd.org Fri May 8 03:07:48 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Fri May 8 03:08:01 2009 Subject: fdescfs brokenness Message-ID: <4A03A202.2050101@freebsd.org> Colin Percival recently pointed out some issues with tar and fdescfs. Part of the problem here is tar; I need to rethink some of the traversal logic. But fdescfs is really wonky: * This is a nit, but: ls /dev/fd/18 should not return EBADF; it should return ENOENT, just like any other reference to a non-existent filename. (Just because a filename reflects a file descriptor does not mean it is a file descriptor.) * The fairly routine recursive directory walker below gets hung in fdescfs. It appears that the two opendir() invocations active at the same time interfere with each other. * A similar chdir()-based version of the directory walker below breaks badly; you can chdir() into a directory under /dev/fd, but you can't chdir("..") to get back out of it. (This is the particular problem that tar is running afoul of.) * Running "find /dev/fd" generates bogus ENOENT errors because you can opendir() a directory inside of /dev/fd, and read the entries, but you can't access those entries because path searches don't work through fdescfs. I think the right solution here is to add a VOP_ACCESS handler to fdescfs that bars all access to directory nodes under /dev/fd. Basically, if your program has a directory open, that should be reflected as a directory node that you can't do anything with. The current implementation allows you to chdir(), opendir(), etc, those directory nodes, but the machinery to fully support those operations is missing so they just screw things up. I have a candidate vop_access handler partly written, but I'm a little new to filesystem work, so it will take me a little while to satisfy myself that it works. /* * Non-chdir directory walker. */ #include #include #include #include #include #include #include #include #include static char curpath[512]; void visit(char *f, int depth) { DIR *d; struct dirent *dp; size_t l = strlen(curpath); strcat(curpath, "/"); strcat(curpath, f); printf("%3d: %s\n", depth, curpath); d = opendir(curpath); if (d != NULL) { while ((dp = readdir(d)) != NULL) { if (dp->d_name[0] == '.') continue; visit(dp->d_name, depth + 1); } closedir(d); } curpath[l] = '\0'; } int main(int argc, char **argv) { visit("/dev/fd", 0); exit(0); } From buganini at gmail.com Fri May 8 05:28:41 2009 From: buganini at gmail.com (Buganini) Date: Fri May 8 05:28:59 2009 Subject: SoC 2009: BSD-licensed libiconv in base system In-Reply-To: <20090506162247.GA23015@britannica.bec.de> References: <20090427183836.GA10793@zim.MIT.EDU> <20090427193326.GA7654@britannica.bec.de> <20090427194904.GA11137@zim.MIT.EDU> <49F6C7A1.6070708@FreeBSD.org> <20090428122225.GA2862@britannica.bec.de> <24e9a86bf5995ba551db8f27aa204191.squirrel@webmail.kovesdan.org> <20090428180624.GA2223@britannica.bec.de> <4A00B897.809@FreeBSD.org> <3cb459ed0905060328n4ad05d98xb5ba0c2e01d356e2@mail.gmail.com> <20090506162247.GA23015@britannica.bec.de> Message-ID: Hi all, I'm also working on a BSDL charset converter library: http://github.com/buganini/bsdconv/ It's *NOT* compatible with iconv. It try to provide more function, including fallback charset, intermediary mapping. Intermediary mapping could be used to do conversion between Traditional/Simplified Chinese, CRLF/CR/LF, etc. Currently iconv can only convert Chinese from/to big5/gb2312, it can't convert Chinese in unicode data. Unicode is prefixed with 0x01 in internal encoding. Therefore charsets like CNS11643, which has symbols not in UNICODE can use other prefix. It's in early developement, supporting few charset by now. Buganini From kheuer2 at gwdg.de Fri May 8 08:25:51 2009 From: kheuer2 at gwdg.de (Konrad Heuer) Date: Fri May 8 08:25:58 2009 Subject: How to invalidate NFS read cache? Message-ID: <20090508101555.J47014@gwdu60.gwdg.de> Hello, sporadically, I observe a strange but serious problem in our large NFS environment. NFS servers are Linux and OS X with StorNext/Xsan cluster filesystems, NFS clients Linux and FreeBSD. NFS client A changes a file, but nfs client B (running on FreeBSD) does still see the old version. On the NFS server itself, everything looks fine. Afaik the FreeBSD kernel invalidates the NFS read cache if file modification time on the server changed which should happen here but doesn't. Can I force FreeBSD (e.g. by sysctl setting) to read file buffers again unconditionally after vfs.nfs.access_cache_timeout seconds have passed? Best regards Konrad Heuer GWDG, Am Fassberg, 37077 Goettingen, Germany, kheuer2@gwdg.de From danny at cs.huji.ac.il Fri May 8 09:11:02 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri May 8 09:11:11 2009 Subject: undetected umass device Message-ID: Hi, this is a: USB to Parallel-ATA bridge(0x0c05), Sunplus Technology Inc.(0x04fc) or in english, a nifty dongle that can be connected to ata/sata disk, but umass fails to detected it fully, and detaches it. any hints? quircks? thanks, danny From hselasky at c2i.net Fri May 8 11:54:19 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Fri May 8 11:54:26 2009 Subject: undetected umass device In-Reply-To: References: Message-ID: <200905081256.47724.hselasky@c2i.net> On Friday 08 May 2009, Danny Braniss wrote: > Hi, > this is a: > USB to Parallel-ATA bridge(0x0c05), Sunplus Technology Inc.(0x04fc) > or in english, a nifty dongle that can be connected to ata/sata disk, but > umass fails to detected it fully, and detaches it. > > any hints? quircks? > thanks, > danny > Can you get dmesg and the USB descriptors of your device? --HPS From danny at cs.huji.ac.il Fri May 8 14:05:04 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri May 8 14:05:11 2009 Subject: undetected umass device In-Reply-To: <200905081256.47724.hselasky@c2i.net> References: <200905081256.47724.hselasky@c2i.net> Message-ID: > On Friday 08 May 2009, Danny Braniss wrote: > > Hi, > > this is a: > > USB to Parallel-ATA bridge(0x0c05), Sunplus Technology Inc.(0x04fc) > > or in english, a nifty dongle that can be connected to ata/sata disk, but > > umass fails to detected it fully, and detaches it. > > > > any hints? quircks? > > thanks, > > danny > > > > Can you get dmesg and the USB descriptors of your device? > sure, but I guess I forgot to mention, this is with 7.2-stable. port 2 addr 3: high speed, self powered, config 1, USB to Parallel-ATA bridge(0x0c05), Sunplus Technology Inc.(0x04fc), rev 10.03 umass1: on uhub7 umass1: SCSI over Bulk-Only; quirks = 0x0000 umass1: Max Lun is 0 umass1:2:-1:-1:XPT_PATH_INQ:. umass1:2:1:-1: Attached to scbus2 umass1: Attach finished scbus2: scanning for umass1:2:1:-1 umass1:2:1:-1:XPT_PATH_INQ:. umass1:2:0:0:XPT_PATH_INQ:. umass1:2:0:0:XPT_PATH_INQ:. umass1:2:0:0:XPT_SET_TRAN_SETTINGS:. umass1:2:0:0:XPT_PATH_INQ:. umass1:2:0:0:XPT_PATH_INQ:. umass1:2:0:0:XPT_SCSI_IO: cmd: 0x12, flags: 0x40, 6b cmd/36b data/18b sense umass1: CBW 1: cmd = 6b (0x120000002400), data = 36b, dir = in umass1: at uhub7 port 2 (addr 3) disconnected umass1: detached umass1: Handling BBB state 2 (BBB CBW), xfer=0xffffff00014c8c00, CANCELLED xpt0: Rescan succeeded umass1: detached cheers, danny From danny at cs.huji.ac.il Fri May 8 14:27:21 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Fri May 8 14:27:28 2009 Subject: fputsock()/fgetsock() replacement Message-ID: hi, Since the comment in the kernel is: 'Note: fputsock() is deprecated, see comment for fgetsock().' I'm looking for a replacement, on the other hand, this quote: deprecated Said of a program or feature that is considered obsolescent and in the process of being phased out, usually in favour of a specified replacement. Deprecated features can, unfortunately, linger on for many years. This term appears with distressing frequency in standards documents when the committees writing the documents realise that large amounts of extant (and presumably happily working) code depend on the feature(s) that have passed out of favour. See also {dusty deck}. [{Jargon File}] (1995-04-19) got me wondering if I should hurry. cheers, danny From brampton+freebsd-hackers at gmail.com Fri May 8 17:24:04 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Fri May 8 17:24:11 2009 Subject: kthreads and sched_relinquish Message-ID: Hi, I'm writing a FreeBSD kernel module and I think I really misunderstand something. My module spawns a thread, which should be running while the module is loaded. The thread does some work and then should yield for other threads. However, if there are no other threads waiting, then I would like to continue to do more work. The problem is that I am getting weird deadlocks so I wrote a simple test app to ask why this doesn't work: ------------------------ #include #include #include #include #include #include #include #include #include static void test_thread(void *blah) { unsigned int i = 100000000; /* Limit the number of iterations */ printf("Test Thread Started\n"); while (i > 0) { sched_relinquish(curthread); i--; } printf("Test Thread Exited\n"); kthread_exit(0); } /* The function called at load/unload. */ static int event_handler(struct module *module, int event, void *arg) { int e = 0; /* Error, 0 for normal return status */ switch (event) { case MOD_LOAD: printf("Test Module Loaded\n"); kthread_create(test_thread, NULL, NULL, 0, 0, "test"); break; case MOD_UNLOAD: printf("Test Module Unloaded\n"); break; default: e = EOPNOTSUPP; /* Error, Operation Not Supported */ } return e; } /* The second argument of DECLARE_MODULE. */ static moduledata_t test_conf = { "test_mod", /* module name */ event_handler, /* event handler */ NULL /* extra data */ }; DECLARE_MODULE(test_mod, test_conf, SI_SUB_DRIVERS, SI_ORDER_MIDDLE); ---------------------------- While my thread is running the rest of the system is unresponsive. The thread should sched_relinquish() every time round the loop, and from my understanding that should yield to allow other threads to run, for example the thread which executes my shell (bash). I suspect my thread is yielding and getting instantly rescheduled. I noticed that poll_idle() in sys/kern_poll.c does something similar to me, but they first lower their priority. This however has not worked for me, since my more complex module interacts with higher priority threads and ends up deadlocking. So I just want to ask, Why does this example code lock the system? Am I using sched_relinquish correctly? Or should I be doing something else? I did try using tsleep(...,1), but I don't want my thread sleeping if there are no other threads waiting. I would also be grateful if people could point me at other examples in the kernel where something like this is done. I have looked in quite a few places, but I can't see why my simple app is wrong. thanks Andrew From rysto32 at gmail.com Fri May 8 18:30:34 2009 From: rysto32 at gmail.com (Ryan Stone) Date: Fri May 8 18:30:41 2009 Subject: kthreads and sched_relinquish In-Reply-To: References: Message-ID: Your kernel thread likely has a higher priority than userspace threads. Ryan Stone From kostikbel at gmail.com Fri May 8 20:12:13 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Fri May 8 20:12:20 2009 Subject: fdescfs brokenness In-Reply-To: <4A03A202.2050101@freebsd.org> References: <4A03A202.2050101@freebsd.org> Message-ID: <20090508201203.GJ1948@deviant.kiev.zoral.com.ua> On Thu, May 07, 2009 at 08:07:46PM -0700, Tim Kientzle wrote: > Colin Percival recently pointed out some issues > with tar and fdescfs. Part of the problem > here is tar; I need to rethink some of the > traversal logic. > > But fdescfs is really wonky: > > * This is a nit, but: ls /dev/fd/18 should not > return EBADF; it should return ENOENT, just > like any other reference to a non-existent filename. > (Just because a filename reflects a file descriptor > does not mean it is a file descriptor.) This is a traditional behaviour for fdescfs. According to man page, open("dev/fd/N") shall be equivalent to fcntl(N, F_DUPFD, 0). Solaris behaviour is the same. > > * The fairly routine recursive directory walker > below gets hung in fdescfs. It appears that > the two opendir() invocations active at the > same time interfere with each other. What you mean by "gets hung" ? In my limited testing, it works. Opendir creates a new directory in /dir/fd by the mere fact of opening the directory. So it walks into that dir, returning to step 1. > > * A similar chdir()-based version of the directory > walker below breaks badly; you can chdir() into > a directory under /dev/fd, but you can't chdir("..") > to get back out of it. (This is the particular > problem that tar is running afoul of.) Not sure about this one. I think that fdescfs vnodes do not support lookup on anything not being root of the fdescfs. > > * Running "find /dev/fd" generates bogus ENOENT errors > because you can opendir() a directory inside of /dev/fd, > and read the entries, but you can't access those entries > because path searches don't work through fdescfs. Again, this may be a consequence of the previous issue. > > I think the right solution here is to add a VOP_ACCESS > handler to fdescfs that bars all access to directory > nodes under /dev/fd. Basically, if your program has a > directory open, that should be reflected as a directory > node that you can't do anything with. The current implementation > allows you to chdir(), opendir(), etc, those directory > nodes, but the machinery to fully support those operations > is missing so they just screw things up. This would chomp the fdescfs functionality, IMHO. Why directory file descriptors should behave differently then any other file descriptor ? I think that the actual solution for the walker problems is to ignore the synthetic filesystems altogether. The information is provided by sysctl vfs.conflist (note that the output is binary), see VFCF_* flags, esp. VFCF_SYNTHETIC. The flag is correctly set at least by procfs, devfs and fdescfs. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090508/6321e52c/attachment.pgp From xorquewasp at googlemail.com Fri May 8 21:11:06 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 8 21:11:13 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090506152222.GC69468@logik.internal.network> References: <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> Message-ID: <20090508211022.GA37475@logik.internal.network> Just an update. Finally managed to get the i386 -> amd64 compiler to compile gcc 4.4.0. It took a few Makefile patches as for some reason, cross compilation breaks gnatmake. About to try to get the amd64 compiler to compile itself and run the test suite. Added a system-freebsd_x86_64.ads profile and checked all the other references to freebsd in the source. I've scripted the entire bootstrap process so it should be easy to produce binaries for a port. xw From ed at 80386.nl Fri May 8 21:41:19 2009 From: ed at 80386.nl (Ed Schouten) Date: Fri May 8 21:41:26 2009 Subject: concurrent sysctl implementation In-Reply-To: References: Message-ID: <20090508214117.GY58540@hoeg.nl> Hi, * vasanth raonaik wrote: > Hello Jt, > > I am a newbee in this alias. I am having a very basic question. It would be > really good if you could give me some of this information. > Could you please elaborate on what is the current architecture of sysctl > implementation and How the concurrency would benefit us. Right now sysctl is synchronized using the sysctl lock. The problem is that certain sysctls just block for a very long time (especially some of the GEOM ones). We also call sysctl when we execute new processes, to obtain a random number for the stack protector. This means we have quite a lot of contention on it. This lock needs to be there, because sysctls can be added and removed on demand. I think I discussed this with John Baldwin (right?) and I think we also have the issue that we cannot allow an unbounded amount of concurrent calls to sysctl, because sysctl wires memory. A solution would be to solve it as follows: - Use a semaphore, initialized to some insane high value to put an upper limit on the amount of concurrent sysctl calls. I'm not sure whether this is really needed. Maybe this issue is not as serious as we think it is. - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up the lock when we traverse parts of the sysctl tree that has dynamically created entries. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090508/f1271d7d/attachment.pgp From jilles at stack.nl Fri May 8 23:05:35 2009 From: jilles at stack.nl (Jilles Tjoelker) Date: Fri May 8 23:05:42 2009 Subject: fdescfs brokenness In-Reply-To: <20090508201203.GJ1948@deviant.kiev.zoral.com.ua> References: <4A03A202.2050101@freebsd.org> <20090508201203.GJ1948@deviant.kiev.zoral.com.ua> Message-ID: <20090508230531.GA8413@stack.nl> On Fri, May 08, 2009 at 11:12:03PM +0300, Kostik Belousov wrote: > On Thu, May 07, 2009 at 08:07:46PM -0700, Tim Kientzle wrote: > > Colin Percival recently pointed out some issues > > with tar and fdescfs. Part of the problem > > here is tar; I need to rethink some of the > > traversal logic. > > But fdescfs is really wonky: > > * This is a nit, but: ls /dev/fd/18 should not > > return EBADF; it should return ENOENT, just > > like any other reference to a non-existent filename. > > (Just because a filename reflects a file descriptor > > does not mean it is a file descriptor.) > This is a traditional behaviour for fdescfs. According to man page, > open("dev/fd/N") shall be equivalent to fcntl(N, F_DUPFD, 0). > Solaris behaviour is the same. On open, yes, but stat behaves differently on a Solaris 10 machine here. A valid but unallocated fd number will still stat as a character device, like an allocated fd. % ls -l /dev/fd/0 /dev/fd/999 crw-rw-rw- 1 root root 320, 0 May 9 00:06 /dev/fd/0 crw-rw-rw- 1 root root 320, 999 May 9 00:06 /dev/fd/999 By the way, both FreeBSD and Solaris also behave strangely if you try to access fd numbers 1<<32 or higher. Linux seems to behave strangely as well: the fds show up as symlinks, some of which do not contain valid file names but can still be opened. However, a command like { read x <&5; read y > * The fairly routine recursive directory walker > > below gets hung in fdescfs. It appears that > > the two opendir() invocations active at the > > same time interfere with each other. > What you mean by "gets hung" ? In my limited testing, it works. > Opendir creates a new directory in /dir/fd by the mere fact of opening > the directory. So it walks into that dir, returning to step 1. > > * A similar chdir()-based version of the directory > > walker below breaks badly; you can chdir() into > > a directory under /dev/fd, but you can't chdir("..") > > to get back out of it. (This is the particular > > problem that tar is running afoul of.) > Not sure about this one. I think that fdescfs vnodes do not support > lookup on anything not being root of the fdescfs. > > * Running "find /dev/fd" generates bogus ENOENT errors > > because you can opendir() a directory inside of /dev/fd, > > and read the entries, but you can't access those entries > > because path searches don't work through fdescfs. > Again, this may be a consequence of the previous issue. > > I think the right solution here is to add a VOP_ACCESS > > handler to fdescfs that bars all access to directory > > nodes under /dev/fd. Basically, if your program has a > > directory open, that should be reflected as a directory > > node that you can't do anything with. The current implementation > > allows you to chdir(), opendir(), etc, those directory > > nodes, but the machinery to fully support those operations > > is missing so they just screw things up. > This would chomp the fdescfs functionality, IMHO. Why directory > file descriptors should behave differently then any other file > descriptor ? Linux and Solaris do not have these problems because their /dev/fd does not copy stat information from the underlying file, instead showing a character device (Solaris) or a symlink (Linux). I think a character device would fit best, because you can do little else with it than open it. The open operation is also different from opening the underlying file because it does not create a new open file description. devfs's /dev/fd/0, /dev/fd/1 and /dev/fd/2 work like this as well: they always show up as character devices no matter what the underlying file is. When opened, they duplicate the respective fd just like the full /dev/fd does. (These are located at the end of /sys/kern/kern_descrip.c.) Apparently someone noticed earlier this could be a problem, because the R and X mode bits are cleared from directories that show up in /dev/fd. It does not come as a surprise to me that that hack does not work. > I think that the actual solution for the walker problems is to > ignore the synthetic filesystems altogether. The information is > provided by sysctl vfs.conflist (note that the output is binary), > see VFCF_* flags, esp. VFCF_SYNTHETIC. The flag is correctly > set at least by procfs, devfs and fdescfs. I think it should be possible to write a directory walker program using only standard interfaces. -- Jilles Tjoelker From jt at 0xabadba.be Sat May 9 00:13:44 2009 From: jt at 0xabadba.be (jt@0xabadba.be) Date: Sat May 9 00:13:50 2009 Subject: concurrent sysctl implementation In-Reply-To: <20090508214117.GY58540@hoeg.nl> References: <20090508214117.GY58540@hoeg.nl> Message-ID: <3C2B6A87-C127-4E1F-9A1E-DADF613B18BF@0xabadba.be> Ed, Thanks :) I'll be implementing this as discussed over the next few months thanks for the technical detail I've been extremely busy with finals. I will write the list with my thoughts within the next week, sorry for the delay. =jt On May 8, 2009, at 17:41, Ed Schouten wrote: > Hi, > > * vasanth raonaik wrote: >> Hello Jt, >> >> I am a newbee in this alias. I am having a very basic question. It >> would be >> really good if you could give me some of this information. >> Could you please elaborate on what is the current architecture of >> sysctl >> implementation and How the concurrency would benefit us. > > Right now sysctl is synchronized using the sysctl lock. The problem is > that certain sysctls just block for a very long time (especially > some of > the GEOM ones). We also call sysctl when we execute new processes, to > obtain a random number for the stack protector. This means we have > quite > a lot of contention on it. This lock needs to be there, because > sysctls > can be added and removed on demand. > > I think I discussed this with John Baldwin (right?) and I think we > also > have the issue that we cannot allow an unbounded amount of concurrent > calls to sysctl, because sysctl wires memory. > > A solution would be to solve it as follows: > > - Use a semaphore, initialized to some insane high value to put an > upper > limit on the amount of concurrent sysctl calls. I'm not sure whether > this is really needed. Maybe this issue is not as serious as we think > it is. > > - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up > the lock when we traverse parts of the sysctl tree that has > dynamically created entries. > > -- > Ed Schouten > WWW: http://80386.nl/ From lars.engels at 0x20.net Sat May 9 11:51:38 2009 From: lars.engels at 0x20.net (Lars Engels) Date: Sat May 9 11:57:27 2009 Subject: concurrent sysctl implementation In-Reply-To: <20090508214117.GY58540@hoeg.nl> References: <20090508214117.GY58540@hoeg.nl> Message-ID: <20090509113459.GD56667@e.0x20.net> On Fri, May 08, 2009 at 11:41:17PM +0200, Ed Schouten wrote: > Hi, > > * vasanth raonaik wrote: > > Hello Jt, > > > > I am a newbee in this alias. I am having a very basic question. It would be > > really good if you could give me some of this information. > > Could you please elaborate on what is the current architecture of sysctl > > implementation and How the concurrency would benefit us. > > Right now sysctl is synchronized using the sysctl lock. The problem is > that certain sysctls just block for a very long time (especially some of > the GEOM ones). We also call sysctl when we execute new processes, to > obtain a random number for the stack protector. This means we have quite > a lot of contention on it. This lock needs to be there, because sysctls > can be added and removed on demand. Why is sysctl used to get a random number? Can't we get a different source for it? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090509/3f2427f5/attachment.pgp From ed at 80386.nl Sat May 9 12:13:15 2009 From: ed at 80386.nl (Ed Schouten) Date: Sat May 9 12:13:22 2009 Subject: concurrent sysctl implementation In-Reply-To: <20090509113459.GD56667@e.0x20.net> References: <20090508214117.GY58540@hoeg.nl> <20090509113459.GD56667@e.0x20.net> Message-ID: <20090509121313.GA58540@hoeg.nl> Hi Lars, * Lars Engels wrote: > Why is sysctl used to get a random number? Can't we get a different > source for it? We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. I think other people (Rink Springer) also reported this issue years ago, even before we had the stack protector enabled. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090509/c95c658d/attachment.pgp From kostikbel at gmail.com Sat May 9 13:34:16 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Sat May 9 13:34:22 2009 Subject: fdescfs brokenness In-Reply-To: <20090508230531.GA8413@stack.nl> References: <4A03A202.2050101@freebsd.org> <20090508201203.GJ1948@deviant.kiev.zoral.com.ua> <20090508230531.GA8413@stack.nl> Message-ID: <20090509133409.GL1948@deviant.kiev.zoral.com.ua> On Sat, May 09, 2009 at 01:05:31AM +0200, Jilles Tjoelker wrote: > On Fri, May 08, 2009 at 11:12:03PM +0300, Kostik Belousov wrote: > > On Thu, May 07, 2009 at 08:07:46PM -0700, Tim Kientzle wrote: > > > Colin Percival recently pointed out some issues > > > with tar and fdescfs. Part of the problem > > > here is tar; I need to rethink some of the > > > traversal logic. > > > > But fdescfs is really wonky: > > > > * This is a nit, but: ls /dev/fd/18 should not > > > return EBADF; it should return ENOENT, just > > > like any other reference to a non-existent filename. > > > (Just because a filename reflects a file descriptor > > > does not mean it is a file descriptor.) > > This is a traditional behaviour for fdescfs. According to man page, > > open("dev/fd/N") shall be equivalent to fcntl(N, F_DUPFD, 0). > > Solaris behaviour is the same. > > On open, yes, but stat behaves differently on a Solaris 10 machine here. > A valid but unallocated fd number will still stat as a character > device, like an allocated fd. > > % ls -l /dev/fd/0 /dev/fd/999 > crw-rw-rw- 1 root root 320, 0 May 9 00:06 /dev/fd/0 > crw-rw-rw- 1 root root 320, 999 May 9 00:06 /dev/fd/999 Yes, this makes sense. > > By the way, both FreeBSD and Solaris also behave strangely if you try to > access fd numbers 1<<32 or higher. The strangeness is purely comsetical, in my opinion, but still. > > Linux seems to behave strangely as well: the fds show up as symlinks, > some of which do not contain valid file names but can still be opened. > However, a command like > { read x <&5; read y which shows the first three lines under FreeBSD and Solaris, > shows the first line three times under Linux, so apparently it does not > duplicate file descriptors (at least in some cases). > I think it should be possible to write a directory walker program using > only standard interfaces. For standard-compiant fses, yes. AFAIR POSIX does not make any claims for the whole namespace. I did liked the idea of turning fdescfs nodes to character devices for stat(2). Besides fixing the issue, it also prevents recursive descent into the vfs, preventing the LOR. Being there, added check for the overflow. diff --git a/sys/fs/fdescfs/fdesc_vnops.c b/sys/fs/fdescfs/fdesc_vnops.c index d1788ae..9846357 100644 --- a/sys/fs/fdescfs/fdesc_vnops.c +++ b/sys/fs/fdescfs/fdesc_vnops.c @@ -264,7 +264,7 @@ fdesc_lookup(ap) struct thread *td = cnp->cn_thread; struct file *fp; int nlen = cnp->cn_namelen; - u_int fd; + u_int fd, fd1; int error; struct vnode *fvp; @@ -296,7 +296,12 @@ fdesc_lookup(ap) error = ENOENT; goto bad; } - fd = 10 * fd + *pname++ - '0'; + fd1 = 10 * fd + *pname++ - '0'; + if (fd1 < fd) { + error = ENOENT; + goto bad; + } + fd = fd1; } if ((error = fget(td, fd, &fp)) != 0) @@ -383,78 +388,34 @@ fdesc_getattr(ap) { struct vnode *vp = ap->a_vp; struct vattr *vap = ap->a_vap; - struct thread *td = curthread; - struct file *fp; - struct stat stb; - u_int fd; - int error = 0; + + vap->va_mode = S_IRUSR|S_IXUSR|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH; + vap->va_fileid = VTOFDESC(vp)->fd_ix; + vap->va_uid = 0; + vap->va_gid = 0; + vap->va_blocksize = DEV_BSIZE; + vap->va_atime.tv_sec = boottime.tv_sec; + vap->va_atime.tv_nsec = 0; + vap->va_mtime = vap->va_atime; + vap->va_ctime = vap->va_mtime; + vap->va_gen = 0; + vap->va_flags = 0; + vap->va_bytes = 0; + vap->va_filerev = 0; switch (VTOFDESC(vp)->fd_type) { case Froot: - vap->va_mode = S_IRUSR|S_IXUSR|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH; vap->va_type = VDIR; vap->va_nlink = 2; vap->va_size = DEV_BSIZE; - vap->va_fileid = VTOFDESC(vp)->fd_ix; - vap->va_uid = 0; - vap->va_gid = 0; - vap->va_blocksize = DEV_BSIZE; - vap->va_atime.tv_sec = boottime.tv_sec; - vap->va_atime.tv_nsec = 0; - vap->va_mtime = vap->va_atime; - vap->va_ctime = vap->va_mtime; - vap->va_gen = 0; - vap->va_flags = 0; vap->va_rdev = NODEV; - vap->va_bytes = 0; - vap->va_filerev = 0; break; case Fdesc: - fd = VTOFDESC(vp)->fd_fd; - - if ((error = fget(td, fd, &fp)) != 0) - return (error); - - bzero(&stb, sizeof(stb)); - error = fo_stat(fp, &stb, td->td_ucred, td); - fdrop(fp, td); - if (error == 0) { - vap->va_type = IFTOVT(stb.st_mode); - vap->va_mode = stb.st_mode; - if (vap->va_type == VDIR) - vap->va_mode &= ~(S_IRUSR|S_IXUSR|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH); - vap->va_nlink = 1; - vap->va_flags = 0; - vap->va_bytes = stb.st_blocks * stb.st_blksize; - vap->va_fileid = VTOFDESC(vp)->fd_ix; - vap->va_size = stb.st_size; - vap->va_blocksize = stb.st_blksize; - vap->va_rdev = stb.st_rdev; - - /* - * If no time data is provided, use the current time. - */ - if (stb.st_atimespec.tv_sec == 0 && - stb.st_atimespec.tv_nsec == 0) - nanotime(&stb.st_atimespec); - - if (stb.st_ctimespec.tv_sec == 0 && - stb.st_ctimespec.tv_nsec == 0) - nanotime(&stb.st_ctimespec); - - if (stb.st_mtimespec.tv_sec == 0 && - stb.st_mtimespec.tv_nsec == 0) - nanotime(&stb.st_mtimespec); - - vap->va_atime = stb.st_atimespec; - vap->va_mtime = stb.st_mtimespec; - vap->va_ctime = stb.st_ctimespec; - vap->va_uid = stb.st_uid; - vap->va_gid = stb.st_gid; - vap->va_gen = 0; - vap->va_filerev = 0; - } + vap->va_type = VCHR; + vap->va_nlink = 1; + vap->va_size = 0; + vap->va_rdev = makedev(0, vap->va_fileid); break; default: @@ -462,9 +423,8 @@ fdesc_getattr(ap) break; } - if (error == 0) - vp->v_type = vap->va_type; - return (error); + vp->v_type = vap->va_type; + return (0); } static int -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090509/cc40a48a/attachment.pgp From subhro.kar at gmail.com Sat May 9 15:06:17 2009 From: subhro.kar at gmail.com (Subhro Kar) Date: Sat May 9 15:06:24 2009 Subject: SoC 2009: Improving ext2fs and making it GPL free In-Reply-To: <20090424013250.GA3672@pcbsd> References: <20090424013250.GA3672@pcbsd> Message-ID: Hello Aditya, What are the things you plan to work on? In other words, what exact improvement are you trying to implement? Thanks Subhro On 24-Apr-09, at 7:02 AM, Aditya Sarawgi wrote: > Hi, > > I'm Aditya Sarawgi from India. I will be working on FreeBSD's ext2fs > as a part of this year's summer of > code program. I will be improving the current implementation and I > will also rewrite parts of ext2fs > under GPL. My mentor is Ulf Lilleengen. For more details you can visit > http://wiki.freebsd.org/SOC2009AdityaSarawgi > > Cheers, > Aditya Sarawgi > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org > " From sarawgi.aditya at gmail.com Sat May 9 16:54:14 2009 From: sarawgi.aditya at gmail.com (Aditya Sarawgi) Date: Sat May 9 16:54:22 2009 Subject: SoC 2009: Improving ext2fs and making it GPL free In-Reply-To: <4A05A273.3010904@cable.net.co> References: <4A05A273.3010904@cable.net.co> Message-ID: <4a05afce.1f588c0a.0ed4.0089@mx.google.com> On Sat, May 09, 2009 at 10:34:11AM -0500, Pedro F. Giffuni wrote: > Hi Aditya; > > Your project is very interesting, specially if you get to freeing us > from the GPL in ext2fs. > > I have a couple of short documents for you. > > First here is the description of the Mach port on which our current code > is based: > http://www.cs.utah.edu/flux/lites/html/ext2fs.html > > Here is the revision document of the NetBSD port: > http://www.feyrer.de/NetBSD/ext2fs-rev1.html > > I am unsure if it will be an easy project or not, or if it's just better > to start from scratch from the complete NetBSD port but if you happen to > have time for additional features, my personal wishlist has: > > - SEEK_HOLE and SEEK_DATA extensions for lseek(2) > - support for gjournal. > > just a wishlist... I know I am too optimistic :). > > best regards, > > Pedro. Hi, Thnaks for the lites document. Porting the entire file system from NetBSD will be quite difficult and the current implementation has only parts under GPL so my plan is to rewrite only those parts. My plan is - freeing ext2fs from GPL. - making it MPSAFE (after SoC). - adding new features like journalling (possibly using gjournal ), extended attribute support etc. Cheers, Aditya Sarawgi From sarawgi.aditya at gmail.com Sat May 9 17:24:53 2009 From: sarawgi.aditya at gmail.com (Aditya Sarawgi) Date: Sat May 9 17:25:05 2009 Subject: SoC 2009: Improving ext2fs and making it GPL free In-Reply-To: References: <20090424013250.GA3672@pcbsd> Message-ID: <4a05bc62.27b38c0a.0a67.ffff81b4@mx.google.com> On Sat, May 09, 2009 at 08:07:15PM +0530, Subhro Kar wrote: > Hello Aditya, > > What are the things you plan to work on? In other words, what exact > improvement are you trying to implement? > > Thanks > Subhro > Hi, I will try to fix the bugs present in the current implementation and a new block allocation policy for it. If time permits I will implement hash_dir support for it and I will work on to make it MPSAFE after SoC :) Cheers, Aditya Sarawgi From brd at FreeBSD.org Sat May 9 19:13:25 2009 From: brd at FreeBSD.org (Brad Davis) Date: Sat May 9 19:13:39 2009 Subject: FreeBSD Status Reports January - March, 2009 Message-ID: <20090509191253.GD40521@valentine.liquidneon.com> FreeBSD Quarterly Status Report Introduction Since the last Status Reports there has been interesting progress in FreeBSD Development. FreeBSD 7.2 was released just a few days ago. Some of the highlights include: Support for superpages in the FreeBSD Virtual Memory subsystem. The FreeBSD Kernel Virtual Address space has been increased to 6GB on amd64. An updated jail(8) subsystem that supports multi-IPv4/IPv6/noIP and much more. Lots of FreeBSD Developers are in Ottawa, Canada attending the FreeBSD Developer Summit that is before BSDCan. BSDCan officially starts tomorrow and should cover lots of interesting topics, see the BSDCan Website for more information. Thanks to all the reporters for the excellent work! We hope you enjoy reading. __________________________________________________________________ Projects * Clang replacing GCC in the base system * Device mmap() Extensions * OpenBSM * Release Engineering * Sysinfo - a set of scripts which document your system * TrustedBSD MAC Framework in GENERIC * VFS/NFS DTrace Probes * VirtualBox on FreeBSD FreeBSD Team Reports * FreeBSD BugBusting Team Architectures * FreeBSD/powerpc G5 Support * FreeBSD/sparc64 UltraSPARC III support Documentation * Dutch Documentation Project * German Documentation Project * Hungarian Documentation Project Google Summer of Code * BSD-licensed text-processing tools __________________________________________________________________ BSD-licensed text-processing tools URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/ soc2008/gabor_textproc Contact: G?bor K?vesd?n Currently, grep is finished and is only waiting for a portbuild test. It is known to be more or less feature complete, while it is much smaller than the GNU version. As for sort, there has been some progress with the complete rewrite and it is lacking few options. Performance is to be measured, as well. Open tasks: 1. Test grep on pointyhat. 2. Complete sort with the missing features. 3. Do performance measurements for sort and look for possible optimization opportunities. 4. Test sort on pointyhat. __________________________________________________________________ Clang replacing GCC in the base system URL: http://wiki.freebsd.org/BuildingFreeBSDWithClang URL: http://git.hoeg.nl/?p=llvm-bmake URL: http://clang.llvm.org/ Contact: Ed Schouten Contact: Roman Divacky Contact: Brooks Davis Contact: Pawel Worach The last 3-4 months we've been working together with the LLVM developers to discuss any bugs and issues we are experiencing with their Clang compiler frontend. The FreeBSD project is looking at the possibility to replace GCC with Clang as a system compiler. It can compile 99% of the FreeBSD world and can compile booting kernel on i386/amd64 but it still contains bugs and its C++ support is still immature. Ed is maintaining a patchset for the FreeBSD sources to replace cc(1) by a Clang binary and bootstrap almost all sources with the Clang compiler. The LLVM developers are very helpful fixing most of the bugs we've reported (over 100). Unfortunately we are currently blocked on some bug reports that prevent us from building libc, libm, libcrypto and various CDDL libraries with Clang but the FreeBSD kernel itself compiles and boots. Open tasks: 1. Testing Clang with compilation of various applications and reporting bugs. 2. Testing the llvm-bmake branch to find more bugs. 3. Arranging an experimental ports build. __________________________________________________________________ Device mmap() Extensions URL: http://www.FreeBSD.org/~jhb/pat/ Contact: John Baldwin GPU device drivers are increasingly requiring more sophisticated support for mapping objects into both userland and the kernel. For example, memory used for textures often needs to be mapped Write-Combining rather than Write-Back. I have recently created three patches to provide several extensions. The first patch allows device drivers to use a different VM object to back specific mmap() calls instead of always using the device pager. The second patch introduces a new VM object type that can map an arbitrary set of physical address ranges. This can be used to let userland mmap PCI BARs, etc. The third patch allows memory mappings to use different caching modes (e.g. Write-Combining or Uncacheable). Together I believe these patches provide the remaining pieces needed for an Nvidia amd64 driver. They will also be useful for future Xorg DRM support as well. The current set of patches can be safely merged back to 7.x as well. Currently I am waiting for review and feedback from several folks. I am hopeful that these patches will be in HEAD soon, prior to the 8.0 freeze. __________________________________________________________________ Dutch Documentation Project URL: http://wiki.freebsd.org/DutchDocumentationProject URL: http://www.freebsd.org/doc/nl/ URL: http://p4web.freebsd.org/@md=d&cd=//&c=pFl@//depot/projects/docproj_nl/ ?ac=83 Contact: Remko Lodder Contact: Ren? Ladan The FreeBSD Dutch Documentation Project is an ongoing project to translate FreeBSD Documentation into the Dutch language. The translation of the Handbook was completed last January. It is kept up-to-date with the English version. Furthermore five articles and the flyer have been translated. Some initial work has been done to translate the website, but most likely more translators are needed to fully realize it. Open tasks: 1. Recruit more translators. 2. Keep the translations up-to-date with the English versions. 3. Finish the translation of the FAQ. 4. Translate more articles and maybe some books. __________________________________________________________________ FreeBSD BugBusting Team URL: http://www.FreeBSD.org/support.html#gnats URL: http://wiki.FreeBSD.org/BugBusting URL: http://people.FreeBSD.org/~linimon/studies/prs/ URL: http://people.freebsd.org/~linimon/studies/prs/recommended_prs.html Contact: Mark Linimon Contact: Remko Lodder We continue to classify PRs as they arrive, with 'tags' corresponding to the kernel subsystem, or man page references for userland PRs. These tags, in turn, produce lists of PRs sorted both by tag and by manpage Mark Linimon (linimon@) has created special reports for the Release Engineering Team to help focus on regressions and other areas of interest relating to the release of FreeBSD 7.2 in the coming weeks. This is a refinement of the 'customized reports for developers' announced in the last status report. A full list of all the automatically generated reports is also available. Any recommendations for reports which do not currently exist but which would be beneficial are welcomed. Mark Linimon also continues attempting to define the general problem and investigating possible new work flow models, and will be presenting on the subject at BSDCan. The list of PRs recommended for committer evaluation by the BugBusting team continues to receive new additions. This list contains PRs, mostly with patches, that the BugBusting team feel are probably ready to be committed as-is, or are probably trivially resolved in the hands of a committer with knowledge of the particular subsystem. All committers are invited to take a look at this list whenever they have a spare 5 minutes and wish to close a PR. Since the last status report, the number of open bugs continued to hover around the 5600 mark, although has began to rise with the 7.2 ports freeze. As always, more help is appreciated, and committers and non-committers alike are invited to join us on #freebsd-bugbusters on EFnet and help close stale PRs or commit patches from valid PRs. Open tasks: 1. Try to find ways to get more committers helping us with closing PRs that the team has already analyzed. 2. Think of some way for committers to only view PRs that have been in some way 'vetted' or 'confirmed'. 3. Generate more publicity for what we've already got in place, and for what we intend to do next. 4. Define new categories, classifications, and states for PRs, that will better match our work flow (in progress). __________________________________________________________________ FreeBSD/powerpc G5 Support Contact: Nathan Whitehorn FreeBSD 8.0-CURRENT now has support for PowerPC CPUs operating in the 64-bit bridge mode. This includes the PowerPC 970 (G5) as well as the POWER3 and POWER4. Currently only Apple systems are known to work. Open tasks: 1. IBM systems currently are not supported due to missing northbridge support. 2. Software fan control on SMU-based Apple G5 systems (G5 iMac, later Powermac G5) is not available. __________________________________________________________________ FreeBSD/sparc64 UltraSPARC III support Contact: Marius Strobl Like announced in the previous status report, support for sun4u-machines based on UltraSPARC III and beyond has been MFC'ed to stable/7 (the last missing piece was r190297) and thus will be present in the upcoming 7.2-RELEASE and can be already tested with 7.2-RC1. Additionally, as of r191076 machfb(4) has been fixed to work with UltraSPARC III and beyond, that fix unfortunately did not make it into 7.2-RC1 but will be in the final version. The X.Org 7.4 and Firefox ports as well as some other gecko-based ones like Seamonkey once again have been fixed to also work and package on sparc64, including on UltraSPARC III and UltraSPARC IIIi based machines equipped with cards driven by creator(4) or machfb(4). The driver for the Sun Cassini/Cassini+ as well as National Semiconductor DP83065 Saturn Gigabit NICs found on-board for example in Fire V440 and as add-on cards is coming along nicely, the last thing which needs to be implemented before it can hit CURRENT is support for jumbo frames. __________________________________________________________________ German Documentation Project URL: https://doc.bsdgroup.de Contact: Johann Kois Contact: Martin Wilke In February 2009 the German version of the FreeBSD Developer's handbook went online. Additionally we managed to update large areas of the FAQ thanks to the contributions of Benedict Reuschling. The website (at least the areas we see as relevant for a translation) is translated and updated constantly. More volunteers are always welcome of course, as there is still plenty of work to be done. Open tasks: 1. Update the existing documentation set (especially the handbook). 2. Read the translations. Check for problems/mistakes. Send feedback. __________________________________________________________________ Hungarian Documentation Project URL: http://www.FreeBSD.org/hu URL: http://www.FreeBSD.org/doc/hu URL: http://wiki.FreeBSD.org/HungarianDocumentationProject URL: http://p4web.freebsd.org/@md=d&cd=//depot/projects/docproj_hu/&c=aXw@// depot/projects/docproj_hu/?ac=83 Contact: G?bor K?vesd?n Contact: G?bor P?li We are proud to announce that the FreeBSD Hungarian web pages have been extended by the following items: * Project news entries, staring from 2009 (HTML, RSS, RDF) * Press releases, starting from 2008 (HTML, RSS) * Events, starting from 2009 (HTML, RSS) * Security advisories (HTML, RSS) We are still hoping that having the FDP Primer translated will encourage others to help our work. Feel free to contribute, every submitted line of translation or feedback is appreciated and is highly welcome. For more information on how to contribute, please read the project's introduction (in Hungarian). Open tasks: 1. Translate news entries, press releases. 2. Translate Release Notes for -CURRENT and 8.X. 3. Translate articles. 4. Translate web pages. 5. Read the translations, send feedback. __________________________________________________________________ OpenBSM URL: http://www.openbsm.org/ Contact: Robert Watson Contact: TrustedBSD audit mailing list The TrustedBSD Project has now released OpenBSM 1.1, the second production release of the OpenBSM code base. OpenBSM 1.1 has been merged to FreeBSD 8-CURRENT, and will be merged to 7-STABLE before FreeBSD 7.3. Major changes since OpenBSM 1.0 include: * Trail files now include the host where the trail is generated. Crash recovery has been improved. Trail expiration based on size and date is now supported; by default trail files will be expired after 10MB of trails. The default individual trail limit is now 2MB. * Mac OS X Snow Leopard is now a fully supported platform; launchd(8) can now be used to launchd auditd(8). Command line tools and libraries are now supported on Mac OS X Leopard. * Extended header tokens are now supported, allowing audit trails to be tagged with a host identifier. IPv6 addresses are now supported in subject tokens. BSM token and record types have been further synchronized to OpenSolaris; support for many new system calls has been added. Local errors and socket types are mapped to and from BSM values. Since the last test release, OpenBSM 1.1 beta 1, 32/64-bit compatibility has been fixed for the auditon(2) system call. A default "expire-after" of 10MB is now set in audit_control(5). Local fcntl(2) arguments are now mapped to wire BSM versions using new APIs. The audit_submit(3) man page has been fixed. A new audit event class has been added for post-login authentication and access control events. Open tasks: 1. Migrate to sbufs in token-encoding. 2. Support for auditing NFS RPCs. __________________________________________________________________ Release Engineering URL: http://www.FreeBSD.org/releng/ Contact: Release Engineering Team The Release Engineering Team (with lots of help from lots of other people) released FreeBSD 7.2 on May 4th, 2009. During this period we have also begun reminding developers of the upcoming FreeBSD 8.0 release cycle which is scheduled to begin in early June 2009 with release targeted at early September 2009. __________________________________________________________________ Sysinfo - a set of scripts which document your system URL: http://danger.rulez.sk/index.php/2009/04/14/sysinfo-a-set-of-scripts-wh ich-document-your-freebsd-system/ URL: https://forums.freebsd.org/showthread.php?p=19321 Contact: Daniel Gerzo Sysinfo a shell script which purpose is to automatically gather system information and document hardware and software configuration of the given host system. The goal is to provide a system operator with descriptive information about an unknown FreeBSD installation. It consists of several modules (also shell scripts), thus is easily extensible and provides an easy way to inspect overall system configuration. It has been written as part of my Bachelor thesis and its development is a work in progress. Therefore, I would appreciate if you could provide me with some feedback as I will defend my thesis soon. Your feedback is welcome at the forums , or alternatively you can send me a private email. The tool itself can now be installed using the Ports tree from the sysutils/sysinfo port. Open tasks: 1. Receive additional feedback. 2. Perform more testing. 3. Extend and improve the tool. __________________________________________________________________ TrustedBSD MAC Framework in GENERIC URL: http://www.trustedBSD.org/mac.html Contact: Robert Watson Contact: TrustedBSD discussion mailing list There is on-going work to allow "options MAC" to be included in the GENERIC kernel for 8.0. This primarily consists of performance work to reduce overhead when policies are used, and eliminate when none are configured. Work to date includes: * The MAC Framework now detects which object types are labeled by policies, and MAC label storage is not allocated when it won't be used. * Add MAC Framework DTrace probes so allow more easy analysis of MAC Framework and policy interactions. * Eliminate mutex-protected reference count used to prevent module unload during entry point invocation, and replace with an sx lock and an rwlock, respectively for long-sleepable and short-sleepable entry points, significantly lowering the overhead of entering the MAC Framework. If no dynamic policies are loaded, no locking overhead is taken. Open tasks: 1. Move to rmlocks for non-sleepable entry points to reduce cache line thrashing under load. 2. Macroize invocation of MAC Framework entry points from the kernel, and perform caller-side determination of whether MAC is enabled in order to avoid additional function call overhead in the caller path if MAC is disabled. __________________________________________________________________ VFS/NFS DTrace Probes Contact: Robert Watson A new DTrace provider, dtnfsclient, has been added to the FreeBSD 8.x kernel, and will be merged to 7.x before 7.3. The following probes are available: * nfsclient:{nfs2,nfs3}:{procname}:start - NFSv2 and NFSv3 RPC start probes * nfsclient:{nfs2,nfs3}:{procname}:done - NFSv2 and NFSv3 RPC done probes * nfsclient:accesscache:: - NFS access cache flush/hit/miss/load probes * nfsclient:attrcache:: - NFS attribute cache flush/hit/miss/done In addition, a number of VFS probes have been added: * vfs:vop:{vopname}:entry - VOP entry probe * vfs:vop:{vopname}:return - VOP return probe * vfs:namei:lookup:entry - VFS name lookup entry probe * vfs:namei:lookup:return - VFS name lookup return probe * vfs:namecache:*:* - VFS namecache enter/enter_negative/fullpath_enter/fullpath_hit/fullpath_miss/full path_return/lookup_hit/lookup_hit_negative/lookup_miss/purge/purge_ negative/purgevfs/zap/zap_negative probes These probes make it much easier to trace NFS and VFS events. Open tasks: 1. Add VFSOP tracing. 2. Add RPC-layer tracing, such as RPC retransmits. 3. Provide decoded NFS RPCs in order to expose transaction IDs and file handles. __________________________________________________________________ VirtualBox on FreeBSD URL: http://miwi.bsdcrew.de/2009/05/virtualbox-on-freebsd/ URL: http://miwi.bsdcrew.de/2009/05/virtualbox-on-freebsd-first-screenshots/ URL: http://vbox.innotek.de/pipermail/vbox-dev/2009-May/001369.html Contact: Beat Gaetzi Contact: Bernhard Froehlich Contact: Dennis Herrmann Contact: Martin Wilke After the first mail from Alexander Eichner on the vbox-dev mailinglist, we started the work on a VirtualBox port. 6 Days was needed to get VirtualBox to start with over 20 patches. We'd like to say thanks to Alexander Eichner, all the VirtualBox Developers, Gustau Perez and Ulf Lilleengen. If you like to play with the current port you can checkout the port here. Please do not ping us about any problems, we know about a lot and are still working to get them all solved before we do an official call for testing. Open tasks: 1. Fix kernel crashes on 7.2-RELEASE. 2. Code cleanup. 3. Fix errors on AMD64. 4. Fix user/permission problems. From raykinsella78 at gmail.com Sun May 10 12:53:53 2009 From: raykinsella78 at gmail.com (Ray Kinsella) Date: Sun May 10 12:54:01 2009 Subject: contigmalloc & access protection failure Message-ID: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> Hi all, I am trying to create a kernel panic with a memory access volition, the memory I am allocating is physically contiguous and is 2 pages in size, I then try to use vm_map_protect to set the access flags of the 2nd page to disables writes, vm_map_protect returns successful but when I write to the page no access volition occurs, what am I missing? My attempt in source code to create the volition is below. Also a question about the FreeBSD memory manager, I am a bit confused, I read the source code of the vm_map_protect function and I see it sets the protection on a vm_map_entry_t, my expectation was protection would be set on vm_page_t, my understanding was this:- each vm_map_t contains 1 or more vm_map_entry_t each vm_map_entry_t contains 1 vm_object_t each vm_object_t contains 1 or more vm_page_t so does this mean that because protection is getting set at vm_map_entry, am I actually protecting more than one page of memory? Thanks Ray Kinsella --------------------------------------------- cut here --------------------------------------------- #include #include #include #include #include #include #include #include #include #include #include #include vm_offset_t palloc_wr; vm_offset_t palloc_r; void _alloc(void); void _free(void); void _alloc(void) { ??? uint32_t retval = 0; ???? ??? palloc_wr = (vm_offset_t) contigmalloc(2 * PAGE_SIZE, ??? ??? ??? M_DEVBUF, 0, 0, (1L << 31), ??? ??? ??? 4096, 1024 * 1024); ??? printf("contigmalloc : 0x%.08x\n", palloc_wr); ??? palloc_r = palloc_wr + PAGE_SIZE; ??? //kernel_map ??? retval = vm_map_protect(&curthread->td_proc->p_vmspace->vm_map ??? ??? ??? , palloc_r, palloc_r + PAGE_SIZE, ??? ??? ??? VM_PROT_ALL, 0); ??? printf("vm_map_protect : %d\n", retval); ??? memset((void *)palloc_r,0xFF, PAGE_SIZE); } void _free(void) { ??? contigfree((void *) palloc_wr, 2 * PAGE_SIZE, M_DEVBUF); } /* The function called at load/unload. */ static int event_handler(struct module *module, int event, void *arg) { ??????? int e = 0; /* Error, 0 for normal return status */ ??????? switch (event) { ??????? case MOD_LOAD: ??????????????? _alloc(); ??????????????? break; ??????? case MOD_UNLOAD: ??? ??? _free(); ??? ??? break; ??????? default: ??????????????? e = EOPNOTSUPP; /* Error, Operation Not Supported */ ??????????????? break; ??????? } ??????? return(e); } /* The second argument of DECLARE_MODULE. */ static moduledata_t mod_conf = { ??? "mod",??? /* module name */ ???? event_handler,? /* event handler */ ???? NULL??????????? /* extra data */ }; DECLARE_MODULE(mod, mod_conf, SI_SUB_DRIVERS, SI_ORDER_MIDDLE); From rysto32 at gmail.com Sun May 10 13:37:57 2009 From: rysto32 at gmail.com (Ryan Stone) Date: Sun May 10 13:38:04 2009 Subject: contigmalloc & access protection failure In-Reply-To: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> References: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> Message-ID: You're loading this through kldload? Then I'd imagine that curthread->td_proc->p_vmspace->vm_map refers to the vm_map of the kldload process, not the kernel. Try using kernel_map directly. Ryan Stone From rysto32 at gmail.com Sun May 10 16:23:01 2009 From: rysto32 at gmail.com (Ryan Stone) Date: Sun May 10 16:23:07 2009 Subject: contigmalloc & access protection failure In-Reply-To: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> References: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> Message-ID: Oh, and you're passing VM_PROT_ALL -- that says "map this read/write/execute". You want VM_PROT_READ Ryan Stone From raykinsella78 at gmail.com Sun May 10 16:58:13 2009 From: raykinsella78 at gmail.com (Ray Kinsella) Date: Sun May 10 16:58:20 2009 Subject: contigmalloc & access protection failure In-Reply-To: References: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> Message-ID: <584ec6bb0905100958m1a0ee883h76eaa41eb566e033@mail.gmail.com> ah I specified VM_PROT_ALL because I though that was "protecting" against reads/writes and executes. I had it the wrong way around, it was that VM_PROT that confused me, my kernel panic works perfectly now thanks. Regards Ray Kinsella On Sun, May 10, 2009 at 5:22 PM, Ryan Stone wrote: > Oh, and you're passing VM_PROT_ALL -- that says "map this > read/write/execute".? You want VM_PROT_READ > > Ryan Stone > From raykinsella78 at gmail.com Sun May 10 17:13:15 2009 From: raykinsella78 at gmail.com (Ray Kinsella) Date: Sun May 10 17:13:22 2009 Subject: contigmalloc & access protection failure In-Reply-To: References: <584ec6bb0905100532n36ae97b1rc5e6e31c23bdb44b@mail.gmail.com> Message-ID: <584ec6bb0905101013p4ee4c3daq68a50d5983d3382a@mail.gmail.com> Hi Ryan, One last thing, thanks for your help, Can you recommend any articles or books on developing with the FreeBSD virtually memory manager? Thanks Ray Kinsella On Sun, May 10, 2009 at 5:22 PM, Ryan Stone wrote: > Oh, and you're passing VM_PROT_ALL -- that says "map this > read/write/execute".? You want VM_PROT_READ > > Ryan Stone > From brampton+freebsd-hackers at gmail.com Mon May 11 10:06:04 2009 From: brampton+freebsd-hackers at gmail.com (Andrew Brampton) Date: Mon May 11 10:06:11 2009 Subject: kthreads and sched_relinquish In-Reply-To: References: Message-ID: 2009/5/8 Ryan Stone : > Your kernel thread likely has a higher priority than userspace threads. > > Ryan Stone > Thanks for your reply Ryan. So that I understand this correctly, if I had two kernel threads, one with a high priority, and one with a low priority, and both are PRI_TIMESHARE, then the high priority thread will run uninterrupted until it sleeps? Or is it that kernel threads trumps userspace threads? >From my experience in userspace, all threads will get a chance to run, even if there is a higher priority thread ready to run. The exact problem I am having is that this code (written by someone else) has implemented their own spin locks (which of course does not sleep), so when the lower priority user thread obtains the lock, and higher priority thread sometimes gets rescheduled before the user thread has released the lock. Now the high priority thread spins forever waiting for it to be released, which doesn't seem to give the lower thread a chance. Of course the correct solution to this is to remove these custom built spin locks and start to use the locking mechanisms provided by FreeBSD. While I've started to do that, I wanted to explore this more for my general understanding. thanks Andrew From jhb at freebsd.org Mon May 11 16:49:42 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon May 11 16:50:49 2009 Subject: concurrent sysctl implementation In-Reply-To: <20090508214117.GY58540@hoeg.nl> References: <20090508214117.GY58540@hoeg.nl> Message-ID: <200905111224.26856.jhb@freebsd.org> On Friday 08 May 2009 5:41:17 pm Ed Schouten wrote: > A solution would be to solve it as follows: > > - Use a semaphore, initialized to some insane high value to put an upper > limit on the amount of concurrent sysctl calls. I'm not sure whether > this is really needed. Maybe this issue is not as serious as we think > it is. Well, one compromise might be to allow concurrent userland requests if the buffer size is small (say < 1 page). This would be a quite simple change and would cover many common syscalls like fetching an int which don't wire memory anyway. > - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up > the lock when we traverse parts of the sysctl tree that has > dynamically created entries. I don't think further work is needed here for the tree, notice that in-kernel sysctls are already concurrent and use a read lock on the tree. -- John Baldwin From jhb at freebsd.org Mon May 11 18:25:08 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon May 11 18:25:14 2009 Subject: POSIXfy readlink() call In-Reply-To: <46FDBAFC.1010000@gmail.com> References: <46FDBAFC.1010000@gmail.com> Message-ID: <200905111405.07289.jhb@freebsd.org> On Friday 28 September 2007 10:39:56 pm Ighighi wrote: > The POXIX prototype for readlink(2) is: > ssize_t readlink(const char *restrict path, char *restrict buf, size_t > bufsize); It can't simply be corrected as it would change the ABI and thus requires a new system call, etc. However, do you really expect a symlink to be longer than 2^31 on a 64-bit machine? -- John Baldwin From jt at 0xabadba.be Mon May 11 18:28:10 2009 From: jt at 0xabadba.be (jt@0xabadba.be) Date: Mon May 11 18:28:17 2009 Subject: concurrent sysctl implementation In-Reply-To: <200905111224.26856.jhb@freebsd.org> References: <20090508214117.GY58540@hoeg.nl> <200905111224.26856.jhb@freebsd.org> Message-ID: John, Thank you for your input on this matter, I'm excited to write some software for this project since its given me great code to learn from as i've grown up (still a kid though :). My questions are a bit more detailed below. On Mon, May 11, 2009 at 12:24 PM, John Baldwin wrote: > On Friday 08 May 2009 5:41:17 pm Ed Schouten wrote: >> A solution would be to solve it as follows: >> >> - Use a semaphore, initialized to some insane high value to put an upper >> ? limit on the amount of concurrent sysctl calls. I'm not sure whether >> ? this is really needed. Maybe this issue is not as serious as we think >> ? it is. > > Well, one compromise might be to allow concurrent userland requests if the > buffer size is small (say < 1 page). ?This would be a quite simple change and > would cover many common syscalls like fetching an int which don't wire memory > anyway. Why is this a compromise? Isn't concurrent sysctl calls from userland a good thing? What materials would be good to read other than the code and the sysctl manual pages? You said it would be relatively easy to implement this; what methods should I be considering to do this in and what part of the code specifically should I be looking at? > >> - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up >> ? the lock when we traverse parts of the sysctl tree that has >> ? dynamically created entries. > > I don't think further work is needed here for the tree, notice that in-kernel > sysctls are already concurrent and use a read lock on the tree. yes i've seen the locking mechanism, it reminds me of a monitor type system... though from my understanding monitors appear seldom compared to semaphores in the kernel. I assume the lock will need a bit of twiddling with in some areas of the code if I'm going to enable concurrency from userland, when its said that we should consider the things that are dynamic would it be better to implement this with more than one "queue" or list? Instead perhaps break them up into several lists or, more fundamentally, two lists -- those that are dynamically created entries and those that are not -- is this even feasible to distinguish between the two originally and then on the fly later? Thanks a lot! Respectfully, /jt From kostikbel at gmail.com Mon May 11 18:33:15 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Mon May 11 18:33:25 2009 Subject: POSIXfy readlink() call In-Reply-To: <200905111405.07289.jhb@freebsd.org> References: <46FDBAFC.1010000@gmail.com> <200905111405.07289.jhb@freebsd.org> Message-ID: <20090511183309.GB1948@deviant.kiev.zoral.com.ua> On Mon, May 11, 2009 at 02:05:07PM -0400, John Baldwin wrote: > On Friday 28 September 2007 10:39:56 pm Ighighi wrote: ^^^^^ > > The POXIX prototype for readlink(2) is: > > ssize_t readlink(const char *restrict path, char *restrict buf, size_t > > bufsize); > > It can't simply be corrected as it would change the ABI and thus requires a > new system call, etc. However, do you really expect a symlink to be longer > than 2^31 on a 64-bit machine? Yes, I agree that this is ABI change. Meantime, r176215 | ru | 2008-02-12 22:09:04 +0200 (Tue, 12 Feb 2008) | 5 lines Change readlink(2)'s return type and type of the last argument to match POSIX. Prodded by: Alexey Lyashkov I tried to convince ru@ that ABI breakage is not good, but has not succeeded. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090511/8ba34479/attachment.pgp From jhb at freebsd.org Mon May 11 18:46:21 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon May 11 18:46:28 2009 Subject: POSIXfy readlink() call In-Reply-To: <20090511183309.GB1948@deviant.kiev.zoral.com.ua> References: <46FDBAFC.1010000@gmail.com> <200905111405.07289.jhb@freebsd.org> <20090511183309.GB1948@deviant.kiev.zoral.com.ua> Message-ID: <200905111446.14439.jhb@freebsd.org> On Monday 11 May 2009 2:33:09 pm Kostik Belousov wrote: > On Mon, May 11, 2009 at 02:05:07PM -0400, John Baldwin wrote: > > On Friday 28 September 2007 10:39:56 pm Ighighi wrote: > ^^^^^ Yes, I had this stuck in the back of my head from when it first appeared. > > > The POXIX prototype for readlink(2) is: > > > ssize_t readlink(const char *restrict path, char *restrict buf, size_t > > > bufsize); > > > > It can't simply be corrected as it would change the ABI and thus requires a > > new system call, etc. However, do you really expect a symlink to be longer > > than 2^31 on a 64-bit machine? > > Yes, I agree that this is ABI change. > > Meantime, > r176215 | ru | 2008-02-12 22:09:04 +0200 (Tue, 12 Feb 2008) | 5 lines > > Change readlink(2)'s return type and type of the last argument > to match POSIX. > > Prodded by: Alexey Lyashkov > > I tried to convince ru@ that ABI breakage is not good, but has not > succeeded. Ugh, is this only in HEAD? If so, I will back it out for 8.0. If this made it into a release then this is a far bigger mess. Oh, good, this is only in 8. I will fix this ASAP. I can just add the new syscall I guess. -- John Baldwin From kostikbel at gmail.com Mon May 11 18:58:20 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Mon May 11 18:58:26 2009 Subject: POSIXfy readlink() call In-Reply-To: <200905111446.14439.jhb@freebsd.org> References: <46FDBAFC.1010000@gmail.com> <200905111405.07289.jhb@freebsd.org> <20090511183309.GB1948@deviant.kiev.zoral.com.ua> <200905111446.14439.jhb@freebsd.org> Message-ID: <20090511185814.GD1948@deviant.kiev.zoral.com.ua> On Mon, May 11, 2009 at 02:46:14PM -0400, John Baldwin wrote: > On Monday 11 May 2009 2:33:09 pm Kostik Belousov wrote: > > On Mon, May 11, 2009 at 02:05:07PM -0400, John Baldwin wrote: > > > On Friday 28 September 2007 10:39:56 pm Ighighi wrote: > > ^^^^^ > > Yes, I had this stuck in the back of my head from when it first appeared. > > > > > The POXIX prototype for readlink(2) is: > > > > ssize_t readlink(const char *restrict path, char *restrict buf, size_t > > > > bufsize); > > > > > > It can't simply be corrected as it would change the ABI and thus requires > a > > > new system call, etc. However, do you really expect a symlink to be > longer > > > than 2^31 on a 64-bit machine? > > > > Yes, I agree that this is ABI change. > > > > Meantime, > > r176215 | ru | 2008-02-12 22:09:04 +0200 (Tue, 12 Feb 2008) | 5 lines > > > > Change readlink(2)'s return type and type of the last argument > > to match POSIX. > > > > Prodded by: Alexey Lyashkov > > > > I tried to convince ru@ that ABI breakage is not good, but has not > > succeeded. > > Ugh, is this only in HEAD? If so, I will back it out for 8.0. If this made > it into a release then this is a far bigger mess. Oh, good, this is only in > 8. I will fix this ASAP. I can just add the new syscall I guess. You need to symver the syscalls. It requires some ugly games with our syscall stubs, because gnu ld only honor .symver in the same object where the symbol is defined. I did prototyped this some time ago, by including a file with appropriate .symver from all stubs. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090511/2335b573/attachment.pgp From jhb at freebsd.org Mon May 11 22:14:26 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon May 11 22:14:45 2009 Subject: POSIXfy readlink() call In-Reply-To: <20090511185814.GD1948@deviant.kiev.zoral.com.ua> References: <46FDBAFC.1010000@gmail.com> <200905111446.14439.jhb@freebsd.org> <20090511185814.GD1948@deviant.kiev.zoral.com.ua> Message-ID: <200905111623.52881.jhb@freebsd.org> On Monday 11 May 2009 2:58:14 pm Kostik Belousov wrote: > On Mon, May 11, 2009 at 02:46:14PM -0400, John Baldwin wrote: > > On Monday 11 May 2009 2:33:09 pm Kostik Belousov wrote: > > > On Mon, May 11, 2009 at 02:05:07PM -0400, John Baldwin wrote: > > > > On Friday 28 September 2007 10:39:56 pm Ighighi wrote: > > > ^^^^^ > > > > Yes, I had this stuck in the back of my head from when it first appeared. > > > > > > > The POXIX prototype for readlink(2) is: > > > > > ssize_t readlink(const char *restrict path, char *restrict buf, size_t > > > > > bufsize); > > > > > > > > It can't simply be corrected as it would change the ABI and thus requires > > a > > > > new system call, etc. However, do you really expect a symlink to be > > longer > > > > than 2^31 on a 64-bit machine? > > > > > > Yes, I agree that this is ABI change. > > > > > > Meantime, > > > r176215 | ru | 2008-02-12 22:09:04 +0200 (Tue, 12 Feb 2008) | 5 lines > > > > > > Change readlink(2)'s return type and type of the last argument > > > to match POSIX. > > > > > > Prodded by: Alexey Lyashkov > > > > > > I tried to convince ru@ that ABI breakage is not good, but has not > > > succeeded. > > > > Ugh, is this only in HEAD? If so, I will back it out for 8.0. If this made > > it into a release then this is a far bigger mess. Oh, good, this is only in > > 8. I will fix this ASAP. I can just add the new syscall I guess. > > You need to symver the syscalls. It requires some ugly games with our > syscall stubs, because gnu ld only honor .symver in the same object where > the symbol is defined. I did prototyped this some time ago, by including > a file with appropriate .symver from all stubs. So, after thinking about this out loud some more, it seems the ABI breakage would only be for 64-bit platforms that passed a -ve value as the buffer size. However, doing so would already either panic due to triggering an assertion, or result in otherwise undefined behavior and that making the new parameter unsigned actually results in the same undefined behavior in the non-panic case. -- John Baldwin From jhb at freebsd.org Mon May 11 22:14:27 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon May 11 22:14:46 2009 Subject: concurrent sysctl implementation In-Reply-To: References: <200905111224.26856.jhb@freebsd.org> Message-ID: <200905111801.18767.jhb@freebsd.org> On Monday 11 May 2009 2:27:48 pm jt@0xabadba.be wrote: > John, > > Thank you for your input on this matter, I'm excited to write > some software for this project since its given me great code to learn > from as i've grown up (still a kid though :). My questions are a bit > more detailed below. > > On Mon, May 11, 2009 at 12:24 PM, John Baldwin wrote: > > On Friday 08 May 2009 5:41:17 pm Ed Schouten wrote: > >> A solution would be to solve it as follows: > >> > >> - Use a semaphore, initialized to some insane high value to put an upper > >> ? limit on the amount of concurrent sysctl calls. I'm not sure whether > >> ? this is really needed. Maybe this issue is not as serious as we think > >> ? it is. > > > > Well, one compromise might be to allow concurrent userland requests if the > > buffer size is small (say < 1 page). ?This would be a quite simple change and > > would cover many common syscalls like fetching an int which don't wire memory > > anyway. > > Why is this a compromise? Isn't concurrent sysctl calls from userland > a good thing? What materials would be good to read other than the > code and the sysctl manual pages? You said it would be relatively > easy to implement this; what methods should I be considering to do > this in and what part of the code specifically should I be looking at? Well, in theory a bunch of "small" requests to SYSCTL_PROC() nodes that used sysctl_wire_old() (or whatever it is called) could cause the amount of user memory wired for sysctls to grow unbounded. Thus, allowing this limited concurrency is a tradeoff as there is a minimal (perhaps only theoretical at the moment) risk of removing the safety net. The patch is quite small, btw, because the locking for the sysctl tree already exists, and by using read locks, one can already allow concurrent sysctl requests. There is no need to add any new locks or restructure the sysctl tree, just to adjust the locking that is already present. It might be clearer, in fact, to split the sysctl memory lock back out into a separate lock. This would allow "small" sysctl requests to run concurrently with a single "large" request whereas in my suggestion in the earlier e-mail, the "large" request will block all other user requests until it finishes. I've actually gone ahead and done this below. --- //depot/projects/smpng/sys/kern/kern_sysctl.c 2009/05/08 11:53:25 +++ //depot/user/jhb/lock/kern/kern_sysctl.c 2009/05/11 21:58:08 @@ -77,11 +77,12 @@ * API rather than using the dynamic API. Use of the dynamic API is * strongly encouraged for most code. * - * This lock is also used to serialize userland sysctl requests. Some - * sysctls wire user memory, and serializing the requests limits the - * amount of wired user memory in use. + * The sysctlmemlock is used to limit the amount of user memory wired for + * sysctl requests. This is implemented by serializing any userland + * sysctl requests larger than a single page via an exclusive lock. */ static struct sx sysctllock; +static struct sx sysctlmemlock; #define SYSCTL_SLOCK() sx_slock(&sysctllock) #define SYSCTL_SUNLOCK() sx_sunlock(&sysctllock) @@ -543,6 +544,7 @@ { struct sysctl_oid **oidp; + sx_init(&sysctlmemlock, "sysctl mem"); SYSCTL_INIT(); SYSCTL_XLOCK(); SET_FOREACH(oidp, sysctl_set) @@ -1563,7 +1565,7 @@ size_t *oldlenp, int inkernel, void *new, size_t newlen, size_t *retval, int flags) { - int error = 0; + int error = 0, memlocked; struct sysctl_req req; bzero(&req, sizeof req); @@ -1603,14 +1605,20 @@ if (KTRPOINT(curthread, KTR_SYSCTL)) ktrsysctl(name, namelen); #endif - - SYSCTL_XLOCK(); + + if (req.oldlen > PAGE_SIZE) { + memlocked = 1; + sx_xlock(&sysctlmemlock); + } else + memlocked = 0; CURVNET_SET(TD_TO_VNET(curthread)); for (;;) { req.oldidx = 0; req.newidx = 0; + SYSCTL_SLOCK(); error = sysctl_root(0, name, namelen, &req); + SYSCTL_SUNLOCK(); if (error != EAGAIN) break; uio_yield(); @@ -1620,7 +1628,8 @@ if (req.lock == REQ_WIRED && req.validlen > 0) vsunlock(req.oldptr, req.validlen); - SYSCTL_XUNLOCK(); + if (memlocked) + sx_xunlock(&sysctlmemlock); if (error && error != ENOMEM) return (error); -- John Baldwin From to.my.trociny at gmail.com Tue May 12 06:27:35 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Tue May 12 06:28:06 2009 Subject: Memory leak on thread removal Message-ID: <814ovqn8dp.fsf@zhuzha.ua1> Hi, The code below is compiled with -fopenmp and run on FreeBSD6/7 (i386, amd64): #include #include int n = 4, m = 2; int main () { for (;;) { int i; //sleep(2); #pragma omp parallel for num_threads(m) for(i = 0; i < 1; i++) {} //sleep(2); #pragma omp parallel for num_threads(n) for(i = 0; i < 1; i++) {} } return 0; } During the run the program's virtual memory usage constantly grows. The growth is observed only when n != m. When running the program with uncommented sleep() and observing the number of threads with 'top -H' I see in turn 2 or 4 threads. So it looks like memory leak when thread is removed. Should I fill PR? -- Mikolaj Golub From James.McPherson at Sun.COM Tue May 12 06:46:24 2009 From: James.McPherson at Sun.COM (James C. McPherson) Date: Tue May 12 11:24:52 2009 Subject: Kernel Conference Australia 2009 - registrations and agenda pages NOW LIVE Message-ID: <20090512164549.000011a9@blinder> Dear friends, It gives me great pleasure to announce that registrations for Kernel Conference Australia 2009 are now open, at http://au.sun.com/sunnews/events/2009/kernel/ The confirmed agenda and the speaker bio pages are here: http://au.sun.com/sunnews/events/2009/kernel/agenda.jsp http://au.sun.com/sunnews/events/2009/kernel/speakers.jsp I'm looking forward to seeing you there. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel From rwatson at FreeBSD.org Tue May 12 18:54:36 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Tue May 12 18:54:43 2009 Subject: How to invalidate NFS read cache? In-Reply-To: <20090508101555.J47014@gwdu60.gwdg.de> References: <20090508101555.J47014@gwdu60.gwdg.de> Message-ID: On Fri, 8 May 2009, Konrad Heuer wrote: > sporadically, I observe a strange but serious problem in our large NFS > environment. NFS servers are Linux and OS X with StorNext/Xsan cluster > filesystems, NFS clients Linux and FreeBSD. > > NFS client A changes a file, but nfs client B (running on FreeBSD) does > still see the old version. On the NFS server itself, everything looks fine. > > Afaik the FreeBSD kernel invalidates the NFS read cache if file modification > time on the server changed which should happen here but doesn't. Can I force > FreeBSD (e.g. by sysctl setting) to read file buffers again unconditionally > after vfs.nfs.access_cache_timeout seconds have passed? Hi Konrad: Normally, NFS clients implement open-to-close consistency, which dictates that when a close() occurs on client A, all pending writes on the file should be issued to the server before close() returns, so that a signal to client B to open() the file can validate its cache before open() returns. This raises the following question: is client A closing the file, and is client B then opening it? If not: relying on writes being visible on the client B before the close() on A and a fresh open() on B is not guaranteed to work, although we can discuss ways to improve behavior with respect to expectation. Try modifying your application and see if it gets the desired behavior, and then we can discuss ways to improve what you're seeing. If you are: this is probably a bug in our caching and or issuing of NFS RPCs. We cache both attribute and access data -- perhaps there is an open() path where we issue neither RPC? In the case of open, we likely should test for a valid access cache entry, and if there is one, issue an attribute read, and otherwise just issue an access check which will piggyback fresh attribute data on the reply. Perhaps there is a bug here somewhere. A few other misc questions: - Could you confirm you're using NFSv3 on all clients. Are there any special mount options in use? - What version of FreeBSD are you running with? In FreeBSD 8.x, we now have DTrace probes for all of the above events -- VOPs, attribute cache hit/miss/load/flush, access cache hit/miss/load/flush, RPCs, etc, which we can use to debug the problem. I haven't yet MFC'd these to 7.x, but if you're able to run a very fresh 7-STABLE, I can probably produce a patch to add it for you in a few days. Robert N M Watson Computer Laboratory University of Cambridge From rwatson at FreeBSD.org Tue May 12 20:56:13 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Tue May 12 20:56:21 2009 Subject: How to invalidate NFS read cache? In-Reply-To: References: <20090508101555.J47014@gwdu60.gwdg.de> Message-ID: On Tue, 12 May 2009, Robert Watson wrote: > Normally, NFS clients implement open-to-close consistency, which dictates > that when a close() occurs on client A, all pending writes on the file > should be issued to the server before close() returns, so that a signal to > client B to open() the file can validate its cache before open() returns. This should, of course, read "close-to-open consistency" -- I plead jetlag after an overnight flight back form Boston to the UK :-) Robert N M Watson Computer Laboratory University of Cambridge > > This raises the following question: is client A closing the file, and is > client B then opening it? > > If not: relying on writes being visible on the client B before the close() on > A and a fresh open() on B is not guaranteed to work, although we can discuss > ways to improve behavior with respect to expectation. Try modifying your > application and see if it gets the desired behavior, and then we can discuss > ways to improve what you're seeing. > > If you are: this is probably a bug in our caching and or issuing of NFS RPCs. > We cache both attribute and access data -- perhaps there is an open() path > where we issue neither RPC? In the case of open, we likely should test for a > valid access cache entry, and if there is one, issue an attribute read, and > otherwise just issue an access check which will piggyback fresh attribute > data on the reply. Perhaps there is a bug here somewhere. > > A few other misc questions: > > - Could you confirm you're using NFSv3 on all clients. Are there any special > mount options in use? > - What version of FreeBSD are you running with? > > In FreeBSD 8.x, we now have DTrace probes for all of the above events -- > VOPs, attribute cache hit/miss/load/flush, access cache hit/miss/load/flush, > RPCs, etc, which we can use to debug the problem. I haven't yet MFC'd these > to 7.x, but if you're able to run a very fresh 7-STABLE, I can probably > produce a patch to add it for you in a few days. > > Robert N M Watson > Computer Laboratory > University of Cambridge > From ru at freebsd.org Tue May 12 21:04:11 2009 From: ru at freebsd.org (Ruslan Ermilov) Date: Tue May 12 21:04:24 2009 Subject: POSIXfy readlink() call In-Reply-To: <200905111623.52881.jhb@freebsd.org> References: <46FDBAFC.1010000@gmail.com> <200905111446.14439.jhb@freebsd.org> <20090511185814.GD1948@deviant.kiev.zoral.com.ua> <200905111623.52881.jhb@freebsd.org> Message-ID: <20090512203244.GA61891@edoofus.dev.vega.ru> On Mon, May 11, 2009 at 04:23:52PM -0400, John Baldwin wrote: > On Monday 11 May 2009 2:58:14 pm Kostik Belousov wrote: > > On Mon, May 11, 2009 at 02:46:14PM -0400, John Baldwin wrote: > > > On Monday 11 May 2009 2:33:09 pm Kostik Belousov wrote: > > > > On Mon, May 11, 2009 at 02:05:07PM -0400, John Baldwin wrote: > > > > > On Friday 28 September 2007 10:39:56 pm Ighighi wrote: > > > > ^^^^^ > > > > > > Yes, I had this stuck in the back of my head from when it first appeared. > > > > > > > > > The POXIX prototype for readlink(2) is: > > > > > > ssize_t readlink(const char *restrict path, char *restrict buf, > size_t > > > > > > bufsize); > > > > > > > > > > It can't simply be corrected as it would change the ABI and thus > requires > > > a > > > > > new system call, etc. However, do you really expect a symlink to be > > > longer > > > > > than 2^31 on a 64-bit machine? > > > > > > > > Yes, I agree that this is ABI change. > > > > > > > > Meantime, > > > > r176215 | ru | 2008-02-12 22:09:04 +0200 (Tue, 12 Feb 2008) | 5 lines > > > > > > > > Change readlink(2)'s return type and type of the last argument > > > > to match POSIX. > > > > > > > > Prodded by: Alexey Lyashkov > > > > > > > > I tried to convince ru@ that ABI breakage is not good, but has not > > > > succeeded. > > > > > > Ugh, is this only in HEAD? If so, I will back it out for 8.0. If this > made > > > it into a release then this is a far bigger mess. Oh, good, this is only > in > > > 8. I will fix this ASAP. I can just add the new syscall I guess. > > > > You need to symver the syscalls. It requires some ugly games with our > > syscall stubs, because gnu ld only honor .symver in the same object where > > the symbol is defined. I did prototyped this some time ago, by including > > a file with appropriate .symver from all stubs. > > So, after thinking about this out loud some more, it seems the ABI breakage > would only be for 64-bit platforms that passed a -ve value as the buffer > size. However, doing so would already either panic due to triggering an > assertion, or result in otherwise undefined behavior and that making the new > parameter unsigned actually results in the same undefined behavior in the > non-panic case. > For the record. I also suggest (re-)reading a thread http://lists.freebsd.org/pipermail/freebsd-current/2008-February/thread.html#83314 that resulted from the original commit where I try to make it clear that a scary ABI breakage Konstantin mentions is pure artificial. Cheers, -- Ruslan Ermilov ru@FreeBSD.org FreeBSD committer From des at des.no Wed May 13 23:59:24 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 13 23:59:36 2009 Subject: PTE modified bit emulation trap Message-ID: <86tz3o4lb9.fsf@ds4.des.no> Coverity complains about the lack of error checking in the following code in sys/kern/kern_sysctl.c, around line 1390: /* * Touch all the wired pages to avoid PTE modified * bit emulation traps on Alpha while holding locks * in the sysctl handler. */ for (i = (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, cp = req->oldptr; i > 0; i--, cp += PAGE_SIZE) { copyin(cp, &dummy, 1); copyout(&dummy, cp, 1); } Since Alpha is dead, can we remove this, or is it still needed for other platforms? DES -- Dag-Erling Sm?rgrav - des@des.no From jgrosch at mooseriver.com Thu May 14 03:25:09 2009 From: jgrosch at mooseriver.com (Josef Grosch) Date: Thu May 14 03:25:16 2009 Subject: In search of a video card Message-ID: <20090514030634.GA1252@mooseriver.com> I'm in search for a decent video card. I currently have an Nvidia GeForce 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into X. It just hangs. I've been poking around and, based on what I read, some FreeBSD developers and Nvidia have gotten into a finger pointing contest as to what is the problem. Its all very nice but doesn't help me much. So, can any one recommend a good video card? What I'm looking for is * Works with amd64 FreeBSD 7.2 * DVI * PCI-E x16 * 512 MB or more * Not going to cost an arm and a leg Thanks Josef -- Josef Grosch | Another day closer to a | FreeBSD 7.2 jgrosch@MooseRiver.com | Micro$oft free world | Berkeley, Ca. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 155 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/1b1411a3/attachment.pgp From jgrosch at mooseriver.com Thu May 14 04:06:06 2009 From: jgrosch at mooseriver.com (Josef Grosch) Date: Thu May 14 04:06:13 2009 Subject: In search of a video card In-Reply-To: <11167f520905132045j57fc8914n40894ceecbf33ee5@mail.gmail.com> References: <20090514030634.GA1252@mooseriver.com> <11167f520905132045j57fc8914n40894ceecbf33ee5@mail.gmail.com> Message-ID: <20090514040604.GA1418@mooseriver.com> On Wed, May 13, 2009 at 10:45:17PM -0500, Sam Fourman Jr. wrote: > On Wed, May 13, 2009 at 10:06 PM, Josef Grosch wrote: > > > > I'm in search for a decent video card. I currently have an Nvidia GeForce > > 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home > > machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into > > X. It just hangs. I've been poking around and, based on what I read, some > > FreeBSD developers and Nvidia have gotten into a finger pointing contest as > > to what is the problem. Its all very nice but doesn't help me much. > > I can shed some light on the amd64 binary nvidia driver issue > John Baldwin has implimented the features the Nvidia people need. > > http://www.freebsd.org/news/status/report-2009-01-2009-03.html#Device-mmap()-Extensions > > if you read the nvnews fourms Zander made refrence to working on a 64bit driver. > > > Sam Fourman Jr. Great! So there is hope for my Nvidia card. Josef -- Josef Grosch | Another day closer to a | FreeBSD 7.2 jgrosch@MooseRiver.com | Micro$oft free world | Berkeley, Ca. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 155 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/7c48734b/attachment.pgp From sfourman at gmail.com Thu May 14 04:10:17 2009 From: sfourman at gmail.com (Sam Fourman Jr.) Date: Thu May 14 04:10:23 2009 Subject: In search of a video card In-Reply-To: <20090514030634.GA1252@mooseriver.com> References: <20090514030634.GA1252@mooseriver.com> Message-ID: <11167f520905132045j57fc8914n40894ceecbf33ee5@mail.gmail.com> On Wed, May 13, 2009 at 10:06 PM, Josef Grosch wrote: > > I'm in search for a decent video card. I currently have an Nvidia GeForce > 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home > machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into > X. It just hangs. I've been poking around and, based on what I read, some > FreeBSD developers and Nvidia have gotten into a finger pointing contest as > to what is the problem. Its all very nice but doesn't help me much. I can shed some light on the amd64 binary nvidia driver issue John Baldwin has implimented the features the Nvidia people need. http://www.freebsd.org/news/status/report-2009-01-2009-03.html#Device-mmap()-Extensions if you read the nvnews fourms Zander made refrence to working on a 64bit driver. Sam Fourman Jr. From jgrosch at mooseriver.com Thu May 14 04:18:36 2009 From: jgrosch at mooseriver.com (Josef Grosch) Date: Thu May 14 04:18:44 2009 Subject: In search of a video card In-Reply-To: <20090514000203.3d053800@bhuda.mired.org> References: <20090514030634.GA1252@mooseriver.com> <20090514000203.3d053800@bhuda.mired.org> Message-ID: <20090514041834.GB1418@mooseriver.com> On Thu, May 14, 2009 at 12:02:03AM -0400, Mike Meyer wrote: > On Wed, 13 May 2009 20:06:34 -0700 > Josef Grosch wrote: > > > > > I'm in search for a decent video card. I currently have an Nvidia GeForce > > 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home > > machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into > > X. It just hangs. I've been poking around and, based on what I read, some > > FreeBSD developers and Nvidia have gotten into a finger pointing contest as > > to what is the problem. Its all very nice but doesn't help me much. > > > > So, can any one recommend a good video card? What I'm looking for is > > > > * Works with amd64 FreeBSD 7.2 > > * DVI > > * PCI-E x16 > > * 512 MB or more > > * Not going to cost an arm and a leg > > You forgot the critical information: you didn't define what works for > you. In particular, does a card that doesn't do 3d acceleration > "work"? How about 2d? > > The open source Nvidia driver pretty much sucks. Probably because the > proprietary Nvidia blob (which you were presumably using on i386) is > one of the most functional X video drivers around, so there's not a > lot of incentive to work on the open source version. But you can't use > the proprietary blob on amd64, so you get the open source driver, and > - well you've experienced it. > > The ATI radeon driver - it's open source, there's no proprietary blob > - is actually pretty good. Except that 2d & 3d acceleration support > is, um, variable. > > If you have to have 2d & 3d acceleration, I don't believe you have a > good option for FreeBSD on amd64, at least not until the kernel tweaks > Nvidia needs are done (there has been some motion on that front > recently). But I don't really need it, so I haven't spent any time > looking for such a card. > > If you're ok with solid - if basic - video performance, I'll recommend > pretty much any radeon card. I've used a number of them to drive dual > 1920x1200 displays on a variety of systems for a couple of years now. > Amazon has cards that meet your listed requirements for under US$40. > > References: <20090514030634.GA1252@mooseriver.com> Message-ID: <20090514000203.3d053800@bhuda.mired.org> On Wed, 13 May 2009 20:06:34 -0700 Josef Grosch wrote: > > I'm in search for a decent video card. I currently have an Nvidia GeForce > 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home > machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into > X. It just hangs. I've been poking around and, based on what I read, some > FreeBSD developers and Nvidia have gotten into a finger pointing contest as > to what is the problem. Its all very nice but doesn't help me much. > > So, can any one recommend a good video card? What I'm looking for is > > * Works with amd64 FreeBSD 7.2 > * DVI > * PCI-E x16 > * 512 MB or more > * Not going to cost an arm and a leg You forgot the critical information: you didn't define what works for you. In particular, does a card that doesn't do 3d acceleration "work"? How about 2d? The open source Nvidia driver pretty much sucks. Probably because the proprietary Nvidia blob (which you were presumably using on i386) is one of the most functional X video drivers around, so there's not a lot of incentive to work on the open source version. But you can't use the proprietary blob on amd64, so you get the open source driver, and - well you've experienced it. The ATI radeon driver - it's open source, there's no proprietary blob - is actually pretty good. Except that 2d & 3d acceleration support is, um, variable. If you have to have 2d & 3d acceleration, I don't believe you have a good option for FreeBSD on amd64, at least not until the kernel tweaks Nvidia needs are done (there has been some motion on that front recently). But I don't really need it, so I haven't spent any time looking for such a card. If you're ok with solid - if basic - video performance, I'll recommend pretty much any radeon card. I've used a number of them to drive dual 1920x1200 displays on a variety of systems for a couple of years now. Amazon has cards that meet your listed requirements for under US$40. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From adrian at freebsd.org Thu May 14 06:10:37 2009 From: adrian at freebsd.org (Adrian Chadd) Date: Thu May 14 06:10:44 2009 Subject: In search of a video card In-Reply-To: <20090514041834.GB1418@mooseriver.com> References: <20090514030634.GA1252@mooseriver.com> <20090514000203.3d053800@bhuda.mired.org> <20090514041834.GB1418@mooseriver.com> Message-ID: 2009/5/14 Josef Grosch : > I don't need 2d & 3d acceleration. I just need a card that will handle > WindowMaker on a 24 inch Dell monitor at 1920x1200 and as long as the flesh > tones on my JPGs don't suck I'm happy. > > Thanks for the advice. I guess I'm going to make a trip to Fry's Friday. I've experienced no-acceleration radeon driver "desktop" under Ubuntu and FreeBSD; let me please point out how sluggish and horribly slow it is. I gave up trying to get accelerated 3d + dualhead support on the card(s) I was using - apparently the hardware just didn't do a single viewport span across 2 1280x1024 screens :( (The max viewport width was 2048 pixels..) 2d acceleration may be a must for that kind of resolution.. YMMV (and obviously, please report back your findings!) adrian From doconnor at gsoft.com.au Thu May 14 07:08:03 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Thu May 14 07:08:41 2009 Subject: In search of a video card In-Reply-To: References: <20090514030634.GA1252@mooseriver.com> <20090514041834.GB1418@mooseriver.com> Message-ID: <200905141610.15344.doconnor@gsoft.com.au> On Thu, 14 May 2009, Adrian Chadd wrote: > 2009/5/14 Josef Grosch : > > I don't need 2d & 3d acceleration. I just need a card that will > > handle WindowMaker on a 24 inch Dell monitor at 1920x1200 and as > > long as the flesh tones on my JPGs don't suck I'm happy. > > > > Thanks for the advice. I guess I'm going to make a trip to Fry's > > Friday. > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > and FreeBSD; let me please point out how sluggish and horribly slow > it is. Depends how unaccelerated it is. VESA is pretty damn slow, but even minimal radeon/radeonhd support is fast enough for desktop use. > I gave up trying to get accelerated 3d + dualhead support on the > card(s) I was using - apparently the hardware just didn't do a single > viewport span across 2 1280x1024 screens :( > (The max viewport width was 2048 pixels..) > > 2d acceleration may be a must for that kind of resolution.. If you have a fast CPU & decent pipe to video memory it isn't necessary, but very nice. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/62103734/attachment.pgp From doconnor at gsoft.com.au Thu May 14 07:08:03 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Thu May 14 07:08:42 2009 Subject: In search of a video card In-Reply-To: References: <20090514030634.GA1252@mooseriver.com> <20090514041834.GB1418@mooseriver.com> Message-ID: <200905141610.15344.doconnor@gsoft.com.au> On Thu, 14 May 2009, Adrian Chadd wrote: > 2009/5/14 Josef Grosch : > > I don't need 2d & 3d acceleration. I just need a card that will > > handle WindowMaker on a 24 inch Dell monitor at 1920x1200 and as > > long as the flesh tones on my JPGs don't suck I'm happy. > > > > Thanks for the advice. I guess I'm going to make a trip to Fry's > > Friday. > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > and FreeBSD; let me please point out how sluggish and horribly slow > it is. Depends how unaccelerated it is. VESA is pretty damn slow, but even minimal radeon/radeonhd support is fast enough for desktop use. > I gave up trying to get accelerated 3d + dualhead support on the > card(s) I was using - apparently the hardware just didn't do a single > viewport span across 2 1280x1024 screens :( > (The max viewport width was 2048 pixels..) > > 2d acceleration may be a must for that kind of resolution.. If you have a fast CPU & decent pipe to video memory it isn't necessary, but very nice. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/62103734/attachment-0001.pgp From rdivacky at freebsd.org Thu May 14 09:13:22 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Thu May 14 09:13:29 2009 Subject: In search of a video card In-Reply-To: <20090514030634.GA1252@mooseriver.com> References: <20090514030634.GA1252@mooseriver.com> Message-ID: <20090514091252.GA4341@freebsd.org> On Wed, May 13, 2009 at 08:06:34PM -0700, Josef Grosch wrote: > > I'm in search for a decent video card. I currently have an Nvidia GeForce > 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home > machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into > X. It just hangs. I've been poking around and, based on what I read, some > FreeBSD developers and Nvidia have gotten into a finger pointing contest as > to what is the problem. Its all very nice but doesn't help me much. > > So, can any one recommend a good video card? What I'm looking for is > > * Works with amd64 FreeBSD 7.2 > * DVI > * PCI-E x16 > * 512 MB or more > * Not going to cost an arm and a leg you can try nouveau driver in the meantime From xorquewasp at googlemail.com Thu May 14 09:57:57 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Thu May 14 09:58:04 2009 Subject: In search of a video card In-Reply-To: <20090514030634.GA1252@mooseriver.com> References: <20090514030634.GA1252@mooseriver.com> Message-ID: <20090514092938.GA14415@logik.internal.network> On 2009-05-13 20:06:34, Josef Grosch wrote: > I'm in search for a decent video card. I had to do the same thing recently. I wanted dual head (2x 1440x800), 3D accelerated, PCI Express, AMD64 capable graphics. Finally settled on a Radeon x1950, specifically the Powercolor x1950. The prices vary wildly, I got mine off ebay for about $80 but I've seen people attempt to charge ~$200. Don't let someone rip you off... So far, the card's been absolutely stable. I have a dual-head setup with two identical monitors and the card gives excellent performance for OpenGL work. Running FreeBSD 7.2-RELEASE-amd64. xw From screwdriver at lxnt.info Thu May 14 11:35:38 2009 From: screwdriver at lxnt.info (Alexander Sabourenkov) Date: Thu May 14 11:35:45 2009 Subject: In search of a video card In-Reply-To: <20090514092938.GA14415@logik.internal.network> References: <20090514030634.GA1252@mooseriver.com> <20090514092938.GA14415@logik.internal.network> Message-ID: <4A0BFCFE.2070907@lxnt.info> xorquewasp@googlemail.com wrote: > Finally settled on a Radeon x1950 I second that. PCIe X1950Pro with 256Mb RAM is best card price/{performance, stability} - wise you can get right now. -- ./lxnt From o.petrachev at sprinthost.ru Thu May 14 11:44:01 2009 From: o.petrachev at sprinthost.ru (=?UTF-8?B?0J7Qu9C10LMg0J/QtdGC0YDQsNGH0ZHQsg==?=) Date: Thu May 14 11:49:41 2009 Subject: ipfw uid rules for lo0 interface Message-ID: <4A0C0187.1030107@sprinthost.ru> Hello! I am using FreeBSD 7.2-RELEASE. I am trying to restrict connections to local smtp daemon to limited number of users. But when I create rules for ipfw with uid pattern, I don't get the desired result: all connections on 25 port are blocked and it is impossible to allow it for anyone. I am using the following rules (let's say only root is allowed send messages): # ipfw flush # ipfw add 100 allow ip from any to me 25 uid root # ipfw add 200 deny ip from any to me 25 # telnet localhost 25 Trying 127.0.0.1... And nothing is happening - the connection is neither allowed nor denied, it just hangs. What am I doing wrong? Thanks in advance! From rnoland at FreeBSD.org Thu May 14 14:40:36 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Thu May 14 14:40:48 2009 Subject: In search of a video card In-Reply-To: <200905141610.15344.doconnor@gsoft.com.au> References: <20090514030634.GA1252@mooseriver.com> <20090514041834.GB1418@mooseriver.com> <200905141610.15344.doconnor@gsoft.com.au> Message-ID: <1242311989.1755.65.camel@balrog.2hip.net> On Thu, 2009-05-14 at 16:10 +0930, Daniel O'Connor wrote: > On Thu, 14 May 2009, Adrian Chadd wrote: > > 2009/5/14 Josef Grosch : > > > I don't need 2d & 3d acceleration. I just need a card that will > > > handle WindowMaker on a 24 inch Dell monitor at 1920x1200 and as > > > long as the flesh tones on my JPGs don't suck I'm happy. > > > > > > Thanks for the advice. I guess I'm going to make a trip to Fry's > > > Friday. > > > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > > and FreeBSD; let me please point out how sluggish and horribly slow > > it is. > > Depends how unaccelerated it is. > > VESA is pretty damn slow, but even minimal radeon/radeonhd support is > fast enough for desktop use. > > > I gave up trying to get accelerated 3d + dualhead support on the > > card(s) I was using - apparently the hardware just didn't do a single > > viewport span across 2 1280x1024 screens :( > > (The max viewport width was 2048 pixels..) > > > > 2d acceleration may be a must for that kind of resolution.. > > If you have a fast CPU & decent pipe to video memory it isn't necessary, > but very nice. Folks, 2d (EXA) should work on virtually any radeon with 7.2 or 8.0. 3d is currently works on r500 and below. AMD has released preliminary code for 3d on r600+, which I've been working with lately. It isn't ready yet, but it will be soon. If you only need 2d, then Nvidia is also an option. At least if you are ok with running my patch to enable nouveau drm. I've recently managed to get 3d sortof working on NV50 as well, not sure what kind of timeline that will be though, since it is using gallium and isn't as well tested as the old dri code. robert. -- Robert Noland FreeBSD -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/f2c3bf27/attachment.pgp From rnoland at FreeBSD.org Thu May 14 14:40:36 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Thu May 14 14:40:49 2009 Subject: In search of a video card In-Reply-To: <200905141610.15344.doconnor@gsoft.com.au> References: <20090514030634.GA1252@mooseriver.com> <20090514041834.GB1418@mooseriver.com> <200905141610.15344.doconnor@gsoft.com.au> Message-ID: <1242311989.1755.65.camel@balrog.2hip.net> On Thu, 2009-05-14 at 16:10 +0930, Daniel O'Connor wrote: > On Thu, 14 May 2009, Adrian Chadd wrote: > > 2009/5/14 Josef Grosch : > > > I don't need 2d & 3d acceleration. I just need a card that will > > > handle WindowMaker on a 24 inch Dell monitor at 1920x1200 and as > > > long as the flesh tones on my JPGs don't suck I'm happy. > > > > > > Thanks for the advice. I guess I'm going to make a trip to Fry's > > > Friday. > > > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > > and FreeBSD; let me please point out how sluggish and horribly slow > > it is. > > Depends how unaccelerated it is. > > VESA is pretty damn slow, but even minimal radeon/radeonhd support is > fast enough for desktop use. > > > I gave up trying to get accelerated 3d + dualhead support on the > > card(s) I was using - apparently the hardware just didn't do a single > > viewport span across 2 1280x1024 screens :( > > (The max viewport width was 2048 pixels..) > > > > 2d acceleration may be a must for that kind of resolution.. > > If you have a fast CPU & decent pipe to video memory it isn't necessary, > but very nice. Folks, 2d (EXA) should work on virtually any radeon with 7.2 or 8.0. 3d is currently works on r500 and below. AMD has released preliminary code for 3d on r600+, which I've been working with lately. It isn't ready yet, but it will be soon. If you only need 2d, then Nvidia is also an option. At least if you are ok with running my patch to enable nouveau drm. I've recently managed to get 3d sortof working on NV50 as well, not sure what kind of timeline that will be though, since it is using gallium and isn't as well tested as the old dri code. robert. -- Robert Noland FreeBSD -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/f2c3bf27/attachment-0001.pgp From rnoland at FreeBSD.org Thu May 14 15:05:43 2009 From: rnoland at FreeBSD.org (Robert Noland) Date: Thu May 14 15:05:49 2009 Subject: In search of a video card In-Reply-To: <20090514030634.GA1252@mooseriver.com> References: <20090514030634.GA1252@mooseriver.com> Message-ID: <1242311248.1755.59.camel@balrog.2hip.net> On Wed, 2009-05-13 at 20:06 -0700, Josef Grosch wrote: > I'm in search for a decent video card. I currently have an Nvidia GeForce > 8400 GS. It worked pretty well i386 FreeBSD 6.2. I have upgraded my home > machine and I am running amd64 FreeBSD 7.2 and it just refuses to go into > X. It just hangs. I've been poking around and, based on what I read, some > FreeBSD developers and Nvidia have gotten into a finger pointing contest as > to what is the problem. Its all very nice but doesn't help me much. > > So, can any one recommend a good video card? What I'm looking for is > > * Works with amd64 FreeBSD 7.2 > * DVI > * PCI-E x16 > * 512 MB or more > * Not going to cost an arm and a leg > radeon robert. > > Thanks > > Josef -- Robert Noland FreeBSD -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/aeea18e5/attachment.pgp From jhb at freebsd.org Thu May 14 16:06:29 2009 From: jhb at freebsd.org (John Baldwin) Date: Thu May 14 16:06:40 2009 Subject: PTE modified bit emulation trap In-Reply-To: <86tz3o4lb9.fsf@ds4.des.no> References: <86tz3o4lb9.fsf@ds4.des.no> Message-ID: <200905140807.35680.jhb@freebsd.org> On Wednesday 13 May 2009 7:47:38 pm Dag-Erling Sm?rgrav wrote: > Coverity complains about the lack of error checking in the following > code in sys/kern/kern_sysctl.c, around line 1390: > > /* > * Touch all the wired pages to avoid PTE modified > * bit emulation traps on Alpha while holding locks > * in the sysctl handler. > */ > for (i = (wiredlen + PAGE_SIZE - 1) / PAGE_SIZE, > cp = req->oldptr; i > 0; i--, cp += PAGE_SIZE) { > copyin(cp, &dummy, 1); > copyout(&dummy, cp, 1); > } > > Since Alpha is dead, can we remove this, or is it still needed for other > platforms? I would check MIPS as it might have similar PTE bits as well (FOR, FOW) (many Alpha things are similar to MIPS). I don't have my See MIPS Run handy or I would check it myself. It might be better to replace the loop with a vm_fault(..., VM_FAULT_DIRTY) though if that would have the same effect. -- John Baldwin From mwm at mired.org Thu May 14 15:16:54 2009 From: mwm at mired.org (Mike Meyer) Date: Thu May 14 16:28:30 2009 Subject: In search of a video card In-Reply-To: References: <20090514030634.GA1252@mooseriver.com> <20090514000203.3d053800@bhuda.mired.org> <20090514041834.GB1418@mooseriver.com> Message-ID: <6FBF1838-3CB3-41EC-9178-0993EAE3AB84@mired.org> On May 14, 2009, at 1:42, Adrian Chadd wrote: > 2009/5/14 Josef Grosch : > >> I don't need 2d & 3d acceleration. I just need a card that will >> handle >> WindowMaker on a 24 inch Dell monitor at 1920x1200 and as long as >> the flesh >> tones on my JPGs don't suck I'm happy. >> >> Thanks for the advice. I guess I'm going to make a trip to Fry's >> Friday. > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > and FreeBSD; let me please point out how sluggish and horribly slow it > Is. Well, the stock Ubuntu desktop is Gnome, which I find sluggish and horribly slow on amd64 hardware even with Nvidias 2d & 3d acceleration at any size above 1024x768 - at least in it's default config. If all depends on your window manager, config and expectations. (John Baldwin's message of "Thu, 14 May 2009 08:07:35 -0400") References: <86tz3o4lb9.fsf@ds4.des.no> <200905140807.35680.jhb@freebsd.org> Message-ID: <86preb39yd.fsf@ds4.des.no> John Baldwin writes: > It might be better to replace the loop with a > vm_fault(..., VM_FAULT_DIRTY) though if that would have the same effect. That was going to be my next question: there must be a more elegant way of doing this :) DES -- Dag-Erling Sm?rgrav - des@des.no From jhs at berklix.org Thu May 14 17:32:38 2009 From: jhs at berklix.org (Julian Stacey) Date: Thu May 14 17:32:47 2009 Subject: FreeBSD jobs Message-ID: <200905141653.n4EGr4AW009720@fire.js.berklix.net> Hi hackers@ A commercial firm asked for _Free_ labour today on jobs@freebsd. The censors passed it. Censors of jobs@freebsd.org then blocked the posting below. jobs@ censors again bad, block wrong things, should all be removed & not replaced. Several suckers have already enquired to that firm. Hope we might get some Free labour to donate time to Freebsd, Not Stock holders ! Exact full copy of what Censors blocked: > > ... employer Juniper Networks. ... > > Extra consideration will be given to applicants that can work more > > than 2 hours/day. > > NOTE: This is an unpaid position! > > Juniper Networks is a commercial company, Not a charity. > http://www.juniper.net/us/en/company/investor-relations/#earnings-sec > Unpaid volunteers better working free for FreeBSD > http://freebsdfoundation.org/activities.shtml > A registered charity, > Projects for _Unpaid_ workers > http://wiki.freebsd.org/ > SOC projects > http://www.freebsd.org/news/newsflash.html#event20090510:01 > Some SOC & other projects that didnt get funding would still benefit from > unpaid help. FreeBSD code projects & server admin > would surely love to have an intern donating 2 or more hours a day > free, to benefit FreeBSD globaly, not just Juniper share holders. Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From raysonlogin at gmail.com Thu May 14 18:37:43 2009 From: raysonlogin at gmail.com (Rayson Ho) Date: Thu May 14 18:37:51 2009 Subject: FreeBSD jobs In-Reply-To: <200905141653.n4EGr4AW009720@fire.js.berklix.net> References: <200905141653.n4EGr4AW009720@fire.js.berklix.net> Message-ID: <73a01bf20905141115s8fffdc3t12060c82982e5812@mail.gmail.com> Thanks for letting us know! I am sure at least a few of us here will buy less products from Juniper Networks. Rayson On Thu, May 14, 2009 at 11:53 AM, Julian Stacey wrote: > Hi hackers@ > A commercial firm asked for _Free_ labour today on jobs@freebsd. > The censors passed it. Censors of jobs@freebsd.org then blocked > the posting below. jobs@ censors again bad, block wrong things, > should all be removed & not replaced. > > Several suckers have already enquired to that firm. Hope we might get > some Free labour to donate time to Freebsd, Not Stock holders ! > > Exact full copy of what Censors blocked: >> > ... employer Juniper Networks. ... >> > Extra consideration will be given to applicants that can work more >> > than 2 hours/day. >> > NOTE: This is an unpaid position! >> >> Juniper Networks is a commercial company, Not a charity. >> http://www.juniper.net/us/en/company/investor-relations/#earnings-sec >> Unpaid volunteers better working free for FreeBSD >> http://freebsdfoundation.org/activities.shtml >> A registered charity, >> Projects for _Unpaid_ workers >> http://wiki.freebsd.org/ >> SOC projects >> http://www.freebsd.org/news/newsflash.html#event20090510:01 >> Some SOC & other projects that didnt get funding would still benefit from >> unpaid help. FreeBSD code projects & server admin >> would surely love to have an intern donating 2 or more hours a day >> free, to benefit FreeBSD globaly, not just Juniper share holders. > > Cheers, > Julian > -- > Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com > Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From matt at ixsystems.com Thu May 14 18:46:07 2009 From: matt at ixsystems.com (Matt Olander) Date: Thu May 14 18:46:14 2009 Subject: FreeBSD jobs In-Reply-To: <200905141653.n4EGr4AW009720@fire.js.berklix.net> References: <200905141653.n4EGr4AW009720@fire.js.berklix.net> Message-ID: <9740caf0905141119o193eda9h1bde4dbbb8f8645e@mail.gmail.com> On Thu, May 14, 2009 at 9:53 AM, Julian Stacey wrote: > Hi hackers@ > A commercial firm asked for _Free_ labour today on jobs@freebsd. > The censors passed it. ?Censors of jobs@freebsd.org then blocked > the posting below. ?jobs@ censors again bad, block wrong things, > should all be removed & not replaced. > > Several suckers have already enquired to that firm. ?Hope we might get > some Free labour to donate time to Freebsd, Not Stock holders ! Hi Julian, Internships are an accepted way for a high school or university student (and nowadays some post grad students and others) to gain a bit of experience in their field before joining the work force or perhaps while switching careers. At my company, we've filled several full-time positions with people that were interns first. It's just a way to fill a part-time, sometimes non-paid job, at a company where there isn't an official requisition for that particular position. Nobody is forcing anybody to take the internship and it is clearly stated that it is a non-paid internship in the post. I imagine that there would be some interested students or unemployed people that would love to work with Alfred on a project at Juniper a few hours a week in their spare time, for free. It will look great on a resume, they will probably learn some valuable skills, and perhaps parlay it into a full-time, paid position. best, -matt > > Exact full copy of what Censors blocked: >> > ... employer Juniper Networks. ... >> > Extra consideration will be given to applicants that can work more >> > than 2 hours/day. >> > NOTE: This is an unpaid position! >> >> Juniper Networks is a commercial company, Not a charity. >> ? ? ? http://www.juniper.net/us/en/company/investor-relations/#earnings-sec >> Unpaid volunteers better working free for FreeBSD >> ? ? ? http://freebsdfoundation.org/activities.shtml >> ? ? ? A registered charity, >> Projects for _Unpaid_ workers >> ? ? ? http://wiki.freebsd.org/ >> SOC projects >> ? ? ? http://www.freebsd.org/news/newsflash.html#event20090510:01 >> Some SOC & other projects that didnt get funding would still benefit from >> unpaid help. ?FreeBSD code projects & server admin >> would surely love to have an intern donating 2 or more hours a day >> free, to benefit FreeBSD globaly, not just Juniper share holders. > > Cheers, > Julian > -- > Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com > ?Mail plain ASCII text. ?HTML & Base64 text are spam. www.asciiribbon.org > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From rick-freebsd2008 at kiwi-computer.com Thu May 14 19:25:14 2009 From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty) Date: Thu May 14 19:25:21 2009 Subject: In search of a video card In-Reply-To: <20090514030634.GA1252@mooseriver.com> References: <20090514030634.GA1252@mooseriver.com> Message-ID: <20090514185826.GA45111@keira.kiwi-computer.com> On Wed, May 13, 2009 at 08:06:34PM -0700, Josef Grosch wrote: > > So, can any one recommend a good video card? What I'm looking for is > > * Works with amd64 FreeBSD 7.2 > * DVI > * PCI-E x16 > * 512 MB or more > * Not going to cost an arm and a leg I just bought an nVidia and an ATI on newegg for under $30 each (plus mail-in rebates), buth PCI x16, DVI+VGA, 512MB. I have tried them both on amd64 and this is what I've discovered: - The ATI is quite fast for xvideo (via mplayer) and somewhat fast on 3D, there seems to be some cursor-related artifacts (I've tried a number of cards with the same results). - The nVidia card required Robert Noland's patches applied against RELENG_7, and the latest git checkouts of the nouveau driver and the libdrm code. I forcibly installed xf86-video-nouveau and then "gmake install"'d the nouveau build on top of it. I did the same for libdrm. It was very slow for xvideo (mplayer) and 3D acceleration was nonexistent. I could not get the "nv" driver to work in dual-head mode nor in single-head mode. Nouveau didn't support my onboard card either. Neither of these drivers comes remotely close to the performance of the same cards using the nvidia driver (on i386). I tried ati and nouveau on i386 for comparisons. The nvidia driver did support my onboard video. I ran these tests on a couple of ATI cards and various nvidia cards. It was very frustrating. I'm using the ati/radeon driver currently on amd64 until nvidia finishes their 64-bit driver. YMMV, -- Rick C. Petty From rick-freebsd2008 at kiwi-computer.com Thu May 14 19:27:07 2009 From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty) Date: Thu May 14 19:27:14 2009 Subject: In search of a video card In-Reply-To: References: <20090514030634.GA1252@mooseriver.com> <20090514000203.3d053800@bhuda.mired.org> <20090514041834.GB1418@mooseriver.com> Message-ID: <20090514190025.GB45111@keira.kiwi-computer.com> On Thu, May 14, 2009 at 01:42:20PM +0800, Adrian Chadd wrote: > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > and FreeBSD; let me please point out how sluggish and horribly slow it > is. > > I gave up trying to get accelerated 3d + dualhead support on the > card(s) I was using - apparently the hardware just didn't do a single > viewport span across 2 1280x1024 screens :( > (The max viewport width was 2048 pixels..) With the nvidia driver I've seen great 3d acceleration work on both heads without using twinview (because I want different resolutions on each head). Perhaps the ATI hardware isn't capable of this, but nvidia's cards seem to be. It's too bad the open source drivers haven't made enough progress in this area yet. -- Rick C. Petty From rick-freebsd2008 at kiwi-computer.com Thu May 14 19:28:55 2009 From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty) Date: Thu May 14 19:29:02 2009 Subject: In search of a video card In-Reply-To: <200905141610.15344.doconnor@gsoft.com.au> References: <20090514030634.GA1252@mooseriver.com> <20090514041834.GB1418@mooseriver.com> <200905141610.15344.doconnor@gsoft.com.au> Message-ID: <20090514190213.GC45111@keira.kiwi-computer.com> On Thu, May 14, 2009 at 04:10:07PM +0930, Daniel O'Connor wrote: > > Depends how unaccelerated it is. > > VESA is pretty damn slow, but even minimal radeon/radeonhd support is > fast enough for desktop use. VESA doesn't support dual-head, if that's something you're looking for. And I agree that it's particularly slow even in 2d. -- Rick C. Petty From rick-freebsd2008 at kiwi-computer.com Thu May 14 20:00:28 2009 From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty) Date: Thu May 14 20:00:34 2009 Subject: In search of a video card In-Reply-To: References: <20090514030634.GA1252@mooseriver.com> <20090514000203.3d053800@bhuda.mired.org> <20090514041834.GB1418@mooseriver.com> Message-ID: <20090514190025.GB45111@keira.kiwi-computer.com> On Thu, May 14, 2009 at 01:42:20PM +0800, Adrian Chadd wrote: > > I've experienced no-acceleration radeon driver "desktop" under Ubuntu > and FreeBSD; let me please point out how sluggish and horribly slow it > is. > > I gave up trying to get accelerated 3d + dualhead support on the > card(s) I was using - apparently the hardware just didn't do a single > viewport span across 2 1280x1024 screens :( > (The max viewport width was 2048 pixels..) With the nvidia driver I've seen great 3d acceleration work on both heads without using twinview (because I want different resolutions on each head). Perhaps the ATI hardware isn't capable of this, but nvidia's cards seem to be. It's too bad the open source drivers haven't made enough progress in this area yet. -- Rick C. Petty From ed at 80386.nl Thu May 14 21:34:27 2009 From: ed at 80386.nl (Ed Schouten) Date: Thu May 14 21:34:35 2009 Subject: concurrent sysctl implementation In-Reply-To: <200905111801.18767.jhb@freebsd.org> References: <200905111224.26856.jhb@freebsd.org> <200905111801.18767.jhb@freebsd.org> Message-ID: <20090514213426.GP58540@hoeg.nl> * John Baldwin wrote: > Well, in theory a bunch of "small" requests to SYSCTL_PROC() nodes that used > sysctl_wire_old() (or whatever it is called) could cause the amount of user > memory wired for sysctls to grow unbounded. Thus, allowing this limited > concurrency is a tradeoff as there is a minimal (perhaps only theoretical at > the moment) risk of removing the safety net. > > The patch is quite small, btw, because the locking for the sysctl tree already > exists, and by using read locks, one can already allow concurrent sysctl > requests. There is no need to add any new locks or restructure the sysctl > tree, just to adjust the locking that is already present. It might be > clearer, in fact, to split the sysctl memory lock back out into a separate > lock. This would allow "small" sysctl requests to run concurrently with a > single "large" request whereas in my suggestion in the earlier e-mail, > the "large" request will block all other user requests until it finishes. > > I've actually gone ahead and done this below. Boohoo. I actually wanted jt to work on this, as a small exercise to figure out the way locking primitives work in the kernel. No problem, because I can think of dozens of other things. Is there a chance we can see this patch in 8.0? I like it that the memlock is being picked up before we pick up the sysctl lock itself, which makes a lot of sense. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090514/5df5fe94/attachment.pgp From jhb at freebsd.org Thu May 14 21:58:21 2009 From: jhb at freebsd.org (John Baldwin) Date: Thu May 14 21:58:27 2009 Subject: concurrent sysctl implementation In-Reply-To: <20090514213426.GP58540@hoeg.nl> References: <200905111801.18767.jhb@freebsd.org> <20090514213426.GP58540@hoeg.nl> Message-ID: <200905141757.52144.jhb@freebsd.org> On Thursday 14 May 2009 5:34:26 pm Ed Schouten wrote: > * John Baldwin wrote: > > Well, in theory a bunch of "small" requests to SYSCTL_PROC() nodes that used > > sysctl_wire_old() (or whatever it is called) could cause the amount of user > > memory wired for sysctls to grow unbounded. Thus, allowing this limited > > concurrency is a tradeoff as there is a minimal (perhaps only theoretical at > > the moment) risk of removing the safety net. > > > > The patch is quite small, btw, because the locking for the sysctl tree already > > exists, and by using read locks, one can already allow concurrent sysctl > > requests. There is no need to add any new locks or restructure the sysctl > > tree, just to adjust the locking that is already present. It might be > > clearer, in fact, to split the sysctl memory lock back out into a separate > > lock. This would allow "small" sysctl requests to run concurrently with a > > single "large" request whereas in my suggestion in the earlier e-mail, > > the "large" request will block all other user requests until it finishes. > > > > I've actually gone ahead and done this below. > > Boohoo. I actually wanted jt to work on this, as a small exercise to > figure out the way locking primitives work in the kernel. No problem, > because I can think of dozens of other things. > > Is there a chance we can see this patch in 8.0? I like it that the > memlock is being picked up before we pick up the sysctl lock itself, > which makes a lot of sense. Yes, I can commit it. -- John Baldwin From kheuer2 at gwdg.de Fri May 15 05:51:35 2009 From: kheuer2 at gwdg.de (Konrad Heuer) Date: Fri May 15 05:51:42 2009 Subject: How to invalidate NFS read cache? In-Reply-To: References: <20090508101555.J47014@gwdu60.gwdg.de> Message-ID: <20090515073949.M70549@gwdu60.gwdg.de> On Tue, 12 May 2009, Robert Watson wrote: > On Fri, 8 May 2009, Konrad Heuer wrote: > >> sporadically, I observe a strange but serious problem in our large NFS >> environment. NFS servers are Linux and OS X with StorNext/Xsan cluster >> filesystems, NFS clients Linux and FreeBSD. >> >> NFS client A changes a file, but nfs client B (running on FreeBSD) does >> still see the old version. On the NFS server itself, everything looks fine. >> >> Afaik the FreeBSD kernel invalidates the NFS read cache if file >> modification time on the server changed which should happen here but >> doesn't. Can I force FreeBSD (e.g. by sysctl setting) to read file buffers >> again unconditionally after vfs.nfs.access_cache_timeout seconds have >> passed? > > Hi Konrad: > > Normally, NFS clients implement open-to-close consistency, which dictates > that when a close() occurs on client A, all pending writes on the file should > be issued to the server before close() returns, so that a signal to client B > to open() the file can validate its cache before open() returns. > > This raises the following question: is client A closing the file, and is > client B then opening it? > > If not: relying on writes being visible on the client B before the close() on > A and a fresh open() on B is not guaranteed to work, although we can discuss > ways to improve behavior with respect to expectation. Try modifying your > application and see if it gets the desired behavior, and then we can discuss > ways to improve what you're seeing. > > If you are: this is probably a bug in our caching and or issuing of NFS RPCs. > We cache both attribute and access data -- perhaps there is an open() path > where we issue neither RPC? In the case of open, we likely should test for a > valid access cache entry, and if there is one, issue an attribute read, and > otherwise just issue an access check which will piggyback fresh attribute > data on the reply. Perhaps there is a bug here somewhere. > > A few other misc questions: > > - Could you confirm you're using NFSv3 on all clients. Are there any special > mount options in use? > - What version of FreeBSD are you running with? > > In FreeBSD 8.x, we now have DTrace probes for all of the above events -- > VOPs, attribute cache hit/miss/load/flush, access cache hit/miss/load/flush, > RPCs, etc, which we can use to debug the problem. I haven't yet MFC'd these > to 7.x, but if you're able to run a very fresh 7-STABLE, I can probably > produce a patch to add it for you in a few days. Hello, Robert, thank you very much for your reply! The problem I observe happens with FreeBSD 6.4-R and 7.0-R with nfsv3. The fstab entry I use is: server:/Volume /local/dir nfs bg,rw,intr,-T,-r32768,-w16384 0 0 The server runs on Mac OSX 10.5. In the meantime, I had the chance to examine a failure a little bit closer. As far as I can see in the moment a file modified on a Linux NFS client gets a new modification time on the NFS server but the FreeBSD client still sees the old timestamp. This obviously happens sporadically only under some circumstances I do not know further. I'll do some further testing the next days. Could you imagine a kind of directory or metadata caching on FreeBSD NFS clients that may cause this behaviour? Best regards Konrad Konrad Heuer GWDG, Am Fassberg, 37077 Goettingen, Germany, kheuer2@gwdg.de From kheuer2 at gwdg.de Fri May 15 07:34:48 2009 From: kheuer2 at gwdg.de (Konrad Heuer) Date: Fri May 15 07:34:55 2009 Subject: How to invalidate NFS read cache? In-Reply-To: <20090515073949.M70549@gwdu60.gwdg.de> References: <20090508101555.J47014@gwdu60.gwdg.de> <20090515073949.M70549@gwdu60.gwdg.de> Message-ID: <20090515093121.W70549@gwdu60.gwdg.de> On Fri, 15 May 2009, Konrad Heuer wrote: > (...) > The problem I observe happens with FreeBSD 6.4-R and 7.0-R with nfsv3. The > fstab entry I use is: > > server:/Volume /local/dir nfs bg,rw,intr,-T,-r32768,-w16384 0 0 > > The server runs on Mac OSX 10.5. > > In the meantime, I had the chance to examine a failure a little bit closer. > As far as I can see in the moment a file modified on a Linux NFS client gets > a new modification time on the NFS server but the FreeBSD client still sees > the old timestamp. This obviously happens sporadically only under some > circumstances I do not know further. I'll do some further testing the next > days. > > Could you imagine a kind of directory or metadata caching on FreeBSD NFS > clients that may cause this behaviour? I forgot to mention one detail: File modification happens with emacs; thus the modified file is indeed a new one with a new inode number whereas the old version keeps its inode and gets renamed. Best regards Konrad Konrad Heuer GWDG, Am Fassberg, 37077 Goettingen, Germany, kheuer2@gwdg.de From marius at nuenneri.ch Fri May 15 11:48:52 2009 From: marius at nuenneri.ch (=?ISO-8859-1?Q?Marius_N=FCnnerich?=) Date: Fri May 15 11:49:07 2009 Subject: Memory leak on thread removal In-Reply-To: <814ovqn8dp.fsf@zhuzha.ua1> References: <814ovqn8dp.fsf@zhuzha.ua1> Message-ID: On Tue, May 12, 2009 at 08:27, Mikolaj Golub wrote: > Hi, > > The code below is compiled with -fopenmp and run on FreeBSD6/7 (i386, amd64): > > #include > #include > > int n = 4, m = 2; > > int main () { > ? ? ? ?for (;;) { > ? ? ? ? ? ? ? ?int i; > > ? ? ? ? ? ? ? ?//sleep(2); > #pragma omp parallel for num_threads(m) > ? ? ? ? ? ? ? ?for(i = 0; i < 1; i++) {} > > ? ? ? ? ? ? ? ?//sleep(2); > #pragma omp parallel for num_threads(n) > ? ? ? ? ? ? ? ?for(i = 0; i < 1; i++) {} > > ? ? ? ?} > > ? ? ? ?return 0; > } > > During the run the program's virtual memory usage constantly grows. The growth > is observed only when n != m. When running the program with uncommented > sleep() and observing the number of threads with 'top -H' I see in turn 2 or 4 > threads. So it looks like memory leak when thread is removed. Should I fill > PR? I can confirm this. I briefly looked through the libgomp code but didn't see the leak. Anybody knows good tools how to investigate this? From andrit at ukr.net Fri May 15 13:18:25 2009 From: andrit at ukr.net (Andriy Tkachuk) Date: Fri May 15 13:18:32 2009 Subject: Memory leak on thread removal In-Reply-To: References: <814ovqn8dp.fsf@zhuzha.ua1> Message-ID: <4A0D67D8.8070401@ukr.net> On 2009-05-15 14:48, Marius N?nnerich wrote: > Anybody knows good tools how to investigate this? Valgrind? From jhs at berklix.org Fri May 15 14:42:58 2009 From: jhs at berklix.org (Julian Stacey) Date: Fri May 15 14:43:06 2009 Subject: FreeBSD jobs In-Reply-To: Your message "Thu, 14 May 2009 11:19:38 PDT." <9740caf0905141119o193eda9h1bde4dbbb8f8645e@mail.gmail.com> Message-ID: <200905151443.n4FEh9Tf027120@fire.js.berklix.net> Hi Matt, > > Several suckers have already enquired to that firm. =A0Hope we might get > > some Free labour to donate time to Freebsd, Not Stock holders ! > > Hi Julian, > Internships are an accepted way for a high school or university In America. America imported unpaid apprenticeships & indentured servitude (time limited slavery) from Europe/Britain centuries ago. Britain dumped both Long since. (Germany still has some interns eg in theatre, last described as a rip off). Condolences that the "Land of the free" retains medieval European intern practice. > it is clearly stated that it is a non-paid internship in the post. Not clear at start. The word "intern" may be clear warning of No Money in American dialect, but to British eyes, it was a strange word out of context & ignored. ( Monika Lewinsky news first brought the quaint old fashioned American word & concept of Intern back across the Atlantic, but in English, Intern just means certain some workers ). Posting near end had: "San Francisco" Posting in final line had: "This is an unpaid position!" Posting should have started "USA/ San Francisco - Unpaid job" to efficiently enable global readership of jobs@ to delete unread, but American-centric censors of global jobs@ messed up as often, > I imagine that there would be some interested students or unemployed > people that would love to work with Alfred on a project at Juniper a > few hours a week in their spare time, for free. http://lists.freebsd.org/pipermail/freebsd-jobs/2009-May/000652.html "more than 2 hours/day" That's 1/4 or more of a full time job - _Unpaid_ ! No offer of money, food, transport, training courses, books, hardware to keep. It undermines labour rates for all BSD workers. The jobs@freebsd.org censors blocked my polite post people instead work free for freebsd.org > It will look great on > a resume, they will probably learn some valuable skills, and perhaps > parlay it into a full-time, paid position. Managers will read: "This keen sucker can be kept at Low rates for a Long time". Better that FreeBSD people work free Not for scrounging commercial firms & share holder profit, but for charity eg: FreeBSD development projects & systems admin (eg send-pr base, (parlay into a job requiring corporate support/ trouble ticket experience ), FSF, Xorg, Greenpeace, computer automating a local library, FreeBSD training lectures to schools, fixing BSD install etc for blind / half blind etc. Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From alfred at freebsd.org Fri May 15 16:30:58 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Fri May 15 16:31:06 2009 Subject: FreeBSD jobs In-Reply-To: <9740caf0905141119o193eda9h1bde4dbbb8f8645e@mail.gmail.com> Message-ID: <20090515163057.GB4062@elvis.mu.org> > On Thu, May 14, 2009 at 9:53 AM, Julian Stacey wrote: > > Hi hackers@ > > A commercial firm asked for _Free_ labour today on jobs at freebsd. > > The censors passed it. Censors of jobs at freebsd.org then blocked > > the posting below. jobs@ censors again bad, block wrong things, > > should all be removed & not replaced. > > > > Several suckers have already enquired to that firm. Hope we might get > > some Free labour to donate time to Freebsd, Not Stock holders ! > > Hi Julian, > > Internships are an accepted way for a high school or university > student (and nowadays some post grad students and others) to gain a > bit of experience in their field before joining the work force or > perhaps while switching careers. At my company, we've filled several > full-time positions with people that were interns first. It's just a > way to fill a part-time, sometimes non-paid job, at a company where > there isn't an official requisition for that particular position. > Nobody is forcing anybody to take the internship and it is clearly > stated that it is a non-paid internship in the post. > > I imagine that there would be some interested students or unemployed > people that would love to work with Alfred on a project at Juniper a > few hours a week in their spare time, for free. It will look great on > a resume, they will probably learn some valuable skills, and perhaps > parlay it into a full-time, paid position. > > best, > -matt Thanks Matt, this is my only intention. A few of the candidates I've spoken too are very excited to get something on their resume with a commercial entity and there is the hope that I may be able to hire one on them in the future. I've also promised the candidates that they will have access to some amazing resources within Juniper if (I can manage it) and at the very least I can mentor them on any FreeBSD endeavors they take on for the other non-2 hours per-day they would be working for me. While I would love to see more students working on FreeBSD, the fact of the matter is that some students already have worked on FreeBSD and would like commercial experience of "worked on a team in an office environment" that is challenging to mimic in our (FreeBSD's) distributed ways. At the end of the day, what FreeBSD-jobs is supposed to be is a place where jobs can be posted and found that will enable a FreeBSD fan to find suitable employment opportunities for a career, or to advance their career. The reason for moderation of FreeBSD-jobs is to prevent people such as Julian turning a well intentioned message into a thread of flames because he's gone imbalanced due to lack of coffee some morning. Effectively it's been a pretty swell system, FreeBSD-jobs has 0 spam (except to the poor moderators) and also insulated job seekers and posters from the typical hecklers who feel the need for extremely abusive emails due to some real or perceived mistake by the recruiter or job-seeker. I honestly feel that we've even saved plenty of people embarrassment by blocking or bouncing messages that they may have sent in haste to freebsd-jobs that after cooling off realized "the Internet is forever, why in g-d's name did I send something so mean with my name on it?!?". It's a shame it doesn't work for cross-list posts. I'm proud to be one of the moderators on FreeBSD-jobs, but I do admit most of the work is done by the other moderators. Thanks again Matt. I'm going to have to pick your brain later about how to deal with interns, care, feeding, hats? :) And Julian, chill out, I still cringe from embarrassment when someone drags out some old email _I_ sent with close to the same tone as the ones I've been seeing from you. Best of luck. -Alfred From ap at bnc.net Fri May 15 16:48:20 2009 From: ap at bnc.net (Achim Patzner) Date: Fri May 15 16:48:27 2009 Subject: FreeBSD jobs In-Reply-To: <200905151443.n4FEh9Tf027120@fire.js.berklix.net> References: <200905151443.n4FEh9Tf027120@fire.js.berklix.net> Message-ID: <790C89F7-2F10-4601-B984-64B3988BAF82@bnc.net> Am 15.05.2009 um 16:43 schrieb Julian Stacey: >> Internships are an accepted way for a high school or university > > In America. America imported unpaid apprenticeships & indentured > servitude (time limited slavery) from Europe/Britain centuries ago. Take this somewhere else, it's getting boring. Grown-ups should know what they're doing without your protection and the rest might learn a bit on their own. Achim Patzner From jhs at berklix.org Fri May 15 16:54:35 2009 From: jhs at berklix.org (Julian Stacey) Date: Fri May 15 16:54:47 2009 Subject: FreeBSD jobs In-Reply-To: Your message "Fri, 15 May 2009 17:48:11 +0200." <790C89F7-2F10-4601-B984-64B3988BAF82@bnc.net> Message-ID: <200905151655.n4FGsrLM028804@fire.js.berklix.net> > >> Internships are an accepted way for a high school or university > > > > In America. America imported unpaid apprenticeships & indentured > > servitude (time limited slavery) from Europe/Britain centuries ago. > > Take this somewhere else, it's getting boring. Grown-ups should know > what they're doing without your protection and the rest might learn > a bit on their own. You'r right on adults & free choice, I'll drop that rather than drift. What I was trying to illustrate is what jobs@ censors pass & block. - jobs@ is censored, so jobs@ censors performance cant be discussed on jobs@. - Those that pushed to censor jobs@ some years ago (& succesors?) are not worth having, jobs@FreeBSD would be better without them. - Censors of jobs@ do not have the courage to announce on footer or header of jobs@ that they censor jobs@freebsd. - Most don't know jobs@freebsd Is censored. Most think only announce@ is moderated , & maybe arch@. - Moving to chat@ is for things that drift off from FreeSBD, but FreeBSD censorship Is relevant to FreeBSD, - Where better than hackers@ to look for support to liberate jobs@freebsd from censors ? Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From xcllnt at mac.com Fri May 15 17:43:29 2009 From: xcllnt at mac.com (Marcel Moolenaar) Date: Fri May 15 17:43:35 2009 Subject: FreeBSD jobs In-Reply-To: <200905151443.n4FEh9Tf027120@fire.js.berklix.net> References: <200905151443.n4FEh9Tf027120@fire.js.berklix.net> Message-ID: <70A19896-8F17-4D20-998B-CE6644B1E969@mac.com> On May 15, 2009, at 7:43 AM, Julian Stacey wrote: > Hi Matt, >>> Several suckers have already enquired to that firm. =A0Hope we >>> might get >>> some Free labour to donate time to Freebsd, Not Stock holders ! >> >> Hi Julian, >> Internships are an accepted way for a high school or university > > In America. America imported unpaid apprenticeships & indentured > servitude (time limited slavery) from Europe/Britain centuries ago. > Britain dumped both Long since. (Germany still has some interns > eg in theatre, last described as a rip off). Condolences that the > "Land of the free" retains medieval European intern practice. *plonk* -- Marcel Moolenaar xcllnt@mac.com From linimon at lonesome.com Fri May 15 17:29:17 2009 From: linimon at lonesome.com (Mark Linimon) Date: Fri May 15 17:52:57 2009 Subject: FreeBSD jobs In-Reply-To: <200905151655.n4FGsrLM028804@fire.js.berklix.net> References: <790C89F7-2F10-4601-B984-64B3988BAF82@bnc.net> <200905151655.n4FGsrLM028804@fire.js.berklix.net> Message-ID: <20090515171002.GA24620@lonesome.com> On Fri, May 15, 2009 at 06:54:53PM +0200, Julian Stacey wrote: > - Those that pushed to censor jobs@ some years ago (& succesors?) > are not worth having, jobs@FreeBSD would be better without them. In your opinion. Not in mine. > - Where better than hackers@ to look for support to liberate > jobs@freebsd from censors ? chat@. mcl From stas at FreeBSD.org Sat May 16 09:37:18 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Sat May 16 09:37:26 2009 Subject: ipfw uid rules for lo0 interface In-Reply-To: <4A0C0187.1030107@sprinthost.ru> References: <4A0C0187.1030107@sprinthost.ru> Message-ID: <20090516133742.0e26a347.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 14 May 2009 15:33:27 +0400 ???? ???????? mentioned: > Hello! > > I am using FreeBSD 7.2-RELEASE. > > I am trying to restrict connections to local smtp daemon to limited > number of users. But when I create rules for ipfw with uid pattern, I > don't get the desired result: all connections on 25 port are blocked and > it is impossible to allow it for anyone. > > I am using the following rules (let's say only root is allowed send > messages): > > # ipfw flush > # ipfw add 100 allow ip from any to me 25 uid root > # ipfw add 200 deny ip from any to me 25 > > # telnet localhost 25 > Trying 127.0.0.1... > > And nothing is happening - the connection is neither allowed nor denied, > it just hangs. > > What am I doing wrong? Thanks in advance! > That should work. I suspect you don't have anything running on 127.0.0.1:25, otherwise you should have been receiving a "permission denied" message. You can inspect what's binded on which ports/addresses by running `sockstat -4`. - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoOiWsACgkQK/VZk+smlYFcpACeMLylEJRGrP7w0ciiHqT+Xhzz QEsAn2AU5chm06vYZBrX8/7mSDfpnD8P =blL4 -----END PGP SIGNATURE----- !DSPAM:4a0e894c994291748722663! From to.my.trociny at gmail.com Sat May 16 17:05:30 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Sat May 16 17:05:36 2009 Subject: Memory leak on thread removal In-Reply-To: ("Marius =?iso-8859-1?Q?N=FCnnerich=22's?= message of "Fri\, 15 May 2009 13\:48\:51 +0200") References: <814ovqn8dp.fsf@zhuzha.ua1> Message-ID: <86k54hvuzv.fsf@kopusha.onet> On Fri, 15 May 2009 13:48:51 +0200 Marius N?nnerich wrote: MN> On Tue, May 12, 2009 at 08:27, Mikolaj Golub wrote: >> Hi, >> >> The code below is compiled with -fopenmp and run on FreeBSD6/7 (i386, amd64): >> >> #include >> #include >> >> int n = 4, m = 2; >> >> int main () { >> ? ? ? ?for (;;) { >> ? ? ? ? ? ? ? ?int i; >> >> ? ? ? ? ? ? ? ?//sleep(2); >> #pragma omp parallel for num_threads(m) >> ? ? ? ? ? ? ? ?for(i = 0; i < 1; i++) {} >> >> ? ? ? ? ? ? ? ?//sleep(2); >> #pragma omp parallel for num_threads(n) >> ? ? ? ? ? ? ? ?for(i = 0; i < 1; i++) {} >> >> ? ? ? ?} >> >> ? ? ? ?return 0; >> } >> >> During the run the program's virtual memory usage constantly grows. The growth >> is observed only when n != m. When running the program with uncommented >> sleep() and observing the number of threads with 'top -H' I see in turn 2 or 4 >> threads. So it looks like memory leak when thread is removed. Should I fill >> PR? It looks like I have found the leak. The problem is in libgomp/team.c. gomp_thread_start() does sem_init() but sem_destroy() is never called. This patch solves the problem for me: --- contrib/gcclibs/libgomp/team.c.orig 2009-05-16 17:32:57.000000000 +0300 +++ contrib/gcclibs/libgomp/team.c 2009-05-16 19:16:37.000000000 +0300 @@ -164,9 +164,12 @@ new_team (unsigned nthreads, struct gomp static void free_team (struct gomp_team *team) { + int i; free (team->work_shares); gomp_mutex_destroy (&team->work_share_lock); gomp_barrier_destroy (&team->barrier); + for(i = 1; i < team->nthreads; i++) + gomp_sem_destroy (team->ordered_release[i]); gomp_sem_destroy (&team->master_release); free (team); } I am going to fill PR to gcc mainstream, but should I also register this in FreeBSD bugtrack as gcc is part of the base? BTW, the problem is not observed under Linux. I have not looked in Linux code but it looks like sem_init() implementation for Linux does not do memory allocation. The memory for the test program below grows under FreeBSD and does not under Linux. #include int main(int argc, char *argv[]) { sem_t sem; for(;;) { sem_init(&sem, 0, 0);} return 0; } MN> I can confirm this. I briefly looked through the libgomp code but MN> didn't see the leak. Anybody knows good tools how to investigate this? http://freshmeat.net/projects/lmdbg This is a small memory leak debugger. It does not provide all functionality you can find in more sophisticated tools but is lightweight, portable and simple in use. It was very useful when I traced this bug. -- Mikolaj Golub From pieter at degoeje.nl Sat May 16 18:22:17 2009 From: pieter at degoeje.nl (Pieter de Goeje) Date: Sat May 16 18:22:24 2009 Subject: Memory leak on thread removal In-Reply-To: <86k54hvuzv.fsf@kopusha.onet> References: <814ovqn8dp.fsf@zhuzha.ua1> <86k54hvuzv.fsf@kopusha.onet> Message-ID: <200905162021.32545.pieter@degoeje.nl> On Saturday 16 May 2009 17:05:24 Mikolaj Golub wrote: > On Fri, 15 May 2009 13:48:51 +0200 Marius N?nnerich wrote: > > MN> On Tue, May 12, 2009 at 08:27, Mikolaj Golub wrote: > >> Hi, > >> > >> The code below is compiled with -fopenmp and run on FreeBSD6/7 (i386, > >> amd64): > >> > >> #include > >> #include > >> > >> int n = 4, m = 2; > >> > >> int main () { > >> for (;;) { > >> int i; > >> > >> //sleep(2); > >> #pragma omp parallel for num_threads(m) > >> for(i = 0; i < 1; i++) {} > >> > >> //sleep(2); > >> #pragma omp parallel for num_threads(n) > >> for(i = 0; i < 1; i++) {} > >> > >> } > >> > >> return 0; > >> } > >> > >> During the run the program's virtual memory usage constantly grows. The > >> growth is observed only when n != m. When running the program with > >> uncommented sleep() and observing the number of threads with 'top -H' I > >> see in turn 2 or 4 threads. So it looks like memory leak when thread is > >> removed. Should I fill PR? > > It looks like I have found the leak. The problem is in libgomp/team.c. > gomp_thread_start() does sem_init() but sem_destroy() is never called. This > patch solves the problem for me: > > --- contrib/gcclibs/libgomp/team.c.orig 2009-05-16 17:32:57.000000000 +0300 > +++ contrib/gcclibs/libgomp/team.c 2009-05-16 19:16:37.000000000 +0300 > @@ -164,9 +164,12 @@ new_team (unsigned nthreads, struct gomp > static void > free_team (struct gomp_team *team) > { > + int i; > free (team->work_shares); > gomp_mutex_destroy (&team->work_share_lock); > gomp_barrier_destroy (&team->barrier); > + for(i = 1; i < team->nthreads; i++) > + gomp_sem_destroy (team->ordered_release[i]); > gomp_sem_destroy (&team->master_release); > free (team); > } > > I am going to fill PR to gcc mainstream, but should I also register this in > FreeBSD bugtrack as gcc is part of the base? > > BTW, the problem is not observed under Linux. I have not looked in Linux > code but it looks like sem_init() implementation for Linux does not do > memory allocation. The memory for the test program below grows under > FreeBSD and does not under Linux. Note that libgomp uses it's own implementation of POSIX semaphores (using phtread mutexes) instead of FreeBSD's sem(4). If the program below is run against FreeBSD's implementation, it quickly stops growing because there is a limit on the number of allocated semaphores. It would be interesting to see if there are any differences performance wise between the two. If the native version is faster, it might be another reason to include sem(4) in the GENERIC kernel... > > #include > > int > main(int argc, char *argv[]) { > > sem_t sem; > > for(;;) { sem_init(&sem, 0, 0);} > > return 0; > } -- Pieter de Goeje From marius at nuenneri.ch Sat May 16 18:24:11 2009 From: marius at nuenneri.ch (=?ISO-8859-1?Q?Marius_N=FCnnerich?=) Date: Sat May 16 18:24:27 2009 Subject: Memory leak on thread removal In-Reply-To: <86k54hvuzv.fsf@kopusha.onet> References: <814ovqn8dp.fsf@zhuzha.ua1> <86k54hvuzv.fsf@kopusha.onet> Message-ID: On Sat, May 16, 2009 at 19:05, Mikolaj Golub wrote: > > On Fri, 15 May 2009 13:48:51 +0200 Marius N?nnerich wrote: > > ?MN> On Tue, May 12, 2009 at 08:27, Mikolaj Golub wrote: > ?>> Hi, > ?>> > ?>> The code below is compiled with -fopenmp and run on FreeBSD6/7 (i386, amd64): > ?>> > ?>> #include > ?>> #include > ?>> > ?>> int n = 4, m = 2; > ?>> > ?>> int main () { > ?>> ? ? ? ?for (;;) { > ?>> ? ? ? ? ? ? ? ?int i; > ?>> > ?>> ? ? ? ? ? ? ? ?//sleep(2); > ?>> #pragma omp parallel for num_threads(m) > ?>> ? ? ? ? ? ? ? ?for(i = 0; i < 1; i++) {} > ?>> > ?>> ? ? ? ? ? ? ? ?//sleep(2); > ?>> #pragma omp parallel for num_threads(n) > ?>> ? ? ? ? ? ? ? ?for(i = 0; i < 1; i++) {} > ?>> > ?>> ? ? ? ?} > ?>> > ?>> ? ? ? ?return 0; > ?>> } > ?>> > ?>> During the run the program's virtual memory usage constantly grows. The growth > ?>> is observed only when n != m. When running the program with uncommented > ?>> sleep() and observing the number of threads with 'top -H' I see in turn 2 or 4 > ?>> threads. So it looks like memory leak when thread is removed. Should I fill > ?>> PR? > > It looks like I have found the leak. The problem is in libgomp/team.c. > gomp_thread_start() does sem_init() but sem_destroy() is never called. This > patch solves the problem for me: > > --- contrib/gcclibs/libgomp/team.c.orig 2009-05-16 17:32:57.000000000 +0300 > +++ contrib/gcclibs/libgomp/team.c ? ? ?2009-05-16 19:16:37.000000000 +0300 > @@ -164,9 +164,12 @@ new_team (unsigned nthreads, struct gomp > ?static void > ?free_team (struct gomp_team *team) > ?{ > + ?int i; > ? free (team->work_shares); > ? gomp_mutex_destroy (&team->work_share_lock); > ? gomp_barrier_destroy (&team->barrier); > + ?for(i = 1; i < team->nthreads; i++) > + ? ?gomp_sem_destroy (team->ordered_release[i]); > ? gomp_sem_destroy (&team->master_release); > ? free (team); > ?} > > I am going to fill PR to gcc mainstream, but should I also register this in > FreeBSD bugtrack as gcc is part of the base? > > BTW, the problem is not observed under Linux. I have not looked in Linux code > but it looks like sem_init() implementation for Linux does not do memory > allocation. The memory for the test program below grows under FreeBSD and does > not under Linux. > > #include > > int > main(int argc, char *argv[]) { > > ? ? ? ?sem_t sem; > > ? ? ? ?for(;;) { sem_init(&sem, 0, 0);} > > ? ? ? ?return 0; > } Wow! Thanks for tracking this down. I think you can file both PR's so that FreeBSD can include your patch before it comes in via upstream. > > > ?MN> I can confirm this. I briefly looked through the libgomp code but > ?MN> didn't see the leak. Anybody knows good tools how to investigate this? > > http://freshmeat.net/projects/lmdbg > > This is a small memory leak debugger. It does not provide all functionality > you can find in more sophisticated tools but is lightweight, portable and > simple in use. It was very useful when I traced this bug. Thanks, I'll take a look at it. From to.my.trociny at gmail.com Sun May 17 08:39:48 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Sun May 17 08:39:55 2009 Subject: Memory leak on thread removal In-Reply-To: <814ovqn8dp.fsf@zhuzha.ua1> (Mikolaj Golub's message of "Tue\, 12 May 2009 09\:27\:30 +0300") References: <814ovqn8dp.fsf@zhuzha.ua1> Message-ID: <86d4a8unqo.fsf@kopusha.onet> On Tue, 12 May 2009 09:27:30 +0300 Mikolaj Golub wrote: MG> Hi, MG> The code below is compiled with -fopenmp and run on FreeBSD6/7 (i386, amd64): MG> #include MG> #include MG> int n = 4, m = 2; MG> int main () { MG> for (;;) { MG> int i; MG> //sleep(2); MG> #pragma omp parallel for num_threads(m) MG> for(i = 0; i < 1; i++) {} MG> //sleep(2); MG> #pragma omp parallel for num_threads(n) MG> for(i = 0; i < 1; i++) {} MG> MG> } MG> return 0; MG> } MG> During the run the program's virtual memory usage constantly grows. The growth MG> is observed only when n != m. When running the program with uncommented MG> sleep() and observing the number of threads with 'top -H' I see in turn 2 or 4 MG> threads. So it looks like memory leak when thread is removed. Should I fill MG> PR? Reported. http://www.freebsd.org/cgi/query-pr.cgi?pr=134604 -- Mikolaj Golub From stas at FreeBSD.org Sun May 17 10:22:11 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Sun May 17 10:22:18 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.081229.1359784281.imp@bsdimp.com> References: <20090430233648.GA95360@keira.kiwi-computer.com> <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> <20090501.081229.1359784281.imp@bsdimp.com> Message-ID: <20090517142231.2968f311.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 01 May 2009 08:12:29 -0600 (MDT) "M. Warner Losh" mentioned: > > This is a religious point, and we're dangerously close to saying my > religion is better than your religion. I don't like this part of the > proposal at all. I can see the value in relaxing it for when you have > a series of variables that are initialized, but relaxing it to the > point where you intermix code and declarations goes way too far. It > is bad enough to have to deal with inner scopes, but tolerable. It is > intolerable to have to look for it anywhere in a big function. It > tends to encourage spaghetti code, which is one of the things that > style(9) tries to discourage in many subtle ways. > Seconded. Furthermore, when declaring the every advanced editor supports jumping to variables declarations, Christoph ignored the point that the code gets written for people and not for compilers and editors. Last ones can live without any style at all, people can't. The thing people love about BSD code is that it is always perfectly known where to look for declarations and specific parts of the code. Strict style implies a lot of implicit knowledge, so you don't have to study a piece of code for a long time before you understand how it works in general. By relaxing style(9) we're in danger of loosing this benefit. - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoP5WwACgkQK/VZk+smlYFocACfTzVHRpQb8H3tAeg97ljqn3bv DZ4An2iOQXXjTNWpivyHrGR3sBaeOfmJ =qz0I -----END PGP SIGNATURE----- !DSPAM:4a0fe550994292383363236! From stas at FreeBSD.org Sun May 17 10:30:07 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Sun May 17 10:30:13 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FBF5F7.7000600@gmx.de> References: <20090430.183727.803597558.imp@bsdimp.com> <49FA8E88.1040905@gmx.de> <20090501.081229.1359784281.imp@bsdimp.com> <20090501.083712.396385864.imp@bsdimp.com> <49FBF5F7.7000600@gmx.de> Message-ID: <20090517143037.9c62ef1f.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 02 May 2009 09:27:51 +0200 Christoph Mallon mentioned: > I also have to object, that it leads to hunting for declarations. > Actually it usually reduces scrolling around in the code: Many variables > are only assigned once. A typical example is getting the softc of a > device; especially there the type is important, because > device_get_softc() returns void*. So it is very convenient to have this > single assignment and its declaration at the same place. Not only the > type of a variable is important, but also its value, so this assignment > needs to be looked up, too. Doing declaration and initialisation at once > you only have to look at one place instead of two. If you use vim as > editor, then the current way makes it hard to find this assignment: "gd" > jumps only to the declaration, the assignment is somewhere else. But if > the declaration and the only assignment are the same, you get both at > the same place and time. You're talking about automatic text processing tools, which is an entirely different subject. If your current tool can't handle the code, it may be the time to improve the tool or change it. Tools can be improved, people can't. By 'hunting for declarations' it usually meant that it is hard to find pieces of the code by looking into it, not that tools can't handle the task. - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoP500ACgkQK/VZk+smlYGO7QCeM3HRKTMqp54SIo28eBN86vc5 fZcAniVfL+cY4rhP/ulp0MQFbxD+twwL =hJYk -----END PGP SIGNATURE----- !DSPAM:4a0fe72c994292017410001! From stas at FreeBSD.org Sun May 17 10:44:47 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Sun May 17 10:44:54 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <49FAE4EA.1010205@gmx.de> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> Message-ID: <20090517144516.331b01a8.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 01 May 2009 14:02:50 +0200 Christoph Mallon mentioned: > > [Don't parenthesize return values] > >> Removed, because it does not improve maintainability in any way. > > > > This change could be made and tested mechanically. But there is > > no justification for making it - stating that the existing rule > > is no better than the proposed rule is no reason to change. > > Just remove the rule. There's no need to add more redundant parentheses. > Again: There is no need to change all existing code at once, but the > rule is removed so that not anymore redundant parentheses are added. If only the rule gets removed this will lead to inconsistent code. Currently it is much easier to experienced leader to notice return statements with parenthesis around the value than without. Recall that people's eyes are build in way that they recognized entire expressions and not letter combinations. > > [ Don't insert an empty line if the function has no local variables.] > > > > This change could be made and tested mechanically. IMHO, this change > > has neglible risk and could be safely implemented. > > Again: No need for immediate global change, but just do not add anymore > of those. There are already quite some functions, which do not have the > unnecessary empty line. > What seems to you as unnecessary rule may be of a great use for other code users. For me it improves the code readability as an empty line at the start clearly points that there're no local variables used. I don't see enough argumentation for removing this rule. > >>> +.Sh LOCAL VARIABLES > > > >> Last, but definitely not least, I added this paragraph about the use of > >> local variables. This is to clarify, how today's compilers handle > >> unaliased local variables. > > > > Every version of gcc that FreeBSD has ever used would do this for > > primitive types when optimisation was enabled. This approach can > > become expensive in stack (and this is an issue for kernel code) when > > using non-primitive types or when optimisation is not enabled (though > > I'm not sure if this is supported). Assuming that gcc (and icc and > > clang) behaves as stated in all supported optimisation modes, this > > change would appear to be quite safe to make. > > Most local variables are scalars (pointers, ints, ...). A struct or > array as local variable is rare and then usually used in one context. So > IMO this is fine. Even considereing the extremly rare case of multiple > non-scalar declarations in a function, then a compiler can coalesce them > if their life ranges are disjoint. It is easier to do so for a compiler > to do so, if they are declared with smaller life ranges, example: > > struct Foo { int x[16]; }; > struct Bar { int* y[16]; }; > > void f(struct Foo*); > void g(struct Bar*); > > void e(int x) > { > struct Foo a; > struct Bar b; > if (x) { > f(&a); > } else { > g(&b); > } > } > > Stack usage: > subl $140, %esp > > moving the declarations into the branches: > subl $76, %esp > > So, apart from improved readability, it also can lead to better code. > But I consider the latter way less important the benefits observed by a > reader of the code. > I particulary like this change. Aliasing behavior is stritcly described in ISO C99 standard, so there's a good point to enforcing strict-aliasing clear code in our kernel. On the other hand the big work should be done on clearing the existing code before any rule on this can be enforced. - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoP6rwACgkQK/VZk+smlYHdAACeJo64Mc0syCLtXq93yg0f87Y7 T2kAn1gLof6OMcHHs3EbRYTx7QE5NwU8 =5waq -----END PGP SIGNATURE----- !DSPAM:4a0fea9d994295559415935! From stas at FreeBSD.org Sun May 17 10:53:04 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Sun May 17 10:53:10 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090501.082020.698246310.imp@bsdimp.com> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> <49FADEF3.5010106@gmx.de> <20090501.082020.698246310.imp@bsdimp.com> Message-ID: <20090517145331.fda0f91f.stas@FreeBSD.org> On Fri, 01 May 2009 08:20:20 -0600 (MDT) "M. Warner Losh" mentioned: > > It is a debugging aid, but one of dubious value for a far more > fundamental reason: > > return; > > will break any macro. > You can use variadic marcos in this case if the piece of code debugged uses void returns. -- Stanislav Sedov ST4096-RIPE !DSPAM:4a0fec8e994291872371064! From christoph.mallon at gmx.de Sun May 17 12:32:05 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sun May 17 12:32:12 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090517145331.fda0f91f.stas@FreeBSD.org> References: <49F4070C.2000108@gmx.de> <20090501112239.GA23199@alchemy.franken.de> <49FADEF3.5010106@gmx.de> <20090501.082020.698246310.imp@bsdimp.com> <20090517145331.fda0f91f.stas@FreeBSD.org> Message-ID: <4A1003C2.8070901@gmx.de> Stanislav Sedov schrieb: > On Fri, 01 May 2009 08:20:20 -0600 (MDT) > "M. Warner Losh" mentioned: >> It is a debugging aid, but one of dubious value for a far more >> fundamental reason: >> >> return; >> >> will break any macro. >> > > You can use variadic marcos in this case if the piece of code debugged > uses void returns. No, you cannot. Function like macros with ellipsis ("variadic macros") cannot be treated as object like macros. See ISO/IEC 9899:1999 (E) ?6.10.3:4. Christoph From christoph.mallon at gmx.de Sun May 17 12:36:06 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sun May 17 12:36:13 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090517144516.331b01a8.stas@FreeBSD.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> <20090517144516.331b01a8.stas@FreeBSD.org> Message-ID: <4A1004B3.5040805@gmx.de> Stanislav Sedov schrieb: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 01 May 2009 14:02:50 +0200 > Christoph Mallon mentioned: > >>> [Don't parenthesize return values] >>>> Removed, because it does not improve maintainability in any way. >>> This change could be made and tested mechanically. But there is >>> no justification for making it - stating that the existing rule >>> is no better than the proposed rule is no reason to change. >> Just remove the rule. There's no need to add more redundant parentheses. >> Again: There is no need to change all existing code at once, but the >> rule is removed so that not anymore redundant parentheses are added. > > If only the rule gets removed this will lead to inconsistent code. Currently > it is much easier to experienced leader to notice return statements with > parenthesis around the value than without. Recall that people's eyes are > build in way that they recognized entire expressions and not letter > combinations. I don't buy this for a simple reason: Parentheses are in many statements (if, while, for). The only thing which distinguishs a return statement from others is the word "return". >>>>> +.Sh LOCAL VARIABLES >>>> Last, but definitely not least, I added this paragraph about the use of >>>> local variables. This is to clarify, how today's compilers handle >>>> unaliased local variables. >>> Every version of gcc that FreeBSD has ever used would do this for >>> primitive types when optimisation was enabled. This approach can >>> become expensive in stack (and this is an issue for kernel code) when >>> using non-primitive types or when optimisation is not enabled (though >>> I'm not sure if this is supported). Assuming that gcc (and icc and >>> clang) behaves as stated in all supported optimisation modes, this >>> change would appear to be quite safe to make. >> Most local variables are scalars (pointers, ints, ...). A struct or >> array as local variable is rare and then usually used in one context. So >> IMO this is fine. Even considereing the extremly rare case of multiple >> non-scalar declarations in a function, then a compiler can coalesce them >> if their life ranges are disjoint. It is easier to do so for a compiler >> to do so, if they are declared with smaller life ranges, example: >> >> struct Foo { int x[16]; }; >> struct Bar { int* y[16]; }; >> >> void f(struct Foo*); >> void g(struct Bar*); >> >> void e(int x) >> { >> struct Foo a; >> struct Bar b; >> if (x) { >> f(&a); >> } else { >> g(&b); >> } >> } >> >> Stack usage: >> subl $140, %esp >> >> moving the declarations into the branches: >> subl $76, %esp >> >> So, apart from improved readability, it also can lead to better code. >> But I consider the latter way less important the benefits observed by a >> reader of the code. >> > > I particulary like this change. Why? > Aliasing behavior is stritcly described in > ISO C99 standard, so there's a good point to enforcing strict-aliasing clear > code in our kernel. If you like this addition because of this reason, I have to disappoint you: This addition has absolutly *nothing* to do with strict-aliasing. > On the other hand the big work should be done on clearing > the existing code before any rule on this can be enforced. This addition is about improving readability for humans, because it simplifies the def-use-chains, and as a side effect sometimes leads to better generated code. It is not sensible to check millions of lines of code whether a variables are used in different contexts within a function before this can added. Anyway, this is moot, because this thread was dead because every suggested improvement was rejected - especially this last improvement was rejected by the guy who requested it in the first place. Christoph From to.my.trociny at gmail.com Sun May 17 15:42:41 2009 From: to.my.trociny at gmail.com (Mikolaj Golub) Date: Sun May 17 15:42:48 2009 Subject: Memory leak on thread removal In-Reply-To: ("Marius =?iso-8859-1?Q?N=FCnnerich=22's?= message of "Sat\, 16 May 2009 20\:24\:09 +0200") References: <814ovqn8dp.fsf@zhuzha.ua1> <86k54hvuzv.fsf@kopusha.onet> Message-ID: <864ovjviqg.fsf@kopusha.onet> On Sat, 16 May 2009 20:24:09 +0200 Marius N?nnerich wrote: >> http://freshmeat.net/projects/lmdbg >> >> This is a small memory leak debugger. It does not provide all functionality >> you can find in more sophisticated tools but is lightweight, portable and >> simple in use. It was very useful when I traced this bug. MN> Thanks, I'll take a look at it. Today I submitted lmdbg port. http://www.freebsd.org/cgi/query-pr.cgi?pr=134617 At present it is waiting to be committed in ports tree, but you can use shar from the PR to build the port yourself. -- Mikolaj Golub From stas at FreeBSD.org Sun May 17 16:04:28 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Sun May 17 16:04:35 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <4A1004B3.5040805@gmx.de> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> <20090517144516.331b01a8.stas@FreeBSD.org> <4A1004B3.5040805@gmx.de> Message-ID: <20090517200456.cefa04fb.stas@FreeBSD.org> On Sun, 17 May 2009 14:36:03 +0200 Christoph Mallon mentioned: > > > Aliasing behavior is stritcly described in > > ISO C99 standard, so there's a good point to enforcing strict-aliasing clear > > code in our kernel. > If you like this addition because of this reason, I have to disappoint > you: This addition has absolutly *nothing* to do with strict-aliasing. > I didn't meant I like this change only from aliasing point of view: certianly, the code readability argument is very important. But this change also works towards the strict aliasing problem solving too: there's just a less chance someone will reuse a variable, address of which was previously taken. > > On the other hand the big work should be done on clearing > > the existing code before any rule on this can be enforced. > This addition is about improving readability for humans, because it > simplifies the def-use-chains, and as a side effect sometimes leads to > better generated code. It is not sensible to check millions of lines of > code whether a variables are used in different contexts within a > function before this can added. > -- Stanislav Sedov ST4096-RIPE !DSPAM:4a103589994292021119546! From christoph.mallon at gmx.de Sun May 17 16:41:17 2009 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Sun May 17 16:41:32 2009 Subject: C99: Suggestions for style(9) In-Reply-To: <20090517200456.cefa04fb.stas@FreeBSD.org> References: <49F4070C.2000108@gmx.de> <20090428114754.GB89235@server.vk2pj.dyndns.org> <49FAE4EA.1010205@gmx.de> <20090517144516.331b01a8.stas@FreeBSD.org> <4A1004B3.5040805@gmx.de> <20090517200456.cefa04fb.stas@FreeBSD.org> Message-ID: <4A103E29.4040309@gmx.de> Stanislav Sedov schrieb: > On Sun, 17 May 2009 14:36:03 +0200 > Christoph Mallon mentioned: > >>> Aliasing behavior is stritcly described in >>> ISO C99 standard, so there's a good point to enforcing strict-aliasing clear >>> code in our kernel. >> If you like this addition because of this reason, I have to disappoint >> you: This addition has absolutly *nothing* to do with strict-aliasing. >> > > I didn't meant I like this change only from aliasing point of view: certianly, > the code readability argument is very important. But this change also > works towards the strict aliasing problem solving too: there's just > a less chance someone will reuse a variable, address of which was > previously taken. Something like this would violate strict-aliasing: int i; short* p = (short*)&i; A short pointer may never point at an int object (ISO/IEC 9899:1999 (E) ?6.5:7). The suggested paragraph has nothing to do with strict-aliasing. It's "just" about reusing the same variable in different contexts. Reusing the same variable in different contexts is bad, because it's harder for a human reader to identify the def-use-chains and additionally if the address of the variable has escaped (just a "normal" alias problem, nothing about type-punning and strict-aliasing) the generated code will be worse. Please, can we stop this now? It was already rejected. It's a pity, but maintaining status quo for style(9) seems to be too holy. *sigh* Christoph From xorquewasp at googlemail.com Mon May 18 08:48:37 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Mon May 18 08:48:44 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090508211022.GA37475@logik.internal.network> References: <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> Message-ID: <20090518084831.GA95354@logik.internal.network> Hi. After a week off, another update: I've realised, too late, that I'm using a version of binutils (2.19) that's incompatible with the system binutils (2.15). Specifically, assembler code emitted by the native GNAT contains .cfi_personality directives (and no doubt other things too) that can't be processed by the system 'as'. I've got two choices now and would appreciate some advice on which to take given that I want to produce a FreeBSD port: 1. Compile binutils-2.15. Unfortunately, compiling these as cross-binutils appear to be problematic: gmake[3]: Entering directory `/root/memfs/c1-bu-obj/gas' gcc -DHAVE_CONFIG_H -I. -I/root/binutils-2.15/gas -I. -D_GNU_SOURCE -I. -I/root/binutils-2.15/gas -I../bfd -I/root/binutils-2.15/gas/config -I/root/binutils-2.15/gas/../include -I/root/binutils-2.15/gas/.. -I/root/binutils-2.15/gas/../bfd -I/root/binutils-2.15/gas/../intl -I../intl -DLOCALEDIR="\"/cross/x86_64/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -g -O2 -c /root/binutils-2.15/gas/app.c In file included from /root/binutils-2.15/gas/as.h:94, from /root/binutils-2.15/gas/app.c:30: /root/binutils-2.15/gas/../include/getopt.h:116: warning: function declaration isn't a prototype In file included from ./targ-cpu.h:1, from /root/binutils-2.15/gas/config/obj-elf.h:42, from ./obj-format.h:1, from /root/binutils-2.15/gas/config/te-freebsd.h:30, from ./targ-env.h:1, from /root/binutils-2.15/gas/as.h:626, from /root/binutils-2.15/gas/app.c:30: /root/binutils-2.15/gas/config/tc-i386.h:451: error: array type has incomplete element type gmake[3]: *** [app.o] Error 1 gmake[3]: Leaving directory `/root/memfs/c1-bu-obj/gas' gmake[2]: *** [all-recursive] Error 1 gmake[2]: Leaving directory `/root/memfs/c1-bu-obj/gas' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory `/root/memfs/c1-bu-obj/gas' gmake: *** [all-gas] Error 2 2. Continue to use binutils-2.19. This would appear to require me to create a binutils-2.19 port just for the GNAT compiler. Seems like it would be preferable to use the system binutils rather than to take this route... xw From xorquewasp at googlemail.com Mon May 18 10:07:38 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Mon May 18 10:07:45 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090518084831.GA95354@logik.internal.network> References: <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> Message-ID: <20090518100734.GA36229@logik.internal.network> On 2009-05-18 09:48:31, xorquewasp@googlemail.com wrote: > 1. Compile binutils-2.15. > > Unfortunately, compiling these as cross-binutils appear to be problematic: > > gmake[3]: Entering directory `/root/memfs/c1-bu-obj/gas' > gcc -DHAVE_CONFIG_H -I. -I/root/binutils-2.15/gas -I. -D_GNU_SOURCE -I. -I/root/binutils-2.15/gas -I../bfd -I/root/binutils-2.15/gas/config -I/root/binutils-2.15/gas/../include -I/root/binutils-2.15/gas/.. -I/root/binutils-2.15/gas/../bfd -I/root/binutils-2.15/gas/../intl -I../intl -DLOCALEDIR="\"/cross/x86_64/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -g -O2 -c /root/binutils-2.15/gas/app.c > In file included from /root/binutils-2.15/gas/as.h:94, > from /root/binutils-2.15/gas/app.c:30: > /root/binutils-2.15/gas/../include/getopt.h:116: warning: function declaration isn't a prototype > In file included from ./targ-cpu.h:1, > from /root/binutils-2.15/gas/config/obj-elf.h:42, > from ./obj-format.h:1, > from /root/binutils-2.15/gas/config/te-freebsd.h:30, > from ./targ-env.h:1, > from /root/binutils-2.15/gas/as.h:626, > from /root/binutils-2.15/gas/app.c:30: > /root/binutils-2.15/gas/config/tc-i386.h:451: error: array type has incomplete element type > gmake[3]: *** [app.o] Error 1 > gmake[3]: Leaving directory `/root/memfs/c1-bu-obj/gas' > gmake[2]: *** [all-recursive] Error 1 > gmake[2]: Leaving directory `/root/memfs/c1-bu-obj/gas' > gmake[1]: *** [all] Error 2 > gmake[1]: Leaving directory `/root/memfs/c1-bu-obj/gas' > gmake: *** [all-gas] Error 2 Correction. Will compile with patches from here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=299671 But doesn't pass test suite: # of expected passes 28 # of unexpected failures 20 Test logs show: Executing /root/binutils-2.15/gas/testsuite/lib/run ../as-new --32 -al /root/binutils-2.15/gas/testsuite/gas/i386/float.s >&dump.out regexp_diff match failure regexp "^.*: Assembler messages:$" line "Assembler messages:" regexp_diff match failure regexp "^.*:3: Warning:.*faddp.*$" line "FATAL: can't create a.out: Invalid bfd target" extra regexps in /root/binutils-2.15/gas/testsuite/gas/i386/float.l starting with "^.*:14: Warning:.*fsubp.*$" EOF from dump.out FAIL: i386 float I have no idea why this happens. xw From chuckr at telenix.org Mon May 18 20:02:38 2009 From: chuckr at telenix.org (Chuck Robey) Date: Mon May 18 20:03:50 2009 Subject: porting info for FreeBSD's kernel? Message-ID: <4A11B893.1000808@telenix.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've been googling, trying to see if I can find notes regarding what needs changing, in what order, to adapt the FreeBSD kernel to a new processor. Anyone know where stuff like that can be found? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoRuJMACgkQz62J6PPcoOmq/gCaAkpfszx/RV6ETjyqsBrYjkKy G4cAniK2BsXTsgTFuvsbPmS7siv2DwTA =Y+ww -----END PGP SIGNATURE----- From deischen at freebsd.org Mon May 18 22:36:18 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon May 18 22:36:44 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090518084831.GA95354@logik.internal.network> References: <20090504185644.GA16315@logik.internal.network> <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> Message-ID: On Mon, 18 May 2009, xorquewasp@googlemail.com wrote: > Hi. > > After a week off, another update: > > I've realised, too late, that I'm using a version of binutils > (2.19) that's incompatible with the system binutils (2.15). > Specifically, assembler code emitted by the native GNAT contains > .cfi_personality directives (and no doubt other things too) that > can't be processed by the system 'as'. > > I've got two choices now and would appreciate some advice on > which to take given that I want to produce a FreeBSD port: > > 1. Compile binutils-2.15. > > Unfortunately, compiling these as cross-binutils appear to be problematic: Hmm, if the system binutils is 2.15, then it should build as a cross. You can do a cross build of all FreeBSD - I think you just set TARGET="amd64" to build amd64 from a different arch. Part of this process should be to create a cross binutils toolset. > 2. Continue to use binutils-2.19. > > This would appear to require me to create a binutils-2.19 port > just for the GNAT compiler. Seems like it would be preferable > to use the system binutils rather than to take this route... Well, I used a newer binutils on sparc when I did the original port. Once I built the cross compiler and binutils toolset, I was done with it. After the native compiler is built using the cross tools, you should be able to rebuild the native compiler _again_ but this time with the system (amd64) binutils. -- DE From glen.j.barber at gmail.com Tue May 19 03:16:50 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Tue May 19 03:16:56 2009 Subject: sshd(8) - alert user when fails to execute from rc.d Message-ID: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> Good evening, hackers. Earlier this evening, I submitted a PR about sshd(8) giving a false-positive when starting on an already occupied socket[1]. I would like to enable some form of console output when the rc.d script is called if the service cannot properly bind to the socket, but I want to make sure I do it "the right way." I was digging through src/crypto/openssh/sshd.c hoping to submit a patch to enable this, but I'm not certain that is the right place to be looking. After digging through erc/etc/rc.d/sshd, I am failing to understand how the service would check the listening port, so now I feel like I am hitting a wall. Any suggestions on how best to enable this? Thanks in advance. [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=134694 -- Glen Barber From xorquewasp at googlemail.com Tue May 19 06:04:09 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Tue May 19 06:04:15 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> Message-ID: <20090519060405.GA43127@logik.internal.network> On 2009-05-18 18:36:15, Daniel Eischen wrote: > Hmm, if the system binutils is 2.15, then it should build > as a cross. You can do a cross build of all FreeBSD - I > think you just set TARGET="amd64" to build amd64 from > a different arch. Part of this process should be to > create a cross binutils toolset. Ok. Silly question - is it actually possible to build contrib/binutils (including TARGET=amd64) without building the whole tree? Trying the obvious: cd /usr/obj /usr/src/contrib/binutils/configure \ --target=x86_64-pc-freebsd7.2 \ --host=i386-pc-freebsd7.2 \ --build=i386-pc-freebsd7.2 \ --prefix=/cross/x86_64 .. Didn't work (didn't really expect it to). xw From hselasky at c2i.net Tue May 19 08:21:07 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Tue May 19 08:21:14 2009 Subject: Which priority do taskqueues have? Message-ID: <200905190923.39298.hselasky@c2i.net> Hi, I'm about to factor out some taskqueue-alike code from USB(II) and I need to know at which priority taskqueues are running. I know there is a priority argument which can be specified for TASK_INIT(), but tracing in the code shows that this is just a queue-priority. At which priority level is taskqueue code being [guaranteed to] run? --HPS From xorquewasp at googlemail.com Tue May 19 11:45:53 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Tue May 19 11:46:00 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> Message-ID: <20090519114548.GA8610@logik.internal.network> On 2009-05-18 18:36:15, Daniel Eischen wrote: > Well, I used a newer binutils on sparc when I did the original > port. Once I built the cross compiler and binutils toolset, > I was done with it. After the native compiler is built using > the cross tools, you should be able to rebuild the native > compiler _again_ but this time with the system (amd64) > binutils. I probably should point out that I don't think this is the case anymore. GCC apparently detects what capabilities the currently selected binutils have so when the first native compiler has been compiled using the cross, it will emit code that can't be assembled using the system binutils (because it uses features from the new binutils that aren't supported by the older system ones). In other words, you can't rebuild the native compiler using the system binutils. If the worst comes to the worst, I can create a dependency on the devel/cross-binutils port. xw From jhb at freebsd.org Tue May 19 13:35:28 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue May 19 13:35:44 2009 Subject: Which priority do taskqueues have? In-Reply-To: <200905190923.39298.hselasky@c2i.net> References: <200905190923.39298.hselasky@c2i.net> Message-ID: <200905190825.13595.jhb@freebsd.org> On Tuesday 19 May 2009 3:23:39 am Hans Petter Selasky wrote: > Hi, > > I'm about to factor out some taskqueue-alike code from USB(II) and I need to > know at which priority taskqueues are running. I know there is a priority > argument which can be specified for TASK_INIT(), but tracing in the code > shows that this is just a queue-priority. At which priority level is > taskqueue code being [guaranteed to] run? It depends on the queue I think. taskqueue_swi runs as a SWI and thus at a software-interrupt priority. taskqueue_thread runs at the default priority for kernel threads (currently a rather bogus PVM I think). -- John Baldwin From eischen at vigrid.com Tue May 19 13:51:10 2009 From: eischen at vigrid.com (Daniel Eischen) Date: Tue May 19 13:51:17 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090519114548.GA8610@logik.internal.network> References: <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> <20090519114548.GA8610@logik.internal.network> Message-ID: On Tue, 19 May 2009, xorquewasp@googlemail.com wrote: > On 2009-05-18 18:36:15, Daniel Eischen wrote: >> Well, I used a newer binutils on sparc when I did the original >> port. Once I built the cross compiler and binutils toolset, >> I was done with it. After the native compiler is built using >> the cross tools, you should be able to rebuild the native >> compiler _again_ but this time with the system (amd64) >> binutils. > > I probably should point out that I don't think this is the case anymore. > > GCC apparently detects what capabilities the currently selected binutils > have so when the first native compiler has been compiled using the > cross, it will emit code that can't be assembled using the system > binutils (because it uses features from the new binutils that aren't > supported by the older system ones). In other words, you can't rebuild the > native compiler using the system binutils. > > If the worst comes to the worst, I can create a dependency on the > devel/cross-binutils port. Even so, you shouldn't need a cross-binutils, only a native (amd64) binutils. Your port won't be a cross port, but a native amd64 port. The native amd64 GNAT will need a native binutils, not a cross binutils. The only thing you will have to make is a minimal bootstrap (native amd64) compiler. Of course you can create a cross port if you want to facilitate cross builds for ports that don't exist yet, but no one running amd64 will want to make a cross build when they can make a faster native build with less dependencies. -- DE From deischen at freebsd.org Tue May 19 13:55:37 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Tue May 19 13:55:43 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: <20090519060405.GA43127@logik.internal.network> References: <20090505005128.GA4519@logik.internal.network> <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> <20090519060405.GA43127@logik.internal.network> Message-ID: On Tue, 19 May 2009, xorquewasp@googlemail.com wrote: > On 2009-05-18 18:36:15, Daniel Eischen wrote: >> Hmm, if the system binutils is 2.15, then it should build >> as a cross. You can do a cross build of all FreeBSD - I >> think you just set TARGET="amd64" to build amd64 from >> a different arch. Part of this process should be to >> create a cross binutils toolset. > > Ok. > > Silly question - is it actually possible to build contrib/binutils > (including TARGET=amd64) without building the whole tree? Trying > the obvious: > > cd /usr/obj > /usr/src/contrib/binutils/configure \ > --target=x86_64-pc-freebsd7.2 \ > --host=i386-pc-freebsd7.2 \ > --build=i386-pc-freebsd7.2 \ > --prefix=/cross/x86_64 > > .. Didn't work (didn't really expect it to). I've not done a cross build before, but I'd look in src/Makefile.inc1 if you want to try to build it piecemeal (see the target for cross-tools). -- DE From xorquewasp at googlemail.com Tue May 19 13:59:21 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Tue May 19 13:59:27 2009 Subject: bootstrapping gnat GCC on amd64 In-Reply-To: References: <20090505022151.GA32477@logik.internal.network> <20090506140325.GA69468@logik.internal.network> <20090506152222.GC69468@logik.internal.network> <20090508211022.GA37475@logik.internal.network> <20090518084831.GA95354@logik.internal.network> <20090519114548.GA8610@logik.internal.network> Message-ID: <20090519135917.GA5391@logik.internal.network> On 2009-05-19 09:51:08, Daniel Eischen wrote: > > Even so, you shouldn't need a cross-binutils, only a native > (amd64) binutils. Your port won't be a cross port, but a > native amd64 port. The native amd64 GNAT will need a native > binutils, not a cross binutils. The only thing you will have > to make is a minimal bootstrap (native amd64) compiler. > > Of course you can create a cross port if you want to facilitate > cross builds for ports that don't exist yet, but no one running > amd64 will want to make a cross build when they can make a > faster native build with less dependencies. 'lo, Sorry, I should have been a bit clearer there. I mean if in the very worst case, I can't get by with the system binutils, I can create a "native" set of recent binutils using the cross-binutils port: cd /usr/ports/devel/cross-binutils make TGTARCH=x86_64 TGTABI=freebsd7.2 install That way, the port can just depend on those and I won't have to create my own binutils port. Like I said, I'm hoping this won't happen. Current status is that I have a working native AMD64 GNAT using 2.19 binutils! $ gcc44 -v Using built-in specs. Target: x86_64-pc-freebsd7.2 Configured with: /usr/jails/i386/root/gcc-4.4.0/configure --build=x86_64-pc-freebsd7.2 --enable-languages=c,ada --disable-nls --with-system-zlib --with-libiconv-prefix=/usr/local --program-suffix=44 --bindir=/usr/local/bin/gcc44 --libdir=/usr/local/lib/gcc-4.4.0 --prefix=/usr/local --mandir=/usr/local/man --infodir=/usr/local/info/gcc44 Thread model: posix gcc version 4.4.0 (GCC) Needless to say, I'm pretty pleased. xw From unixmania at gmail.com Tue May 19 16:30:07 2009 From: unixmania at gmail.com (Carlos A. M. dos Santos) Date: Tue May 19 16:30:39 2009 Subject: IPv6 support on BSNMP Message-ID: Hello, Is there any ongoing work on adding support for IPv6 to BSNMP? Do you have an idea of how much effort it would need? Thanks in advance. -- My preferred quotation of Robert Louis Stevenson is "You cannot make an omelette without breaking eggs". Not because I like the omelettes, but because I like the sound of eggs being broken. From emorras at xroff.net Tue May 19 19:56:18 2009 From: emorras at xroff.net (Eduardo Morras) Date: Tue May 19 19:56:25 2009 Subject: Question about PCIe networks Message-ID: <20090519193727.7512E4FC814@xroff.net> Hello, don't know if this has been discussed but here it goes. I have read recently this http://www.wwpi.com/hardware/hardware/6540-ethernet-tunneling-through-pci-express-inter-processor-communication-low-latency-storage-io It's about using PCIe to connect 2 servers directly, without using ethernet or other hardware. Can it be done in FreeBSD? What is needed to know? TIA From steve at Watt.COM Tue May 19 21:22:12 2009 From: steve at Watt.COM (Steve Watt) Date: Tue May 19 21:22:24 2009 Subject: Question about PCIe networks In-Reply-To: <20090519193727.7512E4FC814@xroff.net> Message-ID: <200905192058.n4JKwSAH031382@wattres.watt.com> In <20090519193727.7512E4FC814@xroff.net>, emorras@xroff.net write: >I have read recently this >http://www.wwpi.com/hardware/hardware/6540-ethernet-tunneling-through-pci-express-inter-processor-communication-low-latency-storage-io (From a company that makes PCIe switches to connect multiple root complexes together.) >It's about using PCIe to connect 2 servers directly, without using >ethernet or other hardware. > >Can it be done in FreeBSD? What is needed to know? Certainly. Non-transparent PCIe bridges basically create windows of memory space into the other side. You'd need the two sides to agree on the data structures, and what signalling mechanism to use for packet availability. Quite straightforward, really. What you'll need is the bridge hardware that connects to the two systems, the two systems, datasheets, and some time. Each side of the bridge would allocate some DMAable memory, and set up the bridge so that is visible to the other side. Set up a pair of rings (one per direction of traffic), and go. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.5" / 37N 20' 15.3" Internet: steve @ Watt.COM Whois: SW32-ARIN Free time? There's no such thing. It just comes in varying prices... From rea-fbsd at codelabs.ru Wed May 20 10:39:04 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Wed May 20 10:39:13 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> Message-ID: Glen, good day. Mon, May 18, 2009 at 10:49:52PM -0400, Glen Barber wrote: > Earlier this evening, I submitted a PR about sshd(8) giving a > false-positive when starting on an already occupied socket[1]. I > would like to enable some form of console output when the rc.d script > is called if the service cannot properly bind to the socket, but I > want to make sure I do it "the right way." Reading through the PR, I can't figure out what do you mean. You're saying that 1. you spawn the other service on a port N; 2. then you're spawning SSH on the same port via rc.d script; 3. after this '/etc/rc.d/sshd status' gives you 'sshd is not running'. But this is completely right: after step 2 there will be no SSH daemon listening, because it fails to bind to the port. And the 'status' command of an rc.d script is perfectly correct -- no SSH daemon is running, really. > I was digging through src/crypto/openssh/sshd.c hoping to submit a > patch to enable this, but I'm not certain that is the right place to > be looking. After digging through erc/etc/rc.d/sshd, I am failing to > understand how the service would check the listening port, so now I > feel like I am hitting a wall. You seem to mix two things: binding to the port and the output from rc.d 'status' command. Binding to the port is done by SSH by the bind(2) system call and if something is already listening on the given address, the socket won't be bound, so SSH daemon terminates. 'status' (for the case of /etc/rc.d/sshd) deduces the status of the service from it's pid file (variable pidfile) with the subroutine check_pidfile. Look at /etc/rc.subr: 'status' is handled via "run_rc_command status" that evaluates _pidcmd that sets $rc_pid. And then $rc_pid it checked for being non-empty, and if emptiness found, command ----- echo "${name} is not running." ----- is executed. It produces the result you're seeing. So, I would say that the PR in question is somewhat false positive. -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From dimitry at andric.com Wed May 20 10:54:56 2009 From: dimitry at andric.com (Dimitry Andric) Date: Wed May 20 10:55:03 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> Message-ID: <4A13E180.1040606@andric.com> On 2009-05-20 12:19, Eygene Ryabinkin wrote: > You seem to mix two things: binding to the port and the output from rc.d > 'status' command. Binding to the port is done by SSH by the bind(2) > system call and if something is already listening on the given address, > the socket won't be bound, so SSH daemon terminates. I think what might be confusing, is the fact that sshd dies due to bind() failing, and it should; but you will only see this in the syslog, NOT on the command line. E.g. the /etc/rc.d/sshd script will NOT give an error, because the /usr/bin/sshd it calls will fork, and as soon as the fork is okay, the original instance with exit with 0. The forked instance is what will die on bind(), so you will not see any failures from it. From dimitry at andric.com Wed May 20 11:27:02 2009 From: dimitry at andric.com (Dimitry Andric) Date: Wed May 20 11:27:08 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4A13E6F7.7070309@glocalnet.net> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> <4A13E180.1040606@andric.com> <4A13E6F7.7070309@glocalnet.net> Message-ID: <4A13E906.7020907@andric.com> On 2009-05-20 13:18, Tobias Fendin wrote: > Does the child really die? I did a little test: > > # /etc/rc.d/sshd status > sshd is not running. > # nc -l 22 >/tmp/ssh_test & > [1] 1733 > # /etc/rc.d/sshd start > Starting sshd. > # /etc/rc.d/sshd status > sshd is running as pid 1740. This is because sshd binds to both IPv4 and IPv6 ports. The IPv4 bind fails, as you will see in syslog, while the IPv6 bind succeeds. Thus sshd keeps on running. If you start two nc's (I don't know any way to do this with one instance), e.g.: nc -4 -l 22 > /tmp/ssh_test4 & nc -6 -l 22 > /tmp/ssh_test6 & and then try starting sshd, you should see it quit. From tobias.fendin at glocalnet.net Wed May 20 11:41:13 2009 From: tobias.fendin at glocalnet.net (Tobias Fendin) Date: Wed May 20 11:41:20 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4A13E180.1040606@andric.com> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> <4A13E180.1040606@andric.com> Message-ID: <4A13E6F7.7070309@glocalnet.net> Dimitry Andric wrote: > On 2009-05-20 12:19, Eygene Ryabinkin wrote: > >> You seem to mix two things: binding to the port and the output from rc.d >> 'status' command. Binding to the port is done by SSH by the bind(2) >> system call and if something is already listening on the given address, >> the socket won't be bound, so SSH daemon terminates. >> > > I think what might be confusing, is the fact that sshd dies due to > bind() failing, and it should; but you will only see this in the syslog, > NOT on the command line. > > E.g. the /etc/rc.d/sshd script will NOT give an error, because the > /usr/bin/sshd it calls will fork, and as soon as the fork is okay, the > original instance with exit with 0. The forked instance is what will > die on bind(), so you will not see any failures from it. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > Does the child really die? I did a little test: # /etc/rc.d/sshd status sshd is not running. # nc -l 22 >/tmp/ssh_test & [1] 1733 # /etc/rc.d/sshd start Starting sshd. # /etc/rc.d/sshd status sshd is running as pid 1740. # ssh someuser@localhost // This didn't timeout or anything, just didn't give any output. I killed it after a couple of minutes. ^C [1]+ Done nc -l 22 > /tmp/ssh_test # ssh someuser@localhost The authenticity of host 'localhost (::1)' can't be established. DSA key fingerprint is 9f:fa:ee:f5:39:c5:de:c4:8f:b9:c5:43:d8:9d:85:23. Are you sure you want to continue connecting (yes/no)? ^C # uname -a FreeBSD asator 7.0-RELEASE-p2 FreeBSD 7.0-RELEASE-p2 #0: Thu Mar 5 03:16:15 CET 2009 root@asator:/usr/obj/usr/src/sys/A_KERNEL i386 As you can see, the first execution of ssh connects to nc (which terminated when I killed the ssh client). And the second execution it gets through to sshd (thus, sshd never failed at it's startup). I don't know if this is the expected behavior, or if it has changed on -CURRENT. From glen.j.barber at gmail.com Wed May 20 14:38:20 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Wed May 20 14:38:27 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> Message-ID: <4ad871310905200738g79989fb6l58616f16495beccb@mail.gmail.com> Hi, Eygene On Wed, May 20, 2009 at 6:19 AM, Eygene Ryabinkin wrote: > Glen, good day. > > Mon, May 18, 2009 at 10:49:52PM -0400, Glen Barber wrote: >> Earlier this evening, I submitted a PR about sshd(8) giving a >> false-positive when starting on an already occupied socket[1]. ?I >> would like to enable some form of console output when the rc.d script >> is called if the service cannot properly bind to the socket, but I >> want to make sure I do it "the right way." > > Reading through the PR, I can't figure out what do you mean. > You're saying that > ?1. you spawn the other service on a port N; > ?2. then you're spawning SSH on the same port via rc.d script; > ?3. after this '/etc/rc.d/sshd status' gives you 'sshd is not running'. > > But this is completely right: after step 2 there will be no SSH daemon > listening, because it fails to bind to the port. ?And the 'status' > command of an rc.d script is perfectly correct -- no SSH daemon is > running, really. > That is correct. There is no daemon running, but there is no output on the console that starting sshd failed -- it is only listed in messages. (And if you don't know it failed, you may never look in messages to realize this.) >> I was digging through src/crypto/openssh/sshd.c hoping to submit a >> patch to enable this, but I'm not certain that is the right place to >> be looking. ?After digging through erc/etc/rc.d/sshd, I am failing to >> understand how the service would check the listening port, so now I >> feel like I am hitting a wall. > > You seem to mix two things: binding to the port and the output from rc.d > 'status' command. ?Binding to the port is done by SSH by the bind(2) > system call and if something is already listening on the given address, > the socket won't be bound, so SSH daemon terminates. > > 'status' (for the case of /etc/rc.d/sshd) deduces the status of the > service from it's pid file (variable pidfile) with the subroutine > check_pidfile. ?Look at /etc/rc.subr: 'status' is handled via > "run_rc_command status" that evaluates _pidcmd that sets $rc_pid. ?And > then $rc_pid it checked for being non-empty, and if emptiness found, > command > ----- > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?echo "${name} is not running." > ----- > is executed. ?It produces the result you're seeing. > > So, I would say that the PR in question is somewhat false positive. > -- > Eygene > ?_ ? ? ? ? ? ? ? ?___ ? ? ? _.--. ? # > ?\`.|\..----...-'` ? `-._.-'_.-'` ? # ?Remember that it is hard > ?/ ?' ` ? ? ? ? , ? ? ? __.--' ? ? ?# ?to read the on-line manual > ?)/' _/ ? ? \ ? `-_, ? / ? ? ? ? ? ?# ?while single-stepping the kernel. > ?`-'" `"\_ ?,_.-;_.-\_ ', ?fsc/as ? # > ? ? _.-'_./ ? {_.' ? ; / ? ? ? ? ? # ? ?-- FreeBSD Developers handbook > ? ?{_.-``-' ? ? ? ? {_/ ? ? ? ? ? ?# > -- Glen Barber From glen.j.barber at gmail.com Wed May 20 14:41:01 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Wed May 20 14:41:08 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4A13E906.7020907@andric.com> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> <4A13E180.1040606@andric.com> <4A13E6F7.7070309@glocalnet.net> <4A13E906.7020907@andric.com> Message-ID: <4ad871310905200740n744f9b83j96db2a3c1a6bec43@mail.gmail.com> Hi, Dimitry On Wed, May 20, 2009 at 7:27 AM, Dimitry Andric wrote: > On 2009-05-20 13:18, Tobias Fendin wrote: >> Does the child really die? I did a little test: >> >> # /etc/rc.d/sshd status >> sshd is not running. >> # nc -l 22 >/tmp/ssh_test & >> [1] 1733 >> # /etc/rc.d/sshd start >> Starting sshd. >> # /etc/rc.d/sshd status >> sshd is running as pid 1740. > > This is because sshd binds to both IPv4 and IPv6 ports. ?The IPv4 bind > fails, as you will see in syslog, while the IPv6 bind succeeds. ?Thus > sshd keeps on running. > > If you start two nc's (I don't know any way to do this with one > instance), e.g.: > > nc -4 -l 22 > /tmp/ssh_test4 & > nc -6 -l 22 > /tmp/ssh_test6 & > > and then try starting sshd, you should see it quit. > It's not an IPv4 versus IPv6 problem. How I tested this, as I had this problem in the past (which was a non-standard setup, but still a problem): sshd was listening on :25, both IPv4 and IPv6 sendmail was listening on :25 (because I had forgotten to disable it) The system boots, and sendmail starts before sshd. When sshd starts (or tries to) there is no console output that it had failed. The only way you realize it is not running, is when you cannot remotely log in. -- Glen Barber From glen.j.barber at gmail.com Wed May 20 14:43:00 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Wed May 20 14:43:09 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4A13E6F7.7070309@glocalnet.net> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> <4A13E180.1040606@andric.com> <4A13E6F7.7070309@glocalnet.net> Message-ID: <4ad871310905200742r10944459i2a0d5ada4df10d91@mail.gmail.com> Hi, Tobias On Wed, May 20, 2009 at 7:18 AM, Tobias Fendin wrote: > > Does the child really die? I did a little test: > > # /etc/rc.d/sshd status > sshd is not running. > # nc -l 22 >/tmp/ssh_test & > [1] 1733 > # /etc/rc.d/sshd start > Starting sshd. > # /etc/rc.d/sshd status > sshd is running as pid 1740. > # ssh someuser@localhost ? ? ? ? ? ? ? ? ? ? ? ? ?// This didn't timeout or > anything, just didn't give any output. I killed it after a couple of > minutes. > ^C > [1]+ ?Done ? ? ? ? ? ? ? ? ? ?nc -l 22 > /tmp/ssh_test > # ssh someuser@localhost > The authenticity of host 'localhost (::1)' can't be established. > DSA key fingerprint is 9f:fa:ee:f5:39:c5:de:c4:8f:b9:c5:43:d8:9d:85:23. > Are you sure you want to continue connecting (yes/no)? ^C > # uname -a > FreeBSD asator 7.0-RELEASE-p2 FreeBSD 7.0-RELEASE-p2 #0: Thu Mar ?5 03:16:15 > CET 2009 ? ? root@asator:/usr/obj/usr/src/sys/A_KERNEL ?i386 > > As you can see, the first execution of ssh connects to nc (which terminated > when I killed the ssh client). And the second execution it gets through to > sshd (thus, sshd never failed at it's startup). > I don't know if this is the expected behavior, or if it has changed on > -CURRENT. > Perhaps sshd is checking for forked processes of itself, but not other daemons listening on that socket? -- Glen Barber From dimitry at andric.com Wed May 20 14:46:11 2009 From: dimitry at andric.com (Dimitry Andric) Date: Wed May 20 14:46:22 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4ad871310905200740n744f9b83j96db2a3c1a6bec43@mail.gmail.com> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> <4A13E180.1040606@andric.com> <4A13E6F7.7070309@glocalnet.net> <4A13E906.7020907@andric.com> <4ad871310905200740n744f9b83j96db2a3c1a6bec43@mail.gmail.com> Message-ID: <4A1417B3.3030303@andric.com> On 2009-05-20 16:40, Glen Barber wrote: > sshd was listening on :25, both IPv4 and IPv6 > sendmail was listening on :25 (because I had forgotten to disable it) > > The system boots, and sendmail starts before sshd. When sshd starts > (or tries to) there is no console output that it had failed. The only > way you realize it is not running, is when you cannot remotely log in. Yes, this is unfortunate, but normal, as I explained in an earlier post. The sshd process does not return any error (and thus the /etc/rc.d script doesn't either), because it has no way to know that its forked copy died. The solution to this PR is "don't run stuff on conflicting ports". :) From kostjn at peterhost.ru Wed May 20 15:51:45 2009 From: kostjn at peterhost.ru (=?UTF-8?B?0JzQtdC90YzRiNC40LrQvtCyINCa0L7QvdGB0YLQsNC90YLQuNC9?=) Date: Wed May 20 15:51:52 2009 Subject: Jail limits under CURRENT In-Reply-To: <49ED55FF.5080306@peterhost.ru> References: <49ED55FF.5080306@peterhost.ru> Message-ID: <4A1423D9.30105@peterhost.ru> ????????? ?????????? wrote: Hi. I`m rewrite jail limit patch under CURRENT. New patch limited CPU, memory, filedesc, process. And allow change limit on the fly You can download tar.gz from http://kostjn.spb.ru/patch-jail-limit-8CURRENT.tar.gz =========================================================================== How to use. =========================================================================== Build cvsup CURRENT cd /usr/src patch -p0 < patch-jail-limit-8CURRENT make buildkernel make buildworld make installkernel reboot make installworld Create new entry in login.conf, for example class jail128 jail128:\ :cputime=10:\ :memoryuse=128M:\ :maxproc=256:\ :openfiles=1024:\ :tc=default: Cputime is percent on 1 core. Openfiles is sum filedesc for all proc in jail. Create new jail. ... Add in /etc/rc.conf jail_test_flags="-Ljail128" Run new jail /etc/rc.d/jail start test =========================================================================== Sysctl =========================================================================== Added sysctl [root@book ~]# sysctl security.jail.limit security.jail.limit.enable: 1 security.jail.limit.memory_exceed_kill: 0 [root@book ~]# sysctl -d security.jail.limit security.jail.limit: Jail limit security.jail.limit.enable: Enable jail limit security.jail.limit.memory_exceed_kill: Kill biggest proc in jail, if jail excee d memory limit =========================================================================== Jset and Jget =========================================================================== jset and jget is program for set new jail limit and get current limit Example [root@book ~]# cat /etc/rc.conf | grep jail2 jail_list="jail1 jail2 jail3 jail4 jail5 jail6 jail7 jail8 jail9 jail10" jail_jail2_rootdir="/usr/jails/jail2/" jail_jail2_hostname="jail2.book.pht" jail_jail2_interface="re0" jail_jail2_ip="192.168.200.22" jail_jail2_flags="-Ljail64" [root@book ~]# /etc/rc.d/jail start jail2 Configuring jails:. Starting jails: jail2.book.pht. [root@book ~]# cd ~kostjn/ [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 Usage: CPU 0, MEM 6M, NPROC 9, NOFILE 65 [root@book /home/kostjn]# ./jset.o 1 jail2048 Set new jail limits, jid = 1 Limits: CPU 30, MEM 2048M, NPROC 1024, NOFILE 2048 [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 30, MEM 2048M, NPROC 1024, NOFILE 2048 Usage: CPU 0, MEM 6M, NPROC 9, NOFILE 65 You see that new limit is set. =========================================================================== Test =========================================================================== Cpu limit <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Script [root@book /home/kostjn]# cat test.sh #!/bin/sh for i in `jot 8 1`; do cpuset -l0 jexec 1 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 2 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 3 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 4 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 5 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 6 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 7 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 8 /a.out & done for i in `jot 8 1`; do cpuset -l0 jexec 9 /a.out & done cpuset -l0 jexec 10 /a.out & Set class for all jail. [root@book /home/kostjn]# for i in `jot 10 1`; do ./jset.o $i jail128 ;done Set new jail limits, jid = 1 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 2 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 3 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 4 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 5 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 6 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 7 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 8 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 9 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 Set new jail limits, jid = 10 Limits: CPU 10, MEM 128M, NPROC 256, NOFILE 1024 [root@book /home/kostjn]# jexec 1 bash [root@jail1 /]# cat cpu.c #include #include #include #include int main(int argc,char *argv[]){ int64_t i,j=0; char *s; for (;;){ } } Run test.sh Result top last pid: 3513; load averages: 70.87, 37.58, 16.40 up 0+00:44:02 14:19:46 185 processes: 74 running, 111 sleeping CPU: 49.9% user, 0.0% nice, 0.0% system, 0.2% interrupt, 49.9% idle Mem: 139M Active, 24M Inact, 47M Wired, 192K Cache, 29M Buf, 1785M Free Swap: 4044M Total, 4044M Free PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAN 3502 10 root 1 97 0 1480K 1244K CPU0 0 0:13 8.79% a.out 3474 6 root 1 97 0 1480K 1244K RUN 0 0:04 4.69% a.out 3431 2 root 1 96 0 1480K 1244K RUN 0 0:03 4.30% a.out 3454 4 root 1 97 0 1480K 1244K RUN 0 0:03 4.05% a.out 3422 1 root 1 96 0 1480K 1244K RUN 0 0:04 3.86% a.out 3482 7 root 1 97 0 1480K 1244K RUN 0 0:03 3.86% a.out 3447 3 root 1 97 0 1480K 1244K RUN 0 0:03 3.86% a.out 3429 1 root 1 96 0 1480K 1244K RUN 0 0:03 3.66% a.out 3485 8 root 1 97 0 1480K 1244K RUN 0 0:05 3.56% a.out 3424 1 root 1 96 0 1480K 1244K RUN 0 0:04 3.56% a.out 3464 5 root 1 97 0 1480K 1244K RUN 0 0:02 3.56% a.out 3438 2 root 1 96 0 1480K 1244K RUN 0 0:03 3.47% a.out 3494 9 root 1 96 0 1480K 1244K RUN 0 0:03 3.27% a.out 3497 9 root 1 97 0 1480K 1244K RUN 0 0:05 3.17% a.out 3433 2 root 1 96 0 1480K 1244K RUN 0 0:03 2.88% a.out 3428 1 root 1 96 0 1480K 1244K RUN 0 0:02 2.88% a.out 3487 8 root 1 97 0 1480K 1244K RUN 0 0:04 2.78% a.out ps auxwwww -ojid | more root 3502 9.0 0.1 1480 1244 v2 RJ 2:15PM 0:07.40 /a.out 10 root 3476 4.4 0.1 1480 1244 v2 RJ 2:15PM 0:04.38 /a.out 7 root 3480 4.1 0.1 1480 1244 v2 RJ 2:15PM 0:03.02 /a.out 7 root 3498 3.9 0.1 1480 1244 v2 RJ 2:15PM 0:04.00 /a.out 9 root 3429 3.7 0.1 1480 1244 v2 RJ 2:15PM 0:01.38 /a.out 1 root 3487 3.6 0.1 1480 1244 v2 RJ 2:15PM 0:03.32 /a.out 8 root 3452 3.5 0.1 1480 1244 v2 RJ 2:15PM 0:01.37 /a.out 4 root 3463 3.5 0.1 1480 1244 v2 RJ 2:15PM 0:01.65 /a.out 5 root 3472 3.3 0.1 1480 1244 v2 RJ 2:15PM 0:02.63 /a.out 6 root 3437 3.2 0.1 1480 1244 v2 RJ 2:15PM 0:01.93 /a.out 2 root 3494 3.0 0.1 1480 1244 v2 RJ 2:15PM 0:02.92 /a.out 9 root 3500 3.0 0.1 1480 1244 v2 RJ 2:15PM 0:03.63 /a.out 9 We see that jail 10 (1 thread), used ~10 % cpu under heavy load. <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Resourse compute [root@book /home/kostjn]# ./jset.o 1 jail64 Set new jail limits, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 Usage: CPU 0, MEM 6M, NPROC 9, NOFILE 65 [root@book /home/kostjn]# [root@book /home/kostjn]# jexec 1 bash [root@jail1 /]# apachectl stop /usr/local/sbin/apachectl stop: httpd stopped [root@jail1 /]# exit [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 Usage: CPU 0, MEM 3M, NPROC 3, NOFILE 24 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Resource limit [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 Usage: CPU 0, MEM 3M, NPROC 3, NOFILE 24 [root@book /home/kostjn]# jexec 1 bash [root@jail1 /]# cat mem.c #include #include #include #include int main(int argc,char *argv[]){ int64_t i,j=0; char *s; for (i=0; i < 1000 ;i++){ s = malloc(100000 * sizeof(char)); } sleep(1000); } [root@jail1 /]# cc mem.c && ./a.out & [1] 1320 [root@jail1 /]# ls bash: fork: Cannot allocate memory [root@jail1 /]# exit [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 Usage: CPU 1, MEM 103M, NPROC 5, NOFILE 31 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< We see that jail exceed memory limit. And new fork, mmap syscall not permitted. If you set sysctl [root@book /home/kostjn]# sysctl security.jail.limit.memory_exceed_kill=1 security.jail.limit.memory_exceed_kill: 1 -> 1 [root@book /home/kostjn]# ./jget.o 1 Jail limits and rusage, jid = 1 Limits: CPU 5, MEM 64M, NPROC 128, NOFILE 512 Usage: CPU 0, MEM 3M, NPROC 3, NOFILE 24 [root@book /home/kostjn]# jexec 1 bash [root@jail1 /]# ./a.out Killed: 9 [root@jail1 /]# exit [root@book /home/kostjn]# tail -n 1 /var/log/messages May 20 14:10:17 book kernel: pid 1337 (a.out), uid 0, jid 1 was killed: Prison e xceed memory limit <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< If you attempt set nonexisten class, limit set to infinity. [root@book /home/kostjn]# ./jset.o 1 jail123 Set new jail limits, jid = 1 Limits: CPU 9223372036854775807, MEM 20M, NPROC 9223372036854775807, NOFILE 9223 372036854775807 =========================================================================== Problem =========================================================================== If you have problem in this patch. Add to kernel config options KTR options KTR_ENTRIES=1024 options KTR_COMPILE=(KTR_PROC|KTR_JAIL|KTR_SCHED|KTR_RUNQ|KTR_LOCK|KTR_CONTENTIO N) options KTR_MASK=KTR_JAIL options KTR_CPUMASK=0x3 options KTR_VERBOSE options PRINTF_BUFR_SIZE=128 Rebuild kernel. Reboot. Set sysctl sysctl debug.ktr.mask=65536 and check /var/log/messages From glen.j.barber at gmail.com Wed May 20 17:39:30 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Wed May 20 17:39:36 2009 Subject: bin/134694: gives false-positive when unable to obtain socket [WAS: sshd(8) - alert user when fails to execute from rc.d] In-Reply-To: <4A1417B3.3030303@andric.com> References: <4ad871310905181949s2874795eoa5ddf425746310bf@mail.gmail.com> <4A13E180.1040606@andric.com> <4A13E6F7.7070309@glocalnet.net> <4A13E906.7020907@andric.com> <4ad871310905200740n744f9b83j96db2a3c1a6bec43@mail.gmail.com> <4A1417B3.3030303@andric.com> Message-ID: <4ad871310905201039nb17251cueedd11f54ad8806@mail.gmail.com> Hi, Dimitry On Wed, May 20, 2009 at 10:46 AM, Dimitry Andric wrote: > On 2009-05-20 16:40, Glen Barber wrote: >> sshd was listening on :25, both IPv4 and IPv6 >> sendmail was listening on :25 (because I had forgotten to disable it) >> >> The system boots, and sendmail starts before sshd. ?When sshd starts >> (or tries to) there is no console output that it had failed. ?The only >> way you realize it is not running, is when you cannot remotely log in. > > Yes, this is unfortunate, but normal, as I explained in an earlier post. > > The sshd process does not return any error (and thus the /etc/rc.d > script doesn't either), because it has no way to know that its forked > copy died. > > The solution to this PR is "don't run stuff on conflicting ports". :) > I absolutely agree about not running sshd on conflicting ports. After a bit more testing, I found that "most" other services will complain when they cannot obtain the requested socket, and you will see a failure notice via the rc.d script. My concern is when someone has a "definite need" to run sshd on a non-standard port less than, say 1024 for example. This is the real reason I initially created the PR and posted to hackers@ about this -- I'd like to fix it. But, I want to fix it the right way, and not hack a crude solution. Regards, -- Glen Barber From alfred at freebsd.org Thu May 21 00:53:16 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Thu May 21 00:53:22 2009 Subject: porting info for FreeBSD's kernel? In-Reply-To: <4A11B893.1000808@telenix.org> References: <4A11B893.1000808@telenix.org> Message-ID: <20090521003646.GS67847@elvis.mu.org> * Chuck Robey [090518 13:03] wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I've been googling, trying to see if I can find notes regarding what needs > changing, in what order, to adapt the FreeBSD kernel to a new processor. Anyone > know where stuff like that can be found? You need a cross compile toolchain of course, look into how FreeBSD is configured for the various arches. Then I would suggest looking at the loaders, followed by kern/init_main.c. If you trace down init_main.c and some of the early sysinits that should give you an idea. You might also be able to backtrack using CVS/svn to follow how mips or arm was done. Note: freebsd has a decent cross-compile setup now, see "make universe" so things should be easier to get started. -- - Alfred Perlstein From yuri at rawbw.com Thu May 21 06:32:51 2009 From: yuri at rawbw.com (Yuri) Date: Thu May 21 06:32:58 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? Message-ID: <4A14F58F.8000801@rawbw.com> Seems like failing system calls (mmap and sbrk) that allocate memory is more graceful and would allow the program to at least issue the reasonable error message. And more intelligent programs would be able to reduce used memory instead of just dying. Yuri From neldredge at math.ucsd.edu Thu May 21 07:15:24 2009 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu May 21 07:15:31 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A14F58F.8000801@rawbw.com> References: <4A14F58F.8000801@rawbw.com> Message-ID: On Wed, 20 May 2009, Yuri wrote: > Seems like failing system calls (mmap and sbrk) that allocate memory is more > graceful and would allow the program to at least issue the reasonable error > message. > And more intelligent programs would be able to reduce used memory instead of > just dying. It's a feature, called "memory overcommit". It has a variety of pros and cons, and is somewhat controversial. One advantage is that programs often allocate memory (in various ways) that they will never use, which under a conservative policy would result in that memory being wasted, or programs failing unnecessarily. With overcommit, you sometimes allocate more memory than you have, on the assumption that some of it will not actually be needed. Although memory allocated by mmap and sbrk usually does get used in fairly short order, there are other ways of allocating memory that are easy to overlook, and which may "allocate" memory that you don't actually intend to use. Probably the best example is fork(). For instance, consider the following program. #define SIZE 1000000000 /* 1 GB */ int main(void) { char *buf = malloc(SIZE); /* 1 GB */ memset(buf, 'x', SIZE); /* touch the buffer */ pid_t pid = fork(); if (pid == 0) { execlp("true", "true", (char *)NULL); perror("true"); _exit(1); } else if (pid > 0) { for (;;); /* do work */ } else { perror("fork"); exit(1); } return 0; } Suppose we run this program on a machine with just over 1 GB of memory. The fork() should give the child a private "copy" of the 1 GB buffer, by setting it to copy-on-write. In principle, after the fork(), the child might want to rewrite the buffer, which would require an additional 1GB to be available for the child's copy. So under a conservative allocation policy, the kernel would have to reserve that extra 1 GB at the time of the fork(). Since it can't do that on our hypothetical 1+ GB machine, the fork() must fail, and the program won't work. However, in fact that memory is not going to be used, because the child is going to exec() right away, which will free the child's "copy". Indeed, this happens most of the time with fork() (but of course the kernel can't know when it will or won't.) With overcommit, we pretend to give the child a writable private copy of the buffer, in hopes that it won't actually use more of it than we can fulfill with physical memory. If it doesn't use it, all is well; if it does use it, then disaster occurs and we have to start killing things. So the advantage is you can run programs like the one above on machines that technically don't have enough memory to do so. The disadvantage, of course, is that if someone calls the bluff, then we kill random processes. However, this is not all that much worse than failing allocations: although programs can in theory handle failed allocations and respond accordingly, in practice they don't do so and just quit anyway. So in real life, both cases result in disaster when memory "runs out"; with overcommit, the disaster is a little less predictable but happens much less often. If you google for "memory overcommit" you will see lots of opinions and debate about this feature on various operating systems. There may be a way to enable the conservative behavior; I know Linux has an option to do this, but am not sure about FreeBSD. This might be useful if you are paranoid, or run programs that you know will gracefully handle running out of memory. IMHO for general use it is better to have overcommit, but I know there are those who disagree. -- Nate Eldredge neldredge@math.ucsd.edu From raysonlogin at gmail.com Thu May 21 07:19:20 2009 From: raysonlogin at gmail.com (Rayson Ho) Date: Thu May 21 07:19:28 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A14F58F.8000801@rawbw.com> References: <4A14F58F.8000801@rawbw.com> Message-ID: <73a01bf20905202356t5fd65eb5n7ec97f6c318e6045@mail.gmail.com> Because the kernel is lazy!! You can google for "lazy algorithm", or find an OS internals book and read about the advantages of doing it this way... Rayson On Thu, May 21, 2009 at 1:32 AM, Yuri wrote: > Seems like failing system calls (mmap and sbrk) that allocate memory is more > graceful and would allow the program to at least issue the reasonable error > message. > And more intelligent programs would be able to reduce used memory instead of > just dying. > > Yuri > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From elias at artx.ru Thu May 21 08:12:51 2009 From: elias at artx.ru (Ilya Orehov) Date: Thu May 21 08:12:58 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A14F58F.8000801@rawbw.com> References: <4A14F58F.8000801@rawbw.com> Message-ID: <20090521073918.GA54618@artx.ru> +------- Yuri, 2009-05-20 ------- | Seems like failing system calls (mmap and sbrk) that allocate memory is more | graceful and would allow the program to at least issue the reasonable | error message. | And more intelligent programs would be able to reduce used memory | instead of just dying. Hi! You can set memory limit to achieve your goal: tcsh% limit vmemoryuse 20M In this case, malloc(1000000000) will return 0. Ilya. | | Yuri | | _______________________________________________ | freebsd-hackers@freebsd.org mailing list | http://lists.freebsd.org/mailman/listinfo/freebsd-hackers | To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" | +----------------------------- From perryh at pluto.rain.com Thu May 21 08:21:50 2009 From: perryh at pluto.rain.com (perryh@pluto.rain.com) Date: Thu May 21 08:21:58 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: References: <4A14F58F.8000801@rawbw.com> Message-ID: <4a150b7b.kwnuIl++HgdJdRWU%perryh@pluto.rain.com> Nate Eldredge wrote: > For instance, consider the following program. > this happens most of the time with fork() ... It may be worthwhile to point out that one extremely common case is the shell itself. Even /bin/sh is large; csh (the default FreeBSD shell) is quite a bit larger and bash larger yet. The case of "big program forks, and the child process execs a small program" arises almost every time a shell command (other than a built-in) is executed. > With overcommit, we pretend to give the child a writable private > copy of the buffer, in hopes that it won't actually use more of it > than we can fulfill with physical memory. I am about 99% sure that the issue involves virtual memory, not physical, at least in the fork/exec case. The incidence of such events under any particular system load scenario can be reduced or eliminated simply by adding swap space. From xorquewasp at googlemail.com Thu May 21 09:53:10 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Thu May 21 09:53:24 2009 Subject: compiling system binutils as cross tools Message-ID: <20090521095305.GA27043@logik.internal.network> Hi. How do I compile the system binutils (contrib/binutils) as i386 -> x86_64 cross utils? That is, binutils that will run on an i386 host but will produce x86_64 binaries? I'm trying to produce a bootstrapping compiler for a port and need to get these working. I've spent a while reading Makefiles but would rather get information from someone who actually knows rather than waste *another* week on this stuff. I'd rather not compile the entire world if it can be avoided. cheers, xw From rwatson at FreeBSD.org Thu May 21 10:20:22 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu May 21 10:20:28 2009 Subject: compiling system binutils as cross tools In-Reply-To: <20090521095305.GA27043@logik.internal.network> References: <20090521095305.GA27043@logik.internal.network> Message-ID: On Thu, 21 May 2009, xorquewasp@googlemail.com wrote: > How do I compile the system binutils (contrib/binutils) as i386 -> x86_64 > cross utils? That is, binutils that will run on an i386 host but will > produce x86_64 binaries? > > I'm trying to produce a bootstrapping compiler for a port and need to get > these working. I've spent a while reading Makefiles but would rather get > information from someone who actually knows rather than waste *another* week > on this stuff. > > I'd rather not compile the entire world if it can be avoided. Not really my area, but if you haven't found "make toolchain" and "make buildenv" then you might want to take a look. Typically these will be combined with TARGET_ARCH=foo, and in your case foo is 'amd64'. The former builds the toolchain required for the architecture, and the latter creates a shell environment with paths appropriately munged and environments appropriately set to cross-compile using that chain. Normally the toolchain step is part of our integrated buildworld/buildkernel/etc process, but you can also use it for other things with buildenv. Robert N M Watson Computer Laboratory University of Cambridge From xorquewasp at googlemail.com Thu May 21 11:55:05 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Thu May 21 11:55:12 2009 Subject: compiling system binutils as cross tools In-Reply-To: References: <20090521095305.GA27043@logik.internal.network> Message-ID: <20090521115458.GA54961@logik.internal.network> On 2009-05-21 11:20:21, Robert Watson wrote: > Not really my area, but if you haven't found "make toolchain" and "make > buildenv" then you might want to take a look. Typically these will be > combined with TARGET_ARCH=foo, and in your case foo is 'amd64'. The > former builds the toolchain required for the architecture, and the latter > creates a shell environment with paths appropriately munged and > environments appropriately set to cross-compile using that chain. > Normally the toolchain step is part of our integrated > buildworld/buildkernel/etc process, but you can also use it for other > things with buildenv. Thanks, 'make toolchain' looks like it'll work. 'make buildenv' gave me the paths to the binaries I needed to tell the compiler I'm working on to use those for cross compilation. What tangled webs we weave... cheers, xw From stas at FreeBSD.org Thu May 21 12:10:28 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Thu May 21 12:10:37 2009 Subject: compiling system binutils as cross tools In-Reply-To: <20090521095305.GA27043@logik.internal.network> References: <20090521095305.GA27043@logik.internal.network> Message-ID: <20090521161018.66b3015c@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 21 May 2009 10:53:05 +0100 xorquewasp@googlemail.com mentioned: > Hi. > > How do I compile the system binutils (contrib/binutils) as i386 -> > x86_64 cross utils? That is, binutils that will run on an i386 host but > will produce x86_64 binaries? > > I'm trying to produce a bootstrapping compiler for a port and need to > get these working. I've spent a while reading Makefiles but would rather > get information from someone who actually knows rather than waste > *another* week on this stuff. > > I'd rather not compile the entire world if it can be avoided. > You can also try using devel/cross-binutils to build cross-tools for x86_64-freebsd. Random people reported they're working fine. - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoVRK4ACgkQK/VZk+smlYGbjwCff1f6hJ+PAE4OSqPV7IIQi8kY 8iwAn2CcQ3H9D5Q+mZdern+11PgRGapq =Amr/ -----END PGP SIGNATURE----- !DSPAM:4a1544b0994291380925937! From neldredge at math.ucsd.edu Thu May 21 16:00:38 2009 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu May 21 16:00:45 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4a150b7b.kwnuIl++HgdJdRWU%perryh@pluto.rain.com> References: <4A14F58F.8000801@rawbw.com> <4a150b7b.kwnuIl++HgdJdRWU%perryh@pluto.rain.com> Message-ID: On Thu, 21 May 2009, perryh@pluto.rain.com wrote: > Nate Eldredge wrote: >> With overcommit, we pretend to give the child a writable private >> copy of the buffer, in hopes that it won't actually use more of it >> than we can fulfill with physical memory. > > I am about 99% sure that the issue involves virtual memory, not > physical, at least in the fork/exec case. The incidence of such > events under any particular system load scenario can be reduced or > eliminated simply by adding swap space. True. When I said "a system with 1GB of memory", I should have said "a system with 1 GB of physical memory + swap". -- Nate Eldredge neldredge@math.ucsd.edu From xorquewasp at googlemail.com Thu May 21 16:44:47 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Thu May 21 16:44:54 2009 Subject: compiling system binutils as cross tools In-Reply-To: <20090521161018.66b3015c@FreeBSD.org> References: <20090521095305.GA27043@logik.internal.network> <20090521161018.66b3015c@FreeBSD.org> Message-ID: <20090521164442.GA59069@logik.internal.network> On 2009-05-21 16:10:18, Stanislav Sedov wrote: > You can also try using devel/cross-binutils to build cross-tools for > x86_64-freebsd. Random people reported they're working fine. Unfortunately, as noted in this thread: http://marc.info/?l=freebsd-hackers&m=124146166902690&w=2 Using that port works but creates a compiler that emits code that can't be assembled by the default system binutils. Not great for a port... xw From julian at elischer.org Thu May 21 17:49:24 2009 From: julian at elischer.org (Julian Elischer) Date: Thu May 21 17:49:31 2009 Subject: compiling system binutils as cross tools In-Reply-To: References: <20090521095305.GA27043@logik.internal.network> Message-ID: <4A159423.2040500@elischer.org> Robert Watson wrote: > > On Thu, 21 May 2009, xorquewasp@googlemail.com wrote: > >> How do I compile the system binutils (contrib/binutils) as i386 -> >> x86_64 cross utils? That is, binutils that will run on an i386 host >> but will produce x86_64 binaries? >> >> I'm trying to produce a bootstrapping compiler for a port and need to >> get these working. I've spent a while reading Makefiles but would >> rather get information from someone who actually knows rather than >> waste *another* week on this stuff. >> >> I'd rather not compile the entire world if it can be avoided. > > Not really my area, but if you haven't found "make toolchain" and "make > buildenv" then you might want to take a look. Typically these will be > combined with TARGET_ARCH=foo, and in your case foo is 'amd64'. The > former builds the toolchain required for the architecture, and the > latter creates a shell environment with paths appropriately munged and > environments appropriately set to cross-compile using that chain. > Normally the toolchain step is part of our integrated > buildworld/buildkernel/etc process, but you can also use it for other > things with buildenv. I munged that once to create a nested jail/chroot set up so that default toolchain was the cross set. so if you did 'cc foo.c' you got a cross binary.. if you needed a native cc you did it in the outside chroot. worked like a charm. from the outside, you just did 'chroot cross cc foo.c' to get cross binary. > Robert N M Watson > Computer Laboratory > University of Cambridge > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From yuri at rawbw.com Thu May 21 17:52:29 2009 From: yuri at rawbw.com (Yuri) Date: Thu May 21 17:52:36 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: References: <4A14F58F.8000801@rawbw.com> Message-ID: <4A1594DA.2010707@rawbw.com> Nate Eldredge wrote: > Suppose we run this program on a machine with just over 1 GB of > memory. The fork() should give the child a private "copy" of the 1 GB > buffer, by setting it to copy-on-write. In principle, after the > fork(), the child might want to rewrite the buffer, which would > require an additional 1GB to be available for the child's copy. So > under a conservative allocation policy, the kernel would have to > reserve that extra 1 GB at the time of the fork(). Since it can't do > that on our hypothetical 1+ GB machine, the fork() must fail, and the > program won't work. I don't have strong opinion for or against "memory overcommit". But I can imagine one could argue that fork with intent of exec is a faulty scenario that is a relict from the past. It can be replaced by some atomic method that would spawn the child without ovecommitting. Are there any other than fork (and mmap/sbrk) situations that would overcommit? Yuri From ticso at cicely7.cicely.de Thu May 21 19:37:12 2009 From: ticso at cicely7.cicely.de (Bernd Walter) Date: Thu May 21 19:37:20 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A1594DA.2010707@rawbw.com> References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: <20090521192356.GA54607@cicely7.cicely.de> On Thu, May 21, 2009 at 10:52:26AM -0700, Yuri wrote: > Nate Eldredge wrote: > >Suppose we run this program on a machine with just over 1 GB of > >memory. The fork() should give the child a private "copy" of the 1 GB > >buffer, by setting it to copy-on-write. In principle, after the > >fork(), the child might want to rewrite the buffer, which would > >require an additional 1GB to be available for the child's copy. So > >under a conservative allocation policy, the kernel would have to > >reserve that extra 1 GB at the time of the fork(). Since it can't do > >that on our hypothetical 1+ GB machine, the fork() must fail, and the > >program won't work. > > I don't have strong opinion for or against "memory overcommit". But I > can imagine one could argue that fork with intent of exec is a faulty > scenario that is a relict from the past. It can be replaced by some > atomic method that would spawn the child without ovecommitting. > > Are there any other than fork (and mmap/sbrk) situations that would > overcommit? If your system has enough virtual memory for working without overcommitment it will run fine with overcommitment as well. If you don't have enough memory it can do much more with overcommitment. A simple apache process needing 1G and serving 1000 Clients would need 1TB swap without ever touching it. Same for small embedded systems with limited swap. So the requirement of overcomittment is not just a requirement of old days. Overcomittment is even used more and more. An example are snapshots, which are popular these days can lead to space failure in case you rewrite a file with new data without growing its length. The old sparse file concept is also one of them, which can confuse unaware software. And then we have geom_virstore since a while. Many modern databases do it as well. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From randy at psg.com Thu May 21 21:12:02 2009 From: randy at psg.com (Randy Bush) Date: Thu May 21 21:17:06 2009 Subject: Installation from USB pen Message-ID: i succeeded with putting 8-current snap on a pen and booting. but i can not figure out how to tell it to use the pen drive for system image loads. do i have to back off to 7 and then upgrade forward after install? rndy From neldredge at math.ucsd.edu Thu May 21 21:37:22 2009 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu May 21 21:37:28 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A1594DA.2010707@rawbw.com> References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: On Thu, 21 May 2009, Yuri wrote: > Nate Eldredge wrote: >> Suppose we run this program on a machine with just over 1 GB of memory. The >> fork() should give the child a private "copy" of the 1 GB buffer, by >> setting it to copy-on-write. In principle, after the fork(), the child >> might want to rewrite the buffer, which would require an additional 1GB to >> be available for the child's copy. So under a conservative allocation >> policy, the kernel would have to reserve that extra 1 GB at the time of the >> fork(). Since it can't do that on our hypothetical 1+ GB machine, the >> fork() must fail, and the program won't work. > > I don't have strong opinion for or against "memory overcommit". But I can > imagine one could argue that fork with intent of exec is a faulty scenario > that is a relict from the past. It can be replaced by some atomic method that > would spawn the child without ovecommitting. I would say rather it's a centerpiece of Unix design, with an unfortunate consequence. Actually, historically this would have been much more of a problem than at present, since early Unix systems had much less memory, no copy-on-write, and no virtual memory (this came in with BSD, it appears; it's before my time.) The modern "atomic" method we have these days is posix_spawn, which has a pretty complicated interface if you want to use pipes or anything. It exists mostly for the benefit of systems whose hardware is too primitive to be able to fork() in a reasonable manner. The old way to avoid the problem of needing this extra memory temporarily was to use vfork(), but this has always been a hack with a number of problems. IMHO neither of these is preferable in principle to fork/exec. Note another good example is a large process that forks, but the child rather than exec'ing performs some simple task that writes to very little of its "copied" address space. Apache does this, as Bernd mentioned. This also is greatly helped by having overcommit, but can't be circumvented by replacing fork() with something else. If it really doesn't need to modify any of its shared address space, a thread can sometimes be used instead of a forked subprocess, but this has issues of its own. Of course all these problems are solved, under any policy, by having more memory or swap. But overcommit allows you to do more with less. > Are there any other than fork (and mmap/sbrk) situations that would > overcommit? Perhaps, but I can't think of good examples offhand. -- Nate Eldredge neldredge@math.ucsd.edu From chuckr at telenix.org Thu May 21 21:56:05 2009 From: chuckr at telenix.org (Chuck Robey) Date: Thu May 21 21:56:11 2009 Subject: porting info for FreeBSD's kernel? In-Reply-To: <20090521003646.GS67847@elvis.mu.org> References: <4A11B893.1000808@telenix.org> <20090521003646.GS67847@elvis.mu.org> Message-ID: <4A15CE00.4040600@telenix.org> Alfred Perlstein wrote: > * Chuck Robey [090518 13:03] wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> I've been googling, trying to see if I can find notes regarding what needs >> changing, in what order, to adapt the FreeBSD kernel to a new processor. Anyone >> know where stuff like that can be found? > > You need a cross compile toolchain of course, look into how FreeBSD > is configured for the various arches. > > Then I would suggest looking at the loaders, followed by > kern/init_main.c. If you trace down init_main.c and some > of the early sysinits that should give you an idea. > > You might also be able to backtrack using CVS/svn to follow > how mips or arm was done. > > Note: freebsd has a decent cross-compile setup now, see > "make universe" so things should be easier to get started. > Thanks. I will *definitely* read all the parts you hint me at, I won't be deleting this mail, and I appreciate it. I was asking on the llvm maillist about Cortex-A8 support. What I got says that it's not there yet, but it's being worked upon, that and the -A9 support (definite differences). So, any crosstools needed today would have to be gcc, from a version at least as new as the 4.3 branch (that's where they brought in the -A8 support). The tool I got by doing the freeBSD crosstools was 4.2.1, which isn't going to do it for the Cortex-A8, and I had someone else (from a FreeBSD list) tell me that bringing in a newer version of gcc wasn't extremely likely, that they'd want llvm instead. I see 3 alternatives for a Cortex-A8 port: using a new gcc port, waiting on the upgrade of llvm, or maybe deciding that the version the llvm that's out now, with the v6 compatibility, would be (for the short term) good enough. Any idea which one to choose? The only one that interests me is for the TI OMAP 3530 (Cortex-A8, among other parts). Maybe if the currently available llvm is good enough, maybe gcc-4.2.1 may creak along well enough for the short term? I need to understand this. My own personal Pandora won't probably won't arrive on my doorstep for maybe as long as 3 more months, so in the meantime, I think I will be reading all I can get my hands on regarding llvm. Maybe I can really learn enough to make a difference. In school, I concentrated very definitely on OSes (I've written 3 of them over the years, of quite varying levels of performance), so for compilers, I'm relying on my old 1988 version of the Aho/Sethi/Ullman compilers book. If anyone knows a more modern book that will show me enough about compilers to be useful, I'd really appreciate the name, maybe Amazon will let me get a cheap used version. From chuckr at telenix.org Thu May 21 22:15:25 2009 From: chuckr at telenix.org (Chuck Robey) Date: Thu May 21 22:15:31 2009 Subject: about building the crosstools Message-ID: <4A15D288.3060008@telenix.org> I got instructions from Warner about how to build my crosstools (the FreeBSD ones) and after a minor startup contretemps, things began to work better. My problem is that on doing the linking step, I'm getting a complaint that it can't figure out how to build the /usr/cross/usr/lib/libc.a (/usr/cross being my toolls destdir). I don't know how to fix this in the build, so I'd appreciate any hints. I mean, it *seems* to me that these tools are meant to run on my current host (i386), not the target (arm) so it really should already know about my /usr/lib/libc.a (or shared version)) right? From imp at bsdimp.com Thu May 21 22:30:55 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Thu May 21 22:31:02 2009 Subject: about building the crosstools In-Reply-To: <4A15D288.3060008@telenix.org> References: <4A15D288.3060008@telenix.org> Message-ID: <20090521.162851.439727948.imp@bsdimp.com> In message: <4A15D288.3060008@telenix.org> Chuck Robey writes: : I got instructions from Warner about how to build my crosstools (the FreeBSD : ones) and after a minor startup contretemps, things began to work better. My : problem is that on doing the linking step, I'm getting a complaint that it can't : figure out how to build the /usr/cross/usr/lib/libc.a (/usr/cross being my : toolls destdir). I don't know how to fix this in the build, so I'd appreciate : any hints. I mean, it *seems* to me that these tools are meant to run on my : current host (i386), not the target (arm) so it really should already know about : my /usr/lib/libc.a (or shared version)) right? You may have some contamination. The xdev targets doesn't use /usr/cross at all. I'd blow that away entirely and try again. Warner From stas at FreeBSD.org Thu May 21 22:46:53 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Thu May 21 22:46:59 2009 Subject: Installation from USB pen In-Reply-To: References: Message-ID: <20090522024721.da1ec85a.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 21 May 2009 14:12:00 -0700 Randy Bush mentioned: > i succeeded with putting 8-current snap on a pen and booting. but i can > not figure out how to tell it to use the pen drive for system image > loads. > What do you mean by system image loads? Does it load kernel succesfully but cannot find root filesystem? - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoV2gEACgkQK/VZk+smlYHVqQCfb0lmeXbKdbk+Ktq1l2Dngz01 HEsAn1U5V1nnnyFs89Yvxo5xbjvIwzmY =gp18 -----END PGP SIGNATURE----- !DSPAM:4a15d9da994292682134302! From stas at FreeBSD.org Thu May 21 22:52:46 2009 From: stas at FreeBSD.org (Stanislav Sedov) Date: Thu May 21 22:52:57 2009 Subject: compiling system binutils as cross tools In-Reply-To: <20090521164442.GA59069@logik.internal.network> References: <20090521095305.GA27043@logik.internal.network> <20090521161018.66b3015c@FreeBSD.org> <20090521164442.GA59069@logik.internal.network> Message-ID: <20090522025322.2acebb01.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 21 May 2009 17:44:42 +0100 xorquewasp@googlemail.com mentioned: > On 2009-05-21 16:10:18, Stanislav Sedov wrote: > > You can also try using devel/cross-binutils to build cross-tools for > > x86_64-freebsd. Random people reported they're working fine. > > Unfortunately, as noted in this thread: > > http://marc.info/?l=freebsd-hackers&m=124146166902690&w=2 > > Using that port works but creates a compiler that emits code > that can't be assembled by the default system binutils. Not > great for a port... > Why not make this compiler to use fresh binutils from cross-binutils instead of using systems binutils? This will also allow to support newer processor families and architectures. Is it possible to tell GNAT where to look for binutils to assembly and link with? - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkoV22IACgkQK/VZk+smlYGJSACghXD2H4iN9HE/DmNDKhdNVfMY /SQAnjQ+HMeyMP9ZKJhF5F09Buex1tOz =VB1I -----END PGP SIGNATURE----- !DSPAM:4a15db3c994295534499307! From randy at psg.com Thu May 21 22:49:09 2009 From: randy at psg.com (Randy Bush) Date: Thu May 21 22:56:49 2009 Subject: Installation from USB pen In-Reply-To: <20090522024721.da1ec85a.stas@FreeBSD.org> References: <20090522024721.da1ec85a.stas@FreeBSD.org> Message-ID: >> i succeeded with putting 8-current snap on a pen and booting. but i can >> not figure out how to tell it to use the pen drive for system image >> loads. > What do you mean by system image loads? Does it load kernel succesfully > but cannot find root filesystem? sorry. no. it wants the cd or ftp or ... to get the install pieces. as it is a snapshot, there are none on net (that i can find). but they went onto the usb. but i can not figure out how to tell it to get them from there. randy From tom.hurst at clara.net Fri May 22 02:09:52 2009 From: tom.hurst at clara.net (Thomas Hurst) Date: Fri May 22 02:10:00 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: References: <4A14F58F.8000801@rawbw.com> Message-ID: <20090522014206.GA62573@voi.aagh.net> * Nate Eldredge (neldredge@math.ucsd.edu) wrote: > There may be a way to enable the conservative behavior; I know Linux > has an option to do this, but am not sure about FreeBSD. I seem to remember a patch to disable overcommit. Here we go: http://people.freebsd.org/~kib/overcommit/ -- Thomas 'Freaky' Hurst http://hur.st/ From dillon at apollo.backplane.com Fri May 22 02:26:16 2009 From: dillon at apollo.backplane.com (Matthew Dillon) Date: Fri May 22 02:26:23 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: <200905220211.n4M2Bg5b036854@apollo.backplane.com> There is no such thing as a graceful way to deal with running out of memory. What is a program supposed to do? Even if it gracefully exits it still fails to perform the function for which it was designed. If such a program is used in a script then the script fails as well. Even the best systems (e.g. space shuttle, mars rover, airplane control systems) which try to deal with unexpected situations still have to have the final option, that being a complete reset. And even a complete reset is no guarantee of recovery (as one of the original airbus accidents during an air-show revealed when the control systems got into a reset loop and the pilot could not regain control of the plane). The most robust systems do things like multiple independant built-to-spec programs and a voting system which require 10 times the man power to code and test, something you will likely never see in the open-source world or even in most of the commercial application world. In fact, it is nearly impossible to write code which gracefully fails even if the intent is to gracefully fail (and even if one can even figure out what a workable graceful failure path would even be). You would have to build code paths to deal with the failure conditions, significantly increasing the size of the code base, and you would have to test every possible failure combination to exercise those code paths to make sure they actually work as expected. If you don't then the code paths designed to deal with the failure will themselves likely be full of bugs and make the problem worse. People who try to program this way but don't have the massive resources required often wind up with seriously bloated and buggy code. So if the system runs out of memory (meaning physical memory + all available swap), having a random subset of programs of any size start to fail will rapidly result in a completely unusable system and only a reboot will get it back into shape. At least until it runs out of memory again. -- The best one can do is make the failures more deterministic. Killing the largest program is one such mechanic. Knowing how the system will react makes it easier to restore the system without necessarily rebooting it. Of course there might have to be exceptions as you don't want your X server to be the program chosen. Generally, though, having some sort of deterministic progression is going to be far better then having half a dozen random programs which happen to be trying to allocate memory suddenly get an unexpected memory allocation failure. Also, it's really a non-problem. Simply configure a lot of swap... like 8G or 16G if you really care. Or 100G. Then you *DO* get a graceful failure which gives you time to figure out what is going on and fix it. The graceful failure is that the system starts to page to swap more and more heavily, getting slower and slower in the process, but doesn't actually have to kill anything for minutes to hours depending on the failure condition. It's a lot easier to write code which reacts to a system which is operating at less then ideal efficiency then it is to write code which reacts to the failure of a core function (that of allocating memory). One could even monitor swap use as ring the alarm bells if it goes above a certain point. Overcommit has never been the problem. The problem is there is no way a large system can gracefully deal with running out of memory, overcommit or not. -Matt From doconnor at gsoft.com.au Fri May 22 06:17:33 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Fri May 22 06:17:45 2009 Subject: Installation from USB pen In-Reply-To: References: Message-ID: <200905221547.27453.doconnor@gsoft.com.au> On Fri, 22 May 2009, Randy Bush wrote: > i succeeded with putting 8-current snap on a pen and booting. but i > can not figure out how to tell it to use the pen drive for system > image loads. > > do i have to back off to 7 and then upgrade forward after install? I don't believe you can install from UFS unless you mount it first and then tell it to do an FS install. I have a 7.x based USB installer that is split in 2 - half FAT32 half UFS and it works. Having half FAT32 is handy if you need to edit/add stuff from Windows. It does make it a PITA to build the install key though. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090522/b533d2ba/attachment.pgp From doconnor at gsoft.com.au Fri May 22 06:17:33 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Fri May 22 06:17:46 2009 Subject: Installation from USB pen In-Reply-To: References: Message-ID: <200905221547.27453.doconnor@gsoft.com.au> On Fri, 22 May 2009, Randy Bush wrote: > i succeeded with putting 8-current snap on a pen and booting. but i > can not figure out how to tell it to use the pen drive for system > image loads. > > do i have to back off to 7 and then upgrade forward after install? I don't believe you can install from UFS unless you mount it first and then tell it to do an FS install. I have a 7.x based USB installer that is split in 2 - half FAT32 half UFS and it works. Having half FAT32 is handy if you need to edit/add stuff from Windows. It does make it a PITA to build the install key though. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090522/b533d2ba/attachment-0001.pgp From doconnor at gsoft.com.au Fri May 22 06:25:48 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Fri May 22 06:25:56 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A1594DA.2010707@rawbw.com> References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: <200905221555.42775.doconnor@gsoft.com.au> On Fri, 22 May 2009, Yuri wrote: > Nate Eldredge wrote: > > Suppose we run this program on a machine with just over 1 GB of > > memory. The fork() should give the child a private "copy" of the 1 > > GB buffer, by setting it to copy-on-write. In principle, after the > > fork(), the child might want to rewrite the buffer, which would > > require an additional 1GB to be available for the child's copy. So > > under a conservative allocation policy, the kernel would have to > > reserve that extra 1 GB at the time of the fork(). Since it can't > > do that on our hypothetical 1+ GB machine, the fork() must fail, > > and the program won't work. > > I don't have strong opinion for or against "memory overcommit". But I > can imagine one could argue that fork with intent of exec is a faulty > scenario that is a relict from the past. It can be replaced by some > atomic method that would spawn the child without ovecommitting. If all you are going to do is call execve() then use vfork(). That explicitly does not copy the parent's address space. Also your example is odd, if you have a program using 1Gb (RAM + swap) and you want to start another (in any way) then that is going to be impossible. If you had a 750Mb process that forked and the child only modified 250Mb you'd be all right because the other pages would be copies. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090522/2271fe8f/attachment.pgp From j.mckeown at ru.ac.za Fri May 22 06:54:25 2009 From: j.mckeown at ru.ac.za (Jonathan McKeown) Date: Fri May 22 06:56:12 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: <200905220854.21917.j.mckeown@ru.ac.za> On Thursday 21 May 2009 23:37:20 Nate Eldredge wrote: > Of course all these problems are solved, under any policy, by having more > memory or swap. ?But overcommit allows you to do more with less. Or to put it another way, 90% of the problems that could be solved by having more memory can also be solved by pretending you have more memory and hoping no-one calls your bluff. Jonathan From alfred at freebsd.org Fri May 22 07:31:31 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Fri May 22 07:31:38 2009 Subject: porting info for FreeBSD's kernel? In-Reply-To: <4A15CE00.4040600@telenix.org> References: <4A11B893.1000808@telenix.org> <20090521003646.GS67847@elvis.mu.org> <4A15CE00.4040600@telenix.org> Message-ID: <20090522073130.GI67847@elvis.mu.org> * Chuck Robey [090521 14:56] wrote: > Alfred Perlstein wrote: > > * Chuck Robey [090518 13:03] wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA1 > >> > >> I've been googling, trying to see if I can find notes regarding what needs > >> changing, in what order, to adapt the FreeBSD kernel to a new processor. Anyone > >> know where stuff like that can be found? > > > > You need a cross compile toolchain of course, look into how FreeBSD > > is configured for the various arches. > > > > Then I would suggest looking at the loaders, followed by > > kern/init_main.c. If you trace down init_main.c and some > > of the early sysinits that should give you an idea. > > > > You might also be able to backtrack using CVS/svn to follow > > how mips or arm was done. > > > > Note: freebsd has a decent cross-compile setup now, see > > "make universe" so things should be easier to get started. > > > > Thanks. I will *definitely* read all the parts you hint me at, I won't be > deleting this mail, and I appreciate it. I was asking on the llvm maillist > about Cortex-A8 support. What I got says that it's not there yet, but it's > being worked upon, that and the -A9 support (definite differences). So, any > crosstools needed today would have to be gcc, from a version at least as new as > the 4.3 branch (that's where they brought in the -A8 support). > > The tool I got by doing the freeBSD crosstools was 4.2.1, which isn't going to > do it for the Cortex-A8, and I had someone else (from a FreeBSD list) tell me > that bringing in a newer version of gcc wasn't extremely likely, that they'd > want llvm instead. I see 3 alternatives for a Cortex-A8 port: using a new gcc > port, waiting on the upgrade of llvm, or maybe deciding that the version the > llvm that's out now, with the v6 compatibility, would be (for the short term) > good enough. Any idea which one to choose? The only one that interests me is > for the TI OMAP 3530 (Cortex-A8, among other parts). Maybe if the currently > available llvm is good enough, maybe gcc-4.2.1 may creak along well enough for > the short term? I need to understand this. > > My own personal Pandora won't probably won't arrive on my doorstep for maybe as > long as 3 more months, so in the meantime, I think I will be reading all I can > get my hands on regarding llvm. Maybe I can really learn enough to make a > difference. In school, I concentrated very definitely on OSes (I've written 3 > of them over the years, of quite varying levels of performance), so for > compilers, I'm relying on my old 1988 version of the Aho/Sethi/Ullman compilers > book. If anyone knows a more modern book that will show me enough about > compilers to be useful, I'd really appreciate the name, maybe Amazon will let me > get a cheap used version. I wouldn't sweat the compiler as much as the actual OS code, I think it should be relatively easy to trick the build to use an external compiler (ie, don't get caught up in the compiler bootstrap quagmire, leave that for later...) Anyhow, you're talking to someone that has studied, but not implemented a port, so take my advice with a few heaps of salt. :) Typically what people focus on is: 1) "how am I going to get the first line of dmesg to come up" 2) "how am I going to get to single user mode" 3) "multi user?" 4) cleanup of compiler and bootstrap issues. If you get sidetracked by #4, you can spend months doing that instead of just rolling with it when you get there. -- - Alfred Perlstein From alfred at freebsd.org Fri May 22 07:34:00 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Fri May 22 07:34:06 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A1594DA.2010707@rawbw.com> References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: <20090522073359.GJ67847@elvis.mu.org> * Yuri [090521 10:52] wrote: > Nate Eldredge wrote: > >Suppose we run this program on a machine with just over 1 GB of > >memory. The fork() should give the child a private "copy" of the 1 GB > >buffer, by setting it to copy-on-write. In principle, after the > >fork(), the child might want to rewrite the buffer, which would > >require an additional 1GB to be available for the child's copy. So > >under a conservative allocation policy, the kernel would have to > >reserve that extra 1 GB at the time of the fork(). Since it can't do > >that on our hypothetical 1+ GB machine, the fork() must fail, and the > >program won't work. > > I don't have strong opinion for or against "memory overcommit". But I > can imagine one could argue that fork with intent of exec is a faulty > scenario that is a relict from the past. It can be replaced by some > atomic method that would spawn the child without ovecommitting. vfork, however that's not sufficient for many scenarios. > Are there any other than fork (and mmap/sbrk) situations that would > overcommit? sysv shm? maybe more. -- - Alfred Perlstein From xorquewasp at googlemail.com Fri May 22 08:15:47 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Fri May 22 08:15:54 2009 Subject: compiling system binutils as cross tools In-Reply-To: <20090522025322.2acebb01.stas@FreeBSD.org> References: <20090521095305.GA27043@logik.internal.network> <20090521161018.66b3015c@FreeBSD.org> <20090521164442.GA59069@logik.internal.network> <20090522025322.2acebb01.stas@FreeBSD.org> Message-ID: <20090522081525.GA59432@logik.internal.network> On 2009-05-22 02:53:22, Stanislav Sedov wrote: > > Why not make this compiler to use fresh binutils from cross-binutils > instead of using systems binutils? This will also allow to support > newer processor families and architectures. Is it possible to tell > GNAT where to look for binutils to assembly and link with? > Well, like I said, at the moment there's a choice of using the system binutils or the cross-binutils port. The compiler isn't actually intended to be a cross compiler but a native amd64 compiler (the lang/gnat-gcc* ports have been marked as i386-only for ages). I'm not sure if it's possible to tell GNAT where to look for binutils at runtime. I have some patches to send to both Adacore and the GCC developers to add support for FreeBSD amd64. I decided to use the system binutils because in order for someone to actually build the port, they have to use bootstrap compiler binaries provided by me (the gnat-gcc* ports do the same thing) and having those binaries depend on a moving target like cross-binutils might create even more complications. I'll be providing full documentation and build scripts to show how the bootstrapping compiler was created so if someone feels the need to use the cross-binutils port, they can. cheers, xw From kostikbel at gmail.com Fri May 22 08:32:24 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Fri May 22 08:32:31 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <20090522014206.GA62573@voi.aagh.net> References: <4A14F58F.8000801@rawbw.com> <20090522014206.GA62573@voi.aagh.net> Message-ID: <20090522083217.GZ1927@deviant.kiev.zoral.com.ua> On Fri, May 22, 2009 at 02:42:06AM +0100, Thomas Hurst wrote: > * Nate Eldredge (neldredge@math.ucsd.edu) wrote: > > > There may be a way to enable the conservative behavior; I know Linux > > has an option to do this, but am not sure about FreeBSD. > > I seem to remember a patch to disable overcommit. Here we go: > > http://people.freebsd.org/~kib/overcommit/ Latest version is at http://people.freebsd.org/~kib/overcommit/vm_overcommit.22.patch applicable to the today CURRENT. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090522/e2f9a7ef/attachment.pgp From chuckr at telenix.org Fri May 22 14:09:34 2009 From: chuckr at telenix.org (Chuck Robey) Date: Fri May 22 14:09:41 2009 Subject: porting info for FreeBSD's kernel? In-Reply-To: <20090522073130.GI67847@elvis.mu.org> References: <4A11B893.1000808@telenix.org> <20090521003646.GS67847@elvis.mu.org> <4A15CE00.4040600@telenix.org> <20090522073130.GI67847@elvis.mu.org> Message-ID: <4A16B22C.6010201@telenix.org> Alfred Perlstein wrote: > * Chuck Robey [090521 14:56] wrote: >> Alfred Perlstein wrote: >>> * Chuck Robey [090518 13:03] wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> I've been googling, trying to see if I can find notes regarding what needs >>>> changing, in what order, to adapt the FreeBSD kernel to a new processor. Anyone >>>> know where stuff like that can be found? >>> You need a cross compile toolchain of course, look into how FreeBSD >>> is configured for the various arches. >>> >>> Then I would suggest looking at the loaders, followed by >>> kern/init_main.c. If you trace down init_main.c and some >>> of the early sysinits that should give you an idea. >>> >>> You might also be able to backtrack using CVS/svn to follow >>> how mips or arm was done. >>> >>> Note: freebsd has a decent cross-compile setup now, see >>> "make universe" so things should be easier to get started. >>> >> Thanks. I will *definitely* read all the parts you hint me at, I won't be >> deleting this mail, and I appreciate it. I was asking on the llvm maillist >> about Cortex-A8 support. What I got says that it's not there yet, but it's >> being worked upon, that and the -A9 support (definite differences). So, any >> crosstools needed today would have to be gcc, from a version at least as new as >> the 4.3 branch (that's where they brought in the -A8 support). >> >> The tool I got by doing the freeBSD crosstools was 4.2.1, which isn't going to >> do it for the Cortex-A8, and I had someone else (from a FreeBSD list) tell me >> that bringing in a newer version of gcc wasn't extremely likely, that they'd >> want llvm instead. I see 3 alternatives for a Cortex-A8 port: using a new gcc >> port, waiting on the upgrade of llvm, or maybe deciding that the version the >> llvm that's out now, with the v6 compatibility, would be (for the short term) >> good enough. Any idea which one to choose? The only one that interests me is >> for the TI OMAP 3530 (Cortex-A8, among other parts). Maybe if the currently >> available llvm is good enough, maybe gcc-4.2.1 may creak along well enough for >> the short term? I need to understand this. >> >> My own personal Pandora won't probably won't arrive on my doorstep for maybe as >> long as 3 more months, so in the meantime, I think I will be reading all I can >> get my hands on regarding llvm. Maybe I can really learn enough to make a >> difference. In school, I concentrated very definitely on OSes (I've written 3 >> of them over the years, of quite varying levels of performance), so for >> compilers, I'm relying on my old 1988 version of the Aho/Sethi/Ullman compilers >> book. If anyone knows a more modern book that will show me enough about >> compilers to be useful, I'd really appreciate the name, maybe Amazon will let me >> get a cheap used version. > > I wouldn't sweat the compiler as much as the actual OS code, I think > it should be relatively easy to trick the build to use an external > compiler (ie, don't get caught up in the compiler bootstrap quagmire, > leave that for later...) > > Anyhow, you're talking to someone that has studied, but not implemented > a port, so take my advice with a few heaps of salt. :) > > Typically what people focus on is: > > 1) "how am I going to get the first line of dmesg to come up" > 2) "how am I going to get to single user mode" > 3) "multi user?" > 4) cleanup of compiler and bootstrap issues. > > If you get sidetracked by #4, you can spend months doing that > instead of just rolling with it when you get there. > I'll admit it's not terribly hard to just get a foreign compiler to work, and I've already gotten a version of gcc-4.3.1 jiggered. I was going to concentrate next on cleaning up the compiler issue, which is why I wanted to get a pronouncement on which way to go. If I simply try to duck as much of that issue as possible, I can use the gcc-4.3.1 without huge problems. I can see that fine ,,, BUT the next part, getting ghe booting working, that does seem to be something which is necessary to do. How could U just duck out of that the way I could easily do for the compiler? I mean, how could you cause the booting to get fooled into thinking it was working? If you could give me an example of any possible way to get past this issue, I'm willing to do as you request, if only I could recognize the action you're asking me to take. In the meantime (Until I understand what you're asking for) I'm rereading my old Dragon book, so I can begin to understand what llvm is doing. From Sandeep Patel, of llvm, btw, he tells me that the -A8 and -A9 work on llvm is going very rapidly, and it may well be ready before we realize, so being able to push off making the compiler decision is actually maybe quite agood thing to contemplate. > From dillon at apollo.backplane.com Fri May 22 20:07:11 2009 From: dillon at apollo.backplane.com (Matthew Dillon) Date: Fri May 22 20:07:18 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> <200905220854.21917.j.mckeown@ru.ac.za> Message-ID: <200905222007.n4MK7ALn046086@apollo.backplane.com> :On Thursday 21 May 2009 23:37:20 Nate Eldredge wrote: :> Of course all these problems are solved, under any policy, by having more :> memory or swap. =A0But overcommit allows you to do more with less. : :Or to put it another way, 90% of the problems that could be solved by havin= :g=20 :more memory can also be solved by pretending you have more memory and hopin= :g=20 :no-one calls your bluff. : :Jonathan It's a bit more complicated then that. Most of the memory duplication (or lack of) which occurs after a fork() is deterministic. It's not a matter of pretending, it's a matter of practical application. For example, when sendmail fork()'s a deterministic subset of the duplicated writable memory will never be modified by the child. Ever. This is what overcommit takes advantage of. Nearly every program which fork()'s has a significant level of duplication of writable memory which deterministically reduces the set of pages which will ever need to be demand-copied. The OS cannot predict which pages these will be, but the effect from a whole-systems point of view is well known and deterministic. Similarly the OS cannot really determine who is responsible for running the system out of memory. Is it that big whopping program X or is it the 200 fork()'ed copies of server Y? Only a human being can really make the determination. This is also why turning off overcommit can easily lead to the system failing even if it is nowhere near running out of actual memory. In otherwords, the only real practical result of turning off overcommit is to make a system less stable and less able to deal with exceptional conditions. Systems which cannot afford to run out of memory are built from the ground-up to not allocate an unbounded amount of memory in the first place. There's no other way to do it. The Mars Rover is a good example of that. In such systems actually running out of memory is often considered to be a fatal fault. -Matt From alfred at freebsd.org Sun May 24 04:04:32 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Sun May 24 04:04:39 2009 Subject: porting info for FreeBSD's kernel? In-Reply-To: <4A16B22C.6010201@telenix.org> References: <4A11B893.1000808@telenix.org> <20090521003646.GS67847@elvis.mu.org> <4A15CE00.4040600@telenix.org> <20090522073130.GI67847@elvis.mu.org> <4A16B22C.6010201@telenix.org> Message-ID: <20090524040432.GQ67847@elvis.mu.org> * Chuck Robey [090522 07:09] wrote: > Alfred Perlstein wrote: > > I wouldn't sweat the compiler as much as the actual OS code, I think > > it should be relatively easy to trick the build to use an external > > compiler (ie, don't get caught up in the compiler bootstrap quagmire, > > leave that for later...) > > > > Anyhow, you're talking to someone that has studied, but not implemented > > a port, so take my advice with a few heaps of salt. :) > > > > Typically what people focus on is: > > > > 1) "how am I going to get the first line of dmesg to come up" > > 2) "how am I going to get to single user mode" > > 3) "multi user?" > > 4) cleanup of compiler and bootstrap issues. > > > > If you get sidetracked by #4, you can spend months doing that > > instead of just rolling with it when you get there. > > > > I'll admit it's not terribly hard to just get a foreign compiler to work, and > I've already gotten a version of gcc-4.3.1 jiggered. I was going to concentrate > next on cleaning up the compiler issue, which is why I wanted to get a > pronouncement on which way to go. If I simply try to duck as much of that > issue as possible, I can use the gcc-4.3.1 without huge problems. I can see > that fine ,,, BUT the next part, getting ghe booting working, that does seem to > be something which is necessary to do. How could U just duck out of that the > way I could easily do for the compiler? I mean, how could you cause the booting > to get fooled into thinking it was working? If you could give me an example of > any possible way to get past this issue, I'm willing to do as you request, if > only I could recognize the action you're asking me to take. Oh, I wasn't suggesting that you somehow fake up the loader part, you'll have to do that too! :) Perhaps a pre-step then should be: 0) get the loader working in some form. :>) > In the meantime (Until I understand what you're asking for) I'm rereading my > old Dragon book, so I can begin to understand what llvm is doing. From Sandeep > Patel, of llvm, btw, he tells me that the -A8 and -A9 work on llvm is going very > rapidly, and it may well be ready before we realize, so being able to push off > making the compiler decision is actually maybe quite agood thing to contemplate. you can really spend forever on this, again, unless you have a pressing need due to the compiler being completely broken, it's a bad idea to focus on cleanliness first. first work on getting it to boot, only stop if you hit a bug, don't "clean" or you'll never finish. again, this has only been my observation, I'm no porting OS master, but I have observed a few ports and my suggestions are what I've observed to have been the course of action of successful porters. I've also observed that whenever someone gets caught up in the details, they usually fail. good luck, -- - Alfred Perlstein From nork at FreeBSD.org Sun May 24 10:26:20 2009 From: nork at FreeBSD.org (Norikatsu Shigemura) Date: Sun May 24 10:26:27 2009 Subject: [CFT] ssh/scp/ssh-add/ssh-agent/ssh-keygen on /rescue Message-ID: <20090524192611.9becb1bc.nork@FreeBSD.org> Hi. I wondered that ssh/scp (at least) was not in /rescue. They are the indispensable tools, and I also often use them in the emergency (single user mode). So I made a patch for src/rescue/rescue/Makefile and src/secure/usr.bin/scp/Makefile. Please review it. Changing point is: o Move some libraries to Common Libraries section. o Add ssh/scp/ssh-add/ssh-agent/ssh-keygen to /rescue member. o Cosmetics Change (labeled Common Libraries section). Sorry, this patch for 8-current. I'll try to MFC after kmacy's zfs/zpool support. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --- rescue/rescue/Makefile.orig 2009-05-23 11:30:38.830094000 +0900 +++ rescue/rescue/Makefile 2009-05-24 18:58:08.925195062 +0900 @@ -72,7 +72,7 @@ CRUNCH_PROGS_bin= cat chflags chio chmod cp date dd df echo \ ed expr getfacl hostname kenv kill ln ls mkdir mv \ pkill ps pwd realpath rm rmdir setfacl sh stty sync test -CRUNCH_LIBS+= -lcrypt -ledit -lkvm -ll -ltermcap -lutil +CRUNCH_LIBS+= -lcrypt -ledit -lkvm -ll -ltermcap # Additional options for specific programs CRUNCH_ALIAS_test= [ @@ -143,7 +143,7 @@ .if ${MK_ZFS} != "no" CRUNCH_LIBS+= -lzfs -lnvpair -luutil -lavl .endif -CRUNCH_LIBS+= -lgeom -lbsdxml -lkiconv -lmd -lreadline -lsbuf -lufs -lz +CRUNCH_LIBS+= -lgeom -lbsdxml -lkiconv -lreadline -lsbuf -lufs .if ${MACHINE_ARCH} == "i386" CRUNCH_PROGS_sbin+= bsdlabel sconfig fdisk @@ -206,13 +206,9 @@ CRUNCH_PROGS_usr.bin+= bzip2 CRUNCH_ALIAS_bzip2= bunzip2 bzcat -CRUNCH_LIBS+= -lbz2 CRUNCH_PROGS_usr.bin+= tar -CRUNCH_LIBS+= -larchive -lmd -.if ${MK_OPENSSL} != "no" -CRUNCH_LIBS+= -lcrypto -.endif +CRUNCH_LIBS+= -larchive CRUNCH_PROGS_usr.bin+= vi CRUNCH_ALIAS_vi= ex @@ -220,6 +216,25 @@ CRUNCH_PROGS_usr.bin+= id CRUNCH_ALIAS_id= groups whoami +.if ${MK_OPENSSL} != "no" && ${MK_OPENSSH} != "no" +CRUNCH_PROGS_usr.bin+= ssh +CRUNCH_PROGS_usr.bin+= scp +CRUNCH_BUILDOPTS_scp= BINDIR=${BINDIR} +CRUNCH_PROGS_usr.bin+= ssh-add +CRUNCH_PROGS_usr.bin+= ssh-agent +CRUNCH_PROGS_usr.bin+= ssh-keygen +CRUNCH_LIBS+= -lssh +.if ${MK_KERBEROS_SUPPORT} != "no" +CRUNCH_LIBS+= -lgssapi +.endif + +CRUNCH_SRCDIR_ssh= ${.CURDIR}/../../secure/usr.bin/ssh +CRUNCH_SRCDIR_scp= ${.CURDIR}/../../secure/usr.bin/scp +CRUNCH_SRCDIR_ssh-add= ${.CURDIR}/../../secure/usr.bin/ssh-add +CRUNCH_SRCDIR_ssh-agent= ${.CURDIR}/../../secure/usr.bin/ssh-agent +CRUNCH_SRCDIR_ssh-keygen= ${.CURDIR}/../../secure/usr.bin/ssh-keygen +.endif + ################################################################## # Programs from stock /usr/sbin # @@ -229,8 +244,14 @@ CRUNCH_PROGS_usr.sbin+= chown CRUNCH_ALIAS_chown= chgrp + ################################################################## -CRUNCH_LIBS+= -lm +# Common Libraries +# +.if ${MK_OPENSSL} != "no" +CRUNCH_LIBS+= -lcrypto +.endif +CRUNCH_LIBS+= -lmd -lutil -lbz2 -lz -lm ################################################################## # The following is pretty nearly a generic crunchgen-handling makefile --- secure/usr.bin/scp/Makefile.orig 2006-05-14 06:38:15.000000000 +0900 +++ secure/usr.bin/scp/Makefile 2009-05-24 15:39:20.594368170 +0900 @@ -1,7 +1,7 @@ # $FreeBSD: src/secure/usr.bin/scp/Makefile,v 1.16 2006/05/13 21:38:15 des Exp $ PROG= scp -CFLAGS+=-I${SSHDIR} -include ssh_namespace.h +CFLAGS+=-I${SSHDIR} -include ssh_namespace.h -D_PATH_SSH_PROGRAM='"${BINDIR}/ssh"' DPADD= ${LIBSSH} ${LIBCRYPT} ${LIBCRYPTO} ${LIBZ} LDADD= -lssh -lcrypt -lcrypto -lz - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - From James.McPherson at Sun.COM Sun May 24 13:19:03 2009 From: James.McPherson at Sun.COM (James C. McPherson) Date: Sun May 24 13:26:18 2009 Subject: Kernel Conference Australia 2009 - one week to go for earlybird registrations Message-ID: <20090524231847.00006275@blinder> Dear friends and colleagues, just a quick note that if you were thinking about coming to Kernel Conference Austalia, then you should register without delay - the earlybird price of $195 expires on the 31st of May. That's this coming Sunday! The registration site is here: https://www.conveneit.com/secure/sun/kernel_jul_09/ In addition to our most excellent keynote speakers Jeff Bonwick, Bill Moore and Max Alt, you can meet, listen to and learn from Fernando Gont Results of a Security Assessment of Common Implementation Strategies of the TCP and IP Protocols Henning Brauer (OpenBSD) Faster Packets: Performance Tuning in the OpenBSD Network Stack and PF Gavin Maltby (Sun Microsystems) Hardware & Software Fault Management Architecture Pawel Dawidek (FreeBSD) GEOM - The FreeBSD way of handling storage John Sonnenschein (Sun Microsystems) Driver and Filesystem Development with the Solaris and OpenSolaris DDI/DKI David Gwynne (University of Queensland) MCLGETI: Effective Network Livelock Mitigation and More Cristina Cifuentes (Sun Microsystems) Finding Bugs in Open Source Kernels Using Parfait Sherry Moore (Sun Microsystems) Fast reboot support (and more) for OpenSolaris Max Bruning (Bruning Systems) Porting USB HID Device Drivers Between Linux and OpenSolaris James Morris (Red Hat) Linux Kernel Security Overview Percy Pari-Salas (Bond University) Automated Testing of OpenSolaris Vivek Joshi (Sun Microsystems) Porting OpenSolaris across architectures Jayakara Kini (Sun Microsystems) Crossbow for OpenSolaris Developers Garrett D'Amore (Sun Microsystems) Boomer: the new OpenSolaris audio system Pramod Batni (Sun Microsystems) Debugging and Diagnosing Interesting Kernel Problems Stewart Smith (Sun Microsystems) (Ab)use the Kernel: what a database server can do to your kernel The registration site is here: https://www.conveneit.com/secure/sun/kernel_jul_09/ I look forward to seeing you at Kernel Conference Australia Best regards, James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel From rodrigo at bebik.net Mon May 25 08:58:07 2009 From: rodrigo at bebik.net (Rodrigo OSORIO (ros)) Date: Mon May 25 08:58:21 2009 Subject: improve my USB knowledge Message-ID: <20090525083845.GA95406@hodja.bebik.net> Hi fellow hackers, This weekend I try to increase my knowledge about USB devices - I start from 0 - playing with one of this funny low cost USB gadgets. I read few articles about writing USB drivers, specially the Linux USB development guide, and I want to know if there is others (BSD related) articles or documents I can/must read. FYI, I work with the new USB stack in 8-CURRENT. regards, Rodrigo OSORIO From hselasky at c2i.net Mon May 25 11:40:48 2009 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon May 25 11:40:55 2009 Subject: improve my USB knowledge In-Reply-To: <20090525083845.GA95406@hodja.bebik.net> References: <20090525083845.GA95406@hodja.bebik.net> Message-ID: <200905251344.46530.hselasky@c2i.net> On Monday 25 May 2009, Rodrigo OSORIO (ros) wrote: > Hi fellow hackers, > > This weekend I try to increase my knowledge about USB devices - I start > from 0 - playing with one of this funny low cost USB gadgets. > I read few articles about writing USB drivers, specially the Linux USB > development guide, and I want to know if there is others (BSD related) > articles or documents I can/must read. FYI, I work with the new USB stack > in 8-CURRENT. > Hi, Maybe a good starting point would be usbconfig utility and the do_request command, which allows you to control your gadget through the control endpoint. Else there is the manpages: man libusb man usb --HPS From laladelausanne at gmail.com Mon May 25 13:10:40 2009 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Mon May 25 13:10:47 2009 Subject: How to visualize cache misses with pmc Message-ID: <5736C5AC-F5EC-4708-8815-1747A55AF81C@gmail.com> Hi, During the tuning of my system, I was using pmc(3) to get various counters. Most important was the number of retired instructions, which helped me to discover some bottlenecks. However, I now need to get L2 cache misses, which I do by running: sudo pmcstat -S L2_LD -O /tmp/sample.out and then run my module with its workload. At the end, I have the output in /tmp/sample.out. However, I don't know how to get that data to anything useful, as gprof output is not very meaningful. Any thoughts, hints, best practices? Cheers, Nikola From doconnor at gsoft.com.au Mon May 25 13:34:33 2009 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Mon May 25 13:34:41 2009 Subject: improve my USB knowledge In-Reply-To: <20090525083845.GA95406@hodja.bebik.net> References: <20090525083845.GA95406@hodja.bebik.net> Message-ID: <200905252304.19081.doconnor@gsoft.com.au> On Mon, 25 May 2009, Rodrigo OSORIO (ros) wrote: > This weekend I try to increase my knowledge about USB devices - I > start from 0 - playing with one of this funny low cost USB gadgets. > I read few articles about writing USB drivers, specially the Linux > USB development guide, and I want to know if there is others (BSD > related) articles or documents I can/must read. FYI, I work with the > new USB stack in 8-CURRENT. libusb is moderately horrible but you can do stuff with it, I wrote a driver for a USB TMC device in Python in an iterative fashion with the pyusb port. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090525/838ccf7c/attachment.pgp From julian at elischer.org Mon May 25 17:57:41 2009 From: julian at elischer.org (Julian Elischer) Date: Mon May 25 17:57:47 2009 Subject: improve my USB knowledge In-Reply-To: <20090525083845.GA95406@hodja.bebik.net> References: <20090525083845.GA95406@hodja.bebik.net> Message-ID: <4A1ADC13.2060702@elischer.org> Rodrigo OSORIO (ros) wrote: > Hi fellow hackers, > > This weekend I try to increase my knowledge about USB devices - I start from 0 - > playing with one of this funny low cost USB gadgets. > I read few articles about writing USB drivers, specially the Linux USB > development guide, and I want to know if there is others (BSD related) > articles or documents I can/must read. FYI, I work with the new USB stack in 8-CURRENT. > > regards, > Rodrigo OSORIO > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" I highly recommend that you rad the mindshare USB book. From hiyorin at gmail.com Tue May 26 03:33:34 2009 From: hiyorin at gmail.com (C. C. Tang) Date: Tue May 26 03:33:41 2009 Subject: Random(?) Fatal trap 12 in Freebsd 7.2 release Message-ID: <4A1B5B80.1070207@gmail.com> Sorry for my bad english. I am having a FreeBSD with Atom processor. It was running for 3 months without any panic when it is 7.1 release. After I upgraded it to 7.2 release weeks ago, it seems fine but it starts to have panic several days ago. The machine will sometimes panic without a heavy load(at least it is not in its peak load but usually it will panic at the time everybody is using it.) I would be grateful if anyone can help me to figure out what is the real problem... uname -a: FreeBSD xxxx 7.2-RELEASE FreeBSD 7.2-RELEASE #0: Fri May 1 08:49:13 UTC 2009 root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 dmesg: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.2-RELEASE #0: Fri May 1 08:49:13 UTC 2009 root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1618.46-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Stepping = 2 Features=0xbfe9fbff Features2=0x40e31d> AMD Features=0x20100000 AMD Features2=0x1 Cores per package: 2 Logical CPUs per core: 2 real memory = 2137391104 (2038 MB) avail memory = 2081775616 (1985 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 ioapic0: Changing APIC ID to 2 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: port 0x30e0-0x30e7 mem 0x90300000-0x9037ffff,0x80000000-0x8fffffff,0x90380000-0x903bffff irq 16 at device 2.0 on pci0 agp0: on vgapci0 agp0: detected 7932k stolen memory agp0: aperture size is 256M pci0: at device 27.0 (no driver attached) pcib1: at device 28.0 on pci0 pci1: on pcib1 re0: port 0x2000-0x20ff mem 0x90200000-0x90200fff,0x90000000-0x9000ffff irq 16 at device 0.0 on pci1 re0: Using 1 MSI messages re0: Chip rev. 0x3c000000 re0: MAC rev. 0x00400000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto re0: Ethernet address: 00:1c:c0:a8:f3:73 re0: [FILTER] pcib2: at device 28.2 on pci0 pci2: on pcib2 pcib3: at device 28.3 on pci0 pci3: on pcib3 uhci0: port 0x3080-0x309f irq 23 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x3060-0x307f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x3040-0x305f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0x3020-0x303f irq 16 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0x903c4000-0x903c43ff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered pcib4: at device 30.0 on pci0 pci4: on pcib4 re1: port 0x1000-0x10ff mem 0x90100000-0x901000ff irq 21 at device 0.0 on pci4 re1: Chip rev. 0x04000000 re1: MAC rev. 0x00000000 miibus1: on re1 rgephy1: PHY 1 on miibus1 rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto re1: Ethernet address: 00:16:01:5c:4a:0f re1: [FILTER] isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30b0-0x30bf irq 18 at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] atapci1: port 0x30c8-0x30cf,0x30ec-0x30ef,0x30c0-0x30c7,0x30e8-0x30eb,0x30a0-0x30af irq 19 at device 31.2 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] pci0: at device 31.3 (no driver attached) sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] cpu0: on acpi0 p4tcc0: on cpu0 cpu1: on acpi0 p4tcc1: on cpu1 cpu2: on acpi0 p4tcc2: on cpu2 cpu3: on acpi0 p4tcc3: on cpu3 pmtimer0 on isa0 orm0: at iomem 0xcb000-0xcbfff pnpid ORM0000 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [ITHREAD] plip0: on ppbus0 plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec ad4: 476940MB at ata2-master SATA150 SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! GEOM_LABEL: Label for provider ad4s2 is ufsid/497231edae4da529. GEOM_LABEL: Label for provider ad4s1a is ufsid/497231ebaad35c34. GEOM_LABEL: Label for provider ad4s1d is ufsid/497231f823755a18. GEOM_LABEL: Label for provider ad4s1e is ufsid/497231eb4cb04b76. GEOM_LABEL: Label for provider ad4s1f is ufsid/497231eb16db295a. Trying to mount root from ufs:/dev/ad4s1a GEOM_LABEL: Label ufsid/497231ebaad35c34 removed. GEOM_LABEL: Label for provider ad4s1a is ufsid/497231ebaad35c34. GEOM_LABEL: Label ufsid/497231eb4cb04b76 removed. GEOM_LABEL: Label for provider ad4s1e is ufsid/497231eb4cb04b76. GEOM_LABEL: Label ufsid/497231eb16db295a removed. GEOM_LABEL: Label for provider ad4s1f is ufsid/497231eb16db295a. GEOM_LABEL: Label ufsid/497231edae4da529 removed. GEOM_LABEL: Label for provider ad4s2 is ufsid/497231edae4da529. GEOM_LABEL: Label ufsid/497231f823755a18 removed. GEOM_LABEL: Label for provider ad4s1d is ufsid/497231f823755a18. GEOM_LABEL: Label ufsid/497231ebaad35c34 removed. GEOM_LABEL: Label ufsid/497231eb4cb04b76 removed. GEOM_LABEL: Label ufsid/497231eb16db295a removed. GEOM_LABEL: Label ufsid/497231edae4da529 removed. GEOM_LABEL: Label ufsid/497231f823755a18 removed. tap0: Ethernet address: 00:bd:9c:3f:00:00 re0: link state changed to DOWN re0: link state changed to UP The following is the info obtained by running crashinfo: ### instance 1 (May 21 about 23:00 local time) Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 02 fault virtual address = 0x19 fault code = supervisor write, page not present instruction pointer = 0x20:0xc085af6c stack pointer = 0x28:0xe5a84bfc frame pointer = 0x28:0xe5a84c18 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 46 (syncer) trap number = 12 panic: page fault cpuid = 2 Uptime: 11d9h43m49s Physical memory: 2025 MB Dumping 294 MB: 279 263 247 231 215 199 183 167 151 135 119 103 87 71 55 39 23 7 #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:196 #1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e2879 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae3ebc in trap_fatal (frame=0xe5a84bbc, eva=25) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4140 in trap_pfault (frame=0xe5a84bbc, usermode=0, eva=25) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae4aec in trap (frame=0xe5a84bbc) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc085af6c in __mnt_vnode_next (mvp=0xe5a84c48, mp=0xc5874b40) at /usr/src/sys/kern/vfs_mount.c:2033 #8 0xc0864202 in vfs_msync (mp=0xc5874b40, flags=2) at /usr/src/sys/kern/vfs_subr.c:3135 #9 0xc08644ba in sync_fsync (ap=0xe5a84cd4) at /usr/src/sys/kern/vfs_subr.c:3389 #10 0xc0af8e72 in VOP_FSYNC_APV (vop=0xc0c598a0, a=0xe5a84cd4) at vnode_if.c:1007 #11 0xc0864cc8 in sched_sync () at vnode_if.h:538 #12 0xc07bd059 in fork_exit (callout=0xc08645c0 , arg=0x0, frame=0xe5a84d38) at /usr/src/sys/kern/kern_fork.c:810 #13 0xc0ac9270 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264 (kgdb) ### instance 2 (May 22 about 20:30 local time) Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xf3e83561 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0880d65 stack pointer = 0x28:0xc53f7be0 frame pointer = 0x28:0xc53f7c00 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 19 (swi5: +) trap number = 12 panic: page fault cpuid = 0 Uptime: 21h4m54s Physical memory: 2025 MB Dumping 291 MB: 276 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4 #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:196 #1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e2879 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae3ebc in trap_fatal (frame=0xc53f7ba0, eva=4092081505) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4140 in trap_pfault (frame=0xc53f7ba0, usermode=0, eva=4092081505) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae4aec in trap (frame=0xc53f7ba0) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc0880d65 in ether_input (ifp=0xc5601800, m=0xc5a2c200) at /usr/src/sys/net/if_ethersubr.c:545 #8 0xc06c4348 in re_rxeof (sc=0xc561b000) at /usr/src/sys/dev/re/if_re.c:1982 #9 0xc06c5dce in re_int_task (arg=0xc561b000, npending=1) at /usr/src/sys/dev/re/if_re.c:2189 #10 0xc0817835 in taskqueue_run (queue=0xc5498400) at /usr/src/sys/kern/subr_taskqueue.c:282 #11 0xc0817973 in taskqueue_fast_run (dummy=0x0) at /usr/src/sys/kern/subr_taskqueue.c:460 #12 0xc07c050b in ithread_loop (arg=0xc55830b0) at /usr/src/sys/kern/kern_intr.c:1088 #13 0xc07bd059 in fork_exit (callout=0xc07c0350 , arg=0xc55830b0, frame=0xc53f7d38) at /usr/src/sys/kern/kern_fork.c:810 #14 0xc0ac9270 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264 (kgdb) ### instance 3 (May 22 about 20:40 local time) Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x200 fault code = supervisor read, page not present instruction pointer = 0x20:0x200 stack pointer = 0x28:0xc53ed8f0 frame pointer = 0x28:0xc53ed910 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 4 (g_down) trap number = 12 (bt is not available since the core dump seems lost. This panic occured immediately after rebooted from instance 2..) ### instance 4 (May 25 about 22:40 local time) Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x3b57860 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0a1cfb6 stack pointer = 0x28:0xe7d00a94 frame pointer = 0x28:0xe7d00ab0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 62673 (httpd) trap number = 12 panic: page fault cpuid = 1 Uptime: 3d1h54m6s Physical memory: 2025 MB Dumping 275 MB: 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4 #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:196 #1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e2879 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae3ebc in trap_fatal (frame=0xe7d00a54, eva=62224480) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4140 in trap_pfault (frame=0xe7d00a54, usermode=0, eva=62224480) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae4aec in trap (frame=0xe7d00a54) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc0a1cfb6 in vm_map_entry_splay (addr=685764608, root=0xc60ac044) at /usr/src/sys/vm/vm_map.c:726 #8 0xc0a1d367 in vm_map_lookup_entry (map=0xc5d0fa24, address=685764608, entry=0xe7d00bf8) at /usr/src/sys/vm/vm_map.c:904 #9 0xc0a1e76a in vm_map_lookup (var_map=0xe7d00bf4, vaddr=685764608, fault_typea=Variable "fault_typea" is not available. ) at /usr/src/sys/vm/vm_map.c:3136 #10 0xc0a17f03 in vm_fault (map=0xc5d0fa24, vaddr=685764608, fault_type=2 '\002', fault_flags=8) at /usr/src/sys/vm/vm_fault.c:233 #11 0xc0ae402b in trap_pfault (frame=0xe7d00d38, usermode=1, eva=685767244) at /usr/src/sys/i386/i386/trap.c:829 #12 0xc0ae4977 in trap (frame=0xe7d00d38) at /usr/src/sys/i386/i386/trap.c:397 #13 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #14 0x2809997e in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) From kostjn at peterhost.ru Tue May 26 06:30:12 2009 From: kostjn at peterhost.ru (Menshikov Konstantin) Date: Tue May 26 06:30:19 2009 Subject: Disk quota for Jail. Discussion. Message-ID: <4A1B8CF8.7030102@peterhost.ru> Hi. Jail now have no disk quotas. Users which use jail, use separate zfs or md device for jail, but it from a hopelessness. It is necessary to discuss possible ways of realisation. I suggest to make disk quotas for jail on the basis of the user quotas. Introduction. User UID and GID are located in inode and are always accessible. No information about jail in ufs is present. All activity jail is limited root path. The work scheme. In structure prison it is added structures containing disk quotas and usage. At start Jail, we calculate the size root path and number of files in it, thus receiving current use of a disk. In functions of allocation of disk blocks and inode, we check quotas and we increase current use. If jail exceeds a quota, inquiry about allocation of the disk block or inode it is rejected. After work end jail the information on disk use is lost. What do you think about it? There are other offers? -- Menshikov Konstantin. From kostikbel at gmail.com Tue May 26 12:03:26 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue May 26 12:03:37 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1B8CF8.7030102@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> Message-ID: <20090526120313.GA1927@deviant.kiev.zoral.com.ua> On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: > Hi. > Jail now have no disk quotas. > Users which use jail, use separate zfs or md device for jail, but it > from a hopelessness. > It is necessary to discuss possible ways of realisation. > I suggest to make disk quotas for jail on the basis of the user quotas. > > Introduction. > User UID and GID are located in inode and are always accessible. > No information about jail in ufs is present. All activity jail is > limited root path. > > The work scheme. > In structure prison it is added structures containing disk quotas and usage. > At start Jail, we calculate the size root path and number of files in > it, thus receiving current use of a disk. > In functions of allocation of disk blocks and inode, we check quotas and > we increase current use. UFS cannot determine whether the new allocation goes under the jail root or not. > If jail exceeds a quota, inquiry about allocation of the disk block or > inode it is rejected. > After work end jail the information on disk use is lost. > > What do you think about it? > There are other offers? > -- > Menshikov Konstantin. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090526/9d2d6cbc/attachment.pgp From kostjn at peterhost.ru Tue May 26 12:33:19 2009 From: kostjn at peterhost.ru (Menshikov Konstantin) Date: Tue May 26 12:33:26 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <20090526120313.GA1927@deviant.kiev.zoral.com.ua> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> Message-ID: <4A1BE1F8.9050804@peterhost.ru> Kostik Belousov wrote: > On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: > >> Hi. >> Jail now have no disk quotas. >> Users which use jail, use separate zfs or md device for jail, but it >> from a hopelessness. >> It is necessary to discuss possible ways of realisation. >> I suggest to make disk quotas for jail on the basis of the user quotas. >> >> Introduction. >> User UID and GID are located in inode and are always accessible. >> No information about jail in ufs is present. All activity jail is >> limited root path. >> >> The work scheme. >> In structure prison it is added structures containing disk quotas and usage. >> At start Jail, we calculate the size root path and number of files in >> it, thus receiving current use of a disk. >> In functions of allocation of disk blocks and inode, we check quotas and >> we increase current use. >> > UFS cannot determine whether the new allocation goes under the jail > root or not. > Yes. But jail cannot allocate block and inode above root path. In allocation functions, whether for example ffs_alloc we have access to ucred process and we can check up there is a process in jail. >> If jail exceeds a quota, inquiry about allocation of the disk block or >> inode it is rejected. >> After work end jail the information on disk use is lost. >> >> What do you think about it? >> There are other offers? >> >> If jail will get access to disk devices in/dev/and will mount file system of a problem also will not arise, I think... Can be eat other problems which are not visible at first sight? From kostikbel at gmail.com Tue May 26 12:36:37 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue May 26 12:36:43 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1BE1F8.9050804@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> Message-ID: <20090526123632.GB1927@deviant.kiev.zoral.com.ua> On Tue, May 26, 2009 at 04:35:04PM +0400, Menshikov Konstantin wrote: > Kostik Belousov wrote: > >On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: > >>In structure prison it is added structures containing disk quotas and > >>usage. > >>At start Jail, we calculate the size root path and number of files in > >>it, thus receiving current use of a disk. > >>In functions of allocation of disk blocks and inode, we check quotas and > >>we increase current use. > >> > >UFS cannot determine whether the new allocation goes under the jail > >root or not. > > > Yes. But jail cannot allocate block and inode above root path. In > allocation functions, whether for example ffs_alloc we have access to > ucred process and we can check up there is a process in jail. Yes, you can check this for jailed process. Think about non-jailed processes that can do allocation below the jail root. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090526/828505a2/attachment.pgp From kostjn at peterhost.ru Tue May 26 12:59:34 2009 From: kostjn at peterhost.ru (Menshikov Konstantin) Date: Tue May 26 12:59:47 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <20090526123632.GB1927@deviant.kiev.zoral.com.ua> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> <20090526123632.GB1927@deviant.kiev.zoral.com.ua> Message-ID: <4A1BE827.2030303@peterhost.ru> Kostik Belousov wrote: > On Tue, May 26, 2009 at 04:35:04PM +0400, Menshikov Konstantin wrote: > >> Kostik Belousov wrote: >> >>> On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: >>> >>>> In structure prison it is added structures containing disk quotas and >>>> usage. >>>> At start Jail, we calculate the size root path and number of files in >>>> it, thus receiving current use of a disk. >>>> In functions of allocation of disk blocks and inode, we check quotas and >>>> we increase current use. >>>> >>>> >>> UFS cannot determine whether the new allocation goes under the jail >>> root or not. >>> >>> >> Yes. But jail cannot allocate block and inode above root path. In >> allocation functions, whether for example ffs_alloc we have access to >> ucred process and we can check up there is a process in jail. >> > > Yes, you can check this for jailed process. Think about non-jailed processes > that can do allocation below the jail root. > Processes out of jail are not considered. I do not understand, these processes have what relation to disk to quotas for jail. Please explain more in detail From kostikbel at gmail.com Tue May 26 13:20:52 2009 From: kostikbel at gmail.com (Kostik Belousov) Date: Tue May 26 13:20:58 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1BE827.2030303@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> <20090526123632.GB1927@deviant.kiev.zoral.com.ua> <4A1BE827.2030303@peterhost.ru> Message-ID: <20090526132046.GC1927@deviant.kiev.zoral.com.ua> On Tue, May 26, 2009 at 05:01:27PM +0400, Menshikov Konstantin wrote: > Kostik Belousov wrote: > >On Tue, May 26, 2009 at 04:35:04PM +0400, Menshikov Konstantin wrote: > > > >>Kostik Belousov wrote: > >> > >>>On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: > >>> > >>>>In structure prison it is added structures containing disk quotas and > >>>>usage. > >>>>At start Jail, we calculate the size root path and number of files in > >>>>it, thus receiving current use of a disk. > >>>>In functions of allocation of disk blocks and inode, we check quotas > >>>>and we increase current use. > >>>> > >>>> > >>>UFS cannot determine whether the new allocation goes under the jail > >>>root or not. > >>> > >>> > >>Yes. But jail cannot allocate block and inode above root path. In > >>allocation functions, whether for example ffs_alloc we have access to > >>ucred process and we can check up there is a process in jail. > >> > > > >Yes, you can check this for jailed process. Think about non-jailed > >processes > >that can do allocation below the jail root. > > > Processes out of jail are not considered. > I do not understand, these processes have what relation to disk to > quotas for jail. Please explain more in detail Since the processes outside of the jail may allocate or delete blocks or inodes under the jail root, doing accounting only for actions initiated by jailed processes means that accounting does not reflect reality. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090526/dfdbb8cd/attachment.pgp From tevans.uk at googlemail.com Tue May 26 13:44:46 2009 From: tevans.uk at googlemail.com (Tom Evans) Date: Tue May 26 13:44:52 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1BE827.2030303@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> <20090526123632.GB1927@deviant.kiev.zoral.com.ua> <4A1BE827.2030303@peterhost.ru> Message-ID: <1243344263.9871.2.camel@strangepork.london.mintel.ad> On Tue, 2009-05-26 at 17:01 +0400, Menshikov Konstantin wrote: > Kostik Belousov wrote: > > On Tue, May 26, 2009 at 04:35:04PM +0400, Menshikov Konstantin wrote: > > > >> Kostik Belousov wrote: > >> > >>> On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: > >>> > >>>> In structure prison it is added structures containing disk quotas and > >>>> usage. > >>>> At start Jail, we calculate the size root path and number of files in > >>>> it, thus receiving current use of a disk. > >>>> In functions of allocation of disk blocks and inode, we check quotas and > >>>> we increase current use. > >>>> > >>>> > >>> UFS cannot determine whether the new allocation goes under the jail > >>> root or not. > >>> > >>> > >> Yes. But jail cannot allocate block and inode above root path. In > >> allocation functions, whether for example ffs_alloc we have access to > >> ucred process and we can check up there is a process in jail. > >> > > > > Yes, you can check this for jailed process. Think about non-jailed processes > > that can do allocation below the jail root. > > > Processes out of jail are not considered. > I do not understand, these processes have what relation to disk to > quotas for jail. Please explain more in detail A process outside of the jail can still write to the file system that you consider to be jailed, depending upon permissions. If all your quota calculations are only triggered by jailed processes writing to the file system, then you can exceed quota trivially. Tom From fhcarron at terra.es Tue May 26 14:00:52 2009 From: fhcarron at terra.es (Fernando Herrero =?ISO-8859-1?Q?Carr=F3n?=) Date: Tue May 26 14:01:00 2009 Subject: Help debugging kernel together with X Message-ID: <1243345847.1007.11.camel@nebet.ii.uam.es> Hi list, I would like to ask for some help debugging the kernel. Here is the problem: I have a computer about five years old with an on-board graphics card (SiS 661). I am trying to install an ATI Radeon 128 on the AGP port. FreeBSD (FreeBSD 7.2-STABLE #10: Tue May 26 15:08:39 CEST 2009) is able to start DRM and AGP without trouble. However, whenever I start X (X.Org X Server 1.6.1; Release Date: 2009-4-14; Build Date: 11 May 2009 12:03:27PM) the system freezes with a blank screen with some apparent noisy green dots on it (reproducible, however). It does not panic, nor reboot. Now I suspect there is some problem on the AGP bridge driver since linux can run X without trouble (i.e. hardware and BIOS are ok). I have found someone reporting problems with this bridge long time ago with no answers (http://www.nabble.com/agp-on-sis-661-td1446998.html). So here I am trying to figure out where the system freezes. I have recompiled my kernel adding the following options: options KDB options DDB options MP_WATCHDOG options SW_WATCHDOG I am able to enter the debugger with ctrl+alt+esc from a console. So what I am trying to do is to have watchdogd time out and the debugger make a dump: ddb script kern.enter.watchdog=call doadump; reboot However, when I start X no timeout occurs, and the system stays freezed. Am I missing something on the workings of watchdogd? Thanks a lot for your help! Fernando From ivoras at freebsd.org Tue May 26 14:05:07 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Tue May 26 14:05:13 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1B8CF8.7030102@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> Message-ID: Menshikov Konstantin wrote: > Hi. > Jail now have no disk quotas. > Users which use jail, use separate zfs or md device for jail, but it > from a hopelessness. Well, the ZFS way of doing things (per-filesystem quotas) is actually the right way to do it - it solves exactly the problem described. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090526/ff5dcd9a/signature.pgp From glen.j.barber at gmail.com Tue May 26 15:39:17 2009 From: glen.j.barber at gmail.com (Glen Barber) Date: Tue May 26 15:39:24 2009 Subject: Help debugging kernel together with X In-Reply-To: <1243345847.1007.11.camel@nebet.ii.uam.es> References: <1243345847.1007.11.camel@nebet.ii.uam.es> Message-ID: <4ad871310905260839u24ce71e9wbaa637712480c390@mail.gmail.com> Hi, Fernando 2009/5/26 Fernando Herrero Carr?n : > Hi list, > > I would like to ask for some help debugging the kernel. > > Here is the problem: > > I have a computer about five years old with an on-board graphics card > (SiS 661). I am trying to install an ATI Radeon 128 on the AGP port. > FreeBSD (FreeBSD 7.2-STABLE #10: Tue May 26 15:08:39 CEST 2009) is able > to start DRM and AGP without trouble. However, whenever I start X (X.Org > X Server 1.6.1; Release Date: 2009-4-14; Build Date: 11 May 2009 > 12:03:27PM) the system freezes with a blank screen with some apparent > noisy green dots on it (reproducible, however). It does not panic, nor > reboot. > > Now I suspect there is some problem on the AGP bridge driver since linux > can run X without trouble (i.e. hardware and BIOS are ok). I have found > someone reporting problems with this bridge long time ago with no > answers (http://www.nabble.com/agp-on-sis-661-td1446998.html). > > So here I am trying to figure out where the system freezes. I have > recompiled my kernel adding the following options: > > options ? ? ? ? KDB > options ? ? ? ? DDB > options ? ? ? ? MP_WATCHDOG > options ? ? ? ? SW_WATCHDOG > > I am able to enter the debugger with ctrl+alt+esc from a console. So > what I am trying to do is to have watchdogd time out and the debugger > make a dump: > > ddb script kern.enter.watchdog=call doadump; reboot > > However, when I start X no timeout occurs, and the system stays freezed. > Am I missing something on the workings of watchdogd? > What you're describing does not appear to be a kernel problem; it sounds like the typical 'Xorg update from hell' problem most experienced a while back. Have a look at /usr/ports/UPDATING, and search for 'AllowEmptyInput'. Adding the following to xorg.conf should correct the problem: Section "ServerFlags" option "AllowEmptyInput" "off" option "AutoAddDevices" "off" EndSection HTH -- Glen Barber From marck at rinet.ru Tue May 26 15:42:51 2009 From: marck at rinet.ru (Dmitry Morozovsky) Date: Tue May 26 15:42:58 2009 Subject: MosChip 7840 dual port ucom Message-ID: Dear colleagues, any hints/directions to get MosChip 7840 dual port USB to RS232 adapter working? In usbdevs output the device is shown as port 1 addr 2: high speed, power 100 mA, config 1, product 0x7840(0x7840), vendor 0x9710(0x9710), rev 0.01 Thanks in advance. -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From kostjn at peterhost.ru Tue May 26 16:09:27 2009 From: kostjn at peterhost.ru (Menshikov Konstantin) Date: Tue May 26 16:09:35 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <1243344263.9871.2.camel@strangepork.london.mintel.ad> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> <20090526123632.GB1927@deviant.kiev.zoral.com.ua> <4A1BE827.2030303@peterhost.ru> <1243344263.9871.2.camel@strangepork.london.mintel.ad> Message-ID: <4A1C14A8.9010104@peterhost.ru> Tom Evans wrote: > On Tue, 2009-05-26 at 17:01 +0400, Menshikov Konstantin wrote: > >> Kostik Belousov wrote: >> >>> On Tue, May 26, 2009 at 04:35:04PM +0400, Menshikov Konstantin wrote: >>> >>> >>>> Kostik Belousov wrote: >>>> >>>> >>>>> On Tue, May 26, 2009 at 10:32:24AM +0400, Menshikov Konstantin wrote: >>>>> >>>>> >>>>>> In structure prison it is added structures containing disk quotas and >>>>>> usage. >>>>>> At start Jail, we calculate the size root path and number of files in >>>>>> it, thus receiving current use of a disk. >>>>>> In functions of allocation of disk blocks and inode, we check quotas and >>>>>> we increase current use. >>>>>> >>>>>> >>>>>> >>>>> UFS cannot determine whether the new allocation goes under the jail >>>>> root or not. >>>>> >>>>> >>>>> >>>> Yes. But jail cannot allocate block and inode above root path. In >>>> allocation functions, whether for example ffs_alloc we have access to >>>> ucred process and we can check up there is a process in jail. >>>> >>>> >>> Yes, you can check this for jailed process. Think about non-jailed processes >>> that can do allocation below the jail root. >>> >>> >> Processes out of jail are not considered. >> I do not understand, these processes have what relation to disk to >> quotas for jail. Please explain more in detail >> > > A process outside of the jail can still write to the file system that > you consider to be jailed, depending upon permissions. If all your quota > calculations are only triggered by jailed processes writing to the file > system, then you can exceed quota trivially. > > Tom > > The primary goal of disk quotas to limit allocation of disk blocks and inode to processes in jail during their work. Jail it is time essence. After end of work Jail, it does not exist. Let's consider disk quotas for Jail, as number of blocks or inode which jail can use during a session. I understand that if process out of jail will create in a root directory jail a file of the sizes in 1 GB, and process in jail will remove this file jail can exceed the limit on 1 GB. But there is no real necessity, in an operating time jail to write down in the root catalogue jail from the outside jail. From kostjn at peterhost.ru Tue May 26 16:23:50 2009 From: kostjn at peterhost.ru (Menshikov Konstantin) Date: Tue May 26 16:23:56 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: References: <4A1B8CF8.7030102@peterhost.ru> Message-ID: <4A1C1805.8070906@peterhost.ru> Ivan Voras wrote: > Menshikov Konstantin wrote: > >> Hi. >> Jail now have no disk quotas. >> Users which use jail, use separate zfs or md device for jail, but it >> from a hopelessness. >> > > Well, the ZFS way of doing things (per-filesystem quotas) is actually > the right way to do it - it solves exactly the problem described. > > > Yes, you are right. But this way, has some lacks. 1. For quota change jail, it is necessary to stop jail and to start procedure of change of the size of file system. In case of use ufs it is not simple. 2. In case of use zfs, as far as I know, it is impossible to use virtual file systems for example unionfs. From des at des.no Tue May 26 20:32:46 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue May 26 20:32:58 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <23727599.post@talk.nabble.com> (Jakub Lach's message of "Tue, 26 May 2009 10:18:50 -0700 (PDT)") References: <23727599.post@talk.nabble.com> Message-ID: <86prdvipwe.fsf@ds4.des.no> [moving from security@ to hackers@] Jakub Lach writes: > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/21768 Like bde@ pointed out, the patch is incorrect. It moves the test for v_type != VDIR up to a point where, in the case of a symlink, v_type is always (by definition) VLNK. The reason why the current code does not work is that, in the symlink case, the v_type != VDIR test is never reached: we will have jumped to either bad2 or success. However, it should be safe to move the test to after the success label, because trailing_slash is only ever true for the last component of the path we were asked to look up (see lines 520 through 535). The attached patch should work. DES -- Dag-Erling Sm?rgrav - des@des.no -------------- next part -------------- A non-text attachment was scrubbed... Name: symlink-slash.diff Type: text/x-patch Size: 748 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090526/2309e5a0/symlink-slash.bin From des at des.no Tue May 26 20:40:35 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue May 26 20:40:41 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86prdvipwe.fsf@ds4.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Tue, 26 May 2009 22:13:21 +0200") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> Message-ID: <86octflhry.fsf@ds4.des.no> Dag-Erling Sm?rgrav writes: > The attached patch should work. Oops. It actually triggers a KASSERT. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Tue May 26 21:20:05 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue May 26 21:20:11 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86prdvipwe.fsf@ds4.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Tue, 26 May 2009 22:13:21 +0200") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> Message-ID: <86my8z8su6.fsf@ds4.des.no> Dag-Erling Sm?rgrav writes: > Like bde@ pointed out, the patch is incorrect. It moves the test for > v_type != VDIR up to a point where, in the case of a symlink, v_type is > always (by definition) VLNK. Hmm, actually, symlinks are resolved in namei(), not lookup(). This is not going to be pretty. I'll be back later... DES -- Dag-Erling Sm?rgrav - des@des.no From danny at cs.huji.ac.il Wed May 27 07:58:28 2009 From: danny at cs.huji.ac.il (Danny Braniss) Date: Wed May 27 07:58:36 2009 Subject: nfs_diskless_valid question Message-ID: Hi, This variable is set by the boot process, and as far as I can tell modified as the boot process progresses, then by /etc/rc, and never again used. Is there some security reason that I'm not aware of, for this variable to be read-only? (sysctl won't change its value). thanks, danny From fhcarron at terra.es Wed May 27 09:39:31 2009 From: fhcarron at terra.es (Fernando Herrero =?ISO-8859-1?Q?Carr=F3n?=) Date: Wed May 27 09:39:39 2009 Subject: Help debugging kernel together with X In-Reply-To: <4ad871310905260839u24ce71e9wbaa637712480c390@mail.gmail.com> References: <1243345847.1007.11.camel@nebet.ii.uam.es> <4ad871310905260839u24ce71e9wbaa637712480c390@mail.gmail.com> Message-ID: <1243417169.996.1.camel@nebet.ii.uam.es> Thanks Glen, Despite having fought many battles with the dreaded X.org update ??, this seems not to be the problem now. I tried to install a new graphics card some time ago, before the upgrade and the result was the same. My concern right now is the behaviour of Watchdogs, I'll start a new, more specific thread. Thanks, Fernando El mar, 26-05-2009 a las 11:39 -0400, Glen Barber escribi?: > Hi, Fernando > > 2009/5/26 Fernando Herrero Carr?n : > > Hi list, > > > > I would like to ask for some help debugging the kernel. > > > > Here is the problem: > > > > I have a computer about five years old with an on-board graphics card > > (SiS 661). I am trying to install an ATI Radeon 128 on the AGP port. > > FreeBSD (FreeBSD 7.2-STABLE #10: Tue May 26 15:08:39 CEST 2009) is able > > to start DRM and AGP without trouble. However, whenever I start X (X.Org > > X Server 1.6.1; Release Date: 2009-4-14; Build Date: 11 May 2009 > > 12:03:27PM) the system freezes with a blank screen with some apparent > > noisy green dots on it (reproducible, however). It does not panic, nor > > reboot. > > > > Now I suspect there is some problem on the AGP bridge driver since linux > > can run X without trouble (i.e. hardware and BIOS are ok). I have found > > someone reporting problems with this bridge long time ago with no > > answers (http://www.nabble.com/agp-on-sis-661-td1446998.html). > > > > So here I am trying to figure out where the system freezes. I have > > recompiled my kernel adding the following options: > > > > options KDB > > options DDB > > options MP_WATCHDOG > > options SW_WATCHDOG > > > > I am able to enter the debugger with ctrl+alt+esc from a console. So > > what I am trying to do is to have watchdogd time out and the debugger > > make a dump: > > > > ddb script kern.enter.watchdog=call doadump; reboot > > > > However, when I start X no timeout occurs, and the system stays freezed. > > Am I missing something on the workings of watchdogd? > > > > What you're describing does not appear to be a kernel problem; it > sounds like the typical 'Xorg update from hell' problem most > experienced a while back. > > Have a look at /usr/ports/UPDATING, and search for 'AllowEmptyInput'. > > Adding the following to xorg.conf should correct the problem: > > Section "ServerFlags" > option "AllowEmptyInput" "off" > option "AutoAddDevices" "off" > EndSection > > HTH > From rea-fbsd at codelabs.ru Wed May 27 10:10:59 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Wed May 27 10:11:09 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86prdvipwe.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> Message-ID: <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> Dag-Erling, *, good day. Tue, May 26, 2009 at 10:13:21PM +0200, Dag-Erling Sm??rgrav wrote: > [moving from security@ to hackers@] > > Jakub Lach writes: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/21768 > > Like bde@ pointed out, the patch is incorrect. It moves the test for > v_type != VDIR up to a point where, in the case of a symlink, v_type is > always (by definition) VLNK. > > The reason why the current code does not work is that, in the symlink > case, the v_type != VDIR test is never reached: we will have jumped to > either bad2 or success. However, it should be safe to move the test to > after the success label, because trailing_slash is only ever true for > the last component of the path we were asked to look up (see lines 520 > through 535). May be the attached patch will fix the thing? It works for me for 7.2 with WITNESS and INVARIANTS enabled. It adds an additional flag, but this was the only thing I was able to invent to avoid ABI breakage. -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-symlink-with-slash.diff Type: text/x-diff Size: 5532 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/b99db350/vfs_lookup-trailing-symlink-with-slash.bin From des at des.no Wed May 27 11:07:17 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 11:07:28 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> (Eygene Ryabinkin's message of "Wed, 27 May 2009 13:52:21 +0400") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> Message-ID: <86vdnmiz30.fsf@ds4.des.no> Eygene Ryabinkin writes: > May be the attached patch will fix the thing? I'm not entirely convinced. Try the regression test I wrote (head/tools/regression/vfs/trailing_slash.t) > It adds an additional flag, but this was the only thing I was able to > invent to avoid ABI breakage. The flag is a good idea, but I think the correct place to handle this is in namei(), around line 290 (don't be fooled by the comment on line 270; the code inside the if statement is for the *non*-symlink case). DES -- Dag-Erling Sm?rgrav - des@des.no From ivoras at freebsd.org Wed May 27 11:47:26 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Wed May 27 11:47:32 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1C1805.8070906@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> <4A1C1805.8070906@peterhost.ru> Message-ID: Menshikov Konstantin wrote: > 2. In case of use zfs, as far as I know, it is impossible to use virtual > file systems for example unionfs. I don't know how stable is it in the long term, but it does work: www:/home/ivoras# mount /dev/mirror/root_a on / (ufs, local, soft-updates) devfs on /dev (devfs, local) data/backups on /backups (zfs, local) data on /data (zfs, local) data/home on /home (zfs, local) data/services on /services (zfs, local) data/storage on /storage (zfs, local) data/usrlocal on /usr/local (zfs, local) data/usrobj on /usr/obj (zfs, local) data/ports on /usr/ports (zfs, local) data/vardb on /var/db (zfs, local) data/varlog on /var/log (zfs, local) /dev/md0 on /tmp (ufs, local) /usr/ports/archivers on /home/tmp (nullfs, local) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/cb2414a4/signature.pgp From ivoras at freebsd.org Wed May 27 11:55:03 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Wed May 27 11:55:26 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: References: <4A1B8CF8.7030102@peterhost.ru> <4A1C1805.8070906@peterhost.ru> Message-ID: Ivan Voras wrote: > Menshikov Konstantin wrote: > > >> 2. In case of use zfs, as far as I know, it is impossible to use virtual >> file systems for example unionfs. > > I don't know how stable is it in the long term, but it does work: > > www:/home/ivoras# mount > /usr/ports/archivers on /home/tmp (nullfs, local) Sorry, I miseread your question as about nullfs. Yes, unionfs doesn't seem to work: www:~# mount_unionfs /usr/ports/archivers /home/tmp mount_unionfs: /home/tmp: : Operation not supported -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/b97f0ca8/signature.pgp From rea-fbsd at codelabs.ru Wed May 27 12:07:14 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Wed May 27 12:07:21 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86vdnmiz30.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> Message-ID: <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> Wed, May 27, 2009 at 01:07:15PM +0200, Dag-Erling Sm??rgrav wrote: > Eygene Ryabinkin writes: > > May be the attached patch will fix the thing? > > I'm not entirely convinced. Try the regression test I wrote > (head/tools/regression/vfs/trailing_slash.t) I see: you mean that the bare '/' at the end of everything but directory should produce ENOTDIR. OK, patch was modified and now it passes all your checks. > > It adds an additional flag, but this was the only thing I was able to > > invent to avoid ABI breakage. > > The flag is a good idea, but I think the correct place to handle this is > in namei(), around line 290 The problem with the check in namei() itself is the cleanup of all locks that were held in the lookup(). If lookup() is finished without error, then the burden of cleanup is ours (namei's). I could duplicate the stuff, but why? lookup() already does it and it's better to keep the things in one place. The logics is laid as follows: if lookup() processes the last component and it had seen the trailing slash, the flag is set. When we have no more targets to get from the current path inside lookup(), check if slashed flag is set and reject anything that is non-directory. Such strategy should also handle the cases of dereferencing (FOLLOWs) of all symbolic links and when some link has slash at the end of the target name: 'ln -s /etc/motd somefile; ln -s somefile/ anotherfile; cat anotherfile' will fail on the last command. If one agrees on such behaviour, such test could be also added to the regression suite. > (don't be fooled by the comment on line 270; > the code inside the if statement is for the *non*-symlink case). Me sees this on the line 226, but may be I hadn't updated my 7.x. And yes, I know what was meant by '(cnp->cn_flags & ISSYMLINK) == 0' ;)) -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-symlink-with-slash.diff Type: text/x-diff Size: 5495 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/cf5a3c5e/vfs_lookup-trailing-symlink-with-slash.bin From des at des.no Wed May 27 12:39:08 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 12:39:15 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> (Eygene Ryabinkin's message of "Wed, 27 May 2009 16:07:09 +0400") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> Message-ID: <86prdug1p0.fsf@ds4.des.no> Eygene Ryabinkin writes: > "Dag-Erling Sm?rgrav" writes: > > (don't be fooled by the comment on line 270; > > the code inside the if statement is for the *non*-symlink case). > Me sees this on the line 226, but may be I hadn't updated my 7.x. I was working on head. The code is (mostly) the same, just shifted somewhere between ~50 and ~90 lines depending on where you look. Your patch should apply cleanly. BTW, you made a lot of whitespace changes in namei.h. This is generally frowned upon, as it makes the functional change almost impossible to spot in the diff. > And yes, I know what was meant by '(cnp->cn_flags & ISSYMLINK) == 0' > ;)) I know you know :) I was just pointing out that the comment is misleading. DES -- Dag-Erling Sm?rgrav - des@des.no From rwatson at FreeBSD.org Wed May 27 12:51:24 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Wed May 27 12:51:31 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: <4A1BE827.2030303@peterhost.ru> References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> <20090526123632.GB1927@deviant.kiev.zoral.com.ua> <4A1BE827.2030303@peterhost.ru> Message-ID: On Tue, 26 May 2009, Menshikov Konstantin wrote: >>> Yes. But jail cannot allocate block and inode above root path. In >>> allocation functions, whether for example ffs_alloc we have access to >>> ucred process and we can check up there is a process in jail. >> >> Yes, you can check this for jailed process. Think about non-jailed >> processes that can do allocation below the jail root. > > Processes out of jail are not considered. I do not understand, these > processes have what relation to disk to quotas for jail. Please explain more > in detail Historic UFS quotas are actually not interested in processes at all, really, except in as much as processes are where exception states are exposed. UFS quotas count blocks and inodes owned by users based on the 'uid' and 'gid' fields in the inode. There's now 'jailid' field, so quotas on this model can't capture the notion of per-jail quotas. In fact, quotacheck relies on being able to walk the file system looking only at file system data in order to establish initial usage accounting. You can imagine adding one, or managing the uid spaces across jails such that all uids are unique, etc, but all of these require some amount of rethinking. Or, some other model of quota. Frankly, I've always been a fan of the AFS model, now accessible locally via ZFS, in which lightweight volumes with quota limits are used for individual user home directories, virtual machines, etc. This was hard to do in FreeBSD before ZFS because (a) UFS didn't want to resize trivially and (b) having lots and lots of mountpoints and file systems wasn't something we made administratively easy. Robert N M Watson Computer Laboratory University of Cambridge From des at des.no Wed May 27 13:10:32 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 13:10:39 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <4A1594DA.2010707@rawbw.com> (yuri@rawbw.com's message of "Thu, 21 May 2009 10:52:26 -0700") References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> Message-ID: <86ljoig08o.fsf@ds4.des.no> Yuri writes: > I don't have strong opinion for or against "memory overcommit". But I > can imagine one could argue that fork with intent of exec is a faulty > scenario that is a relict from the past. It can be replaced by some > atomic method that would spawn the child without ovecommitting. You will very rarely see something like this: if ((pid = fork()) == 0) { execve(path, argv, envp); _exit(1); } Usually, what you see is closer to this: if ((pid = fork()) == 0) { for (int fd = 3; fd < getdtablesize(); ++fd) (void)close(fd); execve(path, argv, envp); _exit(1); } ...with infinite variation depending on whether the parent needs to communicate with the child, whether the child needs std{in,out,err} at all, etc. For the trivial case, there is always vfork(), which does not duplicate the address space, and blocks the parent until the child has execve()d. This allows you to pull cute tricks like this: volatile int error = 0; if ((pid = vfork()) == 0) { error = execve(path, argv, envp); _exit(1); } if (pid == -1 || error != 0) perror("Failed to start subprocess"); DES -- Dag-Erling Sm?rgrav - des@des.no From rea-fbsd at codelabs.ru Wed May 27 13:16:30 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Wed May 27 13:16:37 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86prdug1p0.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> Message-ID: Wed, May 27, 2009 at 02:39:07PM +0200, Dag-Erling Sm??rgrav wrote: > I was working on head. The code is (mostly) the same, just shifted > somewhere between ~50 and ~90 lines depending on where you look. Your > patch should apply cleanly. > > BTW, you made a lot of whitespace changes in namei.h. This is generally > frowned upon, as it makes the functional change almost impossible to > spot in the diff. Yes, spit the patch into two pieces. Thanks for the reminder! > > And yes, I know what was meant by '(cnp->cn_flags & ISSYMLINK) == 0' > > ;)) > > I know you know :) I was just pointing out that the comment is > misleading. Changed it too. All three pieces are attached. Regarding the 'ln -s /etc/motd file; ln -s file/ anotherone': do you (or anyone reading this) think that 'cat anotherone' should really show the contents of /etc/motd or patch's behaviour is good? -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-symlink-with-slash.diff Type: text/x-diff Size: 3069 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/dd0e9849/vfs_lookup-trailing-symlink-with-slash.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-symlink-with-slash-fix-whitespace.diff Type: text/x-diff Size: 3364 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/dd0e9849/vfs_lookup-trailing-symlink-with-slash-fix-whitespace.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-symlink-with-slash-fix-comment.diff Type: text/x-diff Size: 856 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/dd0e9849/vfs_lookup-trailing-symlink-with-slash-fix-comment.bin From des at des.no Wed May 27 14:31:00 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 14:31:07 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: (Eygene Ryabinkin's message of "Wed, 27 May 2009 17:16:25 +0400") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> Message-ID: <86ab4yfwil.fsf@ds4.des.no> Eygene Ryabinkin writes: > Regarding the 'ln -s /etc/motd file; ln -s file/ anotherone': do you > (or anyone reading this) think that 'cat anotherone' should really > show the contents of /etc/motd or patch's behaviour is good? if you mean $ ln -fs /etc/motd foo $ ln -fs foo/ bar $ readlink foo bar /etc/motd foo/ $ cat foo then IMHO it should produce an error. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Wed May 27 16:44:38 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 16:44:46 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: (Eygene Ryabinkin's message of "Wed, 27 May 2009 17:16:25 +0400") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> Message-ID: <86vdnmijgs.fsf@ds4.des.no> Eygene Ryabinkin writes: > [new three-part patch] I committed the namei.h cleanup patch and the vfs_lookup.c comment patch. I made a number of changes to the trailing-slash patch. Can you double-check it before I commit it? DES -- Dag-Erling Sm?rgrav - des@des.no -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-slash.diff Type: text/x-patch Size: 1848 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/f8c4ea82/vfs_lookup-trailing-slash.bin From des at des.no Wed May 27 16:46:02 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 16:46:08 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <20090527233110.E4243@delplex.bde.org> (Bruce Evans's message of "Thu, 28 May 2009 01:15:17 +1000 (EST)") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <20090527233110.E4243@delplex.bde.org> Message-ID: <86r5yaijef.fsf@ds4.des.no> Bruce Evans writes: > This seems to be equivalent to the patch in the PR at the time of PR, > except it risks breaking some other cases, so I don't see how it can > work. As discussed on -hackers, it doesn't. This one does, though. DES -- Dag-Erling Sm?rgrav - des@des.no -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs_lookup-trailing-slash.diff Type: text/x-patch Size: 1848 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/7a21e710/vfs_lookup-trailing-slash.bin From rea-fbsd at codelabs.ru Wed May 27 17:28:54 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Wed May 27 17:29:00 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86vdnmijgs.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> <86vdnmijgs.fsf@ds4.des.no> Message-ID: Wed, May 27, 2009 at 06:44:35PM +0200, Dag-Erling Sm??rgrav wrote: > Eygene Ryabinkin writes: > > [new three-part patch] > > I committed the namei.h cleanup patch and the vfs_lookup.c comment > patch. Thanks! > I made a number of changes to the trailing-slash patch. Can you > double-check it before I commit it? Yes, comments are below. > Index: sys/kern/vfs_lookup.c > =================================================================== > --- sys/kern/vfs_lookup.c (revision 192899) > +++ sys/kern/vfs_lookup.c (working copy) > @@ -147,6 +147,9 @@ > cnp->cn_flags &= ~LOCKSHARED; > fdp = p->p_fd; > > + /* We will set this ourselves if we need it. */ > + cnp->cn_flags &= ~TRAILINGSLASH; > + > /* > * Get a buffer for the name to be translated, and copy the > * name into the buffer. > @@ -533,6 +536,8 @@ > if (*cp == '\0') { > trailing_slash = 1; > *ndp->ni_next = '\0'; /* XXX for direnter() ... */ > + if (cnp->cn_flags & ISLASTCN) > + cnp->cn_flags |= TRAILINGSLASH; 'if ()' looks suspicious: ISLASTCN is set some lines below so it could be not yet flagged. Seems like we could omit 'if ()' clause but leave it's body for the current state of the code -- it will be equivalent to the mine's check. But for the clarity, I will leave the full condition, 'trailing_slash && (cnp->cn_flags & ISLASTCN)' somewhere below the block with ----- if (*ndp->ni_next == 0) cnp->cn_flags |= ISLASTCN; else cnp->cn_flags &= ~ISLASTCN; ----- My original intent was to push it to the bottom of the code to slightly optimize code path: some checks above could already fail and we won't have to perform our test. But now I feel that the best place for the test is immediately below the cited chunk of the code. The rest looks fine. Had you tried your variant of patch? May be I am missing something and the test for ISLASTCN really in place? By the way, I had somewhat extended your regression tests with the intermediate symlink tests, directory tests and device-as-a-target tests. Patches are attached. Will they go? Thanks! -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs-testsuite-dirs-and-double-links.diff Type: text/x-diff Size: 2822 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/60322528/vfs-testsuite-dirs-and-double-links.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: vfs-testsuite-devices-as-destination.diff Type: text/x-diff Size: 934 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090527/60322528/vfs-testsuite-devices-as-destination.bin From oliver.pntr at gmail.com Wed May 27 17:51:09 2009 From: oliver.pntr at gmail.com (Oliver Pinter) Date: Wed May 27 17:51:17 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86vdnmijgs.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> <86vdnmijgs.fsf@ds4.des.no> Message-ID: <6101e8c40905271051r3bb9d633kec6d198d45fc9cf6@mail.gmail.com> Hi! This is a redefinitions of PARAMASK in the patch, that you attached -------8<--------- ... #define PARAMASK 0x0ffffe00 /* mask of parameter descriptors */ +#define TRAILINGSLASH 0x10000000 /* path ended in a slash */ +#define PARAMASK 0x1ffffe00 /* mask of parameter descriptors */ ... -------8<--------- On 5/27/09, Dag-Erling Sm?rgrav wrote: > Eygene Ryabinkin writes: >> [new three-part patch] > > I committed the namei.h cleanup patch and the vfs_lookup.c comment > patch. > > I made a number of changes to the trailing-slash patch. Can you > double-check it before I commit it? > > DES > -- > Dag-Erling Sm?rgrav - des@des.no > > From des at des.no Wed May 27 17:59:02 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 17:59:10 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <6101e8c40905271051r3bb9d633kec6d198d45fc9cf6@mail.gmail.com> (Oliver Pinter's message of "Wed, 27 May 2009 19:51:07 +0200") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> <86vdnmijgs.fsf@ds4.des.no> <6101e8c40905271051r3bb9d633kec6d198d45fc9cf6@mail.gmail.com> Message-ID: <86eiuaig0t.fsf@ds4.des.no> Oliver Pinter writes: > This is a redefinitions of PARAMASK in the patch, that you attached Sorry, I forgot to regenerate the patch after fixing it. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Wed May 27 18:03:59 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed May 27 18:04:07 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: (Eygene Ryabinkin's message of "Wed, 27 May 2009 21:28:49 +0400") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <0vGjPHEq7MqxjtFmBufY+mBxlR4@7oUjtCwN654QcDr16CH+kAk8bJg> <86vdnmiz30.fsf@ds4.des.no> <15QQC+1YeDzOjf35dqyJmioc1ik@XX1fo6zQUfC4h0jjRC6IBz3oNH4> <86prdug1p0.fsf@ds4.des.no> <86vdnmijgs.fsf@ds4.des.no> Message-ID: <86ab4yifsi.fsf@ds4.des.no> Eygene Ryabinkin writes: > 'if ()' looks suspicious: ISLASTCN is set some lines below so it could > be not yet flagged. Seems like we could omit 'if ()' clause but leave > it's body for the current state of the code -- it will be equivalent to > the mine's check. Yes, I was a little too quick there. You're right, we can just drop the if(). Actually, the reason why I moved this up is that I was considering eliminating the trailing_slash variable entirely. > By the way, I had somewhat extended your regression tests with the > intermediate symlink tests, directory tests and device-as-a-target > tests. Patches are attached. Will they go? I'll take a look at them later. DES -- Dag-Erling Sm?rgrav - des@des.no From kostjn at peterhost.ru Thu May 28 06:58:29 2009 From: kostjn at peterhost.ru (Menshikov Konstantin) Date: Thu May 28 06:58:37 2009 Subject: Disk quota for Jail. Discussion. In-Reply-To: References: <4A1B8CF8.7030102@peterhost.ru> <20090526120313.GA1927@deviant.kiev.zoral.com.ua> <4A1BE1F8.9050804@peterhost.ru> <20090526123632.GB1927@deviant.kiev.zoral.com.ua> <4A1BE827.2030303@peterhost.ru> Message-ID: <4A1E3688.8050300@peterhost.ru> Robert Watson wrote: > > On Tue, 26 May 2009, Menshikov Konstantin wrote: > >>>> Yes. But jail cannot allocate block and inode above root path. In >>>> allocation functions, whether for example ffs_alloc we have access >>>> to ucred process and we can check up there is a process in jail. >>> >>> Yes, you can check this for jailed process. Think about non-jailed >>> processes that can do allocation below the jail root. >> >> Processes out of jail are not considered. I do not understand, these >> processes have what relation to disk to quotas for jail. Please >> explain more in detail > > Historic UFS quotas are actually not interested in processes at all, > really, except in as much as processes are where exception states are > exposed. UFS quotas count blocks and inodes owned by users based on > the 'uid' and 'gid' fields in the inode. There's now 'jailid' field, > so quotas on this model can't capture the notion of per-jail quotas. > In fact, quotacheck relies on being able to walk the file system > looking only at file system data in order to establish initial usage > accounting. You can imagine adding one, or managing the uid spaces > across jails such that all uids are unique, etc, but all of these > require some amount of rethinking. > > Or, some other model of quota. Frankly, I've always been a fan of the > AFS model, now accessible locally via ZFS, in which lightweight > volumes with quota limits are used for individual user home > directories, virtual machines, etc. This was hard to do in FreeBSD > before ZFS because (a) UFS didn't want to resize trivially and (b) > having lots and lots of mountpoints and file systems wasn't something > we made administratively easy. > > Robert N M Watson > Computer Laboratory > University of Cambridge > Actually realisation of quotas for jail, is reduced to realisation of quotas on catalogue contents. And it is difficult, it is not necessary. Many thanks for answers and explanations. Menshikov Konstantin From mel.flynn+fbsd.hackers at mailing.thruhere.net Thu May 28 09:25:20 2009 From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn) Date: Thu May 28 09:25:26 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86my8z8su6.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <86my8z8su6.fsf@ds4.des.no> Message-ID: <200905281107.12864.mel.flynn+fbsd.hackers@mailing.thruhere.net> On Tuesday 26 May 2009 23:20:01 Dag-Erling Sm?rgrav wrote: > Dag-Erling Sm?rgrav writes: > > Like bde@ pointed out, the patch is incorrect. It moves the test for > > v_type != VDIR up to a point where, in the case of a symlink, v_type is > > always (by definition) VLNK. > > Hmm, actually, symlinks are resolved in namei(), not lookup(). This is > not going to be pretty. I'll be back later... I don't pretend to comprehend the kernel side of things fully, but wouldn't it be easier to append a dot to all trailing slashes inside or before passing to namei? This works in userland at present and lighttpd could use something similar as a work around until it's fixed: % echo this is foo > foo % ln -fs foo bar % cat bar/ this is foo % cat bar/. cat: bar/.: Not a directory -- Mel From rea-fbsd at codelabs.ru Thu May 28 09:57:05 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Thu May 28 09:57:12 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <200905281107.12864.mel.flynn+fbsd.hackers@mailing.thruhere.net> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <86my8z8su6.fsf@ds4.des.no> <200905281107.12864.mel.flynn+fbsd.hackers@mailing.thruhere.net> Message-ID: Mel, good day. Thu, May 28, 2009 at 11:07:12AM +0200, Mel Flynn wrote: > On Tuesday 26 May 2009 23:20:01 Dag-Erling Sm??rgrav wrote: > > Dag-Erling Sm??rgrav writes: > > > Like bde@ pointed out, the patch is incorrect. It moves the test for > > > v_type != VDIR up to a point where, in the case of a symlink, v_type is > > > always (by definition) VLNK. > > > > Hmm, actually, symlinks are resolved in namei(), not lookup(). This is > > not going to be pretty. I'll be back later... > I don't pretend to comprehend the kernel side of things fully, but > wouldn't it be easier to append a dot to all trailing slashes inside > or before passing to namei? A dirty hack that puts some additional burden on the namei() ;-/ > This works in userland at present and lighttpd could use something > similar as a work around until it's fixed: Yes, this will work, but it is better to apply the real fix ;)) Dirty hacks aren't good at the long timescales -- they tend to obfuscate the code and put unneeded interprocedure constraints (you should prepend dot to the slash if you want to call namei()/we should add dot to slash to make our life easier/etc). -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From alfred at freebsd.org Thu May 28 21:30:18 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Thu May 28 21:30:24 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <86ljoig08o.fsf@ds4.des.no> References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> <86ljoig08o.fsf@ds4.des.no> Message-ID: <20090528213017.GX67847@elvis.mu.org> * Dag-Erling Sm??rgrav [090527 06:10] wrote: > Yuri writes: > > I don't have strong opinion for or against "memory overcommit". But I > > can imagine one could argue that fork with intent of exec is a faulty > > scenario that is a relict from the past. It can be replaced by some > > atomic method that would spawn the child without ovecommitting. > > You will very rarely see something like this: > > if ((pid = fork()) == 0) { > execve(path, argv, envp); > _exit(1); > } > > Usually, what you see is closer to this: > > if ((pid = fork()) == 0) { > for (int fd = 3; fd < getdtablesize(); ++fd) > (void)close(fd); > execve(path, argv, envp); > _exit(1); > } I'm probably missing something, but couldn't you iterate in the parent setting the close-on-exec flag then vfork? I guess that wouldn't work for threads AND you'd have to undo it after the fork if you didn't want to retain that behavior? thanks, -Alfred From davidn04 at gmail.com Fri May 29 04:54:08 2009 From: davidn04 at gmail.com (David N) Date: Fri May 29 04:54:14 2009 Subject: Debugging via DDB Message-ID: <4d7dd86f0905282127u5215979akdd8ea286c73f090e@mail.gmail.com> Hi, I know this might sound like a newbie question. I'm trying to debug a "Lockup" on 7.2-RELEASE. I've compiled DDB and KDB into the kernel and make installkernel. (Can this be called a deadlock?) The machine still response to pings, but it looks like all disk activity has stopped. I can break into the debugger using CTRL-ALT-ESC, but after that, i dont know what I'm looking for in particular. bt (backtrace) gives me the stuff like the keyboard mutex ( i assume it was the last thing that happened, but obviously it was the break) Should I be including WITNESS? Regards David N From kmacy at freebsd.org Fri May 29 06:04:31 2009 From: kmacy at freebsd.org (Kip Macy) Date: Fri May 29 06:04:44 2009 Subject: Debugging via DDB In-Reply-To: <4d7dd86f0905282127u5215979akdd8ea286c73f090e@mail.gmail.com> References: <4d7dd86f0905282127u5215979akdd8ea286c73f090e@mail.gmail.com> Message-ID: <3c1674c90905282304s223349c4p8523abbbed8ade0b@mail.gmail.com> On Thu, May 28, 2009 at 9:27 PM, David N wrote: > Hi, > > I know this might sound like a newbie question. > > I'm trying to debug a "Lockup" on 7.2-RELEASE. I've compiled DDB and > KDB into the kernel and make installkernel. (Can this be called a > deadlock?) > > The machine still response to pings, but it looks like all disk > activity has stopped. > > I can break into the debugger using CTRL-ALT-ESC, but after that, i > dont know what I'm looking for in particular. > > bt (backtrace) gives me the stuff like the keyboard mutex ( i assume > it was the last thing that happened, but obviously it was the break) > > Should I be including WITNESS? Include WITNESS. "ps" to list processes, "show proc" to see proc info, "thread " to switch to a thread, "show locks" to see locks held, show sleepc to see what you're blocked on (if a sleepq), and "show alllocks" to see all locks held -Kip From des at des.no Fri May 29 09:49:23 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri May 29 09:49:35 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <20090528213017.GX67847@elvis.mu.org> (Alfred Perlstein's message of "Thu, 28 May 2009 14:30:17 -0700") References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> <86ljoig08o.fsf@ds4.des.no> <20090528213017.GX67847@elvis.mu.org> Message-ID: <863aaow866.fsf@ds4.des.no> Alfred Perlstein writes: > Dag-Erling Sm?rgrav writes: > > Usually, what you see is closer to this: > > > > if ((pid = fork()) == 0) { > > for (int fd = 3; fd < getdtablesize(); ++fd) > > (void)close(fd); > > execve(path, argv, envp); > > _exit(1); > > } > > I'm probably missing something, but couldn't you iterate > in the parent setting the close-on-exec flag then vfork? This is an example, Alfred. Like most examples, it is greatly simplified. I invite you to peruse the source to find real-world instances of non-trivial fork() / execve() usage. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Fri May 29 16:53:24 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri May 29 16:53:33 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <20090529210855.V1643@besplex.bde.org> (Bruce Evans's message of "Sat, 30 May 2009 02:01:47 +1000 (EST)") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no> <20090529210855.V1643@besplex.bde.org> Message-ID: <86vdnju9z1.fsf@ds4.des.no> Bruce Evans writes: > Dag-Erling Sm?rgrav writes: > % Index: sys/kern/vfs_lookup.c > % =================================================================== > % --- sys/kern/vfs_lookup.c (revision 192899) > % +++ sys/kern/vfs_lookup.c (working copy) > % @@ -147,6 +147,9 @@ > % cnp->cn_flags &= ~LOCKSHARED; > % fdp = p->p_fd; > % % + /* We will set this ourselves if we need it. */ > % + cnp->cn_flags &= ~TRAILINGSLASH; > % + > > Can TRAILINGSLASH ever be set here? Is namei() ever called recursively? "suspenders and a belt" It is hypothetically possible for the caller to have set it. > % /* > % * Get a buffer for the name to be translated, and copy the > % * name into the buffer. > % @@ -533,6 +536,8 @@ > % if (*cp == '\0') { > % trailing_slash = 1; > > I thought at first that this flag can go away. I intend to remove it later - I just wanted to get the bug fixed first. I'm happy to hear that removing it will fix the two bugs introduced by the patch I committed :) DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Fri May 29 16:58:09 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri May 29 16:58:16 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86vdnju9z1.fsf@ds4.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Fri, 29 May 2009 18:53:22 +0200") References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no> <20090529210855.V1643@besplex.bde.org> <86vdnju9z1.fsf@ds4.des.no> Message-ID: <86r5y7u9r3.fsf@ds4.des.no> How's this? Index: sys/kern/vfs_lookup.c =================================================================== --- sys/kern/vfs_lookup.c (revision 193028) +++ sys/kern/vfs_lookup.c (working copy) @@ -454,7 +454,6 @@ int docache; /* == 0 do not cache last component */ int wantparent; /* 1 => wantparent or lockparent flag */ int rdonly; /* lookup read-only flag bit */ - int trailing_slash; int error = 0; int dpunlocked = 0; /* dp has already been unlocked */ struct componentname *cnp = &ndp->ni_cnd; @@ -529,12 +528,10 @@ * trailing slashes to handle symlinks, existing non-directories * and non-existing files that won't be directories specially later. */ - trailing_slash = 0; while (*cp == '/' && (cp[1] == '/' || cp[1] == '\0')) { cp++; ndp->ni_pathlen--; if (*cp == '\0') { - trailing_slash = 1; *ndp->ni_next = '\0'; /* XXX for direnter() ... */ cnp->cn_flags |= TRAILINGSLASH; } @@ -711,7 +708,7 @@ error = EROFS; goto bad; } - if (*cp == '\0' && trailing_slash && + if (*cp == '\0' && (cnp->cn_flags & TRAILINGSLASH) && !(cnp->cn_flags & WILLBEDIR)) { error = ENOENT; goto bad; @@ -788,7 +785,7 @@ * Check for symbolic link */ if ((dp->v_type == VLNK) && - ((cnp->cn_flags & FOLLOW) || trailing_slash || + ((cnp->cn_flags & FOLLOW) || (cnp->cn_flags & TRAILINGSLASH) || *ndp->ni_next == '/')) { cnp->cn_flags |= ISSYMLINK; if (dp->v_iflag & VI_DOOMED) { BTW, what does the "XXX for direnter()" comment mean? DES -- Dag-Erling Sm?rgrav - des@des.no From rea-fbsd at codelabs.ru Fri May 29 18:35:17 2009 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Fri May 29 18:35:24 2009 Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file symlink) vulnerability In-Reply-To: <86vdnju9z1.fsf@ds4.des.no> References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no> <20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no> <20090529210855.V1643@besplex.bde.org> <86vdnju9z1.fsf@ds4.des.no> Message-ID: <7Wfi244TRj6h0BU0G5CUnAA6n1Y@BpFm1zkZmHABxHH1eUOcQSRoWTc> Fri, May 29, 2009 at 06:53:22PM +0200, Dag-Erling Sm??rgrav wrote: > Bruce Evans writes: > > % /* > > % * Get a buffer for the name to be translated, and copy the > > % * name into the buffer. > > % @@ -533,6 +536,8 @@ > > % if (*cp == '\0') { > > % trailing_slash = 1; > > > > I thought at first that this flag can go away. > > I intend to remove it later - I just wanted to get the bug fixed first. > I'm happy to hear that removing it will fix the two bugs introduced by > the patch I committed :) What are those bugs? -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # From alfred at freebsd.org Fri May 29 19:31:38 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Fri May 29 19:31:45 2009 Subject: Why kernel kills processes that run out of memory instead of just failing memory allocation system calls? In-Reply-To: <863aaow866.fsf@ds4.des.no> References: <4A14F58F.8000801@rawbw.com> <4A1594DA.2010707@rawbw.com> <86ljoig08o.fsf@ds4.des.no> <20090528213017.GX67847@elvis.mu.org> <863aaow866.fsf@ds4.des.no> Message-ID: <20090529193137.GH67847@elvis.mu.org> * Dag-Erling Sm??rgrav [090529 02:49] wrote: > Alfred Perlstein writes: > > Dag-Erling Sm??rgrav writes: > > > Usually, what you see is closer to this: > > > > > > if ((pid = fork()) == 0) { > > > for (int fd = 3; fd < getdtablesize(); ++fd) > > > (void)close(fd); > > > execve(path, argv, envp); > > > _exit(1); > > > } > > > > I'm probably missing something, but couldn't you iterate > > in the parent setting the close-on-exec flag then vfork? > > This is an example, Alfred. Like most examples, it is greatly > simplified. I invite you to peruse the source to find real-world > instances of non-trivial fork() / execve() usage. It wasn't meant to critisize, just ask a question for the specific instance because it made me curious. I know how bad it can be with vfork as I observed a few fixes involving mistaken use of vfork at another job. So yes, there's more than one way to skin a cat for this particular example... but in practice using vfork()+exec() is hard to get right? -- - Alfred Perlstein From tam.sergio at gmail.com Fri May 29 23:50:37 2009 From: tam.sergio at gmail.com (Sergio Tam) Date: Fri May 29 23:50:43 2009 Subject: Log message Message-ID: Hi On the /var/log/messages show this: kernel: pid 33785: corrected slot count (4->1) ?Whats means? Thanks for your time. Regards From julian at elischer.org Sat May 30 00:00:29 2009 From: julian at elischer.org (Julian Elischer) Date: Sat May 30 00:00:36 2009 Subject: Log message In-Reply-To: References: Message-ID: <4A20771C.9040008@elischer.org> Sergio Tam wrote: > Hi > > On the /var/log/messages show this: > > kernel: pid 33785: corrected slot count (4->1) > > ?Whats means? > > > Thanks for your time. > > Regards > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" It means that you are running an old system, using M:N threads and that the threading code somehow lost track of how may threads were running on that process. The system scheduler corrected it. I never figured out the situation when this occured but it goes away if you move to a newer version of the OS and go to linking with libthr. From tam.sergio at gmail.com Sat May 30 02:23:30 2009 From: tam.sergio at gmail.com (Sergio Tam) Date: Sat May 30 02:23:36 2009 Subject: Log message Message-ID: 2009/5/29 Julian Elischer : > Sergio Tam wrote: >> On the /var/log/messages show this: >> >> kernel: pid 33785: corrected slot count (4->1) >> >> ?Whats means? > > It means that you are running an old system, using M:N threads and that the > threading code somehow lost track of how may threads were running on that > process. The system scheduler corrected it. > > I never figured out the situation when this occured but it goes away if you > move to a newer version of the OS and go to linking with libthr. > Thank you very much. Regards. From gemochka at gmail.com Sat May 30 12:45:18 2009 From: gemochka at gmail.com (Gema niskazhu) Date: Sat May 30 12:45:24 2009 Subject: pf nat+bridge Message-ID: <84133fac0905300512ja548f95v756eb4e006f06ac8@mail.gmail.com> Hi all! First of all sorry for my bad english again =) I've got some problems with nat gw with pf. My situation is pretty simple: I've got 2 networks: external - 10.7.240.0/20 and a internal qemu network with a tap networking 192.168.0/24 External and internal ifaces are bridged cloned_interfaces="tap0 bridge0" autobridge_interfaces="bridge0" autobridge_bridge0="tap0 nfe0" I've dhcpd on nfe0 and it could be accesed through bridge. I've such a simple rule in pf.conf: qemu_if = "tap0" ext_if = "nfe0" nat on $ext_if from $qemu_if:network to any -> ($ext_if) pass from {lo0,$qemu_if:network } to any keep state But none packet forwarded if we try to acces external host from internal network. Is there any specificity of nat'ing bridged networks? Or i mistaken some where else? Sorry for dumb question. Thanks a lot in advance! From xorquewasp at googlemail.com Sat May 30 17:52:48 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Sat May 30 17:52:55 2009 Subject: Request for opinions - gvinum or ccd? Message-ID: <20090530175239.GA25604@logik.internal.network> Hello. I'm planning to stripe two disks into a RAID0 configuration. As far as I can tell, my hardware has no hardware RAID support and therefore I'll be going the software route. The machine in question is a workstation used to process large datasets (audio and video) and do lots of compilation. Simple question then as the handbook describes both ccd and gvinum - which should I pick? From mwm-keyword-freebsdhackers2.e313df at mired.org Sat May 30 19:12:16 2009 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sat May 30 19:12:22 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090530175239.GA25604@logik.internal.network> References: <20090530175239.GA25604@logik.internal.network> Message-ID: <20090530144354.2255f722@bhuda.mired.org> On Sat, 30 May 2009 18:52:39 +0100 xorquewasp@googlemail.com wrote: > Simple question then as the handbook describes both ccd and gvinum - > which should I pick? My first reaction was "neither", then I realized - you didn't say what version of FreeBSD you're running. But if you're running a supported version of FreeBSD, that doesn't change my answer. If you're running 5.3 or later, you probably want gstripe. If you're running something older than that, then gvinum won't be available either, so you'll need to use ccd. I always figured gvinum was a transition tool to help move from vinum to geom, which is why it's managed to get to the 7.0 release with some pretty painful bugs in it, which don't show up in gstripe. The handbook clearly needs to be rewritten - ccd isn't supported anymore, except via the geom ccd class. However, I think zfs is going to change it all again, so such a rewrite wont' be useful for very long. I don't think zfs supports a two-disk stripe, thought it does do JBOD. If you're running a 7.X 64-bit system with a couple of GIG of ram, expect it to be in service for years without having to reformat the disks, and can afford another drive, I'd recommend going to raidz on a three-drive system. That will give you close to the size/performance of your RAID0 system, but let you lose a disk without losing data. The best you can do with zfs on two disks is a mirror, which means write throughput will suffer. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From xorquewasp at googlemail.com Sat May 30 19:18:44 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Sat May 30 19:18:56 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090530144354.2255f722@bhuda.mired.org> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> Message-ID: <20090530191840.GA68514@logik.internal.network> On 2009-05-30 14:43:54, Mike Meyer wrote: > On Sat, 30 May 2009 18:52:39 +0100 > xorquewasp@googlemail.com wrote: > > Simple question then as the handbook describes both ccd and gvinum - > > which should I pick? > > My first reaction was "neither", then I realized - you didn't say what > version of FreeBSD you're running. But if you're running a supported > version of FreeBSD, that doesn't change my answer. Sorry, yeah. FreeBSD 7.2-RELEASE on AMD64. > If you're running 5.3 or later, you probably want gstripe. If you're > running something older than that, then gvinum won't be available > either, so you'll need to use ccd. I always figured gvinum was a > transition tool to help move from vinum to geom, which is why it's > managed to get to the 7.0 release with some pretty painful bugs in it, > which don't show up in gstripe. That sounds like the kind of entertainment I don't particularly want! > The handbook clearly needs to be rewritten - ccd isn't supported > anymore, except via the geom ccd class. However, I think zfs is going > to change it all again, so such a rewrite wont' be useful for very > long. I don't think zfs supports a two-disk stripe, thought it does do > JBOD. > > If you're running a 7.X 64-bit system with a couple of GIG of ram, > expect it to be in service for years without having to reformat the > disks, and can afford another drive, I'd recommend going to raidz on a > three-drive system. That will give you close to the size/performance > of your RAID0 system, but let you lose a disk without losing data. The > best you can do with zfs on two disks is a mirror, which means write > throughput will suffer. Certainly a lot to think about. The system has 12gb currently, with room to upgrade. I currently have two 500gb drives and one 1tb drive. I wanted the setup to be essentially two drives striped, backed up onto one larger one nightly. I wanted the large backup drive to be as "isolated" as possible, eg, in the event of some catastrophic hardware failure, I can remove it and place it in another machine without a lot of stressful configuration to recover the data (not possible with a RAID configuration involving all three drives, as far as I'm aware). xw From mwm-keyword-freebsdhackers2.e313df at mired.org Sat May 30 20:29:19 2009 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sat May 30 20:29:26 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090530191840.GA68514@logik.internal.network> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> Message-ID: <20090530162744.5d77e9d1@bhuda.mired.org> On Sat, 30 May 2009 20:18:40 +0100 xorquewasp@googlemail.com wrote: > > If you're running a 7.X 64-bit system with a couple of GIG of ram, > > expect it to be in service for years without having to reformat the > > disks, and can afford another drive, I'd recommend going to raidz on a > > three-drive system. That will give you close to the size/performance > > of your RAID0 system, but let you lose a disk without losing data. The > > best you can do with zfs on two disks is a mirror, which means write > > throughput will suffer. > > Certainly a lot to think about. > > The system has 12gb currently, with room to upgrade. I currently have > two 500gb drives and one 1tb drive. I wanted the setup to be essentially > two drives striped, backed up onto one larger one nightly. I wanted the > large backup drive to be as "isolated" as possible, eg, in the event of > some catastrophic hardware failure, I can remove it and place it in > another machine without a lot of stressful configuration to recover the > data (not possible with a RAID configuration involving all three drives, > as far as I'm aware). The last bit is wrong. Moving a zfs pool between two systems is pretty straightforward. The configuration information is on the drives; you just do "zpool import " after plugging them in, and if the mount point exists, it'll mount it. If the system crashed with the zfs pool active, you might have to do -f to force an import. Geom is pretty much the same way, except you can configure it to not write the config data to disk, thus forcing you to do it manually (what you expect). I'm not sure geom is as smart if the drives change names, though. RAID support and volume management has come a long way from the days of ccd and vinum. zfs in particular is a major advance. If you aren't aware of it's advantages, take the time to read the zfs & zpool man pages, at the very least, before committing to geom (not that geom isn't pretty slick in and of itself, but zfs solves a more pressing problem). Hmm. Come to think of it, you ought to be able to use gstrip to stripe your disks, then put a zpool on that, which should get you the advantages of zfs with a striped disk. But that does seem odd to me. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From rpaulo at freebsd.org Sat May 30 21:33:36 2009 From: rpaulo at freebsd.org (Rui Paulo) Date: Sat May 30 21:34:07 2009 Subject: MosChip 7840 dual port ucom In-Reply-To: References: Message-ID: <121BC59A-10A0-4894-A0BD-F47C75457096@freebsd.org> On 26 May 2009, at 16:23, Dmitry Morozovsky wrote: > Dear colleagues, > > any hints/directions to get MosChip 7840 dual port USB to RS232 > adapter > working? In usbdevs output the device is shown as > > port 1 addr 2: high speed, power 100 mA, config 1, product > 0x7840(0x7840), > vendor 0x9710(0x9710), rev 0.01 > > Thanks in advance. I believe this isn't supported under FreeBSD. The Linux driver is mos7840.c. Might not be very hard to do a FreeBSD driver. Regards, -- Rui Paulo -- Rui Paulo -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090530/599acfed/PGP.pgp From xorquewasp at googlemail.com Sat May 30 21:36:46 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Sat May 30 21:36:52 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090530162744.5d77e9d1@bhuda.mired.org> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> Message-ID: <20090530213643.GA1478@logik.internal.network> On 2009-05-30 16:27:44, Mike Meyer wrote: > > The last bit is wrong. Moving a zfs pool between two systems is pretty > straightforward. The configuration information is on the drives; you > just do "zpool import " after plugging them in, and if the mount > point exists, it'll mount it. If the system crashed with the zfs pool > active, you might have to do -f to force an import. Geom is pretty > much the same way, except you can configure it to not write the config > data to disk, thus forcing you to do it manually (what you > expect). I'm not sure geom is as smart if the drives change names, > though. > > RAID support and volume management has come a long way from the days > of ccd and vinum. zfs in particular is a major advance. If you aren't > aware of it's advantages, take the time to read the zfs & zpool man > pages, at the very least, before committing to geom (not that geom > isn't pretty slick in and of itself, but zfs solves a more pressing > problem). > > Hmm. Come to think of it, you ought to be able to use gstrip to stripe > your disks, then put a zpool on that, which should get you the > advantages of zfs with a striped disk. But that does seem odd to me. I'll definitely be looking at ZFS. Thanks for the info. I've never been dead set on any option in particular, it's just that I wasn't aware of anything that would do what I wanted that wasn't just simple RAID0 and manual backups. From mike at reifenberger.com Sun May 31 08:02:38 2009 From: mike at reifenberger.com (Michael Reifenberger) Date: Sun May 31 08:02:45 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090530213643.GA1478@logik.internal.network> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> <20090530213643.GA1478@logik.internal.network> Message-ID: On Sat, 30 May 2009, xorquewasp@googlemail.com wrote: ... > I'll definitely be looking at ZFS. Thanks for the info. > > I've never been dead set on any option in particular, it's just that I > wasn't aware of anything that would do what I wanted that wasn't just > simple RAID0 and manual backups. > Just for the record: ZFS is the only FreeBSD filesystem which ensures your data integrity. I've had dozends of silent block corruptions on SAMSUNG HD103UJ drives. Thanks to 'zfs scrub' I was able to recognize and repair the corruptions. Traditional RAID's only work if the whole drive fails. Bye/2 --- Michael Reifenberger Michael@Reifenberger.com http://www.Reifenberger.com From kraduk at googlemail.com Sun May 31 12:34:31 2009 From: kraduk at googlemail.com (krad) Date: Sun May 31 14:55:52 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090530162744.5d77e9d1@bhuda.mired.org> References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> Message-ID: Please don't whack gstripe and zfs together. It should work but is ugly and you might run into issues. Getting out of them will be harder than a pure zfs solution ZFS does support striping by default across vdevs Eg Zpool create data da1 Zpool add data da2 Would create a striped data set across da1 and da2 Zpool create data mirror da1 da2 Zpool add data mirror da3 da4 This would create a raid 10 across all drives Zpool create data raidz2 da1 da2 da3 da5 Zpool add data raidz2 da6 da7 da8 da9 Would create a raid 60 If you replace the add keyword with attach, mirroring is performed rather than striping Just for fun here is one of the configs off one of our sun x4500 at work, its opensolaris not freebsd, but it is zfs. One whoping big array of ~ 28 TB zpool create -O compression=lzjb -O atime=off data raidz2 c3t0d0 c4t0d0 c8t0d0 c10t0d0 c11t0d0 c3t1d0 c4t1d0 c8t1d0 c9t1d0 c10t1d0 c11t1d0 raidz2 c3t2d0 c4t2d0 c8t2d0 c9t2d0 c11t2d0 c3t3d0 c4t3d0 c8t3d0 c9t3d0 c10t3d0 c11t3d0 raidz2 c3t4d0 c4t4d0 c8t4d0 c10t4d0 c11t4d0 c3t5d0 c4t5d0 c8t5d0 c9t5d0 c10t5d0 c11t5d0 raidz2 c3t6d0 c4t6d0 c8t6d0 c9t6d0 c10t6d0 c11t6d0 c3t7d0 c4t7d0 c9t7d0 c10t7d0 c11t7d0 spare c10t2d0 c8t7d0 $ zpool status pool: archive-2 state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scrub: scrub completed after 11h9m with 0 errors on Sun May 31 01:09:22 2009 config: NAME STATE READ WRITE CKSUM archive-2 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c3t0d0 ONLINE 0 0 0 c4t0d0 ONLINE 0 0 0 c8t0d0 ONLINE 0 0 0 c10t0d0 ONLINE 0 0 0 c11t0d0 ONLINE 0 0 0 c3t1d0 ONLINE 0 0 0 c4t1d0 ONLINE 0 0 0 c8t1d0 ONLINE 0 0 0 c9t1d0 ONLINE 0 0 0 c10t1d0 ONLINE 0 0 0 c11t1d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c4t2d0 ONLINE 0 0 0 c8t2d0 ONLINE 0 0 0 c9t2d0 ONLINE 0 0 0 c11t2d0 ONLINE 0 0 0 c3t3d0 ONLINE 0 0 0 c4t3d0 ONLINE 0 0 0 c8t3d0 ONLINE 0 0 0 c9t3d0 ONLINE 0 0 0 c10t3d0 ONLINE 0 0 0 c11t3d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c3t4d0 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 c8t4d0 ONLINE 0 0 0 c10t4d0 ONLINE 0 0 0 c11t4d0 ONLINE 0 0 0 c3t5d0 ONLINE 0 0 0 c4t5d0 ONLINE 0 0 0 c8t5d0 ONLINE 0 0 0 c9t5d0 ONLINE 0 0 0 c10t5d0 ONLINE 0 0 0 c11t5d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c3t6d0 ONLINE 0 0 0 c4t6d0 ONLINE 0 0 0 c8t6d0 ONLINE 0 0 0 c9t6d0 ONLINE 0 0 0 c10t6d0 ONLINE 0 0 0 c11t6d0 ONLINE 0 0 0 c3t7d0 ONLINE 0 0 0 c4t7d0 ONLINE 0 0 0 c9t7d0 ONLINE 0 0 0 c10t7d0 ONLINE 0 0 0 c11t7d0 ONLINE 0 0 0 spares c10t2d0 AVAIL c8t7d0 AVAIL errors: No known data errors ZFS also check sums all data blocks written to the drive so data integrity is guaranteed. If you are paranoid you can also set it to keep multiple copies of each file. This will eat up loads of disk space so its best to use it sparingly one the most important stuff. You can only do it on a fs basis but this inst a big deal with zfs Zfs create data/important_stuff Zfs set copies=3 data/important_stuff You can also do compression as well, the big example above has this. In the near future encryption and deduping are also getting integrated into zfs. This is probably happening in the next few months on opensolaris, but if you want those features in freebsd I guess it will take at least 6 months after that. With regards to your backup I suggest you definitely look at doing regular fs snapshots. To be real safe, id install the tb drive (probably worth getting another as well as they are cheap) into another machine, and have it in another room, or building if possible. Replicate you data using incremental zfs sends, as this is the most efficient way. You can easily push it through ssh for security as well. Rsync will work fine but you will loose all you zfs fs settings with it as it works at the user level not the fs level. Hope this helps, im really looking forward to zfs maturing on bsd and having pure zfs systems 8) -----Original Message----- From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Mike Meyer Sent: 30 May 2009 21:28 To: xorquewasp@googlemail.com Cc: freebsd-hackers@freebsd.org Subject: Re: Request for opinions - gvinum or ccd? On Sat, 30 May 2009 20:18:40 +0100 xorquewasp@googlemail.com wrote: > > If you're running a 7.X 64-bit system with a couple of GIG of ram, > > expect it to be in service for years without having to reformat the > > disks, and can afford another drive, I'd recommend going to raidz on a > > three-drive system. That will give you close to the size/performance > > of your RAID0 system, but let you lose a disk without losing data. The > > best you can do with zfs on two disks is a mirror, which means write > > throughput will suffer. > > Certainly a lot to think about. > > The system has 12gb currently, with room to upgrade. I currently have > two 500gb drives and one 1tb drive. I wanted the setup to be essentially > two drives striped, backed up onto one larger one nightly. I wanted the > large backup drive to be as "isolated" as possible, eg, in the event of > some catastrophic hardware failure, I can remove it and place it in > another machine without a lot of stressful configuration to recover the > data (not possible with a RAID configuration involving all three drives, > as far as I'm aware). The last bit is wrong. Moving a zfs pool between two systems is pretty straightforward. The configuration information is on the drives; you just do "zpool import " after plugging them in, and if the mount point exists, it'll mount it. If the system crashed with the zfs pool active, you might have to do -f to force an import. Geom is pretty much the same way, except you can configure it to not write the config data to disk, thus forcing you to do it manually (what you expect). I'm not sure geom is as smart if the drives change names, though. RAID support and volume management has come a long way from the days of ccd and vinum. zfs in particular is a major advance. If you aren't aware of it's advantages, take the time to read the zfs & zpool man pages, at the very least, before committing to geom (not that geom isn't pretty slick in and of itself, but zfs solves a more pressing problem). Hmm. Come to think of it, you ought to be able to use gstrip to stripe your disks, then put a zpool on that, which should get you the advantages of zfs with a striped disk. But that does seem odd to me. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From xorquewasp at googlemail.com Sun May 31 20:14:52 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Sun May 31 20:14:59 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> Message-ID: <20090531201445.GA82420@logik.internal.network> On 2009-05-31 13:13:24, krad wrote: > Please don't whack gstripe and zfs together. It should work but is ugly and > you might run into issues. Getting out of them will be harder than a pure > zfs solution Yeah, will be using pure ZFS having read everything I can find on it so far. I was skeptical of ZFS at first as it appeared to have come out of nowhere but it seems it's older (and more mature) than I thought. > ZFS does support striping by default across vdevs > > Eg > > Zpool create data da1 > Zpool add data da2 > > Would create a striped data set across da1 and da2 What kind of performance gain can I expect from this? I'm purely thinking about performance now - the integrity checking stuff of ZFS is a pleasant extra. > Just for fun here is one of the configs off one of our sun x4500 at work, > its opensolaris not freebsd, but it is zfs. One whoping big array of ~ 28 TB Impressive! > Hope this helps, im really looking forward to zfs maturing on bsd and having > pure zfs systems 8) Absolutely. xw From mwm-keyword-freebsdhackers2.e313df at mired.org Sun May 31 20:34:40 2009 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sun May 31 20:34:47 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> Message-ID: <20090531163305.6c7d0cd5@bhuda.mired.org> On Sun, 31 May 2009 13:13:24 +0100 krad wrote: > Please don't whack gstripe and zfs together. It should work but is ugly and > you might run into issues. Getting out of them will be harder than a pure > zfs solution Yeah, I sorta suspected that might be the case. > ZFS does support striping by default across vdevs This isn't documented - at least not in my copies of the manual page. Not being able to find that was the only reason to even consider mixing technologies like that. Thanks, http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From wojtek at wojtek.tensor.gdynia.pl Sun May 31 21:57:37 2009 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sun May 31 21:57:44 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090531201445.GA82420@logik.internal.network> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> <20090531201445.GA82420@logik.internal.network> Message-ID: >> Would create a striped data set across da1 and da2 > > What kind of performance gain can I expect from this? I'm purely thinking > about performance now - the integrity checking stuff of ZFS is a pleasant > extra. with stripping - as much as with gstripe, ZFS do roughly the same. with RAID-z - faster transfer, rougly same IOps as single disk. After i read ZFS papers i know that RAID-z is actually more like RAID-3 not RAID-5. From eitanadlerlist at gmail.com Sun May 31 22:04:27 2009 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun May 31 22:04:33 2009 Subject: pkg_info segfault Revision: 193189 Message-ID: <4A22F7CD.2070500@gmail.com> pkg_info --IwantAcookie Segmentation fault: 11 (core dumped) on FreeBSD 7.2-STABLE i386 From gamato at users.sf.net Sun May 31 22:32:52 2009 From: gamato at users.sf.net (martinko) Date: Sun May 31 22:33:00 2009 Subject: CFT: Graphics support for /boot/loader In-Reply-To: <200902081252.n18CqhZf016155@lurza.secnetix.de> References: <20090208073927.77e2c829@bhuda.mired.org> <200902081252.n18CqhZf016155@lurza.secnetix.de> Message-ID: Oliver Fromme wrote: > Mike Meyer wrote: > > I'm curious - is there a reason that the numbers from the old screen > > have turned into function keys on this one? > > No. That screen shot is an old one. In the current code, > the number keys are used as usual, no function keys. > > In fact, it is not possible to use function keys from the > FORTH code without resorting to dirty hacks. That's what I was thinking about -- whether one could use plain number keys along with function keys and also wanted to ask you to align them with the original text boot menu -- I'm too used to #4 being single user mode. ;-) Cheers and thanks for great work! Martin From kraduk at googlemail.com Sun May 31 22:00:40 2009 From: kraduk at googlemail.com (krad) Date: Sun May 31 23:23:42 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <20090531163305.6c7d0cd5@bhuda.mired.org> References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org> <20090531163305.6c7d0cd5@bhuda.mired.org> Message-ID: <8FFF597348E24C8085BEDE423DC4A6A3@uk.tiscali.intl> Yep it probably isn't clear enough, it does mention stuff about spreading it across vdevs, but doesn't say striped. But that's sun for you. The man page should probably be bsdified more as its more or less pulled from solaris. Note the devices don't look anything like bsd ones (c0t0d0) -----Original Message----- From: Mike Meyer [mailto:mwm@mired.org] Sent: 31 May 2009 21:33 To: krad Cc: 'Mike Meyer'; xorquewasp@googlemail.com; freebsd-hackers@freebsd.org Subject: Re: Request for opinions - gvinum or ccd? On Sun, 31 May 2009 13:13:24 +0100 krad wrote: > Please don't whack gstripe and zfs together. It should work but is ugly and > you might run into issues. Getting out of them will be harder than a pure > zfs solution Yeah, I sorta suspected that might be the case. > ZFS does support striping by default across vdevs This isn't documented - at least not in my copies of the manual page. Not being able to find that was the only reason to even consider mixing technologies like that. Thanks, http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From wojtek at wojtek.tensor.gdynia.pl Sun May 31 23:55:59 2009 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sun May 31 23:56:06 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> <20090531201445.GA82420@logik.internal.network> <0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl> Message-ID: > should really use raidz2 in zfs (or some double parity raid on other > systems) if you are worried about data integrity. The reason being the odds > of the crc checking not detecting an error are much more likely these days. > The extra layer of parity pushes these odds into being much bigger you are right with capacity but not performance. once again - RAIDz is more like RAID-3 not RAID-5, RAIDz2 is somehow like RAID3 with double parity disk. you will get IOps from RAIDz/RAIDz2 set not much more than from single drive, even on reads. But if it's used for mostly linear reading big files you are right. From xorquewasp at googlemail.com Sun May 31 23:59:47 2009 From: xorquewasp at googlemail.com (xorquewasp@googlemail.com) Date: Sun May 31 23:59:53 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: <0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> <20090531201445.GA82420@logik.internal.network> <0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl> Message-ID: <20090531235943.GA77374@logik.internal.network> There is one last thing I'd like clarified. From the zpool manpage: In order to take advantage of these features, a pool must make use of some form of redundancy, using either mirrored or raidz groups. While ZFS supports running in a non-redundant configuration, where each root vdev is simply a disk or file, this is strongly discouraged. A single case of bit corruption can render some or all of your data unavailable. Is this supposed to mean: "ZFS is more fragile than most. If you don't use redundancy, one case of bit corruption will destroy the filesystem" Or: "Hard disks explode often. Use redundancy." From kraduk at googlemail.com Sun May 31 22:13:23 2009 From: kraduk at googlemail.com (krad) Date: Mon Jun 1 00:17:39 2009 Subject: Request for opinions - gvinum or ccd? In-Reply-To: References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> <20090531201445.GA82420@logik.internal.network> Message-ID: <0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl> Yep, its also worth noting that with the capacities of drives these days you should really use raidz2 in zfs (or some double parity raid on other systems) if you are worried about data integrity. The reason being the odds of the crc checking not detecting an error are much more likely these days. The extra layer of parity pushes these odds into being much bigger -----Original Message----- From: Wojciech Puchar [mailto:wojtek@wojtek.tensor.gdynia.pl] Sent: 31 May 2009 22:57 To: xorquewasp@googlemail.com Cc: krad; freebsd-hackers@freebsd.org Subject: Re: Request for opinions - gvinum or ccd? >> Would create a striped data set across da1 and da2 > > What kind of performance gain can I expect from this? I'm purely thinking > about performance now - the integrity checking stuff of ZFS is a pleasant > extra. with stripping - as much as with gstripe, ZFS do roughly the same. with RAID-z - faster transfer, rougly same IOps as single disk. After i read ZFS papers i know that RAID-z is actually more like RAID-3 not RAID-5. From kalinoj1 at iem.pw.edu.pl Sun May 31 22:36:17 2009 From: kalinoj1 at iem.pw.edu.pl (Jedrzej Kalinowski) Date: Mon Jun 1 00:17:54 2009 Subject: Getting rid of pxeboot loader requests for .gz an .bz2 files Message-ID: <4A230199.6090107@iem.pw.edu.pl> Hello, I've got a FreeBSD CURRENT diskless boot set up. In the logs of tftpd-hpa I can see that every and each file from boot.4th to kernel modules is tried to be fetched in compressed format (.gz, .bz2) before it is loaded in its pure form by the pxe-enabled loader. So the sequence is: file.gz->file.bz2->file. I don't have compressed versions of these files in my environment, so I would like to get rid of these requests. Just to make my boot lighter - thus faster. I'm looking for some hints on how to do it in the sources and I only found it on lines 88 - 93 of /usr/src/sys/boot/i386/loader/conf.c, namely: #ifdef LOADER_GZIP_SUPPORT &gzipfs_fsops, #endif #ifdef LOADER_BZIP2_SUPPORT &bzipfs_fsops, #endif Is it the correct idea to: make -C /usr/src/sys/boot -DLOADER_NO_GZIP_SUPPORT -DLOADER_NO_BZIP2_SUPPORT .... ? Or is it the other place I should look for? Thank you -- J?drzej Kalinowski