From rizzo at iet.unipi.it Sun Mar 1 07:25:14 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sun Mar 1 07:25:20 2009 Subject: spliting kernel ipfw source ? (also involves sctp) Message-ID: <20090301153010.GA58942@onelab2.iet.unipi.it> Hi, I am planning to split netinet/ip_fw2.c in a number of smaller files to make it more manageable, and while i do this I would also like to move the files related to ipfw2 (namely ip_fw*c) to a better place. Any objection to moving them to sys/netinet/ipfw2 ? Also, I can't help noticing that sys/netinet/ contains 36 files related to sctp -- wouldn't it be the case to move them (perhaps with the exception of the userland headers) to a separate subdirectory ? (I know the same reasoning would apply to tcp, which has 23 files, but the issue here is that there is 25 years of userland code expecting the tcp headers in netinet/ and moving them would be a nightmare for ports...) cheers luigi From sam at freebsd.org Sun Mar 1 10:15:46 2009 From: sam at freebsd.org (Sam Leffler) Date: Sun Mar 1 10:15:52 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: <20090301153010.GA58942@onelab2.iet.unipi.it> References: <20090301153010.GA58942@onelab2.iet.unipi.it> Message-ID: <49AAC849.9060105@freebsd.org> Luigi Rizzo wrote: > Hi, > I am planning to split netinet/ip_fw2.c in a number of smaller files > to make it more manageable, and while i do this I would also like > to move the files related to ipfw2 (namely ip_fw*c) to a better place. > Any objection to moving them to sys/netinet/ipfw2 ? > > Also, I can't help noticing that sys/netinet/ contains 36 files > related to sctp -- wouldn't it be the case to move them > (perhaps with the exception of the userland headers) > to a separate subdirectory ? > > (I know the same reasoning would apply to tcp, which has 23 files, > but the issue here is that there is 25 years of userland code expecting > the tcp headers in netinet/ and moving them would be a > nightmare for ports...) > I think sctp belongs in it's own directory. I'd vote for just ipfw; the "2" was an artifact of previous code. Sam From julian at elischer.org Sun Mar 1 13:26:44 2009 From: julian at elischer.org (Julian Elischer) Date: Sun Mar 1 13:26:51 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: <20090301153010.GA58942@onelab2.iet.unipi.it> References: <20090301153010.GA58942@onelab2.iet.unipi.it> Message-ID: <49AAFD92.105@elischer.org> Luigi Rizzo wrote: > Hi, > I am planning to split netinet/ip_fw2.c in a number of smaller files > to make it more manageable, and while i do this I would also like > to move the files related to ipfw2 (namely ip_fw*c) to a better place. > Any objection to moving them to sys/netinet/ipfw2 ? > > Also, I can't help noticing that sys/netinet/ contains 36 files > related to sctp -- wouldn't it be the case to move them > (perhaps with the exception of the userland headers) > to a separate subdirectory ? for that matter it would be nice to put ALL teh protocols in their own subdirectories. tcp related files are also multiplying, with things like syncookies etc. > > (I know the same reasoning would apply to tcp, which has 23 files, > but the issue here is that there is 25 years of userland code expecting > the tcp headers in netinet/ and moving them would be a > nightmare for ports...) > > cheers > luigi > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From rpaulo at gmail.com Sun Mar 1 16:41:38 2009 From: rpaulo at gmail.com (Rui Paulo) Date: Sun Mar 1 16:41:45 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: <49AAFD92.105@elischer.org> References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> Message-ID: <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> On 1 Mar 2009, at 21:26, Julian Elischer wrote: > Luigi Rizzo wrote: >> Hi, >> I am planning to split netinet/ip_fw2.c in a number of smaller files >> to make it more manageable, and while i do this I would also like >> to move the files related to ipfw2 (namely ip_fw*c) to a better >> place. >> Any objection to moving them to sys/netinet/ipfw2 ? >> Also, I can't help noticing that sys/netinet/ contains 36 files >> related to sctp -- wouldn't it be the case to move them >> (perhaps with the exception of the userland headers) >> to a separate subdirectory ? > > for that matter it would be nice to put ALL teh protocols in their > own subdirectories. Yes, that would be the perfect scenario, but I don't think that's doable. SCTP can be moved because it hasn't matured enough to cause a "moving nightmare". I vote for "ipfw" like Sam, BTW. -- Rui Paulo -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090302/0279c871/PGP.pgp From perryh at pluto.rain.com Sun Mar 1 18:54:48 2009 From: perryh at pluto.rain.com (perryh@pluto.rain.com) Date: Sun Mar 1 18:55:00 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> Message-ID: <49ab42ca.YZ1mfcCDO0pvdDhv%perryh@pluto.rain.com> Rui Paulo wrote: > On 1 Mar 2009, at 21:26, Julian Elischer wrote: > > Luigi Rizzo wrote: > >> Hi, > >> I am planning to split netinet/ip_fw2.c in a number of smaller > >> files to make it more manageable, and while i do this I would > >> also like to move the files related to ipfw2 (namely ip_fw*c) > >> to a better place. > >> Any objection to moving them to sys/netinet/ipfw2 ? > >> Also, I can't help noticing that sys/netinet/ contains 36 > >> files related to sctp -- wouldn't it be the case to move > >> them (perhaps with the exception of the userland headers) > >> to a separate subdirectory ? > > > > for that matter it would be nice to put ALL teh protocols in > > their own subdirectories. > > Yes, that would be the perfect scenario, but I don't think that's > doable. > > SCTP can be moved because it hasn't matured enough to cause a > "moving nightmare". Perhaps everything can be moved, if hardlinks or symlinks are left in sys/netinet for those parts (mostly .h files, presumably) which have too much legacy to be moved outright. From bugmaster at FreeBSD.org Mon Mar 2 03:07:29 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 2 03:10:36 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200903021106.n22B6mAY057237@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From gnn at neville-neil.com Mon Mar 2 08:50:06 2009 From: gnn at neville-neil.com (George V. Neville-Neil) Date: Mon Mar 2 08:50:12 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: <49ab42ca.YZ1mfcCDO0pvdDhv%perryh@pluto.rain.com> References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> <49ab42ca.YZ1mfcCDO0pvdDhv%perryh@pluto.rain.com> Message-ID: At Sun, 01 Mar 2009 18:22:02 -0800, perryh@pluto.rain.com wrote: > > Rui Paulo wrote: > > On 1 Mar 2009, at 21:26, Julian Elischer wrote: > > > Luigi Rizzo wrote: > > >> Hi, > > >> I am planning to split netinet/ip_fw2.c in a number of smaller > > >> files to make it more manageable, and while i do this I would > > >> also like to move the files related to ipfw2 (namely ip_fw*c) > > >> to a better place. > > >> Any objection to moving them to sys/netinet/ipfw2 ? > > >> Also, I can't help noticing that sys/netinet/ contains 36 > > >> files related to sctp -- wouldn't it be the case to move > > >> them (perhaps with the exception of the userland headers) > > >> to a separate subdirectory ? > > > > > > for that matter it would be nice to put ALL teh protocols in > > > their own subdirectories. > > > > Yes, that would be the perfect scenario, but I don't think that's > > doable. > > > > SCTP can be moved because it hasn't matured enough to cause a > > "moving nightmare". > > Perhaps everything can be moved, if hardlinks or symlinks are > left in sys/netinet for those parts (mostly .h files, presumably) > which have too much legacy to be moved outright. I do not believe we want to go down the path of doing links. I do think that moving protocols is OK, and SCTP is a good candidate. I have cc'd the maintainer in case he's not on arch@. As to the original question about ipfw, if it can e done cleanly then yes that's fine. Best, George From rwatson at FreeBSD.org Mon Mar 2 10:30:28 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Mon Mar 2 10:30:40 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> Message-ID: On Mon, 2 Mar 2009, Rui Paulo wrote: > On 1 Mar 2009, at 21:26, Julian Elischer wrote: > >> Luigi Rizzo wrote: >>> Hi, I am planning to split netinet/ip_fw2.c in a number of smaller files >>> to make it more manageable, and while i do this I would also like to move >>> the files related to ipfw2 (namely ip_fw*c) to a better place. Any >>> objection to moving them to sys/netinet/ipfw2 ? Also, I can't help >>> noticing that sys/netinet/ contains 36 files related to sctp -- wouldn't >>> it be the case to move them (perhaps with the exception of the userland >>> headers) to a separate subdirectory ? >> >> for that matter it would be nice to put ALL teh protocols in their own >> subdirectories. > > Yes, that would be the perfect scenario, but I don't think that's doable. > > SCTP can be moved because it hasn't matured enough to cause a "moving > nightmare". > > I vote for "ipfw" like Sam, BTW. I think massively rearranging things doesn't by us much in terms of beauty, but does give us a lot in terms of hassle, given current assumptions of alignment between the layout of /usr/include and the layout of /usr/src/sys. We'd need to introduce new explicit mappings to install include files in their old locations (which are required by applications), etc. For a change with such a minor benefit, the hassle will be huge. I'm fine with renaming the ipfw .c files and leaving in netinet, but let's not get carried away. Robert N M Watson Computer Laboratory University of Cambridge From rizzo at iet.unipi.it Mon Mar 2 10:57:04 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Mon Mar 2 10:57:23 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> Message-ID: <20090302190157.GA33704@onelab2.iet.unipi.it> On Mon, Mar 02, 2009 at 06:30:27PM +0000, Robert Watson wrote: .. > >>Luigi Rizzo wrote: > >>>Hi, I am planning to split netinet/ip_fw2.c in a number of smaller files > >>>to make it more manageable, and while i do this I would also like to > >>>move the files related to ipfw2 (namely ip_fw*c) to a better place. Any > >>>objection to moving them to sys/netinet/ipfw2 ? Also, I can't help > >>>noticing that sys/netinet/ contains 36 files related to sctp -- wouldn't > >>>it be the case to move them (perhaps with the exception of the userland > >>>headers) to a separate subdirectory ? ... > I think massively rearranging things doesn't by us much in terms of beauty, > but does give us a lot in terms of hassle, given current assumptions of > alignment between the layout of /usr/include and the layout of > /usr/src/sys. We'd need to introduce new explicit mappings to install > include files in their old locations (which are required by applications), > etc. For a change with such a minor benefit, the hassle will be huge. I'm > fine with renaming the ipfw .c files and leaving in netinet, but let's not > get carried away. I suspect that the mention of symlinks (which is not mine and absolutely not what I want to do) triggered some alarms :) I have very clear the potential nightmare with careless moving of files, so I tried to make it clear that public headers are excluded: as you see i wrote "ip_fw*c" and "exception of the userland headers". To further clarify, my plan is the following: - leave ip_fw.h and ip_dummynet.h in /sys/netinet in case userland code is dependent on their location; - create /sys/netinet/ipfw/ to hold the kernel .c files related to ipfw and possibly dummynet (and also their private headers if any); I am not going to touch other people's stuff, though I repeat that sctp would clearly deserve a cleanup -- i doubt userland needs more than one or possibly two headers (one for the basic stuff, one for the fancy and possibly non-portable stuff). Tcp has similar issues especially for recent additions. More than the number of files in /sys/netinet/, i am worried by the amount of private headers that are exposed to userland for no good reason. In fact, some of them are contain: #ifndef _KERNEL #error "no user-serviceable parts inside" #endif Part of this is for historical reasons, but we should try to move to a better structure. Take e.g. tcp_var.h contains the stats (that some userland program may want to have) but also a ton of internal structures (e.g. those for SACK support) that have appeared very recently and have no reason to be there. cheers luigi From delphij at delphij.net Tue Mar 3 15:33:28 2009 From: delphij at delphij.net (Xin LI) Date: Tue Mar 3 15:33:35 2009 Subject: SCCS IDs in old code Message-ID: <49ADBE3D.10508@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'd like to know if there would be any objections if things like: static char sccsid[] = "@(#)rec_utils.c 8.6 (Berkeley) 7/16/94" Be moved into the comment right after the copyright header? Personally I don't see much point keeping 'em just in order to make -DLIBC_SCCS work, since these code changes from time to time and the version number and dates are no longer relevant, perhaps, they should be better just kept in the comment for reference from historical code. Comments? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEARECAAYFAkmtvjwACgkQi+vbBBjt66CR+gCgh80Q1b1+x4BsKq4k6V/v28Io g0YAn3UWTtYShVL9ZPbZKJbUzTwDKlja =yTlj -----END PGP SIGNATURE----- From brde at optusnet.com.au Tue Mar 3 19:20:18 2009 From: brde at optusnet.com.au (Bruce Evans) Date: Tue Mar 3 19:20:25 2009 Subject: SCCS IDs in old code In-Reply-To: <49ADBE3D.10508@delphij.net> References: <49ADBE3D.10508@delphij.net> Message-ID: <20090304141253.V17817@delplex.bde.org> On Tue, 3 Mar 2009, Xin LI wrote: > I'd like to know if there would be any objections if things like: > > static char sccsid[] = "@(#)rec_utils.c 8.6 (Berkeley) 7/16/94" > > Be moved into the comment right after the copyright header? Personally Of course there would be objections. > I don't see much point keeping 'em just in order to make -DLIBC_SCCS > work, since these code changes from time to time and the version number > and dates are no longer relevant, perhaps, they should be better just > kept in the comment for reference from historical code. style(9) requires keeping vendor ids unedited and unmoved and uncommented-out and not attached to the FreeBSD id in `#if 0' blocks. rec_utils.c follows the no-edit no-move rules but not the other 2. Bruce From das at FreeBSD.ORG Tue Mar 3 19:26:41 2009 From: das at FreeBSD.ORG (David Schultz) Date: Tue Mar 3 19:26:46 2009 Subject: SCCS IDs in old code In-Reply-To: <49ADBE3D.10508@delphij.net> References: <49ADBE3D.10508@delphij.net> Message-ID: <20090304032850.GA8551@zim.MIT.EDU> On Tue, Mar 03, 2009, Xin LI wrote: > I'd like to know if there would be any objections if things like: > > static char sccsid[] = "@(#)rec_utils.c 8.6 (Berkeley) 7/16/94" > > Be moved into the comment right after the copyright header? Personally > I don't see much point keeping 'em just in order to make -DLIBC_SCCS > work, since these code changes from time to time and the version number > and dates are no longer relevant, perhaps, they should be better just > kept in the comment for reference from historical code. I don't think it matters much either way, but -DLIBC_SCCS hasn't compiled for a long time, so I doubt anyone will object. From matthew.fleming at isilon.com Wed Mar 4 15:31:48 2009 From: matthew.fleming at isilon.com (Matthew Fleming) Date: Wed Mar 4 15:31:55 2009 Subject: knotes and force unmount Message-ID: <06D5F9F6F655AD4C92E28B662F7F853E0275F637@seaxch09.desktop.isilon.com> At close() time knote_fdclose() will take care of any outstanding knotes for the file by looping over the list, calling kn_fop->f_detach and then knote_drop(). For a umount -f, there may still be open files; these will be handled in vgonel() by a call to VOP_CLOSE(). knotes are untouched in this case. If the filesystem is from a loadable module, then the knotes need to be cleaned up before the module is unloaded to prevent a callout into code that no longer exists, when the file is later closed. So it seems like there is code missing from vgonel() that should be doing this for the case where it calls VOP_CLOSE(). I realize this isn't a problem for ufs (never unloaded because it's used for /) and NFS (no events), but for any other filesystem this seems like a hole in the force unmount story. Is this interpretation correct? It would always be possible to add code to a filesystem's vop_close routine or its unmount routine, but architecturally this seems like something that should be handled at vgonel->VOP_CLOSE time, because it's a problem for any loadable filesystem with events. Thanks, matthew From jroberson at jroberson.net Wed Mar 4 21:44:45 2009 From: jroberson at jroberson.net (Jeff Roberson) Date: Wed Mar 4 21:44:56 2009 Subject: knotes and force unmount In-Reply-To: <06D5F9F6F655AD4C92E28B662F7F853E0275F637@seaxch09.desktop.isilon.com> References: <06D5F9F6F655AD4C92E28B662F7F853E0275F637@seaxch09.desktop.isilon.com> Message-ID: On Wed, 4 Mar 2009, Matthew Fleming wrote: > At close() time knote_fdclose() will take care of any outstanding knotes > for the file by looping over the list, calling kn_fop->f_detach and then > knote_drop(). For a umount -f, there may still be open files; these > will be handled in vgonel() by a call to VOP_CLOSE(). knotes are > untouched in this case. > > If the filesystem is from a loadable module, then the knotes need to be > cleaned up before the module is unloaded to prevent a callout into code > that no longer exists, when the file is later closed. So it seems like > there is code missing from vgonel() that should be doing this for the > case where it calls VOP_CLOSE(). I realize this isn't a problem for ufs > (never unloaded because it's used for /) and NFS (no events), but for > any other filesystem this seems like a hole in the force unmount story. > > Is this interpretation correct? It would always be possible to add code > to a filesystem's vop_close routine or its unmount routine, but > architecturally this seems like something that should be handled at > vgonel->VOP_CLOSE time, because it's a problem for any loadable > filesystem with events. Hi Matthew, This is actually a problem for any VOP with "U U U" in the vnode_if.src specification. Forced unmounts are delayed by vnode locks held but several vops don't hold vnode locks and instead rely on other locking. We have the same problem with loadable syscalls. How can you tell when the code is no longer in use? The right answer would probably be to wait for every thread that is currently in a syscall to leave once before unloading completely. This would solve syscalls and your loadable module case. Although it would be complicated by nfs which doesn't use syscalls but does issue VOPs. I believe a general solution for syscalls/loadable modules would solve this case. Don't you think? I thought Isilon already did something for this? Or is this something you're only discovering since you've merged? If you're interested in working on this I can help you implement a general solution. Thanks, Jeff > > Thanks, > matthew > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From jhb at freebsd.org Thu Mar 5 08:32:29 2009 From: jhb at freebsd.org (John Baldwin) Date: Thu Mar 5 08:32:36 2009 Subject: [PATCH] Set SYSDIR for modules built with kernel Message-ID: <200903051041.10678.jhb@freebsd.org> One of the things each kernel module build has to do is figure out where the source tree it belongs to lives. /usr/share/mk/bsd.kmod.mk uses a heuristic where it walks up N levels of the tree looking for a kernel source directory. If that fails it falls back to /sys or /usr/src/sys. One side effect of this (besides lots of directory lookups during a build as each kernel module build stage has to do this) is that if CURRENT adds an even deeper level of nesting (such as with the recent ata chipset modules) then you can't build a kernel with those modules until /usr/share/mk/bsd.kmod.mk on your build machine has been updated. This means you can't build an 8.0 kernel on 7.0 simply because ata chipset modules don't find the right headers. However, the SYSDIR is already known during a kernel build (we depend on this to make modules built in ports DTRT when they are rebuilt via buildkernel). I just changed the kernel build Makefile to always set SYSDIR. With this change, I was able to build an 8.0 kernel + modules on a 7.1-ish machine with an unpatched /usr/share/mk/bsd.kmod.mk. This should also have the side effect of reducing the number of pathname lookup operations done during a kernel build with modules. Thoughts? --- //depot/projects/smpng/sys/conf/kern.post.mk 2009/01/15 22:41:24 +++ //depot/user/jhb/lock/conf/kern.post.mk 2009/03/04 21:25:21 @@ -12,7 +12,8 @@ .if defined(DESTDIR) MKMODULESENV+= DESTDIR="${DESTDIR}" .endif -MKMODULESENV+= KERNBUILDDIR="${.CURDIR}" +SYSDIR?= ${S:C;^[^/];${.CURDIR}/&;} +MKMODULESENV+= KERNBUILDDIR="${.CURDIR}" SYSDIR="${SYSDIR}" .MAIN: all @@ -29,7 +30,6 @@ # Handle out of tree ports .if !defined(NO_MODULES) && defined(PORTS_MODULES) -SYSDIR?= ${S:C;^[^/];${.CURDIR}/&;} PORTSMODULESENV=SYSDIR=${SYSDIR} .for __target in all install reinstall clean ${__target}: ports-${__target} -- John Baldwin From rpaulo at gmail.com Fri Mar 6 02:48:37 2009 From: rpaulo at gmail.com (Rui Paulo) Date: Fri Mar 6 02:48:44 2009 Subject: [PATCH] Set SYSDIR for modules built with kernel In-Reply-To: <200903051041.10678.jhb@freebsd.org> References: <200903051041.10678.jhb@freebsd.org> Message-ID: <7B12E246-F284-4436-BDAD-5263769E339E@gmail.com> On 5 Mar 2009, at 15:41, John Baldwin wrote: > One of the things each kernel module build has to do is figure out > where the > source tree it belongs to lives. /usr/share/mk/bsd.kmod.mk uses a > heuristic > where it walks up N levels of the tree looking for a kernel source > directory. > If that fails it falls back to /sys or /usr/src/sys. One side > effect of this > (besides lots of directory lookups during a build as each kernel > module build > stage has to do this) is that if CURRENT adds an even deeper level > of nesting > (such as with the recent ata chipset modules) then you can't build a > kernel > with those modules until /usr/share/mk/bsd.kmod.mk on your build > machine has > been updated. This means you can't build an 8.0 kernel on 7.0 > simply because > ata chipset modules don't find the right headers. > > However, the SYSDIR is already known during a kernel build (we > depend on this > to make modules built in ports DTRT when they are rebuilt via > buildkernel). > I just changed the kernel build Makefile to always set SYSDIR. With > this > change, I was able to build an 8.0 kernel + modules on a 7.1-ish > machine with > an unpatched /usr/share/mk/bsd.kmod.mk. This should also have the > side > effect of reducing the number of pathname lookup operations done > during a > kernel build with modules. > > Thoughts? I think this should go in. Thanks. -- Rui Paulo -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090306/9fdf3552/PGP.pgp From vadim_nuclight at mail.ru Fri Mar 6 04:40:59 2009 From: vadim_nuclight at mail.ru (Vadim Goncharov) Date: Fri Mar 6 04:41:06 2009 Subject: spliting kernel ipfw source ? (also involves sctp) References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> <20090302190157.GA33704@onelab2.iet.unipi.it> Message-ID: Hi Luigi Rizzo! On Mon, 2 Mar 2009 20:01:57 +0100; Luigi Rizzo wrote about 'Re: spliting kernel ipfw source ? (also involves sctp)': >>>>>Hi, I am planning to split netinet/ip_fw2.c in a number of smaller files >>>>>to make it more manageable, and while i do this I would also like to >>>>>move the files related to ipfw2 (namely ip_fw*c) to a better place. Any >>>>>objection to moving them to sys/netinet/ipfw2 ? Also, I can't help [...] > To further clarify, my plan is the following: > - leave ip_fw.h and ip_dummynet.h in /sys/netinet in case > userland code is dependent on their location; > - create /sys/netinet/ipfw/ to hold the kernel .c files related to > ipfw and possibly dummynet (and also their private headers if any); Exactly that and nothing more? I'm currently working on extending ipfw for Foundation with userland interface will be changed (and I thinking about introducing modules), what else do you plan to do? -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From rizzo at iet.unipi.it Fri Mar 6 08:21:29 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Fri Mar 6 08:21:34 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> <20090302190157.GA33704@onelab2.iet.unipi.it> Message-ID: <20090306161028.GA12322@onelab2.iet.unipi.it> On Fri, Mar 06, 2009 at 12:40:47PM +0000, Vadim Goncharov wrote: > Hi Luigi Rizzo! > > On Mon, 2 Mar 2009 20:01:57 +0100; Luigi Rizzo wrote about 'Re: spliting kernel ipfw source ? (also involves sctp)': > > >>>>>Hi, I am planning to split netinet/ip_fw2.c in a number of smaller files > >>>>>to make it more manageable, and while i do this I would also like to > >>>>>move the files related to ipfw2 (namely ip_fw*c) to a better place. Any > >>>>>objection to moving them to sys/netinet/ipfw2 ? Also, I can't help > > [...] > > > To further clarify, my plan is the following: > > > - leave ip_fw.h and ip_dummynet.h in /sys/netinet in case > > userland code is dependent on their location; > > - create /sys/netinet/ipfw/ to hold the kernel .c files related to > > ipfw and possibly dummynet (and also their private headers if any); > > Exactly that and nothing more? I'm currently working on extending ipfw for > Foundation with userland interface will be changed (and I thinking about > introducing modules), what else do you plan to do? Generally speaking, the kernel/userland interface will remain unchanged both at binary and source level, which means no backward incompatible changes in the sockopt numbers and messages, and no changes in the location and userland-visible parts of the headers. In practical terms, ip_fw.h might lose the definition of struct ip_fw_args, or the prototypes for the various kernel functions. The #ifdef _KERNEL part of ip_dummynet.h should also go to some other file. If you want to contact me, on the list or offline, to discuss what you want to do or what kind of 'modules' (kernel or userland ?) are you thinking about, i'd be more than happy to help. cheers luigi From das at FreeBSD.ORG Sat Mar 7 02:27:32 2009 From: das at FreeBSD.ORG (David Schultz) Date: Sat Mar 7 02:27:39 2009 Subject: C99 inlines Message-ID: <20090307103138.GA34456@zim.MIT.EDU> I'd like the gcc in our tree to use the C99 semantics instead of GNU semantics for inline functions in C99 and GNU99 mode. The following patch implements this behavior. It is based on a snapshot of the gcc 4.3 branch from March 2007, prior to the GPLv3 switch. http://www.freebsd.org/~das/c99inline.diff I ran the inline function regression tests from the latest gcc snapshot and fixed two bugs. All the tests pass now, except that programs that bogusly try to instantiate an extern definition of an inline function inside another function cause an ICE. The object files produced by buildworld on amd64 in gnu89 mode are identical with and without this patch (excepting things like OpenSSL that include the compilation time in the binary), so it should be safe for the base system. (One small bugfix in lib/msun and one change in reiserfs are required to make universe without errors.) Ports may be affected if they are compiled with -std=c99 or -std=gnu99 and they're not actually prepared to handle it. The patch also enables some additional error checking (e.g., gcc now complains about inline functions that are declared but never defined), so that could lead to a few build errors. I expect that any port that fails as a result of this patch would fail under gcc 4.3.0 anyway. I'll try to coordinate with portmgr if it looks like there are major issues. What do people think about this? From rizzo at iet.unipi.it Sat Mar 7 03:19:45 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sat Mar 7 03:19:51 2009 Subject: C99 inlines In-Reply-To: <20090307103138.GA34456@zim.MIT.EDU> References: <20090307103138.GA34456@zim.MIT.EDU> Message-ID: <20090307112449.GB45088@onelab2.iet.unipi.it> On Sat, Mar 07, 2009 at 05:31:38AM -0500, David Schultz wrote: > I'd like the gcc in our tree to use the C99 semantics instead of GNU > semantics for inline functions in C99 and GNU99 mode. The following > patch implements this behavior. It is based on a snapshot of the gcc > 4.3 branch from March 2007, prior to the GPLv3 switch. > > http://www.freebsd.org/~das/c99inline.diff ... > What do people think about this? would you be able to provide a short summary of how the C99 and GNU99 semantics differ, so that people will have a easier time figuring out how to handle the change ? Especially, is there anything that a developer should worry about, or except from some corner cases the switch will (and should) be completely transparent to the average developer without a specific interest in compilers ? I know what I am asking is probably in the diff you supplied, but it's a bit difficult to follow there thanks luigi From sam at freebsd.org Sat Mar 7 09:39:06 2009 From: sam at freebsd.org (Sam Leffler) Date: Sat Mar 7 09:39:12 2009 Subject: C99 inlines In-Reply-To: <20090307103138.GA34456@zim.MIT.EDU> References: <20090307103138.GA34456@zim.MIT.EDU> Message-ID: <49B2B139.6010104@freebsd.org> David Schultz wrote: > I'd like the gcc in our tree to use the C99 semantics instead of GNU > semantics for inline functions in C99 and GNU99 mode. The following > patch implements this behavior. It is based on a snapshot of the gcc > 4.3 branch from March 2007, prior to the GPLv3 switch. > Why? I can imagine it improving portability to other toolchains like LLVM, ICC, etc. I'm not opposed but seems like you should be clear. Sam From rdivacky at FreeBSD.ORG Sat Mar 7 11:06:33 2009 From: rdivacky at FreeBSD.ORG (Roman Divacky) Date: Sat Mar 7 11:06:39 2009 Subject: C99 inlines In-Reply-To: <20090307103138.GA34456@zim.MIT.EDU> References: <20090307103138.GA34456@zim.MIT.EDU> Message-ID: <20090307190620.GA45956@freebsd.org> On Sat, Mar 07, 2009 at 05:31:38AM -0500, David Schultz wrote: > I'd like the gcc in our tree to use the C99 semantics instead of GNU > semantics for inline functions in C99 and GNU99 mode. The following > patch implements this behavior. It is based on a snapshot of the gcc > 4.3 branch from March 2007, prior to the GPLv3 switch. I definitely support this... thnx for working on it! From das at FreeBSD.ORG Sat Mar 7 11:16:50 2009 From: das at FreeBSD.ORG (David Schultz) Date: Sat Mar 7 11:16:57 2009 Subject: C99 inlines In-Reply-To: <20090307112449.GB45088@onelab2.iet.unipi.it> References: <20090307103138.GA34456@zim.MIT.EDU> <20090307112449.GB45088@onelab2.iet.unipi.it> Message-ID: <20090307192100.GA36158@zim.MIT.EDU> On Sat, Mar 07, 2009, Luigi Rizzo wrote: > On Sat, Mar 07, 2009 at 05:31:38AM -0500, David Schultz wrote: > > I'd like the gcc in our tree to use the C99 semantics instead of GNU > > semantics for inline functions in C99 and GNU99 mode. The following > > patch implements this behavior. It is based on a snapshot of the gcc > > 4.3 branch from March 2007, prior to the GPLv3 switch. > > > > http://www.freebsd.org/~das/c99inline.diff > ... > > What do people think about this? > > would you be able to provide a short summary of how the C99 and > GNU99 semantics differ, so that people will have a easier time > figuring out how to handle the change ? > > Especially, is there anything that a developer should worry about, > or except from some corner cases the switch will (and should) be > completely transparent to the average developer without a specific > interest in compilers ? > > I know what I am asking is probably in the diff you supplied, > but it's a bit difficult to follow there Good point; it isn't clear at all from the diffs. Basically it all has to do with the interpretation of `inline' versus `extern inline'. Historical gcc semantics are incompatible with C99, but this was fixed in gcc 4.3. This patch essentially backports that patch from a development snapshot of gcc 4.3 prior to the GPLv3 switch. This page summarizes the differences pretty well, but ignore the last section: http://www.greenend.org.uk/rjk/2003/03/inline.html The most important point for developers is that the one kind of inline where gcc and C99 agree is `static inline', so if you use that, you won't have to worry. Otherwise, the semantics you get depend on many things: - gcc prior to 4.1.3: You always get GNU inline rules. - gcc 4.1.3 to 4.2.1 (the one in the tree): In -std=gnu89 mode, you get GNU inline rules. In -std=gnu99 or -std=c99 mode, you get a warning that GNU inline rules aren't implemented. - gcc 4.3+ or gcc with this patch: In -std=gnu89 mode, you get GNU inline rules. In -std=gnu99 or -std=c99 mode, you get C99 rules. You can use __gnu89_inline (defined in sys/cdefs.h) or -fgnu-inline to force the GNU rules regardless, but you can't get the C99 rules with an unpatched pre-4.3 compiler. In gcc 4.2+, the macros __GNUC_{GNU,STDC}_INLINE__ tell you what you're getting. With this patch, you basically get the semantics you asked for instead of a warning that they're not implemented. If you weren't getting warnings with gcc 4.2 before, then nothing changes. Another minor consequence of this patch is that it enables a few new warnings and errors involving non-static inlines. For instance, if you have an extern inline function that references a static variable, such that the function might refer to a *different* version of the variable depending on whether it's inlined or not, gcc will now tell you you're trying to do something bogus. It will also complain if you declare a function inline but never define it. From das at FreeBSD.ORG Sat Mar 7 23:09:25 2009 From: das at FreeBSD.ORG (David Schultz) Date: Sat Mar 7 23:09:32 2009 Subject: C99 inlines In-Reply-To: <49B2B139.6010104@freebsd.org> References: <20090307103138.GA34456@zim.MIT.EDU> <49B2B139.6010104@freebsd.org> Message-ID: <20090308070924.GA39236@zim.MIT.EDU> On Sat, Mar 07, 2009, Sam Leffler wrote: > David Schultz wrote: > >I'd like the gcc in our tree to use the C99 semantics instead of GNU > >semantics for inline functions in C99 and GNU99 mode. The following > >patch implements this behavior. It is based on a snapshot of the gcc > >4.3 branch from March 2007, prior to the GPLv3 switch. > > > > Why? I can imagine it improving portability to other toolchains like > LLVM, ICC, etc. > > I'm not opposed but seems like you should be clear. My main motivation is that currently there's no easy way to use non-static inline functions that works with both gcc and other compilers. Furthermore, even GNU wants to move to using the C99 semantics by default. Once that happens, continuing to be dependent upon the old GNU inline semantics is likely to cause porting headaches. From andrew-freebsd at areilly.bpc-users.org Mon Mar 9 06:28:04 2009 From: andrew-freebsd at areilly.bpc-users.org (Andrew Reilly) Date: Mon Mar 9 06:28:11 2009 Subject: C99 inlines In-Reply-To: <20090308070924.GA39236@zim.MIT.EDU> References: <20090307103138.GA34456@zim.MIT.EDU> <49B2B139.6010104@freebsd.org> <20090308070924.GA39236@zim.MIT.EDU> Message-ID: <20090309105541.GA2464@duncan.reilly.home> On Sun, Mar 08, 2009 at 03:09:24AM -0400, David Schultz wrote: > My main motivation is that currently there's no easy way to use > non-static inline functions that works with both gcc and other > compilers. Please pardon my ignorance: what *is* non-static inline behaviour? I've only ever used static inlines myself: they're the only sort that make sense (to me), in the world of standard C static compilation and linkage. What happens elsewhen? Does the compiler generate a "real" function with an exportable name that can be linked-to? Why would you want to do that, when that's what perfectly ordinary functions do? I can't imagine an extern inline meaning anything useful unless one can do LLVM-style link-time optimization. Is that on the cards? > Furthermore, even GNU wants to move to using the C99 > semantics by default. Once that happens, continuing to be > dependent upon the old GNU inline semantics is likely to cause > porting headaches. Well, we don't want to be depending on non-standard semantics, if we can help it. Sure. Are we? Where? Cheers, -- Andrew From stefan at fafoe.narf.at Mon Mar 9 06:58:21 2009 From: stefan at fafoe.narf.at (Stefan Farfeleder) Date: Mon Mar 9 06:58:27 2009 Subject: C99 inlines In-Reply-To: <20090309105541.GA2464@duncan.reilly.home> References: <20090307103138.GA34456@zim.MIT.EDU> <49B2B139.6010104@freebsd.org> <20090308070924.GA39236@zim.MIT.EDU> <20090309105541.GA2464@duncan.reilly.home> Message-ID: <20090309134041.GA23152@lizard.fafoe.narf.at> On Mon, Mar 09, 2009 at 09:55:41PM +1100, Andrew Reilly wrote: > On Sun, Mar 08, 2009 at 03:09:24AM -0400, David Schultz wrote: > > My main motivation is that currently there's no easy way to use > > non-static inline functions that works with both gcc and other > > compilers. > > Please pardon my ignorance: what *is* non-static inline > behaviour? I've only ever used static inlines myself: they're > the only sort that make sense (to me), in the world of standard > C static compilation and linkage. What happens elsewhen? Does > the compiler generate a "real" function with an exportable name > that can be linked-to? Why would you want to do that, when > that's what perfectly ordinary functions do? I can't imagine an > extern inline meaning anything useful unless one can do > LLVM-style link-time optimization. Is that on the cards? With static inline functions you end up with a copy in each object file where the compiler decided to not inline at least one call, with an inline function with external linkage all these copies are coalesced to a single one. From bugmaster at FreeBSD.org Mon Mar 9 10:15:02 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 9 10:15:37 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200903091715.n29HF0o3045179@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From ambrisko at ambrisko.com Tue Mar 10 10:02:22 2009 From: ambrisko at ambrisko.com (Doug Ambrisko) Date: Tue Mar 10 10:02:29 2009 Subject: rtld enhancement to add osversion sub-directory search Message-ID: <200903101702.n2AH2LJK086298@ambrisko.com> Doug Ambrisko writes: | Brooks Davis writes: | | On Fri, Feb 13, 2009 at 01:08:56PM -0800, Doug Ambrisko wrote: | | > David Schultz writes: | | > | On Thu, Feb 12, 2009, Doug Ambrisko wrote: | | > | > Kostik Belousov writes: | | > | > | There is a popular feature, unfortunately, not supported by FreeBSD | | > | > | ld.so, called Dynamic String Tokens, see | | > | > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view | | > | > | | | > | > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, | | > | > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT | | > | > | without serious conflicts. | | > | > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch | | > | > | | > | > That is an interesting feature, however, it almost seems backwards for | | > | > me if I understand it correctly. I need old binaries to find the library | | > | > it was built with and not new ones based on the base OS. The plus that | | > | > I see with their feature is for a library that has been optimized for a | | > | > specific type of CPU etc. | | > | | | > | The Solaris rtld features are very useful when you want to | | > | export a volume with a bunch of apps over NFS, and the clients are | | > | running different releases or different architectures. It can | | > | probably also solve your problem if you bother to place different | | > | library versions in different directories and set your library | | > | path appropriately. | | > | | > Unless I missed something it seems to be an inverse feature to | | > what I want since it expands based on the current kernel's | | > uname type results. I need it to expand based on the original OS | | > that it was built on. I'll have to see if my Solaris box at work | | > has this feature or not. I can't change LD_LIBRARY_PATH etc. | | > type things. Doing ldconfig -m of various things doesn't help | | > since that can find the wrong one. My idea was to do something | | > like what happens for 32bit on 64bit and Linux on FreeBSD in that | | > it looks at osversion specific places then the standard. | | | | While I commented on the Dynamic String Tokens and think it might be | | useful, that's a distraction for your proposal. I think you proposal | | sounds useful as well. If it was in 7 by the time I upgrade my cluster | | from 6, I'd probably use it to ease the transition. | | Since my scheme seems useful, then I'll proceed to clean up my patch | some more then send it out for people to play with and comment on. Below is the patch, got tied up with other issues. It include the LD32_ then LD_ search as well. I use LD_MAP_UNKNOWN_OS_VERSION to set the default version if the .note version is missing. Once this gets into shape then I'll update the man page. Thanks for looking, Doug A. Index: rtld.c =================================================================== RCS file: /cvs/src/libexec/rtld-elf/rtld.c,v retrieving revision 1.128 diff -u -p -w -r1.128 rtld.c --- rtld.c 10 Oct 2008 00:16:32 -0000 1.128 +++ rtld.c 24 Feb 2009 00:05:10 -0000 @@ -103,7 +103,7 @@ static bool is_exported(const Elf_Sym *) static void linkmap_add(Obj_Entry *); static void linkmap_delete(Obj_Entry *); static int load_needed_objects(Obj_Entry *); -static int load_preload_objects(void); +static int load_preload_objects(char *); static Obj_Entry *load_object(const char *, const Obj_Entry *); static Obj_Entry *obj_from_addr(const void *); static void objlist_call_fini(Objlist *, int *lockstate); @@ -157,6 +157,11 @@ static char *ld_debug; /* Environment v static char *ld_library_path; /* Environment variable for search path */ static char *ld_preload; /* Environment variable for libraries to load first */ +#ifdef COMPAT_32BIT +static char *ld32_library_path; /* Environment variable for search path */ +static char *ld32_preload; /* Environment variable for libraries to + load first */ +#endif static char *ld_tracing; /* Called from ldd to print libs */ static char *ld_utrace; /* Use utrace() to log events. */ static Obj_Entry *obj_list; /* Head of linked list of shared objects */ @@ -284,6 +289,15 @@ ld_utrace_log(int event, void *handle, v } /* + * OS Versions to try. Will iterate through the string list looking for + * sub-directories with that name. When a NULL is found it will stop + * iterating through the list. To search the standard location a "" + * version needs to be listed before the NULL. + */ +static char *os_version_try[] = {NULL, NULL, NULL}; +#define OS_VERSION_LEN 10 /* max ascii size of int */ +#define OS_VERSION_MAJOR 100000 /* divisor to get os major */ +/* * Main entry point for dynamic linking. The first argument is the * stack pointer. The stack is expected to be laid out as described * in the SVR4 ABI specification, Intel 386 Processor Supplement. @@ -363,6 +377,13 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_ unsetenv(LD_ "LIBRARY_PATH"); unsetenv(LD_ "LIBMAP_DISABLE"); unsetenv(LD_ "DEBUG"); +#ifdef COMPAT_32BIT + unsetenv(LD_32_ "PRELOAD"); + unsetenv(LD_32_ "LIBMAP"); + unsetenv(LD_32_ "LIBRARY_PATH"); + unsetenv(LD_32_ "LIBMAP_DISABLE"); + unsetenv(LD_32_ "DEBUG"); +#endif } ld_debug = getenv(LD_ "DEBUG"); libmap_disable = getenv(LD_ "LIBMAP_DISABLE") != NULL; @@ -373,6 +394,23 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_ (ld_library_path != NULL) || (ld_preload != NULL); ld_tracing = getenv(LD_ "TRACE_LOADED_OBJECTS"); ld_utrace = getenv(LD_ "UTRACE"); +#ifdef COMPAT_32BIT + ld_debug = getenv(LD_32_ "DEBUG"); + libmap_disable = getenv(LD_32_ "LIBMAP_DISABLE") != NULL; + libmap_override = getenv(LD_32_ "LIBMAP"); + ld32_library_path = getenv(LD_32_ "LIBRARY_PATH"); + ld32_preload = getenv(LD_32_ "PRELOAD"); + if (ld32_library_path == NULL) + ld32_library_path = ld_library_path; + if (ld32_preload == NULL) + ld32_preload = ld_preload; + dangerous_ld_env = libmap_disable || (libmap_override != NULL) || + (ld32_library_path != NULL) || (ld32_preload != NULL) || + (ld_library_path != NULL) || (ld_preload != NULL); + dangerous_ld_env = 0; + ld_tracing = getenv(LD_32_ "TRACE_LOADED_OBJECTS"); + ld_utrace = getenv(LD_32_ "UTRACE"); +#endif if (ld_debug != NULL && *ld_debug != '\0') debug = 1; @@ -409,6 +447,25 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_ if ((obj_main = digest_phdr(phdr, phnum, entry, argv0)) == NULL) die(); } + if (obj_main->ndesc != NULL && strcmp(obj_main->ndesc, "FreeBSD") == 0) { + os_version_try[0] = xmalloc(OS_VERSION_LEN); + snprintf(os_version_try[0], OS_VERSION_LEN, "%d", obj_main->nver); + os_version_try[1] = xmalloc(OS_VERSION_LEN); + snprintf(os_version_try[1], OS_VERSION_LEN, "%d", obj_main->nver + / OS_VERSION_MAJOR); + os_version_try[2] = ""; + } else { + /* Deal with FreeBSD 6 binaries that are missing version */ + os_version_try[0] = getenv(LD_ "MAP_UNKNOWN_OS_VERSION"); + if (os_version_try[0] == NULL) { + os_version_try[0] = ""; + } else { + os_version_try[1] = xmalloc(OS_VERSION_LEN); + snprintf(os_version_try[1], OS_VERSION_LEN, "%d", + atoi(os_version_try[0]) / OS_VERSION_MAJOR); + os_version_try[2] = ""; + } + } obj_main->path = xstrdup(argv0); obj_main->mainprog = true; @@ -447,8 +504,14 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_ libmap_disable = (bool)lm_init(libmap_override); dbg("loading LD_PRELOAD libraries"); - if (load_preload_objects() == -1) +#ifdef COMPAT_32BIT + if (load_preload_objects(ld32_preload) == -1) + if (load_preload_objects(ld_preload) == -1) die(); +#else + if (load_preload_objects(ld_preload) == -1) + die(); +#endif preload_tail = obj_tail; dbg("loading needed objects"); @@ -861,12 +924,24 @@ digest_phdr(const Elf_Phdr *phdr, int ph Obj_Entry *obj; const Elf_Phdr *phlimit = phdr + phnum; const Elf_Phdr *ph; + const Elf_Note *nh; int nsegs = 0; obj = obj_new(); for (ph = phdr; ph < phlimit; ph++) { switch (ph->p_type) { + case PT_NOTE: + nh = ((const Elf_Note *)(const Elf_Phdr *)ph->p_vaddr); + if (nh->n_type == NT_PRSTATUS) { + obj->ndesc = &((char *)((const Elf_Phdr *)ph->p_vaddr)) + [sizeof(nh->n_namesz) + sizeof(nh->n_descsz) + sizeof(nh->n_type)]; + obj->nver = *(u_int32_t *)(&((const char *)((const Elf_Phdr *)ph->p_vaddr)) + [sizeof(nh->n_namesz) + sizeof(nh->n_descsz) + sizeof(nh->n_type) + + ((const Elf_Note *)(const Elf_Phdr *)ph->p_vaddr)->n_namesz]); + } + break; + case PT_PHDR: if ((const Elf_Phdr *)ph->p_vaddr != phdr) { _rtld_error("%s: invalid PT_PHDR", path); @@ -994,6 +1069,9 @@ find_library(const char *xname, const Ob { char *pathname; char *name; + int i; + char *temp; + char new_path[PATH_MAX]; if (strchr(xname, '/') != NULL) { /* Hard coded pathname */ if (xname[0] != '/' && !trust) { @@ -1001,7 +1079,24 @@ find_library(const char *xname, const Ob xname); return NULL; } - return xstrdup(xname); + for (i = 0; os_version_try[i] != NULL; i++) { + if (os_version_try[i][0] == '\0') + temp = (char *)xname; + else { + bzero(new_path, sizeof(new_path)); + bcopy(xname, new_path, strlen(xname) + 1); + temp = rindex(new_path, '/'); + temp++; + *temp = '\0'; + strncat(new_path, os_version_try[i], sizeof(new_path)); + temp = rindex(xname, '/'); + strncat(new_path, temp, sizeof(new_path)); + temp = new_path; + } + if (access(temp, F_OK) == 0) + return xstrdup(temp); + } + return NULL; } if (libmap_disable || (refobj == NULL) || @@ -1010,6 +1105,14 @@ find_library(const char *xname, const Ob dbg(" Searching for \"%s\"", name); +#ifdef COMPAT_32BIT + if ((pathname = search_library_path(name, ld32_library_path)) != NULL || + (refobj != NULL && + (pathname = search_library_path(name, refobj->rpath)) != NULL) || + (pathname = search_library_path(name, gethints())) != NULL || + (pathname = search_library_path(name, STANDARD_LIBRARY_PATH)) != NULL) + return pathname; +#endif if ((pathname = search_library_path(name, ld_library_path)) != NULL || (refobj != NULL && (pathname = search_library_path(name, refobj->rpath)) != NULL) || @@ -1311,9 +1414,9 @@ load_needed_objects(Obj_Entry *first) } static int -load_preload_objects(void) +load_preload_objects(char *preload) { - char *p = ld_preload; + char *p = preload; static const char delim[] = " \t:;"; if (p == NULL) @@ -1326,6 +1429,7 @@ load_preload_objects(void) savech = p[len]; p[len] = '\0'; + if (find_library(p, NULL) != NULL) if (load_object(p, NULL) == NULL) return -1; /* XXX - cleanup */ p[len] = savech; @@ -1731,26 +1835,36 @@ static void * try_library_path(const char *dir, size_t dirlen, void *param) { struct try_library_args *arg; + char *version = NULL; + int temp_dirlen, i; arg = param; if (*dir == '/' || trust) { char *pathname; - if (dirlen + 1 + arg->namelen + 1 > arg->buflen) - return (NULL); - + for (i = 0; os_version_try[i] != NULL; i++) { + if (dirlen + 1 + arg->namelen + 1 + strlen(os_version_try[i]) + 1 > arg->buflen) + continue; + version = os_version_try[i]; + temp_dirlen = dirlen; pathname = arg->buffer; - strncpy(pathname, dir, dirlen); - pathname[dirlen] = '/'; - strcpy(pathname + dirlen + 1, arg->name); - + pathname[0] = '\000'; + strncpy(pathname, dir, temp_dirlen); + pathname[temp_dirlen] = '/'; + if (version[0] != '\000') { + strcpy(pathname + temp_dirlen + 1, version); + temp_dirlen += strlen(os_version_try[i]) + 1; + pathname[temp_dirlen] = '/'; + } + strcpy(pathname + temp_dirlen + 1, arg->name); dbg(" Trying \"%s\"", pathname); if (access(pathname, F_OK) == 0) { /* We found it */ - pathname = xmalloc(dirlen + 1 + arg->namelen + 1); + pathname = xmalloc(temp_dirlen + 1 + arg->namelen + 1); strcpy(pathname, arg->buffer); return (pathname); } } + } return (NULL); } @@ -2729,8 +2843,9 @@ trace_loaded_objects(Obj_Entry *obj) printf("%s:\n", obj->path); for (needed = obj->needed; needed; needed = needed->next) { if (needed->obj != NULL) { - if (needed->obj->traced && !list_containers) + if (needed->obj->traced && !list_containers) { continue; + } needed->obj->traced = true; path = needed->obj->path; } else Index: rtld.h =================================================================== RCS file: /cvs/src/libexec/rtld-elf/rtld.h,v retrieving revision 1.39 diff -u -p -w -r1.39 rtld.h --- rtld.h 4 Apr 2008 20:59:26 -0000 1.39 +++ rtld.h 24 Feb 2009 00:05:10 -0000 @@ -45,7 +45,7 @@ #define _PATH_ELF_HINTS "/var/run/ld-elf32.so.hints" /* For running 32 bit binaries */ #define STANDARD_LIBRARY_PATH "/lib32:/usr/lib32" -#define LD_ "LD_32_" +#define LD_32_ "LD_32_" #endif #ifndef STANDARD_LIBRARY_PATH @@ -223,6 +223,9 @@ typedef struct Struct_Obj_Entry { dev_t dev; /* Object's filesystem's device */ ino_t ino; /* Object's inode number */ void *priv; /* Platform-dependant */ + + char *ndesc; /* Note Description */ + u_int32_t nver; /* FreeBSD Version */ } Obj_Entry; #define RTLD_MAGIC 0xd550b87a From das at FreeBSD.ORG Tue Mar 10 11:44:07 2009 From: das at FreeBSD.ORG (David Schultz) Date: Tue Mar 10 11:44:14 2009 Subject: C99 inlines In-Reply-To: <20090309105541.GA2464@duncan.reilly.home> References: <20090307103138.GA34456@zim.MIT.EDU> <49B2B139.6010104@freebsd.org> <20090308070924.GA39236@zim.MIT.EDU> <20090309105541.GA2464@duncan.reilly.home> Message-ID: <20090310184433.GA72420@zim.MIT.EDU> On Mon, Mar 09, 2009, Andrew Reilly wrote: > On Sun, Mar 08, 2009 at 03:09:24AM -0400, David Schultz wrote: > > My main motivation is that currently there's no easy way to use > > non-static inline functions that works with both gcc and other > > compilers. > > Please pardon my ignorance: what *is* non-static inline > behaviour? I've only ever used static inlines myself: they're > the only sort that make sense (to me), in the world of standard > C static compilation and linkage. What happens elsewhen? Does > the compiler generate a "real" function with an exportable name > that can be linked-to? Why would you want to do that, when > that's what perfectly ordinary functions do? I can't imagine an > extern inline meaning anything useful unless one can do > LLVM-style link-time optimization. Is that on the cards? > > > Furthermore, even GNU wants to move to using the C99 > > semantics by default. Once that happens, continuing to be > > dependent upon the old GNU inline semantics is likely to cause > > porting headaches. > > Well, we don't want to be depending on non-standard semantics, > if we can help it. Sure. Are we? Where? `static inline' is the easiest to use and understand, and it's the only kind where the old GNU rules and C99 agree. So if you're using `static inline', nothing changes. The relevance of non-static inlines becomes apparent if the compiler decides *not* to inline your function. With static inlines, you get a separate static definition in each object file in this case. With non-static inlines, you can arrange to have the compiler generate an external reference to a single module that defines the function. You could argue that the extra copies that `static inline' implies aren't a big deal, and I'd agree with you. So why bother getting the semantics of `inline' and `extern inline' sorted out? Here are a few reasons: - Sometimes standards require libraries to have an externally-visible version of a symbol, even if the header provides an inline definition. In this case, the compiler should pick up the external version if the function isn't inlined. - As gcc 4.3+ becomes more common in the wild, third-party apps may increasingly assume C99 semantics for inlines, and we have no control over this. - In some situations, you want the compiler to inline a function in a few cases where it matters, and use an external reference everywhere else. For example, bde@ arranged for some of the trig innards of libm to be inlined in sin() and cos() because function calls with floating point are incredibly slow on i386. But for functions like lgamma() that nobody uses, the inlining is pointless. From imp at bsdimp.com Tue Mar 10 15:10:27 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Tue Mar 10 15:10:33 2009 Subject: Final sanity pass: xdev Message-ID: <20090310.210746.1649765883.imp@bsdimp.com> OK. I've had this reviewed before, and have been using variations of this patch since 6.1. However, since it has been a while, I thought I'd see if there were any objections before I committed this patch. It installs all the FreeBSD cross build tools as $(ARCH)-freebsd-$TOOL, which is what autoconf looks for. This may help cross building ports, ultimately, but it is also nice for cross compiling kernels (other changes to follow for that). To build, just say 'sudo make xdev XDEV=mips XDEV_ARCH=mips' and away you go. Note: sudo is required, since we have to install the compilers before building the libraries for the xbuild environment. Each tool chain takes up about 75-100M of space. Last call for Comments on this patch... Warner P.S. See you at AsiaBSDCon '09, right? :) -------------- next part -------------- Index: Makefile =================================================================== --- Makefile (revision 189571) +++ Makefile (working copy) @@ -88,7 +88,7 @@ obj objlink regress rerelease showconfig tags toolchain update \ _worldtmp _legacy _bootstrap-tools _cleanobj _obj \ _build-tools _cross-tools _includes _libraries _depend \ - build32 distribute32 install32 + build32 distribute32 install32 xdev xdev-build xdev-install TGTS+= ${SUBDIR_TARGETS} BITGTS= files includes Index: Makefile.inc1 =================================================================== --- Makefile.inc1 (revision 189571) +++ Makefile.inc1 (working copy) @@ -1328,3 +1328,89 @@ # showconfig: @${MAKE} -n -f bsd.own.mk -V dummy -dg1 | grep ^MK_ | sort + + +############### + +.if defined(XDEV) && defined(XDEV_ARCH) + +NOFUN=-DNO_FSCHG -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT \ + -DWITHOUT_MAN -DWITHOUT_NLS -DWITHOUT_PROFILE \ + -DWITHOUT_KERBEROS -DWITHOUT_RESCUE -DNO_WARNS + +XDDIR=${XDEV}-freebsd +XDTP=/usr/${XDDIR} +CDBENV=MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX}/${XDDIR} \ + TARGET=${XDEV} TARGET_ARCH=${XDEV_ARCH} +CDENV= ${CDBENV} \ + _SHLIBDIRPREFIX=${XDTP} \ + TOOLS_PREFIX=${XDTP} +CD2ENV=${CDENV} \ + MACHINE=${XDEV} MACHINE_ARCH=${XDEV_ARCH} + +CDTMP= ${MAKEOBJDIRPREFIX}/${XDEV}/${.CURDIR}/tmp +CDMAKE=${CDENV} ${MAKE} ${NOFUN} +CD2MAKE=${CD2ENV} PATH=${CDTMP}/usr/bin:${XDTP}/usr/bin:${PATH} ${MAKE} ${NOFUN} +XDDESTDIR=${DESTDIR}${XDTP} + +.ORDER: xdev-build xdev-install +xdev: xdev-build xdev-install + +.ORDER: _xb-build-tools _xb-cross-tools +xdev-build: _xb-build-tools _xb-cross-tools + +_xb-build-tools: + ${_+_}cd ${.CURDIR}; \ + ${CDBENV} ${MAKE} -f Makefile.inc1 ${NOFUN} build-tools + +_xb-cross-tools: +.for _tool in \ + gnu/usr.bin/binutils \ + gnu/usr.bin/cc + ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ + cd ${.CURDIR}/${_tool}; \ + ${CDMAKE} DIRPRFX=${_tool}/ obj; \ + ${CDMAKE} DIRPRFX=${_tool}/ depend; \ + ${CDMAKE} DIRPRFX=${_tool}/ all +.endfor + +_xi-mtree: + ${_+_}@${ECHODIR} "mtree populating ${XDDESTDIR}" + mkdir -p ${XDDESTDIR} + mtree -deU -f ${.CURDIR}/etc/mtree/BSD.root.dist \ + -p ${XDDESTDIR} >/dev/null + mtree -deU -f ${.CURDIR}/etc/mtree/BSD.usr.dist \ + -p ${XDDESTDIR}/usr >/dev/null + mtree -deU -f ${.CURDIR}/etc/mtree/BSD.include.dist \ + -p ${XDDESTDIR}/usr/include >/dev/null + +.ORDER: _xi-mtree _xi-cross-tools _xi-includes _xi-libraries _xi-links +xdev-install: _xi-mtree _xi-cross-tools _xi-includes _xi-libraries _xi-links + +_xi-cross-tools: + @echo "_xi-cross-tools" +.for _tool in \ + gnu/usr.bin/binutils \ + gnu/usr.bin/cc + ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ + cd ${.CURDIR}/${_tool}; \ + ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} +.endfor + +_xi-includes: + ${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 par-includes \ + DESTDIR=${XDDESTDIR} + +_xi-libraries: + ${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 libraries \ + DESTDIR=${XDDESTDIR} + +_xi-links: + ${_+_}cd ${XDDESTDIR}/usr/bin; \ + for i in *; do \ + ln -sf ../../${XDTP}/usr/bin/$$i \ + ../../../../usr/bin/${XDDIR}-$$i; \ + ln -sf ../../${XDTP}/usr/bin/$$i \ + ../../../../usr/bin/${XDDIR}${OSREL}-$$i; \ + done +.endif From Alexander at Leidinger.net Wed Mar 11 00:55:45 2009 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Wed Mar 11 00:55:52 2009 Subject: Final sanity pass: xdev In-Reply-To: <20090310.210746.1649765883.imp@bsdimp.com> References: <20090310.210746.1649765883.imp@bsdimp.com> Message-ID: <20090311083823.14600802dxfcspyo@webmail.leidinger.net> Quoting "M. Warner Losh" (from Tue, 10 Mar 2009 21:07:46 +0900 (JST)): > OK. I've had this reviewed before, and have been using variations of > this patch since 6.1. However, since it has been a while, I thought > I'd see if there were any objections before I committed this patch. No objections, just a "What about an else case with usage instructions when XDEV and XDEV_ARCH are not defined?". Bye, Alexander. -- It's not whether you win or lose but how you played the game. -- Grantland Rice http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From vadim_nuclight at mail.ru Thu Mar 12 04:21:52 2009 From: vadim_nuclight at mail.ru (Vadim Goncharov) Date: Thu Mar 12 04:21:59 2009 Subject: spliting kernel ipfw source ? (also involves sctp) References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> <20090302190157.GA33704@onelab2.iet.unipi.it> <20090306161028.GA12322@onelab2.iet.unipi.it> Message-ID: Hi Luigi Rizzo! On Fri, 6 Mar 2009 17:10:28 +0100; Luigi Rizzo wrote about 'Re: spliting kernel ipfw source ? (also involves sctp)': >>>>>>>Hi, I am planning to split netinet/ip_fw2.c in a number of smaller files >>>>>>>to make it more manageable, and while i do this I would also like to >>>>>>>move the files related to ipfw2 (namely ip_fw*c) to a better place. Any >>>>>>>objection to moving them to sys/netinet/ipfw2 ? Also, I can't help >> >> [...] >> >>> To further clarify, my plan is the following: >> >>> - leave ip_fw.h and ip_dummynet.h in /sys/netinet in case >>> userland code is dependent on their location; >>> - create /sys/netinet/ipfw/ to hold the kernel .c files related to >>> ipfw and possibly dummynet (and also their private headers if any); >> >> Exactly that and nothing more? I'm currently working on extending ipfw for >> Foundation with userland interface will be changed (and I thinking about >> introducing modules), what else do you plan to do? > Generally speaking, the kernel/userland interface will remain > unchanged both at binary and source level, which means no > backward incompatible changes in the sockopt numbers and messages, > and no changes in the location and userland-visible parts of > the headers. > In practical terms, ip_fw.h might lose the definition of > struct ip_fw_args, or the prototypes for the various kernel > functions. The #ifdef _KERNEL part of ip_dummynet.h should > also go to some other file. > If you want to contact me, on the list or offline, to discuss what > you want to do or what kind of 'modules' (kernel or userland ?) are > you thinking about, i'd be more than happy to help. I do not know whether this will be polite to discuss in details while Foundation has not yet announced my work :-/ I hope they'll do it in a week or so... I could say that at least dynamic rules and userland API/ABI will go under serious incompatible changes, so any your changing headers is OK, but what do you want to change inside kernel *.c is interesting to me. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From rizzo at iet.unipi.it Thu Mar 12 05:39:34 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Thu Mar 12 05:39:40 2009 Subject: spliting kernel ipfw source ? (also involves sctp) In-Reply-To: References: <20090301153010.GA58942@onelab2.iet.unipi.it> <49AAFD92.105@elischer.org> <8EBEEE24-6473-411D-AE3F-C4D1D3897E51@gmail.com> <20090302190157.GA33704@onelab2.iet.unipi.it> <20090306161028.GA12322@onelab2.iet.unipi.it> Message-ID: <20090312124446.GB28491@onelab2.iet.unipi.it> On Thu, Mar 12, 2009 at 11:21:38AM +0000, Vadim Goncharov wrote: > Hi Luigi Rizzo! ... > > In practical terms, ip_fw.h might lose the definition of > > struct ip_fw_args, or the prototypes for the various kernel > > functions. The #ifdef _KERNEL part of ip_dummynet.h should > > also go to some other file. > > > If you want to contact me, on the list or offline, to discuss what > > you want to do or what kind of 'modules' (kernel or userland ?) are > > you thinking about, i'd be more than happy to help. > > I do not know whether this will be polite to discuss in details while > Foundation has not yet announced my work :-/ I hope they'll do it in a week I see no reason to keep things secret. In fact, I'd try to be as open as possible as there is plenty of people on this list with with a lot of experience on the various issues at hand, and could give you valuable feedback. > or so... I could say that at least dynamic rules and userland API/ABI will > go under serious incompatible changes, so any your changing headers is OK, > but what do you want to change inside kernel *.c is interesting to me. First of all I want to split the files because ip_fw.2 is currently too large to be effectively maintained. There are more things that should be addressed, after proper performance assessment, e.g replacing the huge switch with a table of function pointers, fixing the way information is passed up in the "ipfw show" and "pipe show" commands, replacing the custom hash table with something already in the kernel, and perhaps a better hash function. Even more work (but this is not something I plan to do) involves the locking: there is contention on the rule counters (which I am not sure if/how is handled now) and contention on the dynamic rules, which possibly might be reduced by locking on the individual buckets. cheers luigi From ed at 80386.nl Fri Mar 13 05:51:17 2009 From: ed at 80386.nl (Ed Schouten) Date: Fri Mar 13 05:51:25 2009 Subject: Final sanity pass: xdev In-Reply-To: <20090310.210746.1649765883.imp@bsdimp.com> References: <20090310.210746.1649765883.imp@bsdimp.com> Message-ID: <20090313125115.GU31961@hoeg.nl> Hi Warner, The last couple of days/weeks I've been working a lot with LLVM+Clang. I'm currently writing some BSD makefiles to see how hard it is to add Clang to our base system, as a proof of concept. A nice thing about Clang is that it seems to be very easy to compile in multiple backends into the same Clang binary. If we ever switch to Clang in the base system (who knows), it would be interesting to see whether it's useful to just compile in all backends by default. Anyway, I took a quick look at the xdev patch and it looks okay. :-) -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090313/2d498e92/attachment.pgp From imp at bsdimp.com Fri Mar 13 08:05:09 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri Mar 13 08:05:16 2009 Subject: Final sanity pass: xdev In-Reply-To: <20090313125115.GU31961@hoeg.nl> References: <20090310.210746.1649765883.imp@bsdimp.com> <20090313125115.GU31961@hoeg.nl> Message-ID: <20090313.090038.-890725739.imp@bsdimp.com> In message: <20090313125115.GU31961@hoeg.nl> Ed Schouten writes: : Hi Warner, : : The last couple of days/weeks I've been working a lot with LLVM+Clang. : I'm currently writing some BSD makefiles to see how hard it is to add : Clang to our base system, as a proof of concept. : : A nice thing about Clang is that it seems to be very easy to compile in : multiple backends into the same Clang binary. If we ever switch to Clang : in the base system (who knows), it would be interesting to see whether : it's useful to just compile in all backends by default. : : Anyway, I took a quick look at the xdev patch and it looks okay. :-) Woot! Please let me know if it causes problems. Warner From jhein at timing.com Sat Mar 14 11:25:09 2009 From: jhein at timing.com (John Hein) Date: Sat Mar 14 11:25:16 2009 Subject: Final sanity pass: xdev In-Reply-To: <20090313.090038.-890725739.imp@bsdimp.com> References: <20090310.210746.1649765883.imp@bsdimp.com> <20090313125115.GU31961@hoeg.nl> <20090313.090038.-890725739.imp@bsdimp.com> Message-ID: <18875.60334.947446.966085@gromit.timing.com> M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: > In message: <20090313125115.GU31961@hoeg.nl> > Ed Schouten writes: > : Hi Warner, > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. > : I'm currently writing some BSD makefiles to see how hard it is to add > : Clang to our base system, as a proof of concept. > : > : A nice thing about Clang is that it seems to be very easy to compile in > : multiple backends into the same Clang binary. If we ever switch to Clang > : in the base system (who knows), it would be interesting to see whether > : it's useful to just compile in all backends by default. > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) > > Woot! Please let me know if it causes problems. I meant to mention this earlier - OSREL (referenced in the _xi-links: target) is not defined. So ports looking for arm-freebsd8.0-cc (for instance) won't find it. An earlier patch of yours had: .if !defined(OSREL) OSREL!= uname -r | sed -e 's/[-(].*//' .endif But the latest (and the one committed) does not. From imp at bsdimp.com Sat Mar 14 16:11:01 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Sat Mar 14 16:11:08 2009 Subject: Final sanity pass: xdev In-Reply-To: <18875.60334.947446.966085@gromit.timing.com> References: <20090313125115.GU31961@hoeg.nl> <20090313.090038.-890725739.imp@bsdimp.com> <18875.60334.947446.966085@gromit.timing.com> Message-ID: <20090315.080814.669286040.imp@bsdimp.com> In message: <18875.60334.947446.966085@gromit.timing.com> John Hein writes: : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: : > In message: <20090313125115.GU31961@hoeg.nl> : > Ed Schouten writes: : > : Hi Warner, : > : : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. : > : I'm currently writing some BSD makefiles to see how hard it is to add : > : Clang to our base system, as a proof of concept. : > : : > : A nice thing about Clang is that it seems to be very easy to compile in : > : multiple backends into the same Clang binary. If we ever switch to Clang : > : in the base system (who knows), it would be interesting to see whether : > : it's useful to just compile in all backends by default. : > : : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) : > : > Woot! Please let me know if it causes problems. : : I meant to mention this earlier - OSREL (referenced in the _xi-links: : target) is not defined. So ports looking for arm-freebsd8.0-cc (for : instance) won't find it. : : : An earlier patch of yours had: : : .if !defined(OSREL) : OSREL!= uname -r | sed -e 's/[-(].*//' : .endif : : But the latest (and the one committed) does not. You are right... Dang. I must have comitted the wrong thing... Warner From rwatson at FreeBSD.org Sun Mar 15 03:08:46 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Sun Mar 15 03:08:53 2009 Subject: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) In-Reply-To: References: <20080526110543.J26343@fledge.watson.org> Message-ID: On Mon, 16 Feb 2009, Robert Watson wrote: > 16 February 2009 HEADS UP to lists (this e-mail) > 01 March 2009 Disable build of all IFF_NEEDSGIANT drivers in 8.x Slightly delayed while USB NDIS was sorted out, I will be moving ahead with disabling the build of all IFF_NEEDSGIANT drivers in HEAD later today. > 01 April 2009 Remove all IFF_NEEDSGIANT drivers from 8.x And this date is probably going to remain about the same. Anyone who is interested is welcome to step in and update/fix drivers on the removal path at any time, and drivers can also be restored later easily if drivers are updated outside the tree as well. Thanks, Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From rmacklem at uoguelph.ca Sun Mar 15 13:03:19 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Sun Mar 15 13:03:26 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT Message-ID: As some of you might already know, I have been working on an implementation of NFSv4.0 for BSD for some time now. My original interest in NFSv4.0 came about because I felt that a component of it called delegations might permit significant performance improvements along the lines of what I had hoped to achieve with an experimental protocol I called Not Quite NFS long long ago. To experiment with the use of delegations, I needed an implementation and I got to work on that around Fall 2001/Winter 2002. (If I had had any idea how long it was going to take to get to a near production quality implementation, I would have never started, but I'm finally there.:-) There has been some indication of interest in incorporating this into FreeBSD-CURRENT/8 and I am posting here to start a discussion of this. I feel (and Robert Watson seems to agree) that replacing the current NFS implementation would be impractical for at least one release cycle. It does include NFSv2 and 3 support in it, but it would take some time to gain experience with the implementation and incorporate all the features in the current release into it. As such, it seems appropriate to have it live side-by-side with the current implementation. Here's a brief overview of the current code base: - approximately 75,000 lines of C in the following subdirs nfscl - generic client code that can be shared by the ports. (This includes doing the RPCs using code that avoids most of my infamous macros and handling functions for the NFSv4 state.) nfsclient - a copy of sys/nfsclient tweaked so that it calls functions in nfscl instead of doing the v2,3 RPCs. (As such, it retains the caching semantics, etc of the current client.) nfs - functions shared by the client and server nfsd - the server code nfscrypto - a couple of small functions for helping with RPCSEC_GSS (these aren't required for Doug Rabson's new RPC code) Current work: I am currently in the process of converting it to use Doug Rabson's new RPC layer and re-organizing things so that the two servers can co-reside in the kernel (the client already does co-reside). This work should be completed in a few weeks. Beyond that, the only NFSv4.0 feature I might be looking at that isn't yet implemented is referrals. Code stability: The server code ran here in two FreeBSD NFS servers for a couple of years without difficulties. The client has been tested fairly extensively for the Mac OS X port (25 iMacs in a lab here + about 350 downloads by others). At this point I suspect bugs will be mostly related to architectures other than i386 and, maybe, issues running on fast, heavily SMP machines. (The code is SMP safe, to the best of my knowledge.) What to call it? I currently call it newnfs with options NFSCL and NFSD, but I could care less what it is called. Calling it nfsv4 seemed inaccurate, since it does have nfsv2 and v3 support in it. So, what do folks think? rick ps: I haven't even looked at NFSv4.1 implementation yet. From kientzle at freebsd.org Sun Mar 15 13:43:48 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Mar 15 13:43:54 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: Message-ID: <49BD6378.9030501@freebsd.org> Ah, yes. I remember reading the NQNFS paper some years ago. Nice work! Q: Does this support NFSv4 ACLs? As I recall, trasz@ has NFSv4 patches in P4 that have yet to be merged; I've been waiting on that to work on NFSv4 support for tar/cpio. Q: How does this relate to the new NFS lockd recently committed? (by Doug Rabson? I can't remember now.) As for schedule, that depends. There was talk about branching 8.0 in April. If that's still the plan, then it might be best to just wait. But the website still announces August. If that's the plan, I'd be happy to push this in now with a build option to select which implementation to use. Have that default to the new implementation now in -CURRENT, but leave the option to ship 8.0 with the old implementation as the default. That would give a fair bit of production exposure during the 8.x cycle (some people will enable the new code, especially by 8.1 when the new code should have stabilized some). Then switch the default to the new code for 9.0. As for naming, I see no problems calling it (lowercase) nfs4 even if it does support (uppercase) NFSv2 and NFSv3. Tim Rick Macklem wrote: > As some of you might already know, I have been working on an > implementation of NFSv4.0 for BSD for some time now. My original > interest in NFSv4.0 came about because I felt that a component > of it called delegations might permit significant performance > improvements along the lines of what I had hoped to achieve with > an experimental protocol I called Not Quite NFS long long ago. > > To experiment with the use of delegations, I needed an implementation > and I got to work on that around Fall 2001/Winter 2002. (If I had > had any idea how long it was going to take to get to a near production > quality implementation, I would have never started, but I'm finally > there.:-) > > There has been some indication of interest in incorporating this > into FreeBSD-CURRENT/8 and I am posting here to start a discussion > of this. > > I feel (and Robert Watson seems to agree) that replacing the current > NFS implementation would be impractical for at least one release > cycle. It does include NFSv2 and 3 support in it, but it would take > some time to gain experience with the implementation and incorporate > all the features in the current release into it. As such, it seems > appropriate to have it live side-by-side with the current implementation. > > Here's a brief overview of the current code base: > - approximately 75,000 lines of C in the following subdirs > nfscl - generic client code that can be shared by the ports. > (This includes doing the RPCs using code that avoids most of my > infamous macros and handling functions for the NFSv4 state.) > nfsclient - a copy of sys/nfsclient tweaked so that it calls functions > in nfscl instead of doing the v2,3 RPCs. (As such, it retains the > caching semantics, etc of the current client.) > nfs - functions shared by the client and server > nfsd - the server code > nfscrypto - a couple of small functions for helping with RPCSEC_GSS > (these aren't required for Doug Rabson's new RPC code) > > Current work: > I am currently in the process of converting it to use Doug Rabson's new > RPC layer and re-organizing things so that the two servers can co-reside > in the kernel (the client already does co-reside). This work should be > completed in a few weeks. Beyond that, the only NFSv4.0 feature I might > be looking at that isn't yet implemented is referrals. > > Code stability: > The server code ran here in two FreeBSD NFS servers for a couple of years > without difficulties. The client has been tested fairly extensively for > the Mac OS X port (25 iMacs in a lab here + about 350 downloads by > others). At this point I suspect bugs will be mostly related to > architectures other than i386 and, maybe, issues running on fast, > heavily SMP machines. (The code is SMP safe, to the best of my > knowledge.) > > What to call it? > I currently call it newnfs with options NFSCL and NFSD, but I could > care less what it is called. Calling it nfsv4 seemed inaccurate, since > it does have nfsv2 and v3 support in it. > > So, what do folks think? rick > ps: I haven't even looked at NFSv4.1 implementation yet. > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > From rmacklem at uoguelph.ca Sun Mar 15 13:56:24 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Sun Mar 15 13:56:31 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: <49BD6378.9030501@freebsd.org> References: <49BD6378.9030501@freebsd.org> Message-ID: On Sun, 15 Mar 2009, Tim Kientzle wrote: > Q: Does this support NFSv4 ACLs? As I recall, > trasz@ has NFSv4 patches in P4 that have yet to > be merged; I've been waiting on that to work on > NFSv4 support for tar/cpio. > It works fine with it. Most of the work w.r.t. ACLs is in the local file systems on the server. (It just required a little bit of tweaking of my code, which was done a few months ago.) > Q: How does this relate to the new NFS lockd > recently committed? (by Doug Rabson? I can't > remember now.) > They will share the same RPC code, once I have completed the conversion. I basically just copied the client side lock code over into nfsclient, but have never tested it. As far as the server goes, I'd have to look. NFSv4 locking doesn't use lockd, but my server does grab byte range locks through the VFS and I suspect lockd sees those, just like any other process sees them. (One advantage of NFSv4 is integrated locking that seems to work well.) [good stuff w.r.t. integration snipped for brevity] rick From alfred at freebsd.org Sun Mar 15 14:07:34 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Sun Mar 15 14:10:10 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: Message-ID: <20090315205229.GV55200@elvis.mu.org> * Rick Macklem [090315 13:03] wrote: > As some of you might already know, I have been working on an > implementation of NFSv4.0 for BSD for some time now. My original > interest in NFSv4.0 came about because I felt that a component > of it called delegations might permit significant performance > improvements along the lines of what I had hoped to achieve with > an experimental protocol I called Not Quite NFS long long ago. ... > So, what do folks think? rick > ps: I haven't even looked at NFSv4.1 implementation yet. I think this is awesome! I think it wise to look at 4.1 and scoping that out before taking the time to integrate this to gain an understanding of: 1) what it would take to get to 4.1? 2) how we would interoperate with other machines until we get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become the defacto standard (is it already?)? 3) trade-off between taking the 4.0 code or how we can get 4.1 first before integration. This is important because the bug reports on 4.0 can swamp someone and prevent new feature development (4.1). You know more about it than most of us, so can you take some time to look at 4.1 and give some guestimates so we can gauge what to do next? Again, this is really cool and I apologize if 4.1 is something far out on the horizon, I just am approaching this from an integration perspective, not as an authority on NFSv4 standardization process. If the case is 4.1 is some number of years out, then I think my answer is "let's get this in ASAP!", otherwise we should think about it. thank you, -- - Alfred Perlstein From alfred at freebsd.org Sun Mar 15 14:07:54 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Sun Mar 15 14:10:21 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: <49BD6378.9030501@freebsd.org> Message-ID: <20090315210753.GX55200@elvis.mu.org> * Rick Macklem [090315 13:57] wrote: > > As far as the server goes, I'd have to look. NFSv4 locking > doesn't use lockd, but my server does grab byte range locks > through the VFS and I suspect lockd sees those, just like > any other process sees them. (One advantage of NFSv4 is > integrated locking that seems to work well.) I think the way that kernel server NFS lockd works is that it sits here: V TCPIP V lockd -> calls ASYNC locks down (expects callback) V VFS -> calls back to lockd when locks are acquired It's _not_ like this: V TCPIP V lockd -> calls SYNC locks down V VFS V lockd -> stuff so unless your server uses the async lock facility, you'll wind up with your server context blocked in VFS waiting for locks. (I think.) -Alfred From rmacklem at uoguelph.ca Sun Mar 15 14:15:06 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Sun Mar 15 14:15:13 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: <20090315205229.GV55200@elvis.mu.org> References: <20090315205229.GV55200@elvis.mu.org> Message-ID: On Sun, 15 Mar 2009, Alfred Perlstein wrote: > > I think it wise to look at 4.1 and scoping that out before taking > the time to integrate this to gain an understanding of: NFSv4.1 is still way out there. It hasn't reached RFC stage yet and vendors are only testing bits and pieces of it. (The current draft of the "minor" revision is over 500 pages.) All the code vendors are currently shipping is running 4.0. > > 1) what it would take to get to 4.1? A lot. A required feature is something for handling RPC transport called sessions. One guy has been looking at doing sessions for FreeBSD (hopefully integrated with Doug Rabson's new RPC code), but I have no idea if he has made any progress. > 2) how we would interoperate with other machines until we > get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become > the defacto standard (is it already?)? Systems should still support 4.0 for a long time. I have no idea when 4.1 will become a defacto standard, but I'd guess years. (For 4.0, the RFC is dated 2003 and the Linux implementation of 4.0 is still listed as experimental, I believe. 4.0 shipped with Solaris10, which was the first production quality implementation, imho. Apple only shipped an incomplete, non-usable 4.0 with Leopard. I think that gives you some idea how long it has taken to get from RFC to shipping. 4.1 is almost an RFC, but not quite yet.) > 3) trade-off between taking the 4.0 code or how we can > get 4.1 first before integration. This is important because > the bug reports on 4.0 can swamp someone and prevent new feature > development (4.1). > > You know more about it than most of us, so can you take some time > to look at 4.1 and give some guestimates so we can gauge what to > do next? > I've tried reading the drafts and got swamped. Honestly, I think a 4.1 implementation would take man years of effort and is beyond what I am capable of. Once you have sessions (non trivial, from what I saw), pNFS is what everyone gets excited about, but it's a major project. To do a server requires back end server<->server protocols that aren't spelled out by the draft. A client side of pNFS needs to know how to handle the various layout schemes (there are 2 other drafts beyond the 500+ page document for 2 other schemes than the most basic whole file one). > Again, this is really cool and I apologize if 4.1 is something far > out on the horizon, I just am approaching this from an integration > perspective, not as an authority on NFSv4 standardization process. > > If the case is 4.1 is some number of years out, then I think my > answer is "let's get this in ASAP!", otherwise we should think > about it. > Yes, I think 4.1 is some years away. rick From alfred at freebsd.org Sun Mar 15 17:37:51 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Sun Mar 15 17:37:57 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: <20090315205229.GV55200@elvis.mu.org> Message-ID: <20090316003750.GY55200@elvis.mu.org> * Rick Macklem [090315 14:15] wrote: > > [[snipped but certainly appreciated]] > >If the case is 4.1 is some number of years out, then I think my > >answer is "let's get this in ASAP!", otherwise we should think > >about it. > > > Yes, I think 4.1 is some years away. That along with the fact that you stated, that everyone is shipping 4.0, means that it makes a lot of sense to proceed with your code. Thanks very much for the summary. -- - Alfred Perlstein From jhein at timing.com Sun Mar 15 22:41:56 2009 From: jhein at timing.com (John Hein) Date: Sun Mar 15 22:42:07 2009 Subject: Final sanity pass: xdev In-Reply-To: <20090315.080814.669286040.imp@bsdimp.com> References: <20090313125115.GU31961@hoeg.nl> <20090313.090038.-890725739.imp@bsdimp.com> <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> Message-ID: <18877.57878.136116.691250@gromit.timing.com> M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: > In message: <18875.60334.947446.966085@gromit.timing.com> > John Hein writes: > : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: > : > In message: <20090313125115.GU31961@hoeg.nl> > : > Ed Schouten writes: > : > : Hi Warner, > : > : > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. > : > : I'm currently writing some BSD makefiles to see how hard it is to add > : > : Clang to our base system, as a proof of concept. > : > : > : > : A nice thing about Clang is that it seems to be very easy to compile in > : > : multiple backends into the same Clang binary. If we ever switch to Clang > : > : in the base system (who knows), it would be interesting to see whether > : > : it's useful to just compile in all backends by default. > : > : > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) > : > > : > Woot! Please let me know if it causes problems. > : > : I meant to mention this earlier - OSREL (referenced in the _xi-links: > : target) is not defined. So ports looking for arm-freebsd8.0-cc (for > : instance) won't find it. > : > : > : An earlier patch of yours had: > : > : .if !defined(OSREL) > : OSREL!= uname -r | sed -e 's/[-(].*//' > : .endif > : > : But the latest (and the one committed) does not. > > You are right... Dang. I must have comitted the wrong thing... One more thing... Now that we've switched to bsd ar, installing gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built ports can't find arm-freebsd8.0-ar Adding usr.bin/ar to the cross tools does the job... Index: Makefile.inc1 =================================================================== RCS file: /base/FreeBSD-CVS/src/Makefile.inc1,v retrieving revision 1.620 diff -u -p -r1.620 Makefile.inc1 --- Makefile.inc1 13 Mar 2009 10:40:38 -0000 1.620 +++ Makefile.inc1 16 Mar 2009 05:11:12 -0000 @@ -1035,6 +1035,7 @@ cross-tools: .for _tool in \ gnu/usr.bin/binutils \ gnu/usr.bin/cc \ + usr.bin/ar \ usr.bin/sed \ usr.bin/xlint/lint1 usr.bin/xlint/lint2 usr.bin/xlint/xlint \ ${_btxld} \ @@ -1370,7 +1371,8 @@ _xb-build-tools: _xb-cross-tools: .for _tool in \ gnu/usr.bin/binutils \ - gnu/usr.bin/cc + gnu/usr.bin/cc \ + usr.bin/ar ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ cd ${.CURDIR}/${_tool}; \ ${CDMAKE} DIRPRFX=${_tool}/ obj; \ @@ -1395,7 +1397,8 @@ _xi-cross-tools: @echo "_xi-cross-tools" .for _tool in \ gnu/usr.bin/binutils \ - gnu/usr.bin/cc + gnu/usr.bin/cc \ + usr.bin/ar ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ cd ${.CURDIR}/${_tool}; \ ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} From Info at Visacard.com.uk Sun Mar 15 23:32:56 2009 From: Info at Visacard.com.uk (Visa Card® / Msn® INTERNATIONAL MEGA JACKPOT 2009) Date: Sun Mar 15 23:33:06 2009 Subject: CONTACT CLAIMS ADMINISTRATOR Message-ID: <200903160324.n2G3OX9c008565@ss37.shared.server-system.net> Visa Card® / Msn® INTERNATIONAL MEGA JACKPOT 2009 PROMOTION/CLAIMS DEPARTMENT 440 THE STRAND LONDON, WC2R 0QS ENGLAND, UNITED KINGDOM DIRECTOR:DR.GLENN EDWARD Attn:Winner Winning No:VCard/877/798/2009 Email Ref No:VCard/699/33/2009 Notification Date:26/01/2009 AMOUNT WON:500,000.00GBP (Five Hundred Thousand Great Britain Pounds). This email address has brought you an unexpected luck,Your e-mail address was selected and confirmed by our co-sponsor Msn International, through their latest internet software.You are therefore been approved by Visa Card Int./Msn UK the sum of 500,000.00 (Five Hundred Thousand Great Britain Pounds). =================================== CONTACT CLAIMS ADMINISTRATOR =================================== NAME: Mr.David Ronald Email: Vicamsprdemet2009@hotmail.com Phone # : +(44) 703 596 7375 Fax # :+(44) 454 464 9443 Visa Card® / Msn® Promotion Department Do email the above Claims Administrator, at once with all the claims requirements below.To avoid unnecessary delay.They are needed to proceed. Claims Requirements: 1. Full Name:_____________________ 2. Address:_____________________ 3. Nationality:___________Sex:________ 4. Age:________Date of Birth:___________ 5. Occupation:_________Martial Status_________ 6. Cell Phone:___________Fax:___________ 7. State of Origin:_________Country:_______ 8.Winning No:_______Email Ref No:______ PROCEEDURES / RIGHTS AND PRECAUTIONS Choose from payment options and Contact the Claims Administrator with all your claims requirements well filled @ Vicamsprdemet2009@hotmail.com (i).Bank Transfer (ii).Delivery of Prepaid Visa card® Valued 500,000.00GBP by a registered Courier Company. Sincerely, Mrs.Dora Lazmon(Secretary) Visa Card® /Msn® Mega Jackpot ©2009 Microsoft Corporation.All rights reserved From bugmaster at FreeBSD.org Mon Mar 16 04:06:52 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 16 04:07:27 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200903161106.n2GB6ppi043183@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From lulf at FreeBSD.org Mon Mar 16 09:58:00 2009 From: lulf at FreeBSD.org (Ulf Lilleengen) Date: Mon Mar 16 09:58:07 2009 Subject: [HEADS UP] Merge of projects/gvinum to head Message-ID: <20090316155800.GA2257@carrot> Hello, This is a heads-up for a merge of gvinum project code into HEAD. This means that gvinum implementation will be changed some. The code is based on the work done by Lukas Ertl as well as the work I did before/during Google SoC 2007 and afterwards. It has been staying in p4 for some time, and then moved to the subversion project repository not long ago. The main reason for the delay of getting this into HEAD have been the lack of reviewers of the code, but after some discussion and help from testers, I've decided this is a good time to get it in (should perhaps have been merged a bit earlier) Testers have spotted several differences from the original gvinum, and I've tried to make it behave as the old implementation wherever that seemed the best way to go. Luckily, the work has gotten a bit of attention lately, thanks to Rick C. Petty for helping out with testing and bugfixing, as well as all others who have dared to run the new gvinum. So, what does this update offer? From the user aspect: - Implementation of many of the missing commands from the old vinum: attach/detach, start (was partially implemented), stop (was partially implemented), concat, mirror, stripe, raid5 (shortcuts for creating volumes with one plex of these organizations). - Support for fixing degraded plexes while mounted. - Support for growing for striped and raid5-plexes, meaning that one can extend the volumes for these plex types in addition to the concat type. Also works while the volume is mounted. - The parity check and rebuild no longer goes between userland/kernel, meaning that your gvinum command will not stay and wait forever for the rebuild to finish. You can instead watch the status with the list command. - Many problems with gvinum have been reported since 5.x, and some has been hard to fix due to the complicated architecture. Hopefully, it should be more stable and better handle edge cases that previously made gvinum crash. - Failed drives no longer disappears entirely, but now leave behind a dummy drive that makes sure the original state is not forgotten in case the system is rebooted between drive failures/swaps. From the technical aspect: - Gvinum now uses one single workerthread instead of one thread for each volume and each plex. The reason for this is that the previous scheme was very complex, and was the cause of many of the bugs discovered in gvinum. Instead, gvinum now uses one worker thread with an event queue, quite similar to what used in gmirror. - The rebuild/grow/initialize/parity check routines no longer runs in separate threads, but are run as regular I/O requests with special flags. This made it easier to support on-mount growing and parity rebuild. Probably, there are many other issues that have been fixed, perhaps new issues introduced. This is why this is dropped in HEAD now, to give it more attention from the uses before the 8.0 branch. All in all, this will make gvinum an easier beast to handle for users. If you have any issues related to this, send in problem reports or drop me an e-mail, and I'll take a look. For interested reviewers, the code resides in svn://svn.freebsd.org/base/projects/gvinum -- Ulf Lilleengen -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090316/8961a2ac/attachment.pgp From lulf at FreeBSD.org Mon Mar 16 09:59:58 2009 From: lulf at FreeBSD.org (Ulf Lilleengen) Date: Mon Mar 16 10:00:15 2009 Subject: [HEADS UP] Merge of projects/gvinum to head In-Reply-To: <20090316155800.GA2257@carrot> References: <20090316155800.GA2257@carrot> Message-ID: <20090316155957.GA2385@carrot> On man, mar 16, 2009 at 04:58:00pm +0100, Ulf Lilleengen wrote: > Hello, > > This is a heads-up for a merge of gvinum project code into HEAD. This means > that gvinum implementation will be changed some. The code is based on the > work done by Lukas Ertl as well as the work I did before/during Google SoC > 2007 and afterwards. It has been staying in p4 for some time, and then moved > to the subversion project repository not long ago. The main reason for the > delay of getting this into HEAD have been the lack of reviewers of the code, > but after some discussion and help from testers, I've decided this is a good > time to get it in (should perhaps have been merged a bit earlier) Testers > have spotted several differences from the original gvinum, and I've tried to > make it behave as the old implementation wherever that seemed the best way to > go. Luckily, the work has gotten a bit of attention lately, thanks to Rick C. > Petty for helping out with testing and bugfixing, as well as all others who > have dared to run the new gvinum. So, what does this update offer? And I plan on importing it within 1-2 weeks :) -- Ulf Lilleengen From imp at bsdimp.com Mon Mar 16 11:55:47 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Mar 16 11:55:54 2009 Subject: Final sanity pass: xdev In-Reply-To: <18877.57878.136116.691250@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> Message-ID: <20090317.035405.1683322897.imp@bsdimp.com> thanks! I'll check into this when I finish returning home... Warner In message: <18877.57878.136116.691250@gromit.timing.com> John Hein writes: : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: : > In message: <18875.60334.947446.966085@gromit.timing.com> : > John Hein writes: : > : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: : > : > In message: <20090313125115.GU31961@hoeg.nl> : > : > Ed Schouten writes: : > : > : Hi Warner, : > : > : : > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. : > : > : I'm currently writing some BSD makefiles to see how hard it is to add : > : > : Clang to our base system, as a proof of concept. : > : > : : > : > : A nice thing about Clang is that it seems to be very easy to compile in : > : > : multiple backends into the same Clang binary. If we ever switch to Clang : > : > : in the base system (who knows), it would be interesting to see whether : > : > : it's useful to just compile in all backends by default. : > : > : : > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) : > : > : > : > Woot! Please let me know if it causes problems. : > : : > : I meant to mention this earlier - OSREL (referenced in the _xi-links: : > : target) is not defined. So ports looking for arm-freebsd8.0-cc (for : > : instance) won't find it. : > : : > : : > : An earlier patch of yours had: : > : : > : .if !defined(OSREL) : > : OSREL!= uname -r | sed -e 's/[-(].*//' : > : .endif : > : : > : But the latest (and the one committed) does not. : > : > You are right... Dang. I must have comitted the wrong thing... : : One more thing... Now that we've switched to bsd ar, installing : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built : ports can't find arm-freebsd8.0-ar : : Adding usr.bin/ar to the cross tools does the job... : : Index: Makefile.inc1 : =================================================================== : RCS file: /base/FreeBSD-CVS/src/Makefile.inc1,v : retrieving revision 1.620 : diff -u -p -r1.620 Makefile.inc1 : --- Makefile.inc1 13 Mar 2009 10:40:38 -0000 1.620 : +++ Makefile.inc1 16 Mar 2009 05:11:12 -0000 : @@ -1035,6 +1035,7 @@ cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : gnu/usr.bin/cc \ : + usr.bin/ar \ : usr.bin/sed \ : usr.bin/xlint/lint1 usr.bin/xlint/lint2 usr.bin/xlint/xlint \ : ${_btxld} \ : @@ -1370,7 +1371,8 @@ _xb-build-tools: : _xb-cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ obj; \ : @@ -1395,7 +1397,8 @@ _xi-cross-tools: : @echo "_xi-cross-tools" : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} : : From rmacklem at uoguelph.ca Wed Mar 18 12:54:47 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Wed Mar 18 12:54:54 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT Message-ID: > Q: How does this relate to the new NFS lockd > recently committed? (by Doug Rabson? I can't > remember now.) My nfsv4 server only does VOP_ADVLOCK() with the F_GETLK and F_SETLK options, that are non-blocking. I checked with Doug R. and he doesn't think that there will be a problem. So, it looks like it should be ok, rick From ivoras at freebsd.org Wed Mar 18 15:55:21 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Wed Mar 18 15:55:28 2009 Subject: Magic symlinks redux In-Reply-To: References: Message-ID: <49C17BB3.1010702@freebsd.org> Hi, I started doing something that would be nicely solved by magic symlinks so I remembered this thread. It started some 6+ months ago here: http://lists.freebsd.org/pipermail/freebsd-arch/2008-August/008473.html Is there any new development? Ivan Voras wrote: > I was reading about new things in NetBSD, and one thing caught my > attention: per-user /tmp. See > http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20080714_0251.html for > example. > > Google says that a discussion about magic symlinks happens every now and > then in FreeBSD but nothing really gets done. I found this > implementation which looks like it's for 7.0: > > http://butcher.heavennet.ru/patches/kernel/magiclinks/ > > As far as I understand the VFS (which isn't much...) this looks like an > trivial patch, and it's compatible with NetBSD. Since I'm interested in > this (specifically for the per-user /tmp and maybe similar gadgetry), > I'd like to nurse this patch into the tree, if there are no objections > (of course, I'll bug anyone I can find who knows VFS to review it :) ). > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090318/feb192e1/signature.pgp From rizzo at iet.unipi.it Wed Mar 18 16:40:04 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Wed Mar 18 16:40:11 2009 Subject: Magic symlinks redux In-Reply-To: <49C17BB3.1010702@freebsd.org> References: <49C17BB3.1010702@freebsd.org> Message-ID: <20090318234447.GA21963@onelab2.iet.unipi.it> On Wed, Mar 18, 2009 at 11:54:43PM +0100, Ivan Voras wrote: > Hi, > > I started doing something that would be nicely solved by magic symlinks > so I remembered this thread. It started some 6+ months ago here: > > http://lists.freebsd.org/pipermail/freebsd-arch/2008-August/008473.html > > Is there any new development? at the time i did some quick tests on probably the same code as you reference below, and it is really trivial (attached). I do remember a discussion of a more complete implementation but was many times more complex and more intrusive and I am afraid that such a thing will never happen. BTW today i hit a problem where some form of magic symlinks would be useful. Say you are preparing a disk image in a subtree of your filesystem, and your image contains symlinks with absolute paths: to do things right, you should navigate the subtree in a chroot'ed environment, but chroot() requires root privs. If we had some form of magic symlinks that support a per-process remapping, one could e.g. set a variable that is prepended to the target of absolute symlinks, and achieve the same effect of a chroot without the need for root privs... cheers luigi > Ivan Voras wrote: > > I was reading about new things in NetBSD, and one thing caught my > > attention: per-user /tmp. See > > http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20080714_0251.html for > > example. > > > > Google says that a discussion about magic symlinks happens every now and > > then in FreeBSD but nothing really gets done. I found this > > implementation which looks like it's for 7.0: > > > > http://butcher.heavennet.ru/patches/kernel/magiclinks/ > > > > As far as I understand the VFS (which isn't much...) this looks like an > > trivial patch, and it's compatible with NetBSD. Since I'm interested in > > this (specifically for the per-user /tmp and maybe similar gadgetry), > > I'd like to nurse this patch into the tree, if there are no objections > > (of course, I'll bug anyone I can find who knows VFS to review it :) ). > > -------------- next part -------------- A non-text attachment was scrubbed... Name: 20090211-symlinks.diff Type: text/x-diff Size: 4279 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090318/e7c34846/20090211-symlinks.bin From imp at bsdimp.com Wed Mar 18 17:37:42 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Wed Mar 18 17:37:48 2009 Subject: Final sanity pass: xdev In-Reply-To: <18877.57878.136116.691250@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> Message-ID: <20090318.183646.-593221015.imp@bsdimp.com> In message: <18877.57878.136116.691250@gromit.timing.com> John Hein writes: : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: : > In message: <18875.60334.947446.966085@gromit.timing.com> : > John Hein writes: : > : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: : > : > In message: <20090313125115.GU31961@hoeg.nl> : > : > Ed Schouten writes: : > : > : Hi Warner, : > : > : : > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. : > : > : I'm currently writing some BSD makefiles to see how hard it is to add : > : > : Clang to our base system, as a proof of concept. : > : > : : > : > : A nice thing about Clang is that it seems to be very easy to compile in : > : > : multiple backends into the same Clang binary. If we ever switch to Clang : > : > : in the base system (who knows), it would be interesting to see whether : > : > : it's useful to just compile in all backends by default. : > : > : : > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) : > : > : > : > Woot! Please let me know if it causes problems. : > : : > : I meant to mention this earlier - OSREL (referenced in the _xi-links: : > : target) is not defined. So ports looking for arm-freebsd8.0-cc (for : > : instance) won't find it. : > : : > : : > : An earlier patch of yours had: : > : : > : .if !defined(OSREL) : > : OSREL!= uname -r | sed -e 's/[-(].*//' : > : .endif : > : : > : But the latest (and the one committed) does not. : > : > You are right... Dang. I must have comitted the wrong thing... There is a subtle point here that we've missed. This gives us the version on the host, not the version of the tree we're building. In practice, these need to be the same. However, if we support building a 6.x cross compiler on a 7.x system, we may have to revisit. : One more thing... Now that we've switched to bsd ar, installing : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built : ports can't find arm-freebsd8.0-ar Doh! Thanks for the patch. I'll run it through my QA cycle. Warner : Adding usr.bin/ar to the cross tools does the job... : : Index: Makefile.inc1 : =================================================================== : RCS file: /base/FreeBSD-CVS/src/Makefile.inc1,v : retrieving revision 1.620 : diff -u -p -r1.620 Makefile.inc1 : --- Makefile.inc1 13 Mar 2009 10:40:38 -0000 1.620 : +++ Makefile.inc1 16 Mar 2009 05:11:12 -0000 : @@ -1035,6 +1035,7 @@ cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : gnu/usr.bin/cc \ : + usr.bin/ar \ : usr.bin/sed \ : usr.bin/xlint/lint1 usr.bin/xlint/lint2 usr.bin/xlint/xlint \ : ${_btxld} \ : @@ -1370,7 +1371,8 @@ _xb-build-tools: : _xb-cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ obj; \ : @@ -1395,7 +1397,8 @@ _xi-cross-tools: : @echo "_xi-cross-tools" : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} : : From jhein at timing.com Wed Mar 18 17:57:09 2009 From: jhein at timing.com (John Hein) Date: Wed Mar 18 17:57:15 2009 Subject: Final sanity pass: xdev In-Reply-To: <20090318.183646.-593221015.imp@bsdimp.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> Message-ID: <18881.38984.133668.539997@gromit.timing.com> M. Warner Losh wrote at 18:36 -0600 on Mar 18, 2009: > In message: <18877.57878.136116.691250@gromit.timing.com> > John Hein writes: > : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: > : > In message: <18875.60334.947446.966085@gromit.timing.com> > : > John Hein writes: > : > : An earlier patch of yours had: > : > : > : > : .if !defined(OSREL) > : > : OSREL!= uname -r | sed -e 's/[-(].*//' > : > : .endif > : > : > : > : But the latest (and the one committed) does not. > : > > : > You are right... Dang. I must have comitted the wrong thing... > > There is a subtle point here that we've missed. This gives us the > version on the host, not the version of the tree we're building. In > practice, these need to be the same. However, if we support building > a 6.x cross compiler on a 7.x system, we may have to revisit. In our build env (I'm sure you recall), it works out fine in practice (for most ports [*]). Building 8.x targets on 7.x makes links like "arm-freebsd7.1-cc" but the cross built ports also look for that name - likely because the config-guess machinery uses uname also. I'm not so sure we need to support building the other direction (7 on 8) since we generally don't support that direction of compatibilty, but if it can be done with minimal effort, that'd be good. So, if it's just a matter of finding the appropriate cross version of cc or nm or whatever, this is okay. [*] But if there is some dependency in a particular port that actually looks into the freebsd version and does something different based on version, it may be an issue. Then on top of that, there's kernel version and userland version that may matter to certain ports. Perhaps we should consider setting UNAME_r in the environment when building across major OS levels (possibly outside the scope of /usr/src/Makefile*). At a certain point in the cross-arch / cross-major-OS-version building dance, we could also just say it's not supported and let people work it out with a native "same major" OS level build machine (possibly a virtual machine). > : One more thing... Now that we've switched to bsd ar, installing > : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built > : ports can't find arm-freebsd8.0-ar > > Doh! Thanks for the patch. I'll run it through my QA cycle. Okay. Thanks for getting the 1000 monkeys to give it a test run ;) From imp at bsdimp.com Wed Mar 18 18:04:52 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Wed Mar 18 18:04:59 2009 Subject: Final sanity pass: xdev In-Reply-To: <18881.38984.133668.539997@gromit.timing.com> References: <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> Message-ID: <20090318.190141.2040345308.imp@bsdimp.com> In message: <18881.38984.133668.539997@gromit.timing.com> John Hein writes: : > : One more thing... Now that we've switched to bsd ar, installing : > : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built : > : ports can't find arm-freebsd8.0-ar : > : > Doh! Thanks for the patch. I'll run it through my QA cycle. : : Okay. Thanks for getting the 1000 monkeys to give it a test run ;) Sadly, I'm an old-time C++ programmer, so I have a virtual base class that causes 1000 virtual monkeys to be created and shoot my foot at random intervals. Not the most useful thing in the world, but it is C++, so what can you do :) Warner From julian at elischer.org Wed Mar 18 18:25:55 2009 From: julian at elischer.org (Julian Elischer) Date: Wed Mar 18 18:26:33 2009 Subject: Final sanity pass: xdev In-Reply-To: <18881.38984.133668.539997@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> Message-ID: <49C19F2A.10406@elischer.org> John Hein wrote: > M. Warner Losh wrote at 18:36 -0600 on Mar 18, 2009: > > In message: <18877.57878.136116.691250@gromit.timing.com> > > John Hein writes: > > : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: > > : > In message: <18875.60334.947446.966085@gromit.timing.com> > > : > John Hein writes: > > : > : An earlier patch of yours had: > > : > : > > : > : .if !defined(OSREL) > > : > : OSREL!= uname -r | sed -e 's/[-(].*//' > > : > : .endif > > : > : > > : > : But the latest (and the one committed) does not. > > : > > > : > You are right... Dang. I must have comitted the wrong thing... > > > > There is a subtle point here that we've missed. This gives us the > > version on the host, not the version of the tree we're building. In > > practice, these need to be the same. However, if we support building > > a 6.x cross compiler on a 7.x system, we may have to revisit. > > In our build env (I'm sure you recall), it works out fine in practice > (for most ports [*]). > > Building 8.x targets on 7.x makes links like "arm-freebsd7.1-cc" but > the cross built ports also look for that name - likely because the > config-guess machinery uses uname also. > > I'm not so sure we need to support building the other direction (7 on 8) > since we generally don't support that direction of compatibilty, but > if it can be done with minimal effort, that'd be good. > > So, if it's just a matter of finding the appropriate cross version of > cc or nm or whatever, this is okay. > > [*] But if there is some dependency in a particular port that actually > looks into the freebsd version and does something different based on > version, it may be an issue. Then on top of that, there's kernel > version and userland version that may matter to certain ports. > > Perhaps we should consider setting UNAME_r in the environment when > building across major OS levels (possibly outside the scope of > /usr/src/Makefile*). > > At a certain point in the cross-arch / cross-major-OS-version building > dance, we could also just say it's not supported and let people work > it out with a native "same major" OS level build machine (possibly a > virtual machine). we also have this problem when running a different revision of software in a Jail. Possibly it should be possible to set the value that uname responds with for a process and its' children, (even more so that the environment variables) I'd like to be able to label a jail as a 7.1 jail on an 8.0 machine.. The problem in using teh single environment variable UNAME_r or friends is that the intermediate tools ARE running on a different environment than the final tools so one value may not be correct everywhere. we sort of need one value of UNAME_r up until teh tools are built, and then another value for teh binary build. But even that may be not general enough. > > > > : One more thing... Now that we've switched to bsd ar, installing > > : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built > > : ports can't find arm-freebsd8.0-ar > > > > Doh! Thanks for the patch. I'll run it through my QA cycle. > > Okay. Thanks for getting the 1000 monkeys to give it a test run ;) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From jhein at timing.com Wed Mar 18 18:56:07 2009 From: jhein at timing.com (John Hein) Date: Wed Mar 18 18:56:13 2009 Subject: Final sanity pass: xdev In-Reply-To: <49C19F2A.10406@elischer.org> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> <49C19F2A.10406@elischer.org> Message-ID: <18881.42546.640583.971867@gromit.timing.com> Julian Elischer wrote at 18:26 -0700 on Mar 18, 2009: > John Hein wrote: > > Perhaps we should consider setting UNAME_r in the environment when > > building across major OS levels (possibly outside the scope of > > /usr/src/Makefile*). > > > > At a certain point in the cross-arch / cross-major-OS-version building > > dance, we could also just say it's not supported and let people work > > it out with a native "same major" OS level build machine (possibly a > > virtual machine). > > we also have this problem when running a different revision of > software in a Jail. Possibly it should be possible to set the value > that uname responds with for a process and its' children, (even more > so that the environment variables) > I'd like to be able to label a jail as a 7.1 jail on an 8.0 machine.. > > The problem in using teh single environment variable UNAME_r > or friends is that the intermediate tools ARE running on a > different environment than the final tools so one value may not be > correct everywhere. > we sort of need one value of UNAME_r up until teh tools are built, and > then another value for teh binary build. But even that may be not > general enough. In our locally grown build env, that's pretty easy to do (pass different UNAME_r at different build stages). Perhaps you could even put it into the default env for a build chroot/jail (via login.conf). As I said, it's probably not in the scope of /usr/src/Makefile*, but because we can pass in UNAME_r from a higher level having Makefile.inc1 use 'uname -r' for OSREL (as it is currently named) in cross builds seems okay to me. From rizzo at iet.unipi.it Thu Mar 19 01:14:52 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Thu Mar 19 01:15:00 2009 Subject: RFC: adding 'proxy' nodes to provider ports (with patch) Message-ID: <20090319081936.GA32750@onelab2.iet.unipi.it> Hi, Fabio Checconi and I have been thinking on how to implement "proxy" geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer port, do not do any data transformation, and can be transparently inserted or removed on top of a provider port while the tree is actively moving data. Our immediate need was to add/remove a scheduler while a disk is mounted, but there are possibly other uses e.g. if one wants to "sniff" the traffic through a disk, or do other ops that are transparent for the data stream. We would like opinion on the following solution, which seems extremely simple in terms of implementation. The idea is to intercept requests coming on a provider port, pp, and redirect them to a geom node acting as a proxy if the port is configured in this way: +=====...===...======+ H H H H H H +=====...====== cp ==+ | +---------------+ V | V +=====.....==== pp ==+ | +======= proxy_pp ==+ H 'ad0s1' H | H H H ------->--+ H H H gp -------<--+ H proxy_node H H H | H H +=======....===...===+ | +======= proxy_cp ==+ | V +---------------+ Normally the proxy does not exist, and the geom tree works as it does now. When we create a 'proxy' node, with something like geom my_proxy_class proxy ad0s1 we do something very similar to a 'create', but: - the proxy node is marked specially in gp->flags, so the core will not generate a g_new_provider_event when the provider port is created (this means there is no taste() call and nobody should be able to attach to the port). - the provider port we attach to is linked, with two pointers, to the provider and consumer ports of the proxy_node. In this situation, g_io_request() finds that port pp has a proxy attached to it, and immediately redirects the requests to the proxy, which does everything a geom node does (cloning requests, etc). When the proxy wants to pass the request down, it sends it again to pp, but now there is no redirection because the source can be identified as the proxy. The pointers in the bio insure a correct flow of the requests on the reverse path. Disconnecting a proxy is almost trivial: apart from handling possible races on the data path, we just need to clear pp->proxy_pp and pp->proxy_cp. After that, we can just send the regular destroy events to the proxy node, who will have to take care of flushing any pending bio's (e.g. see our geom_sched node that already does this). Overall the change is very small (see attached patch): a couple of lines in g_io_request, two extra fields in the g_provider, and the addition of a flag to gp->flags to control the generation of g_new_provider_event. There is basically no overhead on regular operation, and only a couple of extra pointers in struct g_provider (we use a spare bit in gp->flags to mark G_GEOM_PROXY nodes). The only things missing in the patch should be: - a check to avoid races on creation&destruction of a proxy. I am not so sure on how to achieve this, but creation and destruction are rare and can normally wait, so we could just piggyback the small critical section (manipulating pp->proxy_cp and pp->proxy_cp) into some other piece of code that is guaranteed to be race-free. - a check to prevent attaching to a provider port of a proxy (not a problem, i believe); - a check to prevent attaching a proxy to a provider port that already has one. Of course you can attach a proxy to another proxy, and if you want to change the order it is as simple as removing the existing proxy and reattaching it after the new one. Feedback welcome. cheers luigi and fabio -------------- next part -------------- A non-text attachment was scrubbed... Name: geom-proxy.patch Type: text/x-diff Size: 3166 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090319/d051c8bb/geom-proxy.bin From pjd at FreeBSD.org Thu Mar 19 03:12:00 2009 From: pjd at FreeBSD.org (Pawel Jakub Dawidek) Date: Thu Mar 19 03:12:37 2009 Subject: RFC: adding 'proxy' nodes to provider ports (with patch) In-Reply-To: <20090319081936.GA32750@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it> Message-ID: <20090319094505.GA1539@garage.freebsd.pl> On Thu, Mar 19, 2009 at 09:19:36AM +0100, Luigi Rizzo wrote: > Hi, > > Fabio Checconi and I have been thinking on how to implement "proxy" > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > port, do not do any data transformation, and can be transparently > inserted or removed on top of a provider port while the tree is > actively moving data. > > Our immediate need was to add/remove a scheduler while a disk is > mounted, but there are possibly other uses e.g. if one wants to > "sniff" the traffic through a disk, or do other ops that are > transparent for the data stream. > > We would like opinion on the following solution, which seems > extremely simple in terms of implementation. > > The idea is to intercept requests coming on a provider port, pp, and > redirect them to a geom node acting as a proxy if the port > is configured in this way: > > +=====...===...======+ > H H > H H > H H > +=====...====== cp ==+ > | +---------------+ > V | V > +=====.....==== pp ==+ | +======= proxy_pp ==+ > H 'ad0s1' H | H H > H ------->--+ H H > H gp -------<--+ H proxy_node H > H H | H H > +=======....===...===+ | +======= proxy_cp ==+ > | V > +---------------+ > > Normally the proxy does not exist, and the geom tree works as it does now. > > When we create a 'proxy' node, with something like > > geom my_proxy_class proxy ad0s1 > > we do something very similar to a 'create', but: > > - the proxy node is marked specially in gp->flags, so the core will > not generate a g_new_provider_event when the provider port is created > (this means there is no taste() call and nobody should be able > to attach to the port). > > - the provider port we attach to is linked, with two pointers, > to the provider and consumer ports of the proxy_node. > > In this situation, g_io_request() finds that port pp has a proxy attached > to it, and immediately redirects the requests to the proxy, which > does everything a geom node does (cloning requests, etc). > When the proxy wants to pass the request down, it sends it again to pp, > but now there is no redirection because the source can be identified > as the proxy. The pointers in the bio insure a correct flow of the > requests on the reverse path. The one advantage I see for this over using regular GEOM rules is that new consumers go through proxy automatically. When I was working on similar functionality I more wanted to do something like this: consumer1 consumer2 \ / \ / provider Insert the proxy in the middle of any provider-consumer pair: consumer1 consumer2 | | proxy_provider | | / proxy_consumer / \ / provider This can be done (almost I think) atomically: /* First attach to the destination provider. */ g_attach(proxy_consumer, provider); /* Then switch original consumer to use proxy_provider (should be almost atomic). */ consumer1->provider = proxy_provider; /* handle access counts */ In-flight I/O requests know how to go back, because they have source and destination stored in bio_from and bio_to fields, so no races here. > Disconnecting a proxy is almost trivial: apart from handling possible > races on the data path, we just need to clear pp->proxy_pp and pp->proxy_cp. > After that, we can just send the regular destroy events to the proxy > node, who will have to take care of flushing any pending bio's (e.g. > see our geom_sched node that already does this). > > Overall the change is very small (see attached patch): > a couple of lines in g_io_request, two extra fields in the g_provider, > and the addition of a flag to gp->flags to control the generation > of g_new_provider_event. > There is basically no overhead on regular operation, and only > a couple of extra pointers in struct g_provider (we use a spare > bit in gp->flags to mark G_GEOM_PROXY nodes). > > The only things missing in the patch should be: > > - a check to avoid races on creation&destruction of a proxy. > I am not so sure on how to achieve this, but creation and destruction > are rare and can normally wait, so we could just piggyback the > small critical section (manipulating pp->proxy_cp and pp->proxy_cp) > into some other piece of code that is guaranteed to be race-free. > > - a check to prevent attaching to a provider port of a proxy > (not a problem, i believe); > > - a check to prevent attaching a proxy to a provider port that already > has one. Of course you can attach a proxy to another proxy, and > if you want to change the order it is as simple as removing the > existing proxy and reattaching it after the new one. Could you provide link for the patch, as it was removed from your e-mail? -- Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090319/8e89dfc7/attachment.pgp From rizzo at iet.unipi.it Thu Mar 19 04:08:53 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Thu Mar 19 04:09:06 2009 Subject: RFC: adding 'proxy' nodes to provider ports (with patch) In-Reply-To: <20090319094505.GA1539@garage.freebsd.pl> References: <20090319081936.GA32750@onelab2.iet.unipi.it> <20090319094505.GA1539@garage.freebsd.pl> Message-ID: <20090319111339.GA38075@onelab2.iet.unipi.it> On Thu, Mar 19, 2009 at 10:45:05AM +0100, Pawel Jakub Dawidek wrote: > > Hi, > > > > Fabio Checconi and I have been thinking on how to implement "proxy" > > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > > port, do not do any data transformation, and can be transparently > > inserted or removed on top of a provider port while the tree is > > actively moving data. ... > > Overall the change is very small (see attached patch): > > a couple of lines in g_io_request, two extra fields in the g_provider, > > and the addition of a flag to gp->flags to control the generation > > of g_new_provider_event. It seems that the attachment was removed so here it is http://info.iet.unipi.it/~luigi/FreeBSD/20090319-geom-proxy.patch cheers luigi From rizzo at iet.unipi.it Thu Mar 19 04:17:08 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Thu Mar 19 04:17:19 2009 Subject: RFC: adding 'proxy' nodes to provider ports (with patch) In-Reply-To: <20090319094505.GA1539@garage.freebsd.pl> References: <20090319081936.GA32750@onelab2.iet.unipi.it> <20090319094505.GA1539@garage.freebsd.pl> Message-ID: <20090319112152.GB38075@onelab2.iet.unipi.it> On Thu, Mar 19, 2009 at 10:45:05AM +0100, Pawel Jakub Dawidek wrote: > On Thu, Mar 19, 2009 at 09:19:36AM +0100, Luigi Rizzo wrote: ... > The one advantage I see for this over using regular GEOM rules is that > new consumers go through proxy automatically. When I was working on > similar functionality I more wanted to do something like this: > > consumer1 consumer2 > \ / > \ / > provider > > Insert the proxy in the middle of any provider-consumer pair: > > consumer1 consumer2 > | | > proxy_provider | > | / > proxy_consumer / > \ / > provider ok this is slightly different from what we have implemented, as we hook into the provider whereas you hook into the consumer. In our case we really need the hook to be in the provider so it intercepts accesses from all consumers, e.g. from /dev/ad0 and from the filesystems mounted on top of it. Given that the geom_disk node does not have a consumer on the bottom, we cannot do it differently. I can imagine that one might want to attach a proxy to a consumer port, but I cannot make a specific case where this would be needed. Also, I wonder how do i name a consumer port in the geom model ?? cheers luigi From marius at nuenneri.ch Thu Mar 19 05:01:39 2009 From: marius at nuenneri.ch (=?ISO-8859-1?Q?Marius_N=FCnnerich?=) Date: Thu Mar 19 05:01:45 2009 Subject: RFC: adding 'proxy' nodes to provider ports (with patch) In-Reply-To: <20090319081936.GA32750@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it> Message-ID: 2009/3/19 Luigi Rizzo : > Hi, > > Fabio Checconi and I have been thinking on how to implement "proxy" > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > port, do not do any data transformation, and can be transparently > inserted or removed on top of a provider port while the tree is > actively moving data. > > Our immediate need was to add/remove a scheduler while a disk is > mounted, but there are possibly other uses e.g. if one wants to > "sniff" the traffic through a disk, or do other ops that are > transparent for the data stream. > > We would like opinion on the following solution, which seems > extremely simple in terms of implementation. > > The idea is to intercept requests coming on a provider port, pp, and > redirect them to a geom node acting as a proxy if the port > is configured in this way: > > ? ? +=====...===...======+ > ? ? H ? ? ? ? ? ? ? ? ? ?H > ? ? H ? ? ? ? ? ? ? ? ? ?H > ? ? H ? ? ? ? ? ? ? ? ? ?H > ? ? +=====...====== cp ==+ > ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ?+---------------+ > ? ? ? ? ? ? ? ? ? ? V ? ? ? ? ?| ? ? ? ? ? ? ? V > ? ? +=====.....==== pp ==+ ? ? | ? ?+======= proxy_pp ==+ > ? ? H ? ? ? ? ? 'ad0s1' ?H ? ? | ? ?H ? ? ? ? ? ? ? ? ? H > ? ? H ? ? ? ? ? ? ? ?------->--+ ? ?H ? ? ? ? ? ? ? ? ? H > ? ? H ? ? ? ?gp ? ? ?-------<--+ ? ?H ? ?proxy_node ? ? H > ? ? H ? ? ? ? ? ? ? ? ? ?H ? ? | ? ?H ? ? ? ? ? ? ? ? ? H > ? ? +=======....===...===+ ? ? | ? ?+======= proxy_cp ==+ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| ? ? ? ? ? ? ? V > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?+---------------+ > > Normally the proxy does not exist, and the geom tree works as it does now. > > When we create a 'proxy' node, with something like > > ? ? ? ?geom my_proxy_class proxy ad0s1 > > we do something very similar to a 'create', but: > > - the proxy node is marked specially in gp->flags, so the core will > ?not generate a g_new_provider_event when the provider port is created > ?(this means there is no taste() call and nobody should be able > ?to attach to the port). > > - the provider port we attach to is linked, with two pointers, > ?to the provider and consumer ports of the proxy_node. > > In this situation, g_io_request() finds that port pp has a proxy attached > to it, and immediately redirects the requests to the proxy, which > does everything a geom node does (cloning requests, etc). > ? When the proxy wants to pass the request down, it sends it again to pp, > but now there is no redirection because the source can be identified > as the proxy. ?The pointers in the bio insure a correct flow of the > requests on the reverse path. > > Disconnecting a proxy is almost trivial: apart from handling possible > races on the data path, we just need to clear pp->proxy_pp and pp->proxy_cp. > After that, we can just send the regular destroy events to the proxy > node, who will have to take care of flushing any pending bio's (e.g. > see our geom_sched node that already does this). > > Overall the change is very small (see attached patch): > a couple of lines in g_io_request, two extra fields in the g_provider, > and the addition of a flag to gp->flags to control the generation > of g_new_provider_event. > There is basically no overhead on regular operation, and only > a couple of extra pointers in struct g_provider (we use a spare > bit in gp->flags to mark G_GEOM_PROXY nodes). > > The only things missing in the patch should be: > > - a check to avoid races on creation&destruction of a proxy. > ?I am not so sure on how to achieve this, but creation and destruction > ?are rare and can normally wait, so we could just piggyback the > ?small critical section (manipulating pp->proxy_cp and pp->proxy_cp) > ?into some other piece of code that is guaranteed to be race-free. > > - a check to prevent attaching to a provider port of a proxy > ?(not a problem, i believe); > > - a check to prevent attaching a proxy to a provider port that already > ?has one. Of course you can attach a proxy to another proxy, and > ?if you want to change the order it is as simple as removing the > ?existing proxy and reattaching it after the new one. > > Feedback welcome. I wonder if it's really necessary to alter the GEOM infrastructure or if it is possible to do this with what's there already. Just an idea: Lock g_topology, put g_down and g_up to sleep, alter the consumer and provider pointers where you need it so the everything is routed through your proxy class (which isn't special in any way) and restart g_down and g_up. From rizzo at iet.unipi.it Thu Mar 19 05:56:52 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Thu Mar 19 05:56:58 2009 Subject: RFC: adding 'proxy' nodes to provider ports (with patch) In-Reply-To: References: <20090319081936.GA32750@onelab2.iet.unipi.it> Message-ID: <20090319130137.GB40489@onelab2.iet.unipi.it> On Thu, Mar 19, 2009 at 12:41:13PM +0100, Marius N?nnerich wrote: > 2009/3/19 Luigi Rizzo : > > Hi, > > > > Fabio Checconi and I have been thinking on how to implement "proxy" > > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > > port, do not do any data transformation, and can be transparently > > inserted or removed on top of a provider port while the tree is > > actively moving data. ... > > The idea is to intercept requests coming on a provider port, pp, and > > redirect them to a geom node acting as a proxy if the port > > is configured in this way: ... > I wonder if it's really necessary to alter the GEOM infrastructure or > if it is possible to do this with what's there already. Just an idea: > Lock g_topology, put g_down and g_up to sleep, alter the consumer and > provider pointers where you need it so the everything is routed > through your proxy class (which isn't special in any way) and restart > g_down and g_up. we'll look into this, thanks. If we can spare the extra fields in the g_provider, the thing is even easier to do. I just don't know how your suggestion interferes with the naming: if I change the pointers, the name of a provider will not be anymore a prefix of the name of the node attached above. But maybe that is not an architectural requirements but just a convenient convention. cheers luigi From zachary.loafman at isilon.com Thu Mar 19 09:54:53 2009 From: zachary.loafman at isilon.com (Zachary Loafman) Date: Thu Mar 19 09:54:59 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: <20090315205229.GV55200@elvis.mu.org> Message-ID: <20090319164251.GA13081@zloafman.west.isilon.com> First off, I wanted to start by saying something that may interest the community at large: We (Isilon) recently staffed a small NFS group. Our intention is to use and extend Rick's awesome effort. We will have three full-time employees working on producitizing it for us "soon" - by mid-May all three employees should be working on v4. It is our intention to give the work back, but we're still trying to work out our branching/upstreaming model. I don't know if that affects the timing on this being merged to CURRENT or not. It might be nice if we had an opportunity to review some things prior to APIs/VOPs being set in stone, but it would also be nice to get wider exposure for Rick's code. On Sun, Mar 15, 2009 at 05:20:20PM -0400, Rick Macklem wrote: > On Sun, 15 Mar 2009, Alfred Perlstein wrote: > > > >I think it wise to look at 4.1 and scoping that out before taking > >the time to integrate this to gain an understanding of: > NFSv4.1 is still way out there. It hasn't reached RFC stage yet and > vendors are only testing bits and pieces of it. (The current draft > of the "minor" revision is over 500 pages.) > > All the code vendors are currently shipping is running 4.0. I think v4.1 is closer than you might think. We've received numerous requests for pNFS, and I think many vendors will ship basic 4.1 stacks this year. > >1) what it would take to get to 4.1? > A lot. A required feature is something for handling RPC transport > called sessions. One guy has been looking at doing sessions for > FreeBSD (hopefully integrated with Doug Rabson's new RPC code), > but I have no idea if he has made any progress. Can you put us in contact? I'd like to avoid duplication of effort here. > >2) how we would interoperate with other machines until we > >get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become > >the defacto standard (is it already?)? > Systems should still support 4.0 for a long time. I have no idea > when 4.1 will become a defacto standard, but I'd guess years. We've idly been considering going 4.1-only given the relatively slow adoption of 4.0. 4.1 has created a fair amount of buzz and may raise adoption of 4.x. I can't really say for sure. Nor can I say for sure what we'd eventually settle on, since the relative cost of 4.0 once you have 4.1 is fairly small. > I've tried reading the drafts and got swamped. Honestly, I think a > 4.1 implementation would take man years of effort and is beyond > what I am capable of. I hope we can help. :) -- Zach Loafman | Staff Engineer | Isilon Systems From rwatson at FreeBSD.org Thu Mar 19 11:04:36 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Mar 19 11:04:42 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: <20090319164251.GA13081@zloafman.west.isilon.com> References: <20090315205229.GV55200@elvis.mu.org> <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: On Thu, 19 Mar 2009, Zachary Loafman wrote: > First off, I wanted to start by saying something that may interest the > community at large: We (Isilon) recently staffed a small NFS group. Our > intention is to use and extend Rick's awesome effort. We will have three > full-time employees working on producitizing it for us "soon" - by mid-May > all three employees should be working on v4. It is our intention to give the > work back, but we're still trying to work out our branching/upstreaming > model. > > I don't know if that affects the timing on this being merged to CURRENT or > not. It might be nice if we had an opportunity to review some things prior > to APIs/VOPs being set in stone, but it would also be nice to get wider > exposure for Rick's code. If we adopt Rick's code as a "second" experimental stack, and use the existing stack as the default, then I think we can reasonably claim that the binary interfaces specific to it can be changed, especially if we do a good job of tagging which interfaces those are. I have proposed Rick for a commit bit with the hopes of getting the NFS code in the tree sooner rather than later so that it can get exposure, etc, however. BTW, this news from Isilon sounds excellent, and is something that the community as a whole will appreciate a great deal. Something I would like to see happen in the quite short term is get more hands looking at Edward's NFSv4 ACL implementation. Rick's code supports it, but since the NFSv4 ACL code requires rolling our ACL ABI, writes things to disk, etc, I think it's actually much more sensitive to binary compatibility concerns. I'd like to get as many hands as possible doing a review of the semantics (especially UFS, POSIX.1e, Solaris/ZFS, NFSv4, Samba, smbfs, Mac OS X, etc compatability concerns) in the next month or so as possible. Having done the POSIX.1e ACL support almost ten years ago now, I can say that you get stuck with a lot of things you regret if you're not careful, and that the most important concern is semantic interoperability with other systems. The plan right now is to have a semi-formal NFSv4 on FreeBSD session (mini-summit) at the developer summit, FYI, with a goal of working out a number of things relating to NFSv4, the future of NFS on FreeBSD, and the 8.x release. Among other things, I hope we can come out with a clear set of "stuff that needs to be done", maybe even a "who's doing it". Robert N M Watson Computer Laboratory University of Cambridge > > On Sun, Mar 15, 2009 at 05:20:20PM -0400, Rick Macklem wrote: >> On Sun, 15 Mar 2009, Alfred Perlstein wrote: >>> >>> I think it wise to look at 4.1 and scoping that out before taking >>> the time to integrate this to gain an understanding of: >> NFSv4.1 is still way out there. It hasn't reached RFC stage yet and >> vendors are only testing bits and pieces of it. (The current draft >> of the "minor" revision is over 500 pages.) >> >> All the code vendors are currently shipping is running 4.0. > > I think v4.1 is closer than you might think. We've received numerous > requests for pNFS, and I think many vendors will ship basic 4.1 stacks > this year. > >>> 1) what it would take to get to 4.1? >> A lot. A required feature is something for handling RPC transport >> called sessions. One guy has been looking at doing sessions for >> FreeBSD (hopefully integrated with Doug Rabson's new RPC code), >> but I have no idea if he has made any progress. > > Can you put us in contact? I'd like to avoid duplication of effort here. > >>> 2) how we would interoperate with other machines until we >>> get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become >>> the defacto standard (is it already?)? >> Systems should still support 4.0 for a long time. I have no idea >> when 4.1 will become a defacto standard, but I'd guess years. > > We've idly been considering going 4.1-only given the relatively slow > adoption of 4.0. 4.1 has created a fair amount of buzz and may raise > adoption of 4.x. I can't really say for sure. Nor can I say for sure > what we'd eventually settle on, since the relative cost of 4.0 once you > have 4.1 is fairly small. > >> I've tried reading the drafts and got swamped. Honestly, I think a >> 4.1 implementation would take man years of effort and is beyond >> what I am capable of. > > I hope we can help. :) > > -- > Zach Loafman | Staff Engineer | Isilon Systems > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From rmacklem at uoguelph.ca Thu Mar 19 11:40:51 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Thu Mar 19 11:40:58 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: <20090319164251.GA13081@zloafman.west.isilon.com> References: <20090315205229.GV55200@elvis.mu.org> <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: On Thu, 19 Mar 2009, Zachary Loafman wrote: > First off, I wanted to start by saying something that may interest the > community at large: We (Isilon) recently staffed a small NFS group. Our > intention is to use and extend Rick's awesome effort. We will have three > full-time employees working on producitizing it for us "soon" - by > mid-May all three employees should be working on v4. It is our intention > to give the work back, but we're still trying to work out our > branching/upstreaming model. > Sounds like good news to me. > I don't know if that affects the timing on this being merged to CURRENT > or not. It might be nice if we had an opportunity to review some things > prior to APIs/VOPs being set in stone, but it would also be nice to get > wider exposure for Rick's code. > The only VFS change I've done is redefining the va_filerev/i_modrev attribute/i-node field such that it satisfies the requirements of the nfsv4 Change attribute. (Way back when, i_modrev was put in for not quite nfs by me, I think? I don't think anything else uses it, but I can't be sure?) The changes are: 1 - move it from the in memory i-node to spare space at the end of the on-disk i-node, so that it will survive a server crash. 2 - incrementing it for metadata changes as well as data changes (I can't think how having it survive a crash could break any app. that might be using it. The fact that it's value would change for metadata changes as well as data changes might affect something?) Does anyone know of an app. that uses the va_filerev attribute? [stuff snipped] > I think v4.1 is closer than you might think. We've received numerous > requests for pNFS, and I think many vendors will ship basic 4.1 stacks > this year. > I'll email on nfsv4@linux-nfs.org and ask. (I haven't been to a testing event, but I'd be under an NDA if I had been. My impression was that no one had SSV working yet and, although there had been a fair amount of pNFS testing, sessions were lagging behind. No sessions (and SSV is the front end part of sessions) and no 4.1. (I wouldn't be surprised if some server vendors are getting close, but without a Linux client...) > > Can you put us in contact? I'd like to avoid duplication of effort here. > I'll ping him via email and then contact you. [good stuff snipped] rick From rmacklem at uoguelph.ca Fri Mar 20 08:58:33 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Fri Mar 20 08:58:39 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: <20090319164251.GA13081@zloafman.west.isilon.com> References: <20090315205229.GV55200@elvis.mu.org> <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: On Thu, 19 Mar 2009, Zachary Loafman wrote: [good stuff snipped] > > I don't know if that affects the timing on this being merged to CURRENT > or not. It might be nice if we had an opportunity to review some things > prior to APIs/VOPs being set in stone, but it would also be nice to get > wider exposure for Rick's code. > Other than the va_filerev/i_modrev issue already mentioned (btw, I grep'd /usr/src and nothing uses va_filerev outside the kernel, it seems), I realized there is another API/VOP issue (kinda in the trivial category, but I should mention it). FreeBSD-CURRENT is out of bits for mnt_flag (the last spare one is used by Edward's ACL code) and it would be nice to add a couple of MNT_EXxxx flags for things like enabling/disabling delegations. Two obvious possible solutions: 1 - bump mnt_flag up to 64 bits 2 - create a separate mnt_exflag field [more stuff snipped] > > I think v4.1 is closer than you might think. We've received numerous > requests for pNFS, and I think many vendors will ship basic 4.1 stacks > this year. > I emailed a request for "predictions" about this on nfsv4@linux-nfs.org and I've only gotten one response sofar: [Tom Haynes wrote] Subject: Re: nfsv4.1 timeframe Rick Macklem wrote: > Anyone feel like making a prediction? (Ideally somewhat realistic:-) > Next year sounds more realistic based on what I saw at Connectathon. I'll post more responses if I get them, rick From jhb at freebsd.org Fri Mar 20 12:46:07 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Mar 20 12:46:13 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: <200903201538.45268.jhb@freebsd.org> On Friday 20 March 2009 12:03:57 pm Rick Macklem wrote: > > On Thu, 19 Mar 2009, Zachary Loafman wrote: > > [good stuff snipped] > > > > I don't know if that affects the timing on this being merged to CURRENT > > or not. It might be nice if we had an opportunity to review some things > > prior to APIs/VOPs being set in stone, but it would also be nice to get > > wider exposure for Rick's code. > > > Other than the va_filerev/i_modrev issue already mentioned (btw, I grep'd > /usr/src and nothing uses va_filerev outside the kernel, it seems), I > realized there is another API/VOP issue (kinda in the trivial category, > but I should mention it). FreeBSD-CURRENT is out of bits for mnt_flag (the > last spare one is used by Edward's ACL code) and it would be nice to add a > couple of MNT_EXxxx flags for things like enabling/disabling delegations. > > Two obvious possible solutions: > 1 - bump mnt_flag up to 64 bits > 2 - create a separate mnt_exflag field FreeBSD in general is moving away from new flags in mnt_flags I believe. At least if you purely need NFS-specific flags you can just add new string options via nmount(2) and set appropriate settings in private flags field in your NFS-specific mount data. -- John Baldwin From rmacklem at uoguelph.ca Fri Mar 20 12:55:12 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Fri Mar 20 12:55:18 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: <200903201538.45268.jhb@freebsd.org> References: <20090319164251.GA13081@zloafman.west.isilon.com> <200903201538.45268.jhb@freebsd.org> Message-ID: On Fri, 20 Mar 2009, John Baldwin wrote: > > FreeBSD in general is moving away from new flags in mnt_flags I believe. At > least if you purely need NFS-specific flags you can just add new string > options via nmount(2) and set appropriate settings in private flags field in > your NFS-specific mount data. > Ok, thanks. I vaguely knew about nmount(2), but didn't realize it was being used for server exports. (The MNT_EXxxx flags are/were used by the NFS server and are relevant when set on an exported local fs like UFS/FFS. As such, they are either generic mount stuff or UFS/FFS specific. I think ZFS keeps its own export stuff, so making it UFS/FFS specific might make sense?) Anyhow, it's not urgent, since the server works ok without any additional flags. rick From jhb at freebsd.org Fri Mar 20 14:38:18 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Mar 20 14:38:24 2009 Subject: NFS version 4.0 for FreeBSD-CURRENT In-Reply-To: References: <200903201538.45268.jhb@freebsd.org> Message-ID: <200903201657.58470.jhb@freebsd.org> On Friday 20 March 2009 4:00:38 pm Rick Macklem wrote: > > On Fri, 20 Mar 2009, John Baldwin wrote: > > > > > FreeBSD in general is moving away from new flags in mnt_flags I believe. At > > least if you purely need NFS-specific flags you can just add new string > > options via nmount(2) and set appropriate settings in private flags field in > > your NFS-specific mount data. > > > Ok, thanks. I vaguely knew about nmount(2), but didn't realize it was > being used for server exports. (The MNT_EXxxx flags are/were used by the > NFS server and are relevant when set on an exported local fs like UFS/FFS. > As such, they are either generic mount stuff or UFS/FFS specific. I > think ZFS keeps its own export stuff, so making it UFS/FFS specific > might make sense?) > > Anyhow, it's not urgent, since the server works ok without any additional > flags. Ahhh, ok, that is different then. I must have misunderstood. If it is flags the NFS server sets on arbitrary filesystems to manage exporting then a 'mnt_exflags' field might be best. -- John Baldwin From rwatson at FreeBSD.org Sat Mar 21 05:46:18 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Sat Mar 21 05:46:23 2009 Subject: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) In-Reply-To: References: <20080526110543.J26343@fledge.watson.org> Message-ID: On Mon, 16 Feb 2009, Robert Watson wrote: > The following schedule is proposed, assuming nothing goes horribly wrong > with the new USB code in the next few weeks, and remaining nits relating to > USB network and 802.11 drivers are handled: > > 16 February 2009 HEADS UP to lists (this e-mail) > 01 March 2009 Disable build of all IFF_NEEDSGIANT drivers in 8.x > 01 April 2009 Remove all IFF_NEEDSGIANT drivers from 8.x Just a teminder that 1 April is gradually approaching. At that point I will remove from the tree the various non-MPSAFE network device drivers that are already disconnected from the build, incuding if_sl, if_ppp, if_ar, and the old USB ethernet drivers. Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From bugmaster at FreeBSD.org Mon Mar 23 04:06:53 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 23 04:07:32 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200903231106.n2NB6qg6003937@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From chuckr at telenix.org Sat Mar 28 17:36:59 2009 From: chuckr at telenix.org (Chuck Robey) Date: Sat Mar 28 17:37:05 2009 Subject: Final sanity pass: xdev In-Reply-To: <18881.42546.640583.971867@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> <49C19F2A.10406@elischer.org> <18881.42546.640583.971867@gromit.timing.com> Message-ID: <49CEBC72.1090601@telenix.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 John Hein wrote: > Julian Elischer wrote at 18:26 -0700 on Mar 18, 2009: > > John Hein wrote: I just ran into this ... I do pretty well with Makefiles, I can do all that playing myself, I was just wondering if you folks have done any benchmarking with new toy, and how it stacks up on FreeBSD current in the area of ... well, in too many areas to even begin to describe! Any mails discussing this (from a FreeBSD standpoint, doing our kernel & world? The thing that mosts concerns me is if it results with a compatible kernel, because I guess I can work out the rest. If you don't care to rehash what seems obvious to you, just tell me if it comes up with a compatible image. What a neat new toy! > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknOvHEACgkQz62J6PPcoOkwyQCdEYnxb5EWZW0BLnVd1wYRQdoU N50An0Cn8bFJLPZiA9231Z4KiN6gd5cl =vCv6 -----END PGP SIGNATURE----- From chuckr at telenix.org Sat Mar 28 17:37:13 2009 From: chuckr at telenix.org (Chuck Robey) Date: Sat Mar 28 17:37:19 2009 Subject: Final sanity pass: xdev In-Reply-To: <18881.42546.640583.971867@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> <49C19F2A.10406@elischer.org> <18881.42546.640583.971867@gromit.timing.com> Message-ID: <49CEC2C0.3020902@telenix.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I was going to ask in ports about this, but you folks really sound like you know the llvm tool a whole lot better, and if, in answering my question, you happened to answer things like benchmark results and if it makes good kernels, well, you wouldn't hear me complaining. Anyhow, my question regards the llvm-devel port. The install phase wants to build python26. Is that because the author of the port likes python26, or would it really work if I used my presently available python25? I could fix my makefile, if it'd work ok, but I need to understand why our llvm port needs python26, before I shoot myself in the foot. Maybe I just ought to be thankful it doesn't go off and try to install python30, which I truly have problems with. I just need to know, I hope one of you folks knows the reason why. In fact, needing that python26 would really stop this from ever getting into the base, wouldn't it? Not that I'd mind a good excuse to get python into our base, but I don't think anyone else would agree with me. Thanks -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknOwsAACgkQz62J6PPcoOn9SgCggZFLfGLcWtxUq9oFKl8OPBgG W4gAni7HrGCrBQ9bgW8mcsCE1DzwBpHc =ZR9q -----END PGP SIGNATURE----- From des at des.no Sat Mar 28 19:00:44 2009 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Sat Mar 28 19:00:52 2009 Subject: Final sanity pass: xdev In-Reply-To: <49CEC2C0.3020902@telenix.org> (Chuck Robey's message of "Sat, 28 Mar 2009 20:37:20 -0400") References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> <49C19F2A.10406@elischer.org> <18881.42546.640583.971867@gromit.timing.com> <49CEC2C0.3020902@telenix.org> Message-ID: <86wsa96r3v.fsf@ds4.des.no> Chuck Robey writes: > In fact, needing that python26 would really stop this from ever > getting into the base, wouldn't it? Not that I'd mind a good excuse > to get python into our base, but I don't think anyone else would agree > with me. Trust me, you *don't* want python (or any other widely used script language) in base. There's a good reason why we removed Tcl and Perl. Imagine a having Python 2.5 is in base, and needing to run a mission- critical application that requires 2.6. You would have no choice but to install 2.6 from ports, and you would run into all sorts of problems with the two versions interfering with each other, scripts mysteriously failing because they were invoked with the wrong $PATH and therefore the wrong interpreter, code running in the 2.5 interpreter trying to load 2.6 modules, etc. DES -- Dag-Erling Sm?rgrav - des@des.no From rdivacky at freebsd.org Sat Mar 28 23:57:20 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Sat Mar 28 23:57:27 2009 Subject: Final sanity pass: xdev In-Reply-To: <49CEC2C0.3020902@telenix.org> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> <49C19F2A.10406@elischer.org> <18881.42546.640583.971867@gromit.timing.com> <49CEC2C0.3020902@telenix.org> Message-ID: <20090329065627.GA4994@freebsd.org> On Sat, Mar 28, 2009 at 08:37:20PM -0400, Chuck Robey wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > > I was going to ask in ports about this, but you folks really sound like you know > the llvm tool a whole lot better, and if, in answering my question, you > happened to answer things like benchmark results and if it makes good kernels, > well, you wouldn't hear me complaining. > > Anyhow, my question regards the llvm-devel port. The install phase wants to > build python26. Is that because the author of the port likes python26, or would > it really work if I used my presently available python25? I could fix my > makefile, if it'd work ok, but I need to understand why our llvm port needs > python26, before I shoot myself in the foot. the llvm/clang ships with a compiler driver that is written in python, the driver is being replaced by a C++ one. the C++ one currently works "just fine". expect the python dependency to be removed in a near future roman From phk at phk.freebsd.dk Mon Mar 30 01:04:28 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 01:04:42 2009 Subject: Final sanity pass: xdev In-Reply-To: Your message of "Sun, 29 Mar 2009 03:43:32 +0200." <86wsa96r3v.fsf@ds4.des.no> Message-ID: <10067.1238399116@critter.freebsd.dk> In message <86wsa96r3v.fsf@ds4.des.no>, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= wr ites: >Chuck Robey writes: >> In fact, needing that python26 would really stop this from ever >> getting into the base, wouldn't it? Not that I'd mind a good excuse >> to get python into our base, but I don't think anyone else would agree >> with me. > >Trust me, you *don't* want python (or any other widely used script >language) in base. There's a good reason why we removed Tcl and Perl. Actually, I think there is an obvious avenue for doing stuff like this: rename it. I faced this problem with libexpat for XML parsing and simply calling the library something else solved the problem entirely: We have had libexpat in src for seven years and not once has a port tripped over it. There is nothing preventing us from importing a scripting language[1] as long as we call it "bsdrun". The important distinction here, is that we import a language to be FreeBSD's built-in scripting language, we do *NOT* do it to make it easier to run tcl, perl or python programs under FreeBSD. Poul-Henning [1] Meet at dawn, with loaded weapons, in front of the cathedral to decide which, last man standing tells us which[2]. [2] If we do it, I would still argue for Tcl, because it is built to be embedded, and small enough that you can put it into programs like inetd, syslogd and similar[3] [3] But I'm not going to be present at the Cathedral, because just like our logo, this will not be decided on technical merit but by who has the larger fan-club. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Mar 30 02:28:18 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 02:28:25 2009 Subject: Final sanity pass: xdev In-Reply-To: Your message of "Mon, 30 Mar 2009 10:13:21 +0100." <49D08D31.1050209@beardz.net> Message-ID: <71785.1238405296@critter.freebsd.dk> In message <49D08D31.1050209@beardz.net>, Jase Thew writes: >Poul-Henning Kamp wrote: > >> [2] If we do it, I would still argue for Tcl, because it is built to >> be embedded, and small enough that you can put it into programs >> like inetd, syslogd and similar[3] >> > >Wouldn't those properties mean that Lua is actually a better choice? ;) http://maps.google.com/maps?ll=48.853024,2.349711&spn=0.001318,0.002701 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From bazerka at beardz.net Mon Mar 30 02:31:12 2009 From: bazerka at beardz.net (Jase Thew) Date: Mon Mar 30 02:31:19 2009 Subject: Final sanity pass: xdev In-Reply-To: <10067.1238399116@critter.freebsd.dk> References: <10067.1238399116@critter.freebsd.dk> Message-ID: <49D08D31.1050209@beardz.net> Poul-Henning Kamp wrote: > [2] If we do it, I would still argue for Tcl, because it is built to > be embedded, and small enough that you can put it into programs > like inetd, syslogd and similar[3] > Wouldn't those properties mean that Lua is actually a better choice? ;) Regards, Jase. From bugmaster at FreeBSD.org Mon Mar 30 04:06:49 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 30 04:07:25 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200903301106.n2UB6mi6054665@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From rmacklem at uoguelph.ca Mon Mar 30 08:04:05 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Mon Mar 30 08:04:12 2009 Subject: Unique machine identifier for nfsv4 client Message-ID: An nfsv4 client needs a unique identifier that is persistent across reboots. My code currently uses the MAC address of the first net interface that it can find that has an Ethernet style MAC. There are two problems with this: 1 - If it can't find a net interface with a MAC, it falls back on a randomly generated identifier, which changes when it is rebooted. 2 - The code that finds the MAC breaks layering (ie knows stuff about the network code like struct ifnet) and only builds if "options VIMAGE_GLOBALS" is set. Can anyone suggest a better way to do this? If not, would it be reasonable to put a function in sys/net that returns a MAC address if it can find one? (At least this gets around the "options VIMAGE_GLOBALS" issue.) Thanks in advance for any help, rick From ed at 80386.nl Mon Mar 30 08:29:17 2009 From: ed at 80386.nl (Ed Schouten) Date: Mon Mar 30 08:29:24 2009 Subject: Unique machine identifier for nfsv4 client In-Reply-To: References: Message-ID: <20090330152914.GW73108@hoeg.nl> * Rick Macklem wrote: > An nfsv4 client needs a unique identifier that is persistent across > reboots. My code currently uses the MAC address of the first net > interface that it can find that has an Ethernet style MAC. Don't we generate a unique uuid on startup? It is stored in the kern.hostuuid sysctl. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090330/125e3b32/attachment.pgp From ed at 80386.nl Mon Mar 30 08:33:53 2009 From: ed at 80386.nl (Ed Schouten) Date: Mon Mar 30 08:34:00 2009 Subject: Unique machine identifier for nfsv4 client In-Reply-To: <20090330152914.GW73108@hoeg.nl> References: <20090330152914.GW73108@hoeg.nl> Message-ID: <20090330153351.GX73108@hoeg.nl> * Ed Schouten wrote: > Don't we generate a unique uuid on startup? It is stored in the ^^^^^^^ > kern.hostuuid sysctl. Errrr... I mean: it gets generated when booting FreeBSD the for first time. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090330/a0369440/attachment.pgp From rpaulo at freebsd.org Mon Mar 30 08:42:00 2009 From: rpaulo at freebsd.org (Rui Paulo) Date: Mon Mar 30 08:42:06 2009 Subject: Unique machine identifier for nfsv4 client In-Reply-To: References: Message-ID: <3E2EA002-4C26-4D44-ACEE-AD0F782F268E@freebsd.org> On 30 Mar 2009, at 16:09, Rick Macklem wrote: > An nfsv4 client needs a unique identifier that is persistent across > reboots. My code currently uses the MAC address of the first net > interface that it can find that has an Ethernet style MAC. > > There are two problems with this: > 1 - If it can't find a net interface with a MAC, it falls back on a > randomly generated identifier, which changes when it is rebooted. > 2 - The code that finds the MAC breaks layering (ie knows stuff about > the network code like struct ifnet) and only builds if > "options VIMAGE_GLOBALS" is set. This doesn't answer the question, but have you thought about the possibility of the using the UUID that the system provides? A lot of new machines come with UUIDs. -- Rui Paulo -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090330/989afb5f/PGP.pgp From gordon at tetlows.org Mon Mar 30 08:47:29 2009 From: gordon at tetlows.org (Gordon Tetlow) Date: Mon Mar 30 08:47:35 2009 Subject: Unique machine identifier for nfsv4 client In-Reply-To: References: Message-ID: <4e571dd70903300825q63b4344bhdee6ba7c54c2deab@mail.gmail.com> On Mon, Mar 30, 2009 at 8:09 AM, Rick Macklem wrote: > An nfsv4 client needs a unique identifier that is persistent across > reboots. My code currently uses the MAC address of the first net > interface that it can find that has an Ethernet style MAC. > > There are two problems with this: > 1 - If it can't find a net interface with a MAC, it falls back on a > randomly generated identifier, which changes when it is rebooted. > 2 - The code that finds the MAC breaks layering (ie knows stuff about > the network code like struct ifnet) and only builds if > "options VIMAGE_GLOBALS" is set. > > Can anyone suggest a better way to do this? > > If not, would it be reasonable to put a function in sys/net that returns > a MAC address if it can find one? (At least this gets around the > "options VIMAGE_GLOBALS" issue.) > There is a uuid that is generated at first boot and persistent. Check /etc/rc.d/hostid. It sets kern.hostuuid and kern.hostid. Gordon From rmacklem at uoguelph.ca Mon Mar 30 10:01:58 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Mon Mar 30 10:02:04 2009 Subject: Unique machine identifier for nfsv4 client In-Reply-To: <4e571dd70903300825q63b4344bhdee6ba7c54c2deab@mail.gmail.com> References: <4e571dd70903300825q63b4344bhdee6ba7c54c2deab@mail.gmail.com> Message-ID: On Mon, 30 Mar 2009, Gordon Tetlow wrote: > > There is a uuid that is generated at first boot and persistent. Check > /etc/rc.d/hostid. It sets kern.hostuuid and kern.hostid. > Duh (as Homer S might say;-). Yep, that should work just fine. From a quick glance at the rc.d script, the sysctl that loads it into the kernel happens before any nfs client stuff. (I also noticed that kern_uuid.c has a routine that gets a MAC, but I won't even need it.) Thanks everyone for the replies, rick From xcllnt at mac.com Mon Mar 30 11:33:23 2009 From: xcllnt at mac.com (Marcel Moolenaar) Date: Mon Mar 30 11:33:33 2009 Subject: On errno Message-ID: <8321954E-5CFF-45F9-9F87-BE83659E4C8D@mac.com> All, I've often found myself going over the existing error codes in search for one that seems to match best or is the least unsuitable. This begs the question: what is stopping us from adding new error codes? With so many drivers returning ENXIO whenever something (i.e anything) is wrong, how meaningful is ENXIO in diagnosing problems? What do the various standards dictate or allow us to do? -- Marcel Moolenaar xcllnt@mac.com From phk at phk.freebsd.dk Mon Mar 30 11:43:30 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 11:43:36 2009 Subject: On errno In-Reply-To: Your message of "Mon, 30 Mar 2009 10:31:56 MST." <8321954E-5CFF-45F9-9F87-BE83659E4C8D@mac.com> Message-ID: <93378.1238438607@critter.freebsd.dk> In message <8321954E-5CFF-45F9-9F87-BE83659E4C8D@mac.com>, Marcel Moolenaar wri tes: >With so many drivers returning ENXIO whenever something (i.e >anything) is wrong, how meaningful is ENXIO in diagnosing >problems? > >What do the various standards dictate or allow us to do? Long time ago, I proposed a scheme where a process can register a userland error-text buffer with the kernel. Whenever a system call returns with error, the kernel has the opportunity to write an explanatory text in the registered buffer (if there is one). That is not only backwards and standards compatible, but it is also much more expressive than errno. If we start with teaching err(3) function about it, we even get a lot of coverage right away. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From kientzle at freebsd.org Mon Mar 30 12:13:53 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Mon Mar 30 12:14:10 2009 Subject: On errno In-Reply-To: <93378.1238438607@critter.freebsd.dk> References: <93378.1238438607@critter.freebsd.dk> Message-ID: <49D115B9.7030501@freebsd.org> Poul-Henning Kamp wrote: > In message <8321954E-5CFF-45F9-9F87-BE83659E4C8D@mac.com>, Marcel Moolenaar wri > tes: > >> With so many drivers returning ENXIO whenever something (i.e >> anything) is wrong, how meaningful is ENXIO in diagnosing >> problems? >> >> What do the various standards dictate or allow us to do? POSIX does specify the range of allowable error codes for a lot of system calls, but not all. In my experience, straying outside of that causes more problems than it's worth. A lot of programs make error-recovery decisions based on errno values and that can get to be a portability headache rather quickly (remember that for most software, the default is going to be "if you don't recognize the errno value, exit with a fatal error.") > Long time ago, I proposed a scheme where a process can register > a userland error-text buffer with the kernel. > > Whenever a system call returns with error, the kernel has the > opportunity to write an explanatory text in the registered > buffer (if there is one). > > That is not only backwards and standards compatible, but it is also > much more expressive than errno. > > If we start with teaching err(3) function about it, we even get > a lot of coverage right away. This is the right direction: Basically, add a new variable that augments errno instead of extending the possible values of errno. One variation, though: I would argue for another integer variable (errno_fine?) so that translations can be done in userland (instead of having to deal with I18N in the kernel) but the principle is still sound. Tim From phk at phk.freebsd.dk Mon Mar 30 12:32:45 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 12:32:51 2009 Subject: On errno In-Reply-To: Your message of "Mon, 30 Mar 2009 11:55:53 MST." <49D115B9.7030501@freebsd.org> Message-ID: <93864.1238441563@critter.freebsd.dk> In message <49D115B9.7030501@freebsd.org>, Tim Kientzle writes: >This is the right direction: Basically, add a new variable >that augments errno instead of extending the possible values of >errno. One variation, though: I would argue for another >integer variable (errno_fine?) so that translations can be >done in userland (instead of having to deal with I18N in >the kernel) but the principle is still sound. The probelm with an integer is that you cannot give details like: "partition 3 overlaps bootcode" without precreating the N^2 possible messages of that kind. I'm very sympathetic to the I18N crowd, but I just don't see a sensible solution apart from: "Live with it". Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ed at 80386.nl Mon Mar 30 12:50:06 2009 From: ed at 80386.nl (Ed Schouten) Date: Mon Mar 30 12:50:21 2009 Subject: On errno In-Reply-To: <49D115B9.7030501@freebsd.org> References: <93378.1238438607@critter.freebsd.dk> <49D115B9.7030501@freebsd.org> Message-ID: <20090330195003.GB73108@hoeg.nl> * Tim Kientzle wrote: > POSIX does specify the range of allowable error codes > for a lot of system calls, but not all. In my experience, > straying outside of that causes more problems than it's > worth. A lot of programs make error-recovery decisions > based on errno values and that can get to be a portability > headache rather quickly (remember that for most software, > the default is going to be "if you don't recognize the errno > value, exit with a fatal error.") Create a new system call that allows you to obtain a more accurate error code afterwards. We should probably call this function GetLastError(). -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090330/1d238f13/attachment.pgp From deischen at freebsd.org Mon Mar 30 12:50:10 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon Mar 30 12:50:22 2009 Subject: On errno In-Reply-To: <49D115B9.7030501@freebsd.org> References: <93378.1238438607@critter.freebsd.dk> <49D115B9.7030501@freebsd.org> Message-ID: On Mon, 30 Mar 2009, Tim Kientzle wrote: > Poul-Henning Kamp wrote: >> In message <8321954E-5CFF-45F9-9F87-BE83659E4C8D@mac.com>, Marcel Moolenaar >> wri >> tes: >> >>> With so many drivers returning ENXIO whenever something (i.e >>> anything) is wrong, how meaningful is ENXIO in diagnosing >>> problems? >>> >>> What do the various standards dictate or allow us to do? > > POSIX does specify the range of allowable error codes > for a lot of system calls, but not all. In my experience, > straying outside of that causes more problems than it's > worth. A lot of programs make error-recovery decisions > based on errno values and that can get to be a portability > headache rather quickly (remember that for most software, > the default is going to be "if you don't recognize the errno > value, exit with a fatal error.") > >> Long time ago, I proposed a scheme where a process can register >> a userland error-text buffer with the kernel. >> >> Whenever a system call returns with error, the kernel has the >> opportunity to write an explanatory text in the registered >> buffer (if there is one). >> >> That is not only backwards and standards compatible, but it is also >> much more expressive than errno. >> >> If we start with teaching err(3) function about it, we even get >> a lot of coverage right away. > > This is the right direction: Basically, add a new variable > that augments errno instead of extending the possible values of > errno. One variation, though: I would argue for another > integer variable (errno_fine?) so that translations can be > done in userland (instead of having to deal with I18N in > the kernel) but the principle is still sound. Just add the other error values in the upper half of the word, and have __errno() return with the upper half masked out. VxWorks doesn't something similar with their errno, the lower half is the error and they use the upper half to identify the module from which it came. -- DE From phk at phk.freebsd.dk Mon Mar 30 13:02:57 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 13:03:04 2009 Subject: On errno In-Reply-To: Your message of "Mon, 30 Mar 2009 21:50:03 +0200." <20090330195003.GB73108@hoeg.nl> Message-ID: <93979.1238443373@critter.freebsd.dk> In message <20090330195003.GB73108@hoeg.nl>, Ed Schouten writes: >Create a new system call that allows you to obtain a more accurate error >code afterwards. We should probably call this function GetLastError(). You have to produce the errorinformation on the spot, we don't want to keep old error messages lingering in case people will ask for them later. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ed at 80386.nl Mon Mar 30 13:47:56 2009 From: ed at 80386.nl (Ed Schouten) Date: Mon Mar 30 13:48:25 2009 Subject: On errno In-Reply-To: <93979.1238443373@critter.freebsd.dk> References: <20090330195003.GB73108@hoeg.nl> <93979.1238443373@critter.freebsd.dk> Message-ID: <20090330204755.GC73108@hoeg.nl> * Poul-Henning Kamp wrote: > In message <20090330195003.GB73108@hoeg.nl>, Ed Schouten writes: > > > >Create a new system call that allows you to obtain a more accurate error > >code afterwards. We should probably call this function GetLastError(). > > You have to produce the errorinformation on the spot, we don't want > to keep old error messages lingering in case people will ask for > them later. You know I was making a joke, right? http://msdn.microsoft.com/en-us/library/ms679360(VS.85).aspx ;-) -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090330/f11ba686/attachment.pgp From rmacklem at uoguelph.ca Mon Mar 30 14:35:23 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Mon Mar 30 14:35:31 2009 Subject: getting a callback ip address for nfsv4 client Message-ID: Well, since the last one turned out to be too easy, here's one I think is a little tougher... The nfsv4 client needs to know an ip address for the machine, that can be used by servers to do callbacks on the client. I currently use the following, which I know isn't correct, but usually works ok: loopb = htonl(INADDR_LOOPBACK); TAILQ_FOREACH(ia, &in_ifaddrhead, ia_link) { if (IA_SIN(ia)->sin_addr.s_addr != loopb) return (&(IA_SIN(ia)->sin_addr.s_addr)); } return (NULL); Now, I could just make it a constant set by an rc script (argument to the callback daemon or a sysctl variable), but that's a bother for laptops using dhcp and similar. I think allowing an argument to the callback daemon is a good fallback, but it would be nice if it didn't normally have to be set for things to work ok. Any ideas on how to do this? Thanks in advance for any help, rick ps: Part of the reason that the above loop doesn't seem to be good enough is that it requires "options VIMAGE_GLOBALS" to build. From julian at elischer.org Mon Mar 30 15:00:41 2009 From: julian at elischer.org (Julian Elischer) Date: Mon Mar 30 15:00:47 2009 Subject: getting a callback ip address for nfsv4 client In-Reply-To: References: Message-ID: <49D13E9C.8010005@elischer.org> Rick Macklem wrote: > Well, since the last one turned out to be too easy, here's one I think > is a little tougher... > > The nfsv4 client needs to know an ip address for the machine, that can > be used by servers to do callbacks on the client. I currently use the > following, which I know isn't correct, but usually works ok: > > loopb = htonl(INADDR_LOOPBACK); > TAILQ_FOREACH(ia, &in_ifaddrhead, ia_link) { > if (IA_SIN(ia)->sin_addr.s_addr != loopb) > return (&(IA_SIN(ia)->sin_addr.s_addr)); > } > return (NULL); It's important that the address you use be an address that is in the same 'security domain' as the client.. The best way to do that is to find an address that is on the interface that will be used to send the packet out. So you want to do a route lookup so use rtalloc or friends (see route.c). you want to use the same code that is used in ip_output (or is it {udp/tcp}_output?) to fidn the local address when teh user specifies INADDR_ANY. > > Now, I could just make it a constant set by an rc script (argument to > the callback daemon or a sysctl variable), but that's a bother for > laptops using dhcp and similar. I think allowing an argument to the > callback daemon is a good fallback, but it would be nice if it didn't > normally have to be set for things to work ok. > > Any ideas on how to do this? > > Thanks in advance for any help, rick > ps: Part of the reason that the above loop doesn't seem to be good > enough is that it requires "options VIMAGE_GLOBALS" to build. > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From kientzle at freebsd.org Mon Mar 30 15:35:26 2009 From: kientzle at freebsd.org (Tim Kientzle) Date: Mon Mar 30 15:35:32 2009 Subject: On errno In-Reply-To: References: <93378.1238438607@critter.freebsd.dk> <49D115B9.7030501@freebsd.org> Message-ID: <49D1492C.5050101@freebsd.org> Marcel Moolenar suggests adding new values of errno to further distinguish error conditions, leading to the following comments: Daniel Eischen wrote: > On Mon, 30 Mar 2009, Tim Kientzle wrote: >> Poul-Henning Kamp wrote: >>> Long time ago, I proposed a scheme where a process can register >>> a userland error-text buffer with the kernel. >> >> This is the right direction: Basically, add a new variable >> that augments errno instead of extending the possible values of >> errno. > > Just add the other error values in the upper half of the > word, and have __errno() return with the upper half > masked out. VxWorks doesn't something similar with > their errno,... I thought of that, but it won't work. Old binaries exist and you don't want to version every single system call. Poul-Henning Kamp wrote: > The probelm with an integer is that you cannot give details > like: > "partition 3 overlaps bootcode" > without precreating the N^2 possible messages of that kind. The standard solution if you need variable parameters, of course, is to pass the parameters back: int code: EPARTITIONOVERLAPSBOOTCODE char *default_text: "partition %1d overlaps bootcode" arg1: 3 This provides everything necessary for libc to generate an appropriate translated text. The "default_text" return is a common tweak, mostly to to make the code at the error source easier to read, though it also helps the consumer to provide something even when the translation catalogs are missing or damaged. Note that the code at the error source is essentially the same either way: generate_error(EPARTITIONOVERLAPSBOOTCODE, "partition %1d overlaps bootcode", partno); but the varargs values get passed back to userland to be formatted instead of being formatted in the kernel. Tim From phk at phk.freebsd.dk Mon Mar 30 22:22:24 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 22:22:30 2009 Subject: On errno In-Reply-To: Your message of "Mon, 30 Mar 2009 15:35:24 MST." <49D1492C.5050101@freebsd.org> Message-ID: <95823.1238476941@critter.freebsd.dk> In message <49D1492C.5050101@freebsd.org>, Tim Kientzle writes: >> The probelm with an integer is that you cannot give details >> like: >> "partition 3 overlaps bootcode" >> without precreating the N^2 possible messages of that kind. > >The standard solution if you need variable parameters, >of course, is to pass the parameters back: > int code: EPARTITIONOVERLAPSBOOTCODE > char *default_text: "partition %1d overlaps bootcode" > arg1: 3 And the "standard" solution is stupid and useless, because more often than not, some language, typically french, will want the arguments in the opposite order... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From das at FreeBSD.ORG Mon Mar 30 23:32:10 2009 From: das at FreeBSD.ORG (David Schultz) Date: Mon Mar 30 23:32:21 2009 Subject: On errno In-Reply-To: <95823.1238476941@critter.freebsd.dk> References: <49D1492C.5050101@freebsd.org> <95823.1238476941@critter.freebsd.dk> Message-ID: <20090331063159.GA9265@zim.MIT.EDU> On Tue, Mar 31, 2009, Poul-Henning Kamp wrote: > In message <49D1492C.5050101@freebsd.org>, Tim Kientzle writes: > > >> The probelm with an integer is that you cannot give details > >> like: > >> "partition 3 overlaps bootcode" > >> without precreating the N^2 possible messages of that kind. > > > >The standard solution if you need variable parameters, > >of course, is to pass the parameters back: > > int code: EPARTITIONOVERLAPSBOOTCODE > > char *default_text: "partition %1d overlaps bootcode" > > arg1: 3 > > And the "standard" solution is stupid and useless, because more > often than not, some language, typically french, will want the > arguments in the opposite order... You could get the extensibility you're looking for without annoying the i18n folks by making the strings conform to a simple machine-parsable grammar. For instance, the above might be: "geom:part:overlaps bootcode(3)" A user-level library function could parse this, look up a natural language translation in a locale-specific database, and fall back on a generic format if no translation is available. From rizzo at iet.unipi.it Mon Mar 30 23:44:55 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Mon Mar 30 23:45:03 2009 Subject: On errno In-Reply-To: <95823.1238476941@critter.freebsd.dk> References: <49D1492C.5050101@freebsd.org> <95823.1238476941@critter.freebsd.dk> Message-ID: <20090331064959.GA3516@onelab2.iet.unipi.it> On Tue, Mar 31, 2009 at 05:22:21AM +0000, Poul-Henning Kamp wrote: > In message <49D1492C.5050101@freebsd.org>, Tim Kientzle writes: > > >> The probelm with an integer is that you cannot give details > >> like: > >> "partition 3 overlaps bootcode" > >> without precreating the N^2 possible messages of that kind. > > > >The standard solution if you need variable parameters, > >of course, is to pass the parameters back: > > int code: EPARTITIONOVERLAPSBOOTCODE > > char *default_text: "partition %1d overlaps bootcode" > > arg1: 3 > > And the "standard" solution is stupid and useless, because more > often than not, some language, typically french, will want the > arguments in the opposite order... we are probably digressing but printf in glibc has specifiers to indicate which argument you want to use for each format. http://www.gnu.org/software/hello/manual/libc/Output-Conversion-Syntax.html I suppose this takes an extra pass over the format string to collect the proper type info for all arguments, so it is not not a dramatic change in the implementation of *printf. cheers luigi From phk at phk.freebsd.dk Mon Mar 30 23:46:30 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 23:46:38 2009 Subject: On errno In-Reply-To: Your message of "Tue, 31 Mar 2009 08:49:59 +0200." <20090331064959.GA3516@onelab2.iet.unipi.it> Message-ID: <96314.1238481987@critter.freebsd.dk> In message <20090331064959.GA3516@onelab2.iet.unipi.it>, Luigi Rizzo writes: >we are probably digressing but printf in glibc has specifiers to >indicate which argument you want to use for each format. > >http://www.gnu.org/software/hello/manual/libc/Output-Conversion-Syntax.html > >I suppose this takes an extra pass over the format string to collect >the proper type info for all arguments, so it is not >not a dramatic change in the implementation of *printf. Yeah, we have that crap too, and you can see how messy and slow our printf became as a result in SVN. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Mar 30 23:48:37 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Mar 30 23:48:43 2009 Subject: On errno In-Reply-To: Your message of "Tue, 31 Mar 2009 02:31:59 -0400." <20090331063159.GA9265@zim.MIT.EDU> Message-ID: <96332.1238482115@critter.freebsd.dk> In message <20090331063159.GA9265@zim.MIT.EDU>, David Schultz writes: >You could get the extensibility you're looking for without >annoying the i18n folks by making the strings conform to a simple >machine-parsable grammar. For instance, the above might be: > "geom:part:overlaps bootcode(3)" >A user-level library function could parse this, look up a natural >language translation in a locale-specific database, and fall back >on a generic format if no translation is available. Or maybe emit all messages in Esparanto and use automatic translation ? Seriously, I don't see anybody advocating that dmesg(1) output be translated, so until such time as somebody starts that pointless project, I think we can leave error messages from the kernel in the same language as pretty much everything else in our system: English. Don't overengineer it guys. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From rizzo at iet.unipi.it Mon Mar 30 23:50:43 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Mon Mar 30 23:50:50 2009 Subject: On errno In-Reply-To: <96314.1238481987@critter.freebsd.dk> References: <20090331064959.GA3516@onelab2.iet.unipi.it> <96314.1238481987@critter.freebsd.dk> Message-ID: <20090331065548.GA3851@onelab2.iet.unipi.it> On Tue, Mar 31, 2009 at 06:46:27AM +0000, Poul-Henning Kamp wrote: > In message <20090331064959.GA3516@onelab2.iet.unipi.it>, Luigi Rizzo writes: > > >we are probably digressing but printf in glibc has specifiers to > >indicate which argument you want to use for each format. > > > >http://www.gnu.org/software/hello/manual/libc/Output-Conversion-Syntax.html > > > >I suppose this takes an extra pass over the format string to collect > >the proper type info for all arguments, so it is not > >not a dramatic change in the implementation of *printf. > > Yeah, we have that crap too, and you can see how messy and slow our > printf became as a result in SVN. I have never run performance tests of printf, but it woudld be definitely interesting to figure out how expensive is the parsing of the format specifiers. cheers luigi From wollman at hergotha.csail.mit.edu Mon Mar 30 23:52:11 2009 From: wollman at hergotha.csail.mit.edu (Garrett Wollman) Date: Mon Mar 30 23:52:18 2009 Subject: On errno In-Reply-To: <95823.1238476941@critter.freebsd.dk> References: <49D1492C.5050101@freebsd.org> Message-ID: <200903310620.n2V6Kudd072936@hergotha.csail.mit.edu> In article <95823.1238476941@critter.freebsd.dk>, phk@phk.freebsd.dk writes: >And the "standard" solution is stupid and useless, because more >often than not, some language, typically french, will want the >arguments in the opposite order... Which is why printf() supports putting the arguments in arbitrary order. A much more serious objection to this model is the desire someone will certainly have to use the %s format, with the attendent pain involved in moving variable-length strings along with the arguments across the user-kernel boundary. I'd much rather have plain integers, thank you very much. At most, a small structure, in which the only pointer is a reference to a user-space argument. (So [ENOENT] and [EACCES] could have an extended error code ECOMPONENTNAME which indicates that the pointer value is a substring of the pathname argument, and identifies the beginning of the pathname component being resolved when the failure occurred. You'd probably also want to have ENOTANERROR to indicate "error return indicates an expected success condition" and EFD to indicate "integer value is the file descriptor which caused the error condition.) But all this is really irrelevant if no other operating system or standard adopts the interface. Interfaces which are peculiar to FreeBSD are rarely useful. -GAWollman From rrs at lakerest.net Tue Mar 31 04:15:12 2009 From: rrs at lakerest.net (Randall Stewart) Date: Tue Mar 31 04:15:18 2009 Subject: On errno In-Reply-To: <96332.1238482115@critter.freebsd.dk> References: <96332.1238482115@critter.freebsd.dk> Message-ID: P: I like your original idea best... its simple and seems to me a very positive way to go. Things that get complex tend to be avoided by application writers.. where simple things get picked up ;-) R On Mar 31, 2009, at 2:48 AM, Poul-Henning Kamp wrote: > In message <20090331063159.GA9265@zim.MIT.EDU>, David Schultz writes: > >> You could get the extensibility you're looking for without >> annoying the i18n folks by making the strings conform to a simple >> machine-parsable grammar. For instance, the above might be: >> "geom:part:overlaps bootcode(3)" >> A user-level library function could parse this, look up a natural >> language translation in a locale-specific database, and fall back >> on a generic format if no translation is available. > > Or maybe emit all messages in Esparanto and use automatic > translation ? > > Seriously, I don't see anybody advocating that dmesg(1) output be > translated, so until such time as somebody starts that pointless > project, I think we can leave error messages from the kernel in the > same language as pretty much everything else in our system: English. > > Don't overengineer it guys. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch- > unsubscribe@freebsd.org" > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From max at love2party.net Tue Mar 31 04:33:54 2009 From: max at love2party.net (Max Laier) Date: Tue Mar 31 04:34:00 2009 Subject: getting a callback ip address for nfsv4 client In-Reply-To: <49D13E9C.8010005@elischer.org> References: <49D13E9C.8010005@elischer.org> Message-ID: <200903311321.14955.max@love2party.net> On Monday 30 March 2009 23:50:20 Julian Elischer wrote: > Rick Macklem wrote: > > Well, since the last one turned out to be too easy, here's one I think > > is a little tougher... > > > > The nfsv4 client needs to know an ip address for the machine, that can > > be used by servers to do callbacks on the client. I currently use the > > following, which I know isn't correct, but usually works ok: > > > > loopb = htonl(INADDR_LOOPBACK); > > TAILQ_FOREACH(ia, &in_ifaddrhead, ia_link) { > > if (IA_SIN(ia)->sin_addr.s_addr != loopb) > > return (&(IA_SIN(ia)->sin_addr.s_addr)); > > } > > return (NULL); > > It's important that the address you use be an address that is in the > same 'security domain' as the client.. > > The best way to do that is to find an address that is on the interface > that will be used to send the packet out. > > So you want to do a route lookup so use rtalloc or friends > (see route.c). > > you want to use the same code that is used in ip_output (or is it > {udp/tcp}_output?) to fidn the local address when teh user specifies > INADDR_ANY. Mostly, but it's a bit more complicated than that. Basically you do a normal source address selection (i.e. your callback address is the address you'd use as source when talking to that server), but you might want to specify more restrictions. e.g. you might not want to use temporary IPv6 addresses etc. RFC 5014 suggests a userland API for source address selection (which we don't implement, yet). There are good guidelines for default behavior in there, as well - and they also apply to IPv4 (eventhough the title suggests otherwise). I'm CC'ing Bjoern who has done some work regarding source address selection recently - IIRC. > > Now, I could just make it a constant set by an rc script (argument to > > the callback daemon or a sysctl variable), but that's a bother for > > laptops using dhcp and similar. I think allowing an argument to the > > callback daemon is a good fallback, but it would be nice if it didn't > > normally have to be set for things to work ok. > > > > Any ideas on how to do this? > > > > Thanks in advance for any help, rick > > ps: Part of the reason that the above loop doesn't seem to be good > > enough is that it requires "options VIMAGE_GLOBALS" to build. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From alc at cs.rice.edu Tue Mar 31 10:48:11 2009 From: alc at cs.rice.edu (Alan Cox) Date: Tue Mar 31 10:48:18 2009 Subject: memory protection and sbrk() Message-ID: <49D252EF.7060102@cs.rice.edu> For as long as I can remember, FreeBSD's sbrk() has provided memory with execute permission enabled. Before we branch for FreeBSD 8.0, I'd like to try removing execute permission on this memory. On (non-PAE) i386 and a few of the embedded processors, this change will have little tangible effect because there is no distinction in the processor's MMU between read and execute permissions. On amd64, the only potential problems are likely with a very few applications, like the JVM, that have their own internal implementation of "malloc()" and generate code on the fly (i.e., JIT compilation). However, I have verified that at least the Sun JVM works ok. I have also checked that cvsup, which is based on Modula-3 and does not use the standard malloc(), works ok. It's also worth noting that our standard malloc() has flip-flopped over the last year or so in terms of whether it uses sbrk() or mmap() by default to acquire memory. When it uses mmap(), it does not request execute permission on the allocated memory. So, depending on whether malloc() used mmap() or sbrk(), malloc() was returning memory with different permissions. Consequently, I think that any application problems due to the lack of execute permission on memory returned by malloc() would have long since been detected. As a final sanity check, I would appreciate it if three kinds of users would test the attached patch: (1) some IA64, PowerPC, and Sparc64 users, (2) amd64-based users of "exotic" languages, like common lisp, haskell, etc. and (3) amd64-based users of other JVMs, like kaffe. My plan is to commit the attached patch to HEAD on the 7th of April unless I hear of problems. Thanks, Alan -------------- next part -------------- Index: vm/vm_unix.c =================================================================== --- vm/vm_unix.c (revision 190506) +++ vm/vm_unix.c (working copy) @@ -117,7 +117,7 @@ goto done; } rv = vm_map_insert(&vm->vm_map, NULL, 0, old, new, - VM_PROT_ALL, VM_PROT_ALL, 0); + VM_PROT_RW, VM_PROT_ALL, 0); if (rv != KERN_SUCCESS) { error = ENOMEM; goto done; From rmacklem at uoguelph.ca Tue Mar 31 13:07:47 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Tue Mar 31 13:07:54 2009 Subject: getting a callback ip address for nfsv4 client In-Reply-To: <49D13E9C.8010005@elischer.org> References: <49D13E9C.8010005@elischer.org> Message-ID: On Mon, 30 Mar 2009, Julian Elischer wrote: > > So you want to do a route lookup so use rtalloc or friends > (see route.c). > First off, I should say that, if there is a better mailing list for this kind of thing, please let me know. Also, if this fails, it is not the end of the world. All that happens is that the server doesn't issue delegations to the client, since the callbacks aren't working. Given that I'd like to avoid having code in nfs fiddling with *inp's and such, since it would have to worry about what global vars it can use, locking on them, etc., I haven't worried about what Max Laier mentioned and just used rtalloc(). Would someone like to review this function and see what you think? nfscl_getmyip(struct nfsmount *nmp, int *isinet6p) { struct route ro; struct sockaddr_in *sin, *rsin; struct rtentry *rt; u_int8_t *retp = NULL; static struct in_addr laddr; *isinet6p = 0; /* * Loop up a route for the destination address. */ if (nmp->nm_nam->sa_family == AF_INET) { bzero(&ro, sizeof (ro)); rsin = (struct sockaddr_in *)&ro.ro_dst; sin = (struct sockaddr_in *)nmp->nm_nam; rsin->sin_family = AF_INET; rsin->sin_len = sizeof (struct sockaddr_in); rsin->sin_addr = sin->sin_addr; rtalloc(&ro); rt = ro.ro_rt; if (rt != NULL) { if (rt->rt_ifp != NULL && rt->rt_ifa != NULL && ((rt->rt_ifp->if_flags & IFF_LOOPBACK) == 0) && rt->rt_ifa->ifa_addr->sa_family == AF_INET) { sin = (struct sockaddr_in *) rt->rt_ifa->ifa_addr; laddr = sin->sin_addr; retp = (u_int8_t *)&laddr; } RTFREE(rt); } #ifdef INET6 } else if (nmp->nm_nam->sa_family == AF_INET6) { struct sockaddr_in6 *sin6, *rsin6; struct route_in6 ro6; static struct in6_addr laddr6; bzero(&ro6, sizeof (ro6)); rsin6 = (struct sockaddr_in6 *)&ro6.ro_dst; sin6 = (struct sockaddr_in6 *)nmp->nm_nam; rsin6->sin6_family = AF_INET6; rsin6->sin6_len = sizeof (struct sockaddr_in6); rsin6->sin6_addr = sin6->sin6_addr; rtalloc((struct route *)&ro6); rt = ro6.ro_rt; if (rt != NULL) { if (rt->rt_ifp != NULL && rt->rt_ifa != NULL && ((rt->rt_ifp->if_flags & IFF_LOOPBACK) == 0) && rt->rt_ifa->ifa_addr->sa_family == AF_INET6) { sin6 = (struct sockaddr_in6 *) rt->rt_ifa->ifa_addr; laddr6 = sin6->sin6_addr; retp = (u_int8_t *)&laddr6; *isinet6p = 1; } RTFREE(rt); } #endif } return (retp); } Thanks everyone, for your help sofar, rick From julian at elischer.org Tue Mar 31 13:32:09 2009 From: julian at elischer.org (Julian Elischer) Date: Tue Mar 31 13:32:15 2009 Subject: getting a callback ip address for nfsv4 client In-Reply-To: References: <49D13E9C.8010005@elischer.org> Message-ID: <49D27DDF.9@elischer.org> Rick Macklem wrote: > > > On Mon, 30 Mar 2009, Julian Elischer wrote: > >> >> So you want to do a route lookup so use rtalloc or friends >> (see route.c). >> > > First off, I should say that, if there is a better mailing list for > this kind of thing, please let me know. Also, if this fails, it is > not the end of the world. All that happens is that the server doesn't > issue delegations to the client, since the callbacks aren't working. possibly freebsd-net@ might be a bit better. > > Given that I'd like to avoid having code in nfs fiddling > with *inp's and such, since it would have to worry about what global > vars it can use, locking on them, etc., I haven't worried about what > Max Laier mentioned and just used rtalloc(). > > Would someone like to review this function and see what you think? looks about right (without doing great analysis). You may be able to make it a bit simpler by calling rtalloc1() directly.. also, on might eventually think about the case where one might want to select a different routing table for NFS as one uses for other stuff.. but that can be ignored for now. As there are possibly many addresses on an interface, one needs to look for the one that has the same network as the gateway..(if it's not p2p). e.g. if you have 10.2.2.2/24 and 10.3.3.3/24 on an interface and the gateway is 10.3.3.1, then you obviously need to select the second address on that interface. > > nfscl_getmyip(struct nfsmount *nmp, int *isinet6p) > { > struct route ro; > struct sockaddr_in *sin, *rsin; > struct rtentry *rt; > u_int8_t *retp = NULL; > static struct in_addr laddr; > > *isinet6p = 0; > /* > * Loop up a route for the destination address. > */ > if (nmp->nm_nam->sa_family == AF_INET) { > bzero(&ro, sizeof (ro)); > rsin = (struct sockaddr_in *)&ro.ro_dst; > sin = (struct sockaddr_in *)nmp->nm_nam; > rsin->sin_family = AF_INET; > rsin->sin_len = sizeof (struct sockaddr_in); > rsin->sin_addr = sin->sin_addr; > rtalloc(&ro); > rt = ro.ro_rt; > if (rt != NULL) { > if (rt->rt_ifp != NULL && > rt->rt_ifa != NULL && > ((rt->rt_ifp->if_flags & IFF_LOOPBACK) == 0) && > rt->rt_ifa->ifa_addr->sa_family == AF_INET) { > sin = (struct sockaddr_in *) > rt->rt_ifa->ifa_addr; > laddr = sin->sin_addr; > retp = (u_int8_t *)&laddr; > } > RTFREE(rt); > } > #ifdef INET6 > } else if (nmp->nm_nam->sa_family == AF_INET6) { > struct sockaddr_in6 *sin6, *rsin6; > struct route_in6 ro6; > static struct in6_addr laddr6; > > bzero(&ro6, sizeof (ro6)); > rsin6 = (struct sockaddr_in6 *)&ro6.ro_dst; > sin6 = (struct sockaddr_in6 *)nmp->nm_nam; > rsin6->sin6_family = AF_INET6; > rsin6->sin6_len = sizeof (struct sockaddr_in6); > rsin6->sin6_addr = sin6->sin6_addr; > rtalloc((struct route *)&ro6); > rt = ro6.ro_rt; > if (rt != NULL) { > if (rt->rt_ifp != NULL && > rt->rt_ifa != NULL && > ((rt->rt_ifp->if_flags & IFF_LOOPBACK) == 0) && > rt->rt_ifa->ifa_addr->sa_family == AF_INET6) { > sin6 = (struct sockaddr_in6 *) > rt->rt_ifa->ifa_addr; > laddr6 = sin6->sin6_addr; > retp = (u_int8_t *)&laddr6; > *isinet6p = 1; > } > RTFREE(rt); > } > #endif > } > return (retp); > } > > Thanks everyone, for your help sofar, rick > > From rmacklem at uoguelph.ca Tue Mar 31 18:24:21 2009 From: rmacklem at uoguelph.ca (Rick Macklem) Date: Tue Mar 31 18:24:27 2009 Subject: getting a callback ip address for nfsv4 client In-Reply-To: <49D27DDF.9@elischer.org> References: <49D13E9C.8010005@elischer.org> <49D27DDF.9@elischer.org> Message-ID: On Tue, 31 Mar 2009, Julian Elischer wrote: > > looks about right (without doing great analysis). > > You may be able to make it a bit simpler by calling rtalloc1() directly.. Yep, I can see it would be. Is there a preferred interface or should I just recode it using rtalloc1()? > > As there are possibly many addresses on an interface, one needs > to look for the one that has the same network as the gateway..(if > it's not p2p). e.g. if you have 10.2.2.2/24 and 10.3.3.3/24 on > an interface and the gateway is 10.3.3.1, then you obviously > need to select the second address on that interface. > I've been looking at this and it seems to me that it "falls out in the wash". It looks like in_pcbladdr() is what udp uses for this case and it just takes the rt_ifa->... as the address. It seems like there is a separate routing table entry for each address on an interface and that the search done by rtalloc() should get the correct one from what I can see. Does that make sense? Anyhow, thanks a lot for the help, rick From julian at elischer.org Tue Mar 31 20:59:53 2009 From: julian at elischer.org (Julian Elischer) Date: Tue Mar 31 21:00:00 2009 Subject: getting a callback ip address for nfsv4 client In-Reply-To: References: <49D13E9C.8010005@elischer.org> <49D27DDF.9@elischer.org> Message-ID: <49D2E6D1.7000106@elischer.org> Rick Macklem wrote: > > > On Tue, 31 Mar 2009, Julian Elischer wrote: > >> >> looks about right (without doing great analysis). >> >> You may be able to make it a bit simpler by calling rtalloc1() directly.. > > Yep, I can see it would be. Is there a preferred interface or should I > just recode it using rtalloc1()? > >> >> As there are possibly many addresses on an interface, one needs >> to look for the one that has the same network as the gateway..(if >> it's not p2p). e.g. if you have 10.2.2.2/24 and 10.3.3.3/24 on >> an interface and the gateway is 10.3.3.1, then you obviously >> need to select the second address on that interface. >> > I've been looking at this and it seems to me that it "falls out in the > wash". It looks like in_pcbladdr() is what udp uses for this case and > it just takes the rt_ifa->... as the address. > > It seems like there is a separate routing table entry for each address > on an interface and that the search done by rtalloc() should get the > correct one from what I can see. Does that make sense? > yes as long as you take the ifa you should end up with the right one. but if you just take the ifp you need to refine to the right ifa. > Anyhow, thanks a lot for the help, rick