From bugmaster at FreeBSD.org Mon Sep 1 11:06:53 2008 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 1 11:07:23 2008 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200809011106.m81B6qAM068370@freefall.freebsd.org> Current FreeBSD problem reports Critical problems Serious problems Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From imp at bsdimp.com Sun Sep 7 04:41:51 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Sun Sep 7 04:41:58 2008 Subject: pax in /rescue Message-ID: <20080906.224018.-1303464793.imp@bsdimp.com> Is there any reason to continue to include both tar and pax in /rescue? Warner From kensmith at cse.Buffalo.EDU Sun Sep 7 05:20:17 2008 From: kensmith at cse.Buffalo.EDU (Ken Smith) Date: Sun Sep 7 05:20:23 2008 Subject: pax in /rescue In-Reply-To: <20080906.224018.-1303464793.imp@bsdimp.com> References: <20080906.224018.-1303464793.imp@bsdimp.com> Message-ID: <1220763849.29265.47.camel@bauer.cse.buffalo.edu> On Sat, 2008-09-06 at 22:40 -0600, M. Warner Losh wrote: > Is there any reason to continue to include both tar and pax in > /rescue? JFYI - despite the cpio/sysinstall issues apparently being fixed I'm still planning to switch sysinstall over to using tar. I know that's not what you're "directly" asking about but it's at least "remotely related" depending on peoples' feelings about which of the two should be removed if the answer to your question is "No". -- Ken Smith - From there to here, from here to | kensmith@cse.buffalo.edu there, funny things are everywhere. | - Theodore Geisel | -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080907/63a174d5/attachment.pgp From chinsan.tw at gmail.com Sun Sep 7 05:45:07 2008 From: chinsan.tw at gmail.com (chinsan) Date: Sun Sep 7 05:45:14 2008 Subject: pax in /rescue In-Reply-To: <20080906.224018.-1303464793.imp@bsdimp.com> References: <20080906.224018.-1303464793.imp@bsdimp.com> Message-ID: <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> On Sun, Sep 7, 2008 at 12:40 PM, M. Warner Losh wrote: > Is there any reason to continue to include both tar and pax in > /rescue? > Maybe for rescue some important file? BTW, could editors/e3 be include into /rescue? I think it's ineed to have a text editor to edit some file for rescue. Because vi(1) lives in /usr/bin. ps. e3 is a full featured text editor written in NASM assembler. It is highly optimized for size. From imp at bsdimp.com Sun Sep 7 05:57:05 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Sun Sep 7 05:57:11 2008 Subject: pax in /rescue In-Reply-To: <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> References: <20080906.224018.-1303464793.imp@bsdimp.com> <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> Message-ID: <20080906.235633.-267228782.imp@bsdimp.com> In message: <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> chinsan writes: : On Sun, Sep 7, 2008 at 12:40 PM, M. Warner Losh wrote: : > Is there any reason to continue to include both tar and pax in : > /rescue? : > : Maybe for rescue some important file? tar can read all the formats that pax can read. Why the duplication in functionality? : BTW, could editors/e3 be include into /rescue? : I think it's ineed to have a text editor to edit some file for rescue. : Because vi(1) lives in /usr/bin. /rescue already has nvi. : ps. e3 is a full featured text editor written in NASM assembler. It is highly : optimized for size. I notice pax was there when I was looking at it on my FreeBSD/arm box :-) Warner From chinsan.tw at gmail.com Sun Sep 7 06:12:14 2008 From: chinsan.tw at gmail.com (chinsan) Date: Sun Sep 7 06:12:21 2008 Subject: pax in /rescue In-Reply-To: <20080906.235633.-267228782.imp@bsdimp.com> References: <20080906.224018.-1303464793.imp@bsdimp.com> <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> <20080906.235633.-267228782.imp@bsdimp.com> Message-ID: <1f27304c0809062312x7656c379gcd994be9fc0ca3e2@mail.gmail.com> On Sun, Sep 7, 2008 at 1:56 PM, M. Warner Losh wrote: > In message: <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> > chinsan writes: > : On Sun, Sep 7, 2008 at 12:40 PM, M. Warner Losh wrote: > : > Is there any reason to continue to include both tar and pax in > : > /rescue? > : > > : Maybe for rescue some important file? > > tar can read all the formats that pax can read. Why the duplication > in functionality? Oh, you are right. :p > > : BTW, could editors/e3 be include into /rescue? > : I think it's ineed to have a text editor to edit some file for rescue. > : Because vi(1) lives in /usr/bin. > > /rescue already has nvi. But nvi needs terminfo(5), so it needs to mount /usr filesystem too. From peterjeremy at optushome.com.au Sun Sep 7 13:59:43 2008 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Sun Sep 7 13:59:51 2008 Subject: pax in /rescue In-Reply-To: <1f27304c0809062312x7656c379gcd994be9fc0ca3e2@mail.gmail.com> References: <20080906.224018.-1303464793.imp@bsdimp.com> <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> <20080906.235633.-267228782.imp@bsdimp.com> <1f27304c0809062312x7656c379gcd994be9fc0ca3e2@mail.gmail.com> Message-ID: <20080907104924.GZ15376@server.vk2pj.dyndns.org> On 2008-Sep-07 14:12:12 +0800, chinsan wrote: >But nvi needs terminfo(5), so it needs to mount /usr filesystem too. There was a discussion on this point in -current in early August. I don't think there's a need to re-hash it here. See http://lists.freebsd.org/pipermail/freebsd-current/2008-August/087355.html -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080907/e264b668/attachment.pgp From bugmaster at FreeBSD.org Mon Sep 8 02:22:17 2008 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 8 02:22:38 2008 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200809080222.m882MHOG006617@freefall.freebsd.org> The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. Bugs can be in one of several states: o - open A problem report has been submitted, no sanity checking performed. a - analyzed The problem is understood and a solution is being sought. f - feedback Further work requires additional information from the originator or the community - possibly confirmation of the effectiveness of a proposed solution. p - patched A patch has been committed, but some issues (MFC and / or confirmation from originator) are still open. r - repocopy The resolution of the problem report is dependent on a repocopy operation within the CVS repository which is awaiting completion. s - suspended The problem is not being worked on, due to lack of information or resources. This is a prime candidate for somebody who is looking for a project to do. If the problem cannot be solved at all, it will be closed, rather than suspended. c - closed A problem report is closed when any changes have been integrated, documented, and tested -- or when fixing the problem is abandoned. From jhs at berklix.org Mon Sep 8 09:46:47 2008 From: jhs at berklix.org (Julian Stacey) Date: Mon Sep 8 09:46:54 2008 Subject: pax in /rescue In-Reply-To: Your message "Sun, 07 Sep 2008 13:22:44 +0800." <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> Message-ID: <200809080925.m889PRLA083055@fire.js.berklix.net> Hi, Reference: > From: chinsan > Date: Sun, 7 Sep 2008 13:22:44 +0800 > Message-id: <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> chinsan wrote: > On Sun, Sep 7, 2008 at 12:40 PM, M. Warner Losh wrote: > > Is there any reason to continue to include both tar and pax in > > /rescue? > > > Maybe for rescue some important file? > BTW, could editors/e3 be include into /rescue? > I think it's ineed to have a text editor to edit some file for rescue. > Because vi(1) lives in /usr/bin. No, use ed, ed has been in Unix 30+ years. /bin/ed > ps. e3 is a full featured text editor written in NASM assembler. It is highly > optimized for size. No, FreeBSD is not just i386. Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From rcoleman at criticalmagic.com Mon Sep 8 18:50:13 2008 From: rcoleman at criticalmagic.com (Richard Coleman) Date: Mon Sep 8 18:50:20 2008 Subject: pax in /rescue In-Reply-To: <20080906.235633.-267228782.imp@bsdimp.com> References: <20080906.224018.-1303464793.imp@bsdimp.com> <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> <20080906.235633.-267228782.imp@bsdimp.com> Message-ID: <48C56E41.40002@criticalmagic.com> M. Warner Losh wrote: > : BTW, could editors/e3 be include into /rescue? > : I think it's ineed to have a text editor to edit some file for rescue. > : Because vi(1) lives in /usr/bin. > > /rescue already has nvi. But this is only partially useful until the root partition/termcap issue is finally fixed. It comes up on the list about every 6 months, and everyone says "yeah, we should move the termcap file". And that's where it sits until the topic comes up again in another 6 months :-) Richard Coleman rcoleman@criticalmagic.com From keramida at freebsd.org Tue Sep 9 16:50:18 2008 From: keramida at freebsd.org (Giorgos Keramidas) Date: Tue Sep 9 16:50:25 2008 Subject: pax in /rescue In-Reply-To: <48C56E41.40002@criticalmagic.com> (Richard Coleman's message of "Mon, 08 Sep 2008 14:26:09 -0400") References: <20080906.224018.-1303464793.imp@bsdimp.com> <1f27304c0809062222m724603dat2fb1af63dba5faa6@mail.gmail.com> <20080906.235633.-267228782.imp@bsdimp.com> <48C56E41.40002@criticalmagic.com> Message-ID: <87ej3tnuiu.fsf@kobe.laptop> On Mon, 08 Sep 2008 14:26:09 -0400, Richard Coleman wrote: > M. Warner Losh wrote: >> : BTW, could editors/e3 be include into /rescue? >> : I think it's ineed to have a text editor to edit some file for rescue. >> : Because vi(1) lives in /usr/bin. >> >> /rescue already has nvi. > > But this is only partially useful until the root partition/termcap issue > is finally fixed. > > It comes up on the list about every 6 months, and everyone says "yeah, > we should move the termcap file". And that's where it sits until the > topic comes up again in another 6 months :-) I'm working on it. The ideas from the last discussion about allowing a minimal termcap in the root filesystem, and modifying ncurses to look for that as a fallback are nice... See the thread in freebsd-current that includes the posts: !A 2008-08-05 [ 28: Alex Kozlov ] Re: termcap under single luser ! 2008-08-06 [ 32: Alex Kozlov ] Re: termcap under single luser From jhb at FreeBSD.org Wed Sep 10 20:13:22 2008 From: jhb at FreeBSD.org (John Baldwin) Date: Wed Sep 10 20:13:29 2008 Subject: PASSERT() - asserting for panics Message-ID: <200809101531.54646.jhb@FreeBSD.org> So one of the things I like to do is use kernel modules that do regression tests. One of the things I want to test is that certain invalid operations will cause a specific panic. In the past I've done nefarious things like '#define panic printf' at the top of kern_rwlock.c and such. :) However, this is not suitable for more widespread use. The other approach of having the tests panic and verifying the panics that way is tedious. So what I came up with is a way to assert that a given panic will be triggered by a chunk of code, and if that panic happens, the kernel doesn't actually panic. The way I implemented this was by having the actual test do a setjmp() and if the "expected" panic triggers, then panic() does a longjmp() before setting panicstr, etc. A simple example of using it would be: PASSERT("foo", panic("foo")); The patch is at http://www.FreeBSD.org/~jhb/patches/passert.patch and below: --- //depot/projects/smpng/sys/kern/kern_shutdown.c 2008/03/18 12:54:14 +++ //depot/user/jhb/lock/kern/kern_shutdown.c 2008/09/10 17:40:44 @@ -531,6 +531,17 @@ if (panicstr) bootopt |= RB_NOSYNC; else { +#ifdef INVARIANT_SUPPORT + if (td->td_expected_panic != NULL && + strcmp(td->td_expected_panic, fmt) == 0) { + va_start(ap, fmt); + printf("expected panic: "); + vprintf(fmt, ap); + printf("\n"); + va_end(ap); + longjmp(td->td_panic_buf, 1); + } +#endif panicstr = fmt; newpanic = 1; } --- //depot/projects/smpng/sys/kern/kern_thread.c 2008/08/25 16:33:41 +++ //depot/user/jhb/lock/kern/kern_thread.c 2008/09/09 18:31:16 @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -48,6 +49,7 @@ #include #include +#include #include #include @@ -161,6 +163,9 @@ td->td_sched = (struct td_sched *)&td[1]; umtx_thread_init(td); td->td_kstack = 0; +#ifdef INVARIANT_SUPPORT + td->td_panic_buf = malloc(sizeof(struct _jmp_buf), M_SUBPROC, M_WAITOK); +#endif return (0); } @@ -178,6 +183,9 @@ sleepq_free(td->td_sleepqueue); umtx_thread_fini(td); seltdfini(td); +#ifdef INVARIANT_SUPPORT + free(td->td_panic_buf, M_SUBPROC); +#endif } /* --- //depot/projects/smpng/sys/sys/proc.h 2008/08/25 16:33:41 +++ //depot/user/jhb/lock/sys/proc.h 2008/09/09 18:31:16 @@ -166,6 +166,7 @@ struct kdtrace_proc; struct kdtrace_thread; struct cpuset; +struct _jmp_buf; /* * Kernel runnable context (thread). @@ -273,6 +274,8 @@ struct lpohead td_lprof[2]; /* (a) lock profiling objects. */ struct kdtrace_thread *td_dtrace; /* (*) DTrace-specific data. */ int td_errno; /* Error returned by last syscall. */ + struct _jmp_buf *td_panic_buf; /* (k) Jump buffer for PASSERT(). */ + const char *td_expected_panic; /* (k) Expected panic. */ }; struct mtx *thread_lock_block(struct thread *); --- //depot/projects/smpng/sys/sys/systm.h 2008/07/25 18:20:23 +++ //depot/user/jhb/lock/sys/systm.h 2008/09/09 18:31:16 @@ -71,9 +71,25 @@ if (__predict_false(!(exp))) \ panic msg; \ } while (0) + +#define PASSERT(panicstr, code) do { \ + switch (setjmp(curthread->td_panic_buf)) { \ + case 0: \ + curthread->td_expected_panic = (panicstr); \ + code; \ + panic("Expected panic '%s' did not trigger", \ + (panicstr)); \ + case 1: \ + curthread->td_expected_panic = NULL; \ + break; \ + default: \ + panic("Unexpected return value from setjmp()"); \ + } \ +} while (0) + #define VNASSERT(exp, vp, msg) do { \ if (__predict_false(!(exp))) { \ vn_printf(vp, "VNASSERT failed\n"); \ panic msg; \ } \ } while (0) @@ -81,6 +97,9 @@ #define KASSERT(exp,msg) do { \ } while (0) +#define PASSERT(panicstr, code) do { \ +} while (0) + #define VNASSERT(exp, vp, msg) do { \ } while (0) #endif Is this too evil for the tree? -- John Baldwin From dillon at apollo.backplane.com Wed Sep 10 21:37:41 2008 From: dillon at apollo.backplane.com (Matthew Dillon) Date: Wed Sep 10 21:37:48 2008 Subject: PASSERT() - asserting for panics References: <200809101531.54646.jhb@FreeBSD.org> Message-ID: <200809102126.m8ALQxdK032024@apollo.backplane.com> : PASSERT("foo", panic("foo")); : :The patch is at http://www.FreeBSD.org/~jhb/patches/passert.patch and below: :.. The general problem with creating a more complex style of assertion is, well, that it is more complex. As a programmer who adds assertions all over the place (HAMMER has 343 KKASSERTs, for example), the last thing I want to do is to have to think about the assertion line itself or have to read through the visual pollution. Most of the time I just want to KKASSERT(condition) and be done with it. I would consider using a DATASET (easily embedded in a statement as it is a static declaration) to back the assertion macro and then simplify it to just: PASSERT(label, condition) The kernel would collect the DATASETs together and provide a sysctl API to allow the action taken to be managed at run-time based on the label. You could even have a version that doesn't use a label at all and is instead indexed by file and line. The output would be the stringtized condition (similar to libc's assert(cond) or DFly's KKASSERT(cond)). You could also have an optional macro to specify a default action for the label, and a master global sysctl to override everything. The actual code generated would simply be the conditional and, say, a call to _passert(&dataset), reducing code pollution as well as visual pollution. I have considered doing something similar in DragonFly for KKASSERT to reduce the code generated by the multi-argument panic() it calls. -Matt From jhb at freebsd.org Wed Sep 10 22:35:39 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Sep 10 22:35:46 2008 Subject: PASSERT() - asserting for panics In-Reply-To: <200809102126.m8ALQxdK032024@apollo.backplane.com> References: <200809101531.54646.jhb@FreeBSD.org> <200809102126.m8ALQxdK032024@apollo.backplane.com> Message-ID: <200809101806.38042.jhb@freebsd.org> On Wednesday 10 September 2008 05:26:59 pm Matthew Dillon wrote: > : PASSERT("foo", panic("foo")); > : > :The patch is at http://www.FreeBSD.org/~jhb/patches/passert.patch and below: > :.. > > The general problem with creating a more complex style of assertion > is, well, that it is more complex. As a programmer who adds assertions > all over the place (HAMMER has 343 KKASSERTs, for example), the last > thing I want to do is to have to think about the assertion line itself > or have to read through the visual pollution. Most of the time I just > want to KKASSERT(condition) and be done with it. > > I would consider using a DATASET (easily embedded in a statement as it > is a static declaration) to back the assertion macro and then simplify > it to just: > > PASSERT(label, condition) > > The kernel would collect the DATASETs together and provide a sysctl > API to allow the action taken to be managed at run-time based on the > label. You could even have a version that doesn't use a label at all > and is instead indexed by file and line. The output would be the > stringtized condition (similar to libc's assert(cond) or DFly's > KKASSERT(cond)). You could also have an optional macro to specify > a default action for the label, and a master global sysctl to override > everything. > > The actual code generated would simply be the conditional and, say, > a call to _passert(&dataset), reducing code pollution as well as visual > pollution. I have considered doing something similar in DragonFly for > KKASSERT to reduce the code generated by the multi-argument panic() > it calls. I think part of the pollution is that what I really want to do is treat a panic like an exception that I can catch. I could probably make that idiom work with two macros so your code can be: PASSERT_BEGIN("foo") { /* do stuff */ } PASSERT_END; It would be more like exceptions in C++ though to have the string at the end: PANIC_TRY { /* do stuff */ } PANIC_CATCH("foo"); That could potentially work with some evil goto's that jumped between the macros (i.e. have the setjmp() in PANIC_CATCH and PANIC_TRY goto's down to that and then CATCH goto's back to TRY when setjmp returns 0), but those would be gross. The other problem is that you would probably be limited to a single TRY/CATCH per function, and I actually have some existing regression tests that do multiple potential panics in a single routine. -- John Baldwin From dillon at apollo.backplane.com Thu Sep 11 00:06:57 2008 From: dillon at apollo.backplane.com (Matthew Dillon) Date: Thu Sep 11 00:07:03 2008 Subject: PASSERT() - asserting for panics References: <200809101531.54646.jhb@FreeBSD.org> <200809102126.m8ALQxdK032024@apollo.backplane.com> <200809101806.38042.jhb@freebsd.org> Message-ID: <200809110006.m8B06vOU033199@apollo.backplane.com> :I think part of the pollution is that what I really want to do is treat a :panic like an exception that I can catch. I could probably make that idiom :work with two macros so your code can be: : : PANIC_TRY { : /* do stuff */ : } PANIC_CATCH("foo"); : :That could potentially work with some evil goto's that jumped between the :macros (i.e. have the setjmp() in PANIC_CATCH and PANIC_TRY goto's down to :-- :John Baldwin Hmm. Almost java-like, where an exception pops back through subroutine levels until it finds a match. That kind of functionality would be a real mess in C. A limited form would be possible, something like this: #define PANIC_CATCH(label) \ (...) static jmpbuf label # _jmpbuf if (0) { for (;;) { longjmp(&label # _jmpbuf); label: #define PANIC_RETRY }} #define PANIC_PANIC panic(...); }} #define PASSERT(label, cond) \ (...) if (__predict_false(!(cond))) { setjmp (&label # _jmpbuf); goto label; } PANIC_CATCH(badthings) { ... PANIC_RETRY; } PANIC_CATCH(badthings) { ... PANIC_PANIC; } PASSERT(badthings, x == 1); Which would allow you some control over the context plus allow multiple PASSERT's with the same label. But, OMG I don't know about doing setjmp/longjmp in the kernel. I don't think it would be worth it. Theoretically one could cross procedural boundaries with the longjmp, and place the jmpbuf in a DATASET and have the kernel glue the longjmp address. That would be a candidate for the C obfuscation contest though. -Matt Matthew Dillon From mehulc87 at gmail.com Thu Sep 11 17:55:58 2008 From: mehulc87 at gmail.com (Mehul Chadha) Date: Thu Sep 11 17:56:04 2008 Subject: Adding Virtual kernel support to FreeBSD Message-ID: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> hello all, We are a group of 3 students doing their graduation in computer science. We were thinking of implementing the virtual kernel support to freeBSD similar to what user mode linux does . We would be very thankful if we can have some suggestions and advice from your side. Thanks and Regards, Mehul From jhb at freebsd.org Thu Sep 11 17:56:41 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu Sep 11 17:56:47 2008 Subject: PASSERT() - asserting for panics In-Reply-To: <200809110006.m8B06vOU033199@apollo.backplane.com> References: <200809101531.54646.jhb@FreeBSD.org> <200809101806.38042.jhb@freebsd.org> <200809110006.m8B06vOU033199@apollo.backplane.com> Message-ID: <200809111028.59610.jhb@freebsd.org> On Wednesday 10 September 2008 08:06:57 pm Matthew Dillon wrote: > > :I think part of the pollution is that what I really want to do is treat a > :panic like an exception that I can catch. I could probably make that idiom > :work with two macros so your code can be: > : > : PANIC_TRY { > : /* do stuff */ > : } PANIC_CATCH("foo"); > : > :That could potentially work with some evil goto's that jumped between the > :macros (i.e. have the setjmp() in PANIC_CATCH and PANIC_TRY goto's down to > :-- > :John Baldwin > > Hmm. Almost java-like, where an exception pops back through subroutine > levels until it finds a match. That kind of functionality would be a > real mess in C. A limited form would be possible, something like this: > > #define PANIC_CATCH(label) \ (...) > static jmpbuf label # _jmpbuf > if (0) { > for (;;) { > longjmp(&label # _jmpbuf); > label: > > #define PANIC_RETRY }} > > #define PANIC_PANIC panic(...); }} > > #define PASSERT(label, cond) \ (...) > if (__predict_false(!(cond))) { > setjmp (&label # _jmpbuf); > goto label; > } > > PANIC_CATCH(badthings) { > ... > PANIC_RETRY; > } > > PANIC_CATCH(badthings) { > ... > PANIC_PANIC; > } > > PASSERT(badthings, x == 1); The problem is I want panics in unmodified code to be caught. For example, I want to write regression tests to make sure assertions on locking primivities fail when the state of the lock doesn't match what the assertion is requiring. I can do something fairly simple though that lets me do this: PANIC_TRY { stuff; } PANIC_CATCH { IGNORE_PANIC("this one is ok"); /* WITNESS can do different panics in foo_assert() sometimes */ IGNORE_PANIC("this one is too"); UNEXPECTED_PANIC(); } Without needing goto's. > Which would allow you some control over the context plus allow multiple > PASSERT's with the same label. But, OMG I don't know about doing > setjmp/longjmp in the kernel. I don't think it would be worth it. We use setjmp already if you get a trap in ddb so ddb doesn't crash. > Theoretically one could cross procedural boundaries with the longjmp, > and place the jmpbuf in a DATASET and have the kernel glue the longjmp > address. That would be a candidate for the C obfuscation contest > though. To catch unmodified panics, I have to have the longjmp in panic() itself, and I have the jmpbuf hung off of curthread. -- John Baldwin From julian at elischer.org Thu Sep 11 18:18:15 2008 From: julian at elischer.org (Julian Elischer) Date: Thu Sep 11 18:18:25 2008 Subject: Adding Virtual kernel support to FreeBSD In-Reply-To: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> References: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> Message-ID: <48C95D6D.80204@elischer.org> Mehul Chadha wrote: > hello all, > We are a group of 3 students doing their graduation in > computer science. We were thinking of implementing the virtual kernel > support to freeBSD similar to what user mode linux does . We would be very > thankful if we can have some suggestions and advice from your side. dragonflyBSD which is derived from freeBSD has this. > > Thanks and Regards, > Mehul > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From alfred at freebsd.org Fri Sep 12 05:03:53 2008 From: alfred at freebsd.org (Alfred Perlstein) Date: Fri Sep 12 05:04:02 2008 Subject: Adding Virtual kernel support to FreeBSD In-Reply-To: <48C95D6D.80204@elischer.org> References: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> <48C95D6D.80204@elischer.org> Message-ID: <20080912050352.GB16977@elvis.mu.org> * Julian Elischer [080911 11:18] wrote: > Mehul Chadha wrote: > >hello all, > > We are a group of 3 students doing their graduation in > >computer science. We were thinking of implementing the virtual kernel > >support to freeBSD similar to what user mode linux does . We would be very > >thankful if we can have some suggestions and advice from your side. > > dragonflyBSD which is derived from freeBSD has this. Really??? :) What kind of porting effort would it take? -Alfred From mehulc87 at gmail.com Fri Sep 12 05:44:39 2008 From: mehulc87 at gmail.com (Mehul Chadha) Date: Fri Sep 12 05:44:46 2008 Subject: Adding Virtual kernel support to FreeBSD In-Reply-To: <20080912050352.GB16977@elvis.mu.org> References: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> <48C95D6D.80204@elischer.org> <20080912050352.GB16977@elvis.mu.org> Message-ID: <251d650c0809112244w1995bc26r4f6622080034051d@mail.gmail.com> A lot of effort to have a virtual kernel running on top of a real kernel. We will be dividing the project in a number of phases. We would be first concentrating on running the processes on virtual kernel (We are working for the algorithms), filesystems interface , next the libc interface that is the functions like printk in virtual kernel need to be mapped to printf of real kernel. Once we are through with this then we will start working with networks through the tun/tap interface . Any Thoughts?? On Fri, Sep 12, 2008 at 10:33 AM, Alfred Perlstein wrote: > * Julian Elischer [080911 11:18] wrote: > > Mehul Chadha wrote: > > >hello all, > > > We are a group of 3 students doing their graduation in > > >computer science. We were thinking of implementing the virtual kernel > > >support to freeBSD similar to what user mode linux does . We would be > very > > >thankful if we can have some suggestions and advice from your side. > > > > dragonflyBSD which is derived from freeBSD has this. > > Really??? :) > > What kind of porting effort would it take? > > -Alfred > From julian at elischer.org Fri Sep 12 06:10:13 2008 From: julian at elischer.org (Julian Elischer) Date: Fri Sep 12 06:10:19 2008 Subject: Adding Virtual kernel support to FreeBSD In-Reply-To: <20080912050352.GB16977@elvis.mu.org> References: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> <48C95D6D.80204@elischer.org> <20080912050352.GB16977@elvis.mu.org> Message-ID: <48CA07C4.3040707@elischer.org> Alfred Perlstein wrote: > * Julian Elischer [080911 11:18] wrote: >> Mehul Chadha wrote: >>> hello all, >>> We are a group of 3 students doing their graduation in >>> computer science. We were thinking of implementing the virtual kernel >>> support to freeBSD similar to what user mode linux does . We would be very >>> thankful if we can have some suggestions and advice from your side. >> dragonflyBSD which is derived from freeBSD has this. > > Really??? :) > > What kind of porting effort would it take? matt says it would be doable.... > > -Alfred From julian at elischer.org Fri Sep 12 06:15:29 2008 From: julian at elischer.org (Julian Elischer) Date: Fri Sep 12 06:15:36 2008 Subject: Adding Virtual kernel support to FreeBSD In-Reply-To: <251d650c0809112244w1995bc26r4f6622080034051d@mail.gmail.com> References: <251d650c0809111026h1b16f5d2u77e8d611ce1b39f7@mail.gmail.com> <48C95D6D.80204@elischer.org> <20080912050352.GB16977@elvis.mu.org> <251d650c0809112244w1995bc26r4f6622080034051d@mail.gmail.com> Message-ID: <48CA0900.4030705@elischer.org> Mehul Chadha wrote: > A lot of effort to have a virtual kernel running on top of a real kernel. > > We will be dividing the project in a number of phases. > > We would be first concentrating on running the processes on virtual > kernel (We are working for the algorithms), filesystems interface , next > the libc interface that is the functions like printk in virtual kernel > need to be mapped to printf of real kernel. Once we are through with > this then we will start working with networks through the tun/tap > interface . Any Thoughts?? > If dragonfly's vkernel support is now what you mean then what DO you mean? > On Fri, Sep 12, 2008 at 10:33 AM, Alfred Perlstein > wrote: > > * Julian Elischer > > [080911 11:18] wrote: > > Mehul Chadha wrote: > > >hello all, > > > We are a group of 3 students doing their > graduation in > > >computer science. We were thinking of implementing the virtual > kernel > > >support to freeBSD similar to what user mode linux does . We > would be very > > >thankful if we can have some suggestions and advice from your side. > > > > dragonflyBSD which is derived from freeBSD has this. > > Really??? :) > > What kind of porting effort would it take? > > -Alfred > > From ed at 80386.nl Fri Sep 12 18:27:24 2008 From: ed at 80386.nl (Ed Schouten) Date: Fri Sep 12 18:27:41 2008 Subject: Expanding vops in vop_vectors during startup Message-ID: <20080912182722.GK1191@hoeg.nl> Hello everyone, Yesterday I was talking with some friends of mine about the FreeBSD VFS layer. As an exercise, Jille was trying to patch nullfs to hide .svn directories (see hackers@), so that's how we got to the subject. After talking about the way our vop_vector works (vop_bypass and vop_default), we were wondering why we don't propagate all pointers in the vop_vector to its children to save the unneeded function pointer resolving inside the VOP_* calls. I've created a patch that adds a new routine to the kernel, vop_vector_init(), which recursively expands the vops. A new macro (DECLARE_VOP_VECTOR) is used to automatically perform this when loading modules/booting. There is no need to reference default_vnodeops anymore, because it is now used implicitly (when vop_vector == NULL). Any comments on the attached patch? I've only tested it with ufs, nullfs, devfs, etc. yet. I haven't tested `make universe' yet, but I suspect it shouldn't break hard. URL: http://80386.nl/files/vnode-vop-expand.diff Thanks! -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080912/13515822/attachment.pgp From bugmaster at FreeBSD.org Mon Sep 15 15:18:45 2008 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 15 15:19:16 2008 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200809151518.m8FFIix2018812@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From sam at errno.com Tue Sep 16 02:43:14 2008 From: sam at errno.com (Sam Leffler) Date: Tue Sep 16 02:43:22 2008 Subject: the more build knobs bikeshed Message-ID: <48CF1979.90507@errno.com> Here is a patch against HEAD to add several knew MK_* knobs that can be used to trim the build for small-footprint applications: http://www.freebsd.org/~sam/build.patch With these changes I can build a nanobsd image for x86 or arm that is <1/2 the size you can get otherwise (using just the standard knobs). If you do not use the knobs you should get exactly what you're used to getting. My goal in doing this is to make nanobsd more useful for building embedded packages without bypassing the build process to purge stuff. This is still a LONG way from small enough to fit in many applications but seems like a useful start. FWIW I chose knobs using two criteria: 1. Does it save an appreciable amount of space. 2. Does it remove applications that you really don't want to be present. I'm sure folks will wonder about some choices. One can also argue that some of the groups I've added are wrong and/or we should have individual knobs for each application. Separate issues that I intend to work on: 1. Build time is way too long; paring down the target config should also reduce the build time but it does not because things like MK_TOOLCHAIN cannot be used when building; only when installing. 2. Some applications that you want are hugely bloated and may benefit from some TLC. bind comes to mind where I can crunchgen all the bits into a single binary but when built normally you get multiple huge executables. 3. Cross-build and package; I can use nanobsd to build arm images but the packaging facilities are inadequate. I've got a start on default setups for some boards/packages and would like to see this grow into a library that people can use for reference and/or extend. Anyway, I'm interested in getting the above patch committed so looking for constructive comments. I haven't included the files that go under src/tools/build/options (I've done them already). I also have mods for RELENG_7 that I've been using for a while. Sam From julian at elischer.org Tue Sep 16 03:35:19 2008 From: julian at elischer.org (Julian Elischer) Date: Tue Sep 16 03:35:26 2008 Subject: the more build knobs bikeshed In-Reply-To: <48CF1979.90507@errno.com> References: <48CF1979.90507@errno.com> Message-ID: <48CF2975.3010908@elischer.org> Sam Leffler wrote: > Here is a patch against HEAD to add several knew MK_* knobs that can be > used to trim the build for small-footprint applications: > > http://www.freebsd.org/~sam/build.patch not bad.. I'd be temoted to make some larger groups e.g. debugstuff and devtools so exclude lots of stuff in one hit, with some smaller groups to bring some of them back more selectively.. maybe too complicated.. > > With these changes I can build a nanobsd image for x86 or arm that is > <1/2 the size you can get otherwise (using just the standard knobs). If > you do not use the knobs you should get exactly what you're used to > getting. tinybsd gets its apps together and then uses ldd on each to get the minimum set of libraries as well. (though that doesn't help with programs that dlopen a bunch of stuff such as natd which dlopens all teh libalias*.so libs). > > My goal in doing this is to make nanobsd more useful for building > embedded packages without bypassing the build process to purge stuff. > This is still a LONG way from small enough to fit in many applications > but seems like a useful start. > > FWIW I chose knobs using two criteria: > > 1. Does it save an appreciable amount of space. > 2. Does it remove applications that you really don't want to be present. > > I'm sure folks will wonder about some choices. One can also argue that > some of the groups I've added are wrong and/or we should have individual > knobs for each application. > > Separate issues that I intend to work on: > > 1. Build time is way too long; paring down the target config should also > reduce the build time but it does not because things like MK_TOOLCHAIN > cannot be used when building; only when installing. Tinybsd uses the binaries on your system rather than building them. Now I know that isn't what you want but it sure is faster.. great for prototyping a setup I find.. maybe you could point to a prebuilt image and just grab from there... > 2. Some applications that you want are hugely bloated and may benefit > from some TLC. bind comes to mind where I can crunchgen all the bits > into a single binary but when built normally you get multiple huge > executables. and there are many programs that have quite a lot of options that add space that are rarely used... a "trim" option might be nice, meaning "build the simple version. > 3. Cross-build and package; I can use nanobsd to build arm images but > the packaging facilities are inadequate. I've got a start on default > setups for some boards/packages and would like to see this grow into a > library that people can use for reference and/or extend. we really should import the makefs utilities from NetBSD.. they are in ports. > > Anyway, I'm interested in getting the above patch committed so looking > for constructive comments. I haven't included the files that go under > src/tools/build/options (I've done them already). I also have mods for > RELENG_7 that I've been using for a while. > > Sam > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From imp at bsdimp.com Tue Sep 16 05:57:19 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Tue Sep 16 05:57:26 2008 Subject: the more build knobs bikeshed In-Reply-To: <48CF2975.3010908@elischer.org> References: <48CF1979.90507@errno.com> <48CF2975.3010908@elischer.org> Message-ID: <20080915.235522.-169059506.imp@bsdimp.com> In message: <48CF2975.3010908@elischer.org> Julian Elischer writes: : > With these changes I can build a nanobsd image for x86 or arm that is : > <1/2 the size you can get otherwise (using just the standard knobs). If : > you do not use the knobs you should get exactly what you're used to : > getting. : : tinybsd gets its apps together and then uses ldd on each to get the : minimum set of libraries as well. : (though that doesn't help with programs that dlopen a bunch of stuff : such as natd which dlopens all teh libalias*.so libs). And it doesn't work with cross building because of its use of LDD. It needs to use the NEEDED lines in objdump to get this information. : Tinybsd uses the binaries on your system rather than building them. Unless you are cross building, in which case it grabs them from an image that you've installed... : Now I know that isn't what you want but it sure is faster.. great : for prototyping a setup I find.. maybe you could point to a prebuilt : image and just grab from there... It is useful for prototyping.. I've used a tinybsd-like system in production, but it didn't do automatic library depends. It had very crude cross build support for ports, but only very stupid or very cooperative ports would work... : > 2. Some applications that you want are hugely bloated and may benefit : > from some TLC. bind comes to mind where I can crunchgen all the bits : > into a single binary but when built normally you get multiple huge : > executables. : : and there are many programs that have quite a lot of options that : add space that are rarely used... a "trim" option might be nice, : meaning "build the simple version. Yes. : > 3. Cross-build and package; I can use nanobsd to build arm images but : > the packaging facilities are inadequate. I've got a start on default : > setups for some boards/packages and would like to see this grow into a : > library that people can use for reference and/or extend. : : we really should import the makefs utilities from NetBSD.. they are in : ports. These work great... : > Anyway, I'm interested in getting the above patch committed so looking : > for constructive comments. I haven't included the files that go under : > src/tools/build/options (I've done them already). I also have mods for : > RELENG_7 that I've been using for a while. All in all, I think this stuff is great... Warner From jhay at meraka.org.za Tue Sep 16 06:05:41 2008 From: jhay at meraka.org.za (John Hay) Date: Tue Sep 16 06:05:47 2008 Subject: the more build knobs bikeshed In-Reply-To: <20080915.235522.-169059506.imp@bsdimp.com> References: <48CF1979.90507@errno.com> <48CF2975.3010908@elischer.org> <20080915.235522.-169059506.imp@bsdimp.com> Message-ID: <20080916060537.GA40711@zibbi.meraka.csir.co.za> On Mon, Sep 15, 2008 at 11:55:22PM -0600, M. Warner Losh wrote: > In message: <48CF2975.3010908@elischer.org> > Julian Elischer writes: > : > With these changes I can build a nanobsd image for x86 or arm that is > : > <1/2 the size you can get otherwise (using just the standard knobs). If > : > you do not use the knobs you should get exactly what you're used to > : > getting. > : > : tinybsd gets its apps together and then uses ldd on each to get the > : minimum set of libraries as well. > : (though that doesn't help with programs that dlopen a bunch of stuff > : such as natd which dlopens all teh libalias*.so libs). > > And it doesn't work with cross building because of its use of LDD. It > needs to use the NEEDED lines in objdump to get this information. I have been using this in a script on an i386 box to build small i386 and arm distros: llst=`readelf --dynamic ${sdir}/${f} 2> /dev/null | awk '/Shared library/ { print $5 }' | sed -E -e 's/\[|\]//g'` John -- John Hay -- John.Hay@meraka.csir.co.za / jhay@FreeBSD.org From phk at phk.freebsd.dk Tue Sep 16 07:41:27 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue Sep 16 07:41:34 2008 Subject: the more build knobs bikeshed In-Reply-To: Your message of "Mon, 15 Sep 2008 19:27:05 MST." <48CF1979.90507@errno.com> Message-ID: <5764.1221549183@critter.freebsd.dk> In message <48CF1979.90507@errno.com>, Sam Leffler writes: >Here is a patch against HEAD to add several knew MK_* knobs that can be >used to trim the build for small-footprint applications: > >http://www.freebsd.org/~sam/build.patch Looks good. You should probably check if the periodic scripts get unhappy if there is no locate binary. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From deischen at freebsd.org Tue Sep 16 16:06:45 2008 From: deischen at freebsd.org (Daniel Eischen) Date: Tue Sep 16 16:09:15 2008 Subject: the more build knobs bikeshed In-Reply-To: <5764.1221549183@critter.freebsd.dk> References: <5764.1221549183@critter.freebsd.dk> Message-ID: On Tue, 16 Sep 2008, Poul-Henning Kamp wrote: > In message <48CF1979.90507@errno.com>, Sam Leffler writes: >> Here is a patch against HEAD to add several knew MK_* knobs that can be >> used to trim the build for small-footprint applications: >> >> http://www.freebsd.org/~sam/build.patch > > Looks good. Yes, I agree. I'd love to see these and other similar patches that can be useful with nanobsd hit the tree and MFC'd. It'd be nice to eventually have something with a tree-like dependency chain, so you could remove the leaves or branches. Adding a leaf would also automatically add any missing branches (warning or informing you what it added). If that sounds too vxWorks'ish, sorry ;-) -- DE From brooks at freebsd.org Tue Sep 16 16:26:21 2008 From: brooks at freebsd.org (Brooks Davis) Date: Tue Sep 16 16:27:44 2008 Subject: the more build knobs bikeshed In-Reply-To: <48CF1979.90507@errno.com> References: <48CF1979.90507@errno.com> Message-ID: <20080916162707.GA35096@lor.one-eyed-alien.net> On Mon, Sep 15, 2008 at 07:27:05PM -0700, Sam Leffler wrote: > Here is a patch against HEAD to add several knew MK_* knobs that can be > used to trim the build for small-footprint applications: > > http://www.freebsd.org/~sam/build.patch > > With these changes I can build a nanobsd image for x86 or arm that is <1/2 > the size you can get otherwise (using just the standard knobs). If you do > not use the knobs you should get exactly what you're used to getting. > > My goal in doing this is to make nanobsd more useful for building embedded > packages without bypassing the build process to purge stuff. This is still > a LONG way from small enough to fit in many applications but seems like a > useful start. > > FWIW I chose knobs using two criteria: > > 1. Does it save an appreciable amount of space. > 2. Does it remove applications that you really don't want to be present. > > I'm sure folks will wonder about some choices. One can also argue that > some of the groups I've added are wrong and/or we should have individual > knobs for each application. > > Separate issues that I intend to work on: > > 1. Build time is way too long; paring down the target config should also > reduce the build time but it does not because things like MK_TOOLCHAIN > cannot be used when building; only when installing. > 2. Some applications that you want are hugely bloated and may benefit from > some TLC. bind comes to mind where I can crunchgen all the bits into a > single binary but when built normally you get multiple huge executables. > 3. Cross-build and package; I can use nanobsd to build arm images but the > packaging facilities are inadequate. I've got a start on default setups > for some boards/packages and would like to see this grow into a library > that people can use for reference and/or extend. > > Anyway, I'm interested in getting the above patch committed so looking for > constructive comments. I haven't included the files that go under > src/tools/build/options (I've done them already). I also have mods for > RELENG_7 that I've been using for a while. Looks generally good to me. Some of us at AsiaBSDCon noted that most of usr.sbin isn't useful on most system so this is a good step forward. One minor dependency nit is that freebsd-update depends on portsnap at the moment because both use phttpget which resides in usr.sbin/portsnap/phttpget. It should probably move to libexec/phttpget. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080916/58a8c5c0/attachment.pgp From remko at FreeBSD.org Tue Sep 16 17:00:08 2008 From: remko at FreeBSD.org (Remko Lodder) Date: Tue Sep 16 17:01:27 2008 Subject: the more build knobs bikeshed In-Reply-To: References: <5764.1221549183@critter.freebsd.dk> Message-ID: <48CFDE92.3030708@FreeBSD.org> Daniel Eischen wrote: > On Tue, 16 Sep 2008, Poul-Henning Kamp wrote: > >> In message <48CF1979.90507@errno.com>, Sam Leffler writes: >>> Here is a patch against HEAD to add several knew MK_* knobs that can be >>> used to trim the build for small-footprint applications: >>> >>> http://www.freebsd.org/~sam/build.patch >> >> Looks good. > > Yes, I agree. I'd love to see these and other similar patches > that can be useful with nanobsd hit the tree and MFC'd. > > It'd be nice to eventually have something with a tree-like > dependency chain, so you could remove the leaves or branches. > Adding a leaf would also automatically add any missing > branches (warning or informing you what it added). If > that sounds too vxWorks'ish, sorry ;-) > I like it a lot Sam! -- /"\ Best regards, | remko@FreeBSD.org \ / Remko Lodder | remko@EFnet X http://www.evilcoder.org/ | / \ ASCII Ribbon Campaign | Against HTML Mail and News From bzeeb-lists at lists.zabbadoz.net Tue Sep 16 17:54:29 2008 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Tue Sep 16 17:56:32 2008 Subject: the more build knobs bikeshed In-Reply-To: <5764.1221549183@critter.freebsd.dk> References: <5764.1221549183@critter.freebsd.dk> Message-ID: <20080916171748.F65801@maildrop.int.zabbadoz.net> On Tue, 16 Sep 2008, Poul-Henning Kamp wrote: > In message <48CF1979.90507@errno.com>, Sam Leffler writes: >> Here is a patch against HEAD to add several knew MK_* knobs that can be >> used to trim the build for small-footprint applications: >> >> http://www.freebsd.org/~sam/build.patch > > Looks good. > > You should probably check if the periodic scripts get unhappy if > there is no locate binary. I have always ignored periodic failures with my minimalized builds as I usually do not expect to run any/much periodic scripts anyway not that anyone would get the mails without MAIL, ... Things like ipfw, ipstat, purgestat, ... are just complaining but things still work (even on those where I have replaced mail with something else(tm)). Never had any problems. Otherwise a customer /etc/periodic.conf.local can always fix things for each individual setup. The patch looks good to me. I wonder if we, at one point want to supergroup for example APM + ACPI and add zzz and powerd and have a MK_POWERMGMT, etc. like MK_MAIL, etc. but those things, for sure, can be refined incrementally. You said you had the descriptions for src.conf already? Put this into HEAD and other people might tune more. -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From jhein at timing.com Tue Sep 16 21:08:42 2008 From: jhein at timing.com (John Hein) Date: Tue Sep 16 21:08:49 2008 Subject: 64 bit time_t Message-ID: <18640.5196.580629.632590@gromit.timing.com> Other than recompiling for -current users (and not being an MFC-able change and possibly breaking a gazillion unfortunately written ports), are their any other issues with switching to 64 bit time_t for i386? I suppose compat libs are a bit dicey. From brooks at freebsd.org Tue Sep 16 21:15:58 2008 From: brooks at freebsd.org (Brooks Davis) Date: Tue Sep 16 21:16:04 2008 Subject: 64 bit time_t In-Reply-To: <18640.5196.580629.632590@gromit.timing.com> References: <18640.5196.580629.632590@gromit.timing.com> Message-ID: <20080916211646.GA35778@lor.one-eyed-alien.net> On Tue, Sep 16, 2008 at 02:17:16PM -0600, John Hein wrote: > Other than recompiling for -current users (and not being an MFC-able > change and possibly breaking a gazillion unfortunately written ports), > are their any other issues with switching to 64 bit time_t for i386? > I suppose compat libs are a bit dicey. Off hand: every syscall that takes a time_t or a structure containing a time_t would have to be reimplemented and a compatability version provided in the old location. The same would be true of every similar function in the libc symbol map. A number of ioctl's and sysctl would probably need to grow compatability interfaces as well. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080916/c7b509c8/attachment.pgp From phk at phk.freebsd.dk Tue Sep 16 21:26:18 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue Sep 16 21:26:28 2008 Subject: 64 bit time_t In-Reply-To: Your message of "Tue, 16 Sep 2008 16:16:47 EST." <20080916211646.GA35778@lor.one-eyed-alien.net> Message-ID: <75968.1221600374@critter.freebsd.dk> In message <20080916211646.GA35778@lor.one-eyed-alien.net>, Brooks Davis writes : > >--PEIAKu/WMn1b1Hv9 >Content-Type: text/plain; charset=us-ascii >Content-Disposition: inline > >On Tue, Sep 16, 2008 at 02:17:16PM -0600, John Hein wrote: >> Other than recompiling for -current users (and not being an MFC-able >> change and possibly breaking a gazillion unfortunately written ports), >> are their any other issues with switching to 64 bit time_t for i386? >> I suppose compat libs are a bit dicey. > >Off hand: every syscall that takes a time_t or a structure containing >a time_t would have to be reimplemented and a compatability version[...] This is a pretty nasty piece of work because it also involves the timespec and timeval structures which appear in ioctls, socket options, socket messages and so on. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jhb at freebsd.org Wed Sep 17 15:31:51 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Sep 17 15:32:02 2008 Subject: 64 bit time_t In-Reply-To: <75968.1221600374@critter.freebsd.dk> References: <75968.1221600374@critter.freebsd.dk> Message-ID: <200809171040.36105.jhb@freebsd.org> On Tuesday 16 September 2008 05:26:14 pm Poul-Henning Kamp wrote: > In message <20080916211646.GA35778@lor.one-eyed-alien.net>, Brooks Davis writes > : > > > >--PEIAKu/WMn1b1Hv9 > >Content-Type: text/plain; charset=us-ascii > >Content-Disposition: inline > > > >On Tue, Sep 16, 2008 at 02:17:16PM -0600, John Hein wrote: > >> Other than recompiling for -current users (and not being an MFC-able > >> change and possibly breaking a gazillion unfortunately written ports), > >> are their any other issues with switching to 64 bit time_t for i386? > >> I suppose compat libs are a bit dicey. > > > >Off hand: every syscall that takes a time_t or a structure containing > >a time_t would have to be reimplemented and a compatability version[...] > > This is a pretty nasty piece of work because it also involves the > timespec and timeval structures which appear in ioctls, socket > options, socket messages and so on. And with amd64/x86-64, it may prove to not really be necessary. -- John Baldwin From jhb at freebsd.org Wed Sep 17 15:31:51 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Sep 17 15:32:03 2008 Subject: 64 bit time_t In-Reply-To: <75968.1221600374@critter.freebsd.dk> References: <75968.1221600374@critter.freebsd.dk> Message-ID: <200809171040.36105.jhb@freebsd.org> On Tuesday 16 September 2008 05:26:14 pm Poul-Henning Kamp wrote: > In message <20080916211646.GA35778@lor.one-eyed-alien.net>, Brooks Davis writes > : > > > >--PEIAKu/WMn1b1Hv9 > >Content-Type: text/plain; charset=us-ascii > >Content-Disposition: inline > > > >On Tue, Sep 16, 2008 at 02:17:16PM -0600, John Hein wrote: > >> Other than recompiling for -current users (and not being an MFC-able > >> change and possibly breaking a gazillion unfortunately written ports), > >> are their any other issues with switching to 64 bit time_t for i386? > >> I suppose compat libs are a bit dicey. > > > >Off hand: every syscall that takes a time_t or a structure containing > >a time_t would have to be reimplemented and a compatability version[...] > > This is a pretty nasty piece of work because it also involves the > timespec and timeval structures which appear in ioctls, socket > options, socket messages and so on. And with amd64/x86-64, it may prove to not really be necessary. -- John Baldwin From jhein at timing.com Wed Sep 17 15:34:29 2008 From: jhein at timing.com (John Hein) Date: Wed Sep 17 15:34:35 2008 Subject: 64 bit time_t In-Reply-To: <75968.1221600374@critter.freebsd.dk> References: <20080916211646.GA35778@lor.one-eyed-alien.net> <75968.1221600374@critter.freebsd.dk> Message-ID: <18641.9074.499346.988999@gromit.timing.com> Poul-Henning Kamp wrote at 21:26 +0000 on Sep 16, 2008: > In message <20080916211646.GA35778@lor.one-eyed-alien.net>, Brooks Davis writes > : > > > >--PEIAKu/WMn1b1Hv9 > >Content-Type: text/plain; charset=us-ascii > >Content-Disposition: inline > > > >On Tue, Sep 16, 2008 at 02:17:16PM -0600, John Hein wrote: > >> Other than recompiling for -current users (and not being an MFC-able > >> change and possibly breaking a gazillion unfortunately written ports), > >> are their any other issues with switching to 64 bit time_t for i386? > >> I suppose compat libs are a bit dicey. > > > >Off hand: every syscall that takes a time_t or a structure containing > >a time_t would have to be reimplemented and a compatability version[...] > > This is a pretty nasty piece of work because it also involves the > timespec and timeval structures which appear in ioctls, socket > options, socket messages and so on. Okay. Sounds "fun". So for systems where we don't care about compatibility (where a product is built from scratch and we don't have to worry about 3rd party binary libs/programs), the problems mentioned by brooks & phk disappear. No one wants to play the performance or atomic access card? From jhein at timing.com Wed Sep 17 15:38:43 2008 From: jhein at timing.com (John Hein) Date: Wed Sep 17 15:38:50 2008 Subject: 64 bit time_t In-Reply-To: <200809171040.36105.jhb@freebsd.org> References: <75968.1221600374@critter.freebsd.dk> <200809171040.36105.jhb@freebsd.org> Message-ID: <18641.9342.134166.77425@gromit.timing.com> John Baldwin wrote at 10:40 -0400 on Sep 17, 2008: > And with amd64/x86-64, it may prove to not really be necessary. I'm not sure I understand the "may" part. Aren't we already using 64 bit time_t natively on amd64? Or maybe you're talking about 32 bit compat on amd64. From phk at phk.freebsd.dk Wed Sep 17 15:43:10 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed Sep 17 15:43:19 2008 Subject: 64 bit time_t In-Reply-To: Your message of "Wed, 17 Sep 2008 09:34:10 CST." <18641.9074.499346.988999@gromit.timing.com> Message-ID: <47836.1221666187@critter.freebsd.dk> In message <18641.9074.499346.988999@gromit.timing.com>, John Hein writes: >So for systems where we don't care about compatibility (where a >product is built from scratch and we don't have to worry about 3rd >party binary libs/programs), the problems mentioned by brooks & phk >disappear. > >No one wants to play the performance or atomic access card? I know of only the kernel variable "time_second" where that would be a concern, and since we have 64bit atomic writes, I consider that a non-concern. At any rate, even if you write it with two 32bit writes, you will only have one chance for a race every 136 years. Next time you have the chance is: critter phk> bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 2^31-1 2147483647 critter phk> date -r 2147483647 Tue Jan 19 03:14:07 UTC 2038 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From keramida at ceid.upatras.gr Wed Sep 17 16:34:06 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Wed Sep 17 16:34:22 2008 Subject: 64 bit time_t In-Reply-To: <75968.1221600374@critter.freebsd.dk> (Poul-Henning Kamp's message of "Tue, 16 Sep 2008 21:26:14 +0000") References: <75968.1221600374@critter.freebsd.dk> Message-ID: <87iqsuk9zn.fsf@kobe.laptop> On Tue, 16 Sep 2008 21:26:14 +0000, "Poul-Henning Kamp" wrote: >In message <20080916211646.GA35778@lor.one-eyed-alien.net>, Brooks Davis writes >>On Tue, Sep 16, 2008 at 02:17:16PM -0600, John Hein wrote: >>> Other than recompiling for -current users (and not being an MFC-able >>> change and possibly breaking a gazillion unfortunately written ports), >>> are their any other issues with switching to 64 bit time_t for i386? >>> I suppose compat libs are a bit dicey. >> >> Off hand: every syscall that takes a time_t or a structure containing >> a time_t would have to be reimplemented and a compatability >> version[...] > > This is a pretty nasty piece of work because it also involves the > timespec and timeval structures which appear in ioctls, socket > options, socket messages and so on. Wire-formats for network protocols are also going to be lots of fun too. icmp4's and tcp4's timestamp options (and other wire-protocol formats) use uint32_t. Presumambly, by the time 32-bits are no longer enough these wire-protools will be replaced by their more modern versions. From jhb at freebsd.org Wed Sep 17 17:44:13 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Sep 17 17:44:19 2008 Subject: 64 bit time_t In-Reply-To: <18641.9342.134166.77425@gromit.timing.com> References: <75968.1221600374@critter.freebsd.dk> <200809171040.36105.jhb@freebsd.org> <18641.9342.134166.77425@gromit.timing.com> Message-ID: <200809171321.45354.jhb@freebsd.org> On Wednesday 17 September 2008 11:38:38 am John Hein wrote: > John Baldwin wrote at 10:40 -0400 on Sep 17, 2008: > > And with amd64/x86-64, it may prove to not really be necessary. > > I'm not sure I understand the "may" part. Aren't we already using 64 > bit time_t natively on amd64? Or maybe you're talking about 32 bit > compat on amd64. Yes, we are, and as newer server-class machines (at least) are predominantly 64-bit, for at least the server-class market it would seem that boxes will probably move to an amd64 kernel with a 64-bit time_t w/o requiring lots of rototilling to support 64-bit time_t on i386. -- John Baldwin From jhein at timing.com Wed Sep 17 20:08:43 2008 From: jhein at timing.com (John Hein) Date: Wed Sep 17 20:08:50 2008 Subject: 64 bit time_t In-Reply-To: <200809171321.45354.jhb@freebsd.org> References: <75968.1221600374@critter.freebsd.dk> <200809171040.36105.jhb@freebsd.org> <18641.9342.134166.77425@gromit.timing.com> <200809171321.45354.jhb@freebsd.org> Message-ID: <18641.24692.875414.533794@gromit.timing.com> John Baldwin wrote at 13:21 -0400 on Sep 17, 2008: > On Wednesday 17 September 2008 11:38:38 am John Hein wrote: > > John Baldwin wrote at 10:40 -0400 on Sep 17, 2008: > > > And with amd64/x86-64, it may prove to not really be necessary. > > > > I'm not sure I understand the "may" part. Aren't we already using 64 > > bit time_t natively on amd64? Or maybe you're talking about 32 bit > > compat on amd64. > > Yes, we are, and as newer server-class machines (at least) are predominantly > 64-bit, for at least the server-class market it would seem that boxes will > probably move to an amd64 kernel with a 64-bit time_t w/o requiring lots of > rototilling to support 64-bit time_t on i386. Right. I'm more concerned with planning now for y2038 on 32-bit embedded boxes that may still be around in 30 years. In this case, I think it's easy enough for me to just change my local FreeBSD tree to have time_t be 64 bit and recompile. But that doesn't help those users desperately clinging to their 7.1-RELEASE i386 boxes 30 years from now ;) From keramida at ceid.upatras.gr Thu Sep 18 09:33:16 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Thu Sep 18 09:33:32 2008 Subject: 64 bit time_t In-Reply-To: <18641.24692.875414.533794@gromit.timing.com> (John Hein's message of "Wed, 17 Sep 2008 13:54:28 -0600") References: <75968.1221600374@critter.freebsd.dk> <200809171040.36105.jhb@freebsd.org> <18641.9342.134166.77425@gromit.timing.com> <200809171321.45354.jhb@freebsd.org> <18641.24692.875414.533794@gromit.timing.com> Message-ID: <874p4dlry4.fsf@kobe.laptop> On Wed, 17 Sep 2008 13:54:28 -0600, John Hein wrote: > But that doesn't help those users desperately clinging to their > 7.1-RELEASE i386 boxes 30 years from now ;) If they are still running 7.1-RELEASE 30 years from now, they can give us their root password now and avoid the wait, I think :) /me ducks and runs From jhb at freebsd.org Fri Sep 19 20:35:16 2008 From: jhb at freebsd.org (John Baldwin) Date: Fri Sep 19 20:35:18 2008 Subject: Trim interrupt messages in dmesg.. Message-ID: <200809191437.28550.jhb@freebsd.org> Personally, I find the MPSAFE/ITHREAD/FILTER messages a bit much in dmesg. I'd like to cut it back to just warning about GIANT-locked handlers. Thoughts? Index: subr_bus.c =================================================================== --- subr_bus.c (revision 183112) +++ subr_bus.c (working copy) @@ -3538,15 +3538,6 @@ return (error); if (handler != NULL && !(flags & INTR_MPSAFE)) device_printf(dev, "[GIANT-LOCKED]\n"); - if (bootverbose && (flags & INTR_MPSAFE)) - device_printf(dev, "[MPSAFE]\n"); - if (filter != NULL) { - if (handler == NULL) - device_printf(dev, "[FILTER]\n"); - else - device_printf(dev, "[FILTER+ITHREAD]\n"); - } else - device_printf(dev, "[ITHREAD]\n"); return (0); } -- John Baldwin From ed at 80386.nl Sat Sep 20 08:29:50 2008 From: ed at 80386.nl (Ed Schouten) Date: Sat Sep 20 08:29:54 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <200809191437.28550.jhb@freebsd.org> References: <200809191437.28550.jhb@freebsd.org> Message-ID: <20080920082949.GX81522@hoeg.nl> Hello John, * John Baldwin wrote: > Personally, I find the MPSAFE/ITHREAD/FILTER messages a bit much in dmesg. > I'd like to cut it back to just warning about GIANT-locked handlers. > Thoughts? What about printing all the flags only when bootverbose is turned on, just like MPSAFE is right now? -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080920/dbe35233/attachment.pgp From rink at FreeBSD.org Sat Sep 20 09:37:13 2008 From: rink at FreeBSD.org (Rink Springer) Date: Sat Sep 20 09:37:21 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <20080920082949.GX81522@hoeg.nl> References: <200809191437.28550.jhb@freebsd.org> <20080920082949.GX81522@hoeg.nl> Message-ID: <20080920092034.GC49919@rink.nu> On Sat, Sep 20, 2008 at 10:29:49AM +0200, Ed Schouten wrote: > What about printing all the flags only when bootverbose is turned on, > just like MPSAFE is right now? I'd prefer just to nuke them completely. It's not as if they are so informative with almost everything finely-locked - plus if you need details, you have The Source(tm) :-) Regards, -- Rink P.W. Springer - http://rink.nu "Anyway boys, this is America. Just because you get more votes doesn't mean you win." - Fox Mulder From piso at FreeBSD.org Sat Sep 20 13:45:58 2008 From: piso at FreeBSD.org (Paolo Pisati) Date: Sat Sep 20 13:46:26 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <20080920092034.GC49919@rink.nu> References: <200809191437.28550.jhb@freebsd.org> <20080920082949.GX81522@hoeg.nl> <20080920092034.GC49919@rink.nu> Message-ID: <20080920132154.GA53882@tin.it> On Sat, Sep 20, 2008 at 11:20:34AM +0200, Rink Springer wrote: > > I'd prefer just to nuke them completely. It's not as if they are so > informative with almost everything finely-locked - plus if you need > details, you have The Source(tm) :-) actually there's nothing related to locking with these messages. with filters you can have 3 different setup: a filter-only driver, an ithread-only driver or a filter+ithread driver. Thus while i was developing intr filtr i wanted to know which driver was using what, and that's where these messges were added. -- bye, P. From sam at freebsd.org Sat Sep 20 17:13:18 2008 From: sam at freebsd.org (Sam Leffler) Date: Sat Sep 20 17:13:21 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <20080920092034.GC49919@rink.nu> References: <200809191437.28550.jhb@freebsd.org> <20080920082949.GX81522@hoeg.nl> <20080920092034.GC49919@rink.nu> Message-ID: <48D52B76.9080606@freebsd.org> Rink Springer wrote: > On Sat, Sep 20, 2008 at 10:29:49AM +0200, Ed Schouten wrote: > >> What about printing all the flags only when bootverbose is turned on, >> just like MPSAFE is right now? >> > > I'd prefer just to nuke them completely. It's not as if they are so > informative with almost everything finely-locked - plus if you need > details, you have The Source(tm) :-) > Actually the code is useless. We need a way to identify how each handler is registered. So long as we have that I'm fine with trimming the msgs. Sticking it under bootverbose is ok too though I'd still like a way to see how the handler is registered w/o have to reboot. Sam From dillon at apollo.backplane.com Sat Sep 20 17:31:26 2008 From: dillon at apollo.backplane.com (Matthew Dillon) Date: Sat Sep 20 17:31:29 2008 Subject: Trim interrupt messages in dmesg.. References: <200809191437.28550.jhb@freebsd.org> <20080920082949.GX81522@hoeg.nl> <20080920092034.GC49919@rink.nu> <48D52B76.9080606@freebsd.org> Message-ID: <200809201731.m8KHVPbq030945@apollo.backplane.com> :Actually the code is useless. We need a way to identify how each :handler is registered. So long as we have that I'm fine with trimming :the msgs. Sticking it under bootverbose is ok too though I'd still like :a way to see how the handler is registered w/o have to reboot. : : Sam I would recommend augmenting vmstat -i to provide the required information. -Matt Matthew Dillon From sam at freebsd.org Sat Sep 20 19:34:36 2008 From: sam at freebsd.org (Sam Leffler) Date: Sat Sep 20 19:34:40 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <200809201731.m8KHVPbq030945@apollo.backplane.com> References: <200809191437.28550.jhb@freebsd.org> <20080920082949.GX81522@hoeg.nl> <20080920092034.GC49919@rink.nu> <48D52B76.9080606@freebsd.org> <200809201731.m8KHVPbq030945@apollo.backplane.com> Message-ID: <48D5504A.7080309@freebsd.org> Matthew Dillon wrote: > :Actually the code is useless. We need a way to identify how each > :handler is registered. So long as we have that I'm fine with trimming > :the msgs. Sticking it under bootverbose is ok too though I'd still like > :a way to see how the handler is registered w/o have to reboot. > : > : Sam > > I would recommend augmenting vmstat -i to provide the required > information. > Doesn't belong there IMO but devinfo (where I hoped it might go) lacks the context to collect the info. Sam From marius at alchemy.franken.de Sat Sep 20 23:26:50 2008 From: marius at alchemy.franken.de (Marius Strobl) Date: Sat Sep 20 23:26:53 2008 Subject: removal of ipi_all() and ipi_self() [Re: cvs commit: src/sys/sparc64/include smp.h src/sys/sparc64/sparc64 genassym.c mp_machdep.c] In-Reply-To: <89B9A8BE-05F2-4DB2-B7B2-AB240AA9F0DD@mac.com> References: <200809181356.m8IDuaxT089888@repoman.freebsd.org> <200809181027.51997.jhb@freebsd.org> <20080918191947.GX94638@alchemy.franken.de> <89B9A8BE-05F2-4DB2-B7B2-AB240AA9F0DD@mac.com> Message-ID: <20080920231152.GA67442@alchemy.franken.de> On Thu, Sep 18, 2008 at 12:48:52PM -0700, Marcel Moolenaar wrote: > > On Sep 18, 2008, at 12:19 PM, Marius Strobl wrote: > > >On Thu, Sep 18, 2008 at 10:27:51AM -0400, John Baldwin wrote: > >>On Thursday 18 September 2008 09:56:30 am Marius Strobl wrote: > >>>marius 2008-09-18 13:56:30 UTC > >>> > >>> FreeBSD src repository > >>> > >>> Modified files: > >>> sys/sparc64/include smp.h > >>> sys/sparc64/sparc64 genassym.c mp_machdep.c > >>> Log: > >>> SVN rev 183142 on 2008-09-18 13:56:30Z by marius > >>> > >>> - Newer firmware versions no longer provide SUNW,stop-self so just > >>> disable interrupts and loop forever with these. > >>> - Hide all MP-related bits in underneath #ifdef > >>>SMP. > >>> - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose > >>>any > >>> additional bits but save a few cycles by doing so. > >>> - Remove ipi_all(9), which actually only called panic(9). It > >>>can't be > >>> implemented natively anyway and having it removed at least causes > >>> MI users to fail already fail when linking. > >> > >>Should we just remove ipi_all() completely? > >> > > > >Well, grepping in the CVS repository shows that there never was > >an actually consumer of ipi_all() (only #ifdef'ed out ones in > >ironically the sparc64 code) so it seems to be a good candidate > >for axing. Generally I can't think of a reason why MI code would > >want a CPU to send an IPI to itself. Actually, ipi_self() also > >isn't and never was used in MI code, only in ia64 and powerpc > >code for testing purposes. > > That's DS (=developer-specific) code rather than MI or MD code :-) > > Sending a test IPI to 'self' helps with bring-up or porting, but > serves no real purpose (other than maybe a POST-like purpose) > once IPIs are known to work... > Okay, I take these as a call for removing ipi_all() and ipi_self() along with the ia64 and powerpc test IPI code completely. A patch doing just that and which passes a universe build just fine is at: http://people.freebsd.org/~marius/nuke_ipi_all_self.diff Does anybody object to committing it? Should __FreeBSD_version be bumped for this? Marius From edwin at freebsd.org Sun Sep 21 09:42:55 2008 From: edwin at freebsd.org (Edwin Groothuis) Date: Sun Sep 21 09:43:00 2008 Subject: tzcode update to 2008e Message-ID: <20080921092704.GC83347@k7.mavetju> Currently the tzcode in the FreeBSD operating system is from 2004. I have updated, on my development machine at home, src/lib/libc/stdtime and src/usr.sbin/zic to tzcode version 2008e. It still works. zic compiles the zonefiles into version 2 format, zdump properly shows the data. The strftime() tests with the date regression tests (bin/127514: [patch] regression tests for date(1)) work fine. The patch can be found at http://www.mavetju.org/~edwin/tzcode2008e-update.txt, it is against current. The MFCs for 7.x and 6.x should be relatively simple. I have done it on i386, so I'm happy to hear how other architectures are going with it. And then, euhm... yeah, what's next? Edwin -- Edwin Groothuis edwin@freebsd.org http://www.mavetju.org From peter at wemm.org Mon Sep 22 05:32:17 2008 From: peter at wemm.org (Peter Wemm) Date: Mon Sep 22 05:32:21 2008 Subject: tzcode update to 2008e In-Reply-To: <20080921092704.GC83347@k7.mavetju> References: <20080921092704.GC83347@k7.mavetju> Message-ID: On Sun, Sep 21, 2008 at 2:27 AM, Edwin Groothuis wrote: > Currently the tzcode in the FreeBSD operating system is from 2004. > I have updated, on my development machine at home, src/lib/libc/stdtime > and src/usr.sbin/zic to tzcode version 2008e. It still works. > > zic compiles the zonefiles into version 2 format, zdump properly > shows the data. The strftime() tests with the date regression tests > (bin/127514: [patch] regression tests for date(1)) work fine. > > The patch can be found at > http://www.mavetju.org/~edwin/tzcode2008e-update.txt, it is against > current. The MFCs for 7.x and 6.x should be relatively simple. > > I have done it on i386, so I'm happy to hear how other architectures > are going with it. And then, euhm... yeah, what's next? > > Edwin What happens when you run old static linked binaries? Will they understand 'version 2 format' zone files? -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From edwin at freebsd.org Mon Sep 22 09:23:16 2008 From: edwin at freebsd.org (Edwin Groothuis) Date: Mon Sep 22 09:23:21 2008 Subject: tzcode update to 2008e In-Reply-To: References: <20080921092704.GC83347@k7.mavetju> Message-ID: <20080922092245.GA3233@k7.mavetju> On Sun, Sep 21, 2008 at 10:32:16PM -0700, Peter Wemm wrote: > On Sun, Sep 21, 2008 at 2:27 AM, Edwin Groothuis wrote: > > Currently the tzcode in the FreeBSD operating system is from 2004. > > I have updated, on my development machine at home, src/lib/libc/stdtime > > and src/usr.sbin/zic to tzcode version 2008e. It still works. > > > > zic compiles the zonefiles into version 2 format, zdump properly > > shows the data. The strftime() tests with the date regression tests > > (bin/127514: [patch] regression tests for date(1)) work fine. > > > > The patch can be found at > > http://www.mavetju.org/~edwin/tzcode2008e-update.txt, it is against > > current. The MFCs for 7.x and 6.x should be relatively simple. > > > > I have done it on i386, so I'm happy to hear how other architectures > > are going with it. And then, euhm... yeah, what's next? > > > > Edwin > > What happens when you run old static linked binaries? Will they > understand 'version 2 format' zone files? If I understand the code and its behaviour correct (which I asusme I did, but feel free to correct me, the relevant data is tzload() in lib/libc/stdtime/localtime.c :-), the version two data is added to the end of the zonefile. A diff of the two shows that this is indeed happening. I have installed the new zonefile on an 7.0 OS and ran the regression tests and old zdump on it without problems. So the conclusion for me is that it is working fine with statically linked libraries. For 7.0 users, the patches can be found at http://www.mavetju.org/~edwin/tzcode2008e-update-7.txt. Edwin -- Edwin Groothuis edwin@freebsd.org http://www.mavetju.org From bugmaster at FreeBSD.org Mon Sep 22 11:06:50 2008 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 22 11:07:01 2008 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200809221106.m8MB6nTt015300@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From rpaulo at FreeBSD.org Mon Sep 22 11:50:35 2008 From: rpaulo at FreeBSD.org (Rui Paulo) Date: Mon Sep 22 11:50:37 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <200809191437.28550.jhb@freebsd.org> References: <200809191437.28550.jhb@freebsd.org> Message-ID: <20080922111546.GB10240@alpha.local> On Fri, Sep 19, 2008 at 02:37:28PM -0400, John Baldwin wrote: > Personally, I find the MPSAFE/ITHREAD/FILTER messages a bit much in dmesg. > I'd like to cut it back to just warning about GIANT-locked handlers. > Thoughts? Go for it. As Sam pointed out, it would be nice to have this information in devinfo, but don't let that stop you. Regards, -- Rui Paulo From edwin at freebsd.org Mon Sep 22 13:19:47 2008 From: edwin at freebsd.org (Edwin Groothuis) Date: Mon Sep 22 13:19:50 2008 Subject: tzcode update to 2008e In-Reply-To: <20080922092245.GA3233@k7.mavetju> References: <20080921092704.GC83347@k7.mavetju> <20080922092245.GA3233@k7.mavetju> Message-ID: <20080922131916.GB3233@k7.mavetju> On Mon, Sep 22, 2008 at 07:22:45PM +1000, Edwin Groothuis wrote: > For 7.0 users, the patches can be found at > http://www.mavetju.org/~edwin/tzcode2008e-update-7.txt. For people who want to try it, after compiling and installing libc and the files in usr.sbin/zic, you need to also (re)install src/share/zoneinfo or ports/misc/zoneinfo, to get the zone-files compiled with the right zic(8) binary, and then run tzsetup(8) to install /etc/localtime again. I just had it tested on an amd64 machine with releng_7 on it and it worked fine on it. Edwin -- Edwin Groothuis | Personal website: http://www.mavetju.org edwin@mavetju.org | Weblog: http://www.mavetju.org/weblog/ From sam at freebsd.org Mon Sep 22 15:44:31 2008 From: sam at freebsd.org (Sam Leffler) Date: Mon Sep 22 15:44:37 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <20080922111546.GB10240@alpha.local> References: <200809191437.28550.jhb@freebsd.org> <20080922111546.GB10240@alpha.local> Message-ID: <48D7BD5E.6060005@freebsd.org> Rui Paulo wrote: > On Fri, Sep 19, 2008 at 02:37:28PM -0400, John Baldwin wrote: > >> Personally, I find the MPSAFE/ITHREAD/FILTER messages a bit much in dmesg. >> I'd like to cut it back to just warning about GIANT-locked handlers. >> Thoughts? >> > > Go for it. As Sam pointed out, it would be nice to have this information > in devinfo, but don't let that stop you. > > Regards, > "nice" is NOT what I said. Until there's a way to find this information please do not remove the printfs. Sam From jhb at freebsd.org Mon Sep 22 21:09:33 2008 From: jhb at freebsd.org (John Baldwin) Date: Mon Sep 22 21:09:41 2008 Subject: removal of ipi_all() and ipi_self() [Re: cvs commit: src/sys/sparc64/include smp.h src/sys/sparc64/sparc64 genassym.c mp_machdep.c] In-Reply-To: <20080920231152.GA67442@alchemy.franken.de> References: <200809181356.m8IDuaxT089888@repoman.freebsd.org> <89B9A8BE-05F2-4DB2-B7B2-AB240AA9F0DD@mac.com> <20080920231152.GA67442@alchemy.franken.de> Message-ID: <200809221639.56429.jhb@freebsd.org> On Saturday 20 September 2008 07:11:53 pm Marius Strobl wrote: > On Thu, Sep 18, 2008 at 12:48:52PM -0700, Marcel Moolenaar wrote: > > > > On Sep 18, 2008, at 12:19 PM, Marius Strobl wrote: > > > > >On Thu, Sep 18, 2008 at 10:27:51AM -0400, John Baldwin wrote: > > >>On Thursday 18 September 2008 09:56:30 am Marius Strobl wrote: > > >>>marius 2008-09-18 13:56:30 UTC > > >>> > > >>> FreeBSD src repository > > >>> > > >>> Modified files: > > >>> sys/sparc64/include smp.h > > >>> sys/sparc64/sparc64 genassym.c mp_machdep.c > > >>> Log: > > >>> SVN rev 183142 on 2008-09-18 13:56:30Z by marius > > >>> > > >>> - Newer firmware versions no longer provide SUNW,stop-self so just > > >>> disable interrupts and loop forever with these. > > >>> - Hide all MP-related bits in underneath #ifdef > > >>>SMP. > > >>> - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose > > >>>any > > >>> additional bits but save a few cycles by doing so. > > >>> - Remove ipi_all(9), which actually only called panic(9). It > > >>>can't be > > >>> implemented natively anyway and having it removed at least causes > > >>> MI users to fail already fail when linking. > > >> > > >>Should we just remove ipi_all() completely? > > >> > > > > > >Well, grepping in the CVS repository shows that there never was > > >an actually consumer of ipi_all() (only #ifdef'ed out ones in > > >ironically the sparc64 code) so it seems to be a good candidate > > >for axing. Generally I can't think of a reason why MI code would > > >want a CPU to send an IPI to itself. Actually, ipi_self() also > > >isn't and never was used in MI code, only in ia64 and powerpc > > >code for testing purposes. > > > > That's DS (=developer-specific) code rather than MI or MD code :-) > > > > Sending a test IPI to 'self' helps with bring-up or porting, but > > serves no real purpose (other than maybe a POST-like purpose) > > once IPIs are known to work... > > > > Okay, I take these as a call for removing ipi_all() and ipi_self() > along with the ia64 and powerpc test IPI code completely. A patch > doing just that and which passes a universe build just fine is at: > http://people.freebsd.org/~marius/nuke_ipi_all_self.diff > Does anybody object to committing it? Should __FreeBSD_version be > bumped for this? I think this is ok. -- John Baldwin From jhb at freebsd.org Mon Sep 22 21:09:41 2008 From: jhb at freebsd.org (John Baldwin) Date: Mon Sep 22 21:09:44 2008 Subject: Trim interrupt messages in dmesg.. In-Reply-To: <20080920082949.GX81522@hoeg.nl> References: <200809191437.28550.jhb@freebsd.org> <20080920082949.GX81522@hoeg.nl> Message-ID: <200809221640.37956.jhb@freebsd.org> On Saturday 20 September 2008 04:29:49 am Ed Schouten wrote: > Hello John, > > * John Baldwin wrote: > > Personally, I find the MPSAFE/ITHREAD/FILTER messages a bit much in dmesg. > > I'd like to cut it back to just warning about GIANT-locked handlers. > > Thoughts? > > What about printing all the flags only when bootverbose is turned on, > just like MPSAFE is right now? I think dmesg is the wrong place for this information. An intrinfo or some such would be better. -- John Baldwin From jhb at freebsd.org Tue Sep 23 19:12:30 2008 From: jhb at freebsd.org (John Baldwin) Date: Tue Sep 23 19:12:33 2008 Subject: Using a file-backed swap via md0 results in panic on reboot.. Message-ID: <200809231509.40279.jhb@freebsd.org> So if you attach an md device to a file and then use that md0 device for swap, you will get a panic on shutdown if the swap file was in use. The reason is that we remove swap devices _after_ unmounting the filesystems in boot(): if (nbusy) { /* * Failed to sync all blocks. Indicate this and don't * unmount filesystems (thus forcing an fsck on reboot). */ printf("Giving up on %d buffers\n", nbusy); DELAY(5000000); /* 5 seconds */ } else { if (!first_buf_printf) printf("Final sync complete\n"); /* * Unmount filesystems */ if (panicstr == 0) vfs_unmountall(); } swapoff_all(); DELAY(100000); /* wait for console output to finish */ Is there any reason we can't move the swapoff_all()? Should we perhaps only do it in "clean" case? Alternatively, should the swapoff_all() during shutdown have a special flag so that it doesn't actually read in any data from disk and just throws away the data instead? Sample panic: Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done All buffers synced. Swap device ad0s1b removed. swap_pager: I/O error - pagein failed; blkno 2097177,size 4096, error 5 panic: swap_pager_force_pagein: read from swap failed cpuid = 0 KDB: stack backtrace: panic() at panic+0x2c5 swapoff_one() at swapoff_one+0x5bb swapoff_all() at swapoff_all+0xe4 boot() at boot+0x871 reboot() at reboot+0x45 syscall() at syscall+0x642 Xfast_syscall() at Xfast_syscall+0xa8 -- John Baldwin From julian at elischer.org Tue Sep 23 19:55:32 2008 From: julian at elischer.org (Julian Elischer) Date: Tue Sep 23 19:55:36 2008 Subject: Using a file-backed swap via md0 results in panic on reboot.. In-Reply-To: <200809231509.40279.jhb@freebsd.org> References: <200809231509.40279.jhb@freebsd.org> Message-ID: <48D949B2.7020001@elischer.org> John Baldwin wrote: > So if you attach an md device to a file and then use that md0 device for swap, > you will get a panic on shutdown if the swap file was in use. The reason is > that we remove swap devices _after_ unmounting the filesystems in boot(): > > if (nbusy) { > /* > * Failed to sync all blocks. Indicate this and don't > * unmount filesystems (thus forcing an fsck on reboot). > */ > printf("Giving up on %d buffers\n", nbusy); > DELAY(5000000); /* 5 seconds */ > } else { > if (!first_buf_printf) > printf("Final sync complete\n"); > /* > * Unmount filesystems > */ > if (panicstr == 0) > vfs_unmountall(); > } > swapoff_all(); > DELAY(100000); /* wait for console output to finish */ > > Is there any reason we can't move the swapoff_all()? Should we perhaps only > do it in "clean" case? Alternatively, should the swapoff_all() during > shutdown have a special flag so that it doesn't actually read in any data > from disk and just throws away the data instead? Sample panic: then what happens with a swap based filesystem? > > Waiting (max 60 seconds) for system process `vnlru' to stop...done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining...2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 > 0 0 done > Waiting (max 60 seconds) for system process `bufdaemon' to stop...done > All buffers synced. > Swap device ad0s1b removed. > swap_pager: I/O error - pagein failed; blkno 2097177,size 4096, error 5 > panic: swap_pager_force_pagein: read from swap failed > cpuid = 0 > KDB: stack backtrace: > panic() at panic+0x2c5 > swapoff_one() at swapoff_one+0x5bb > swapoff_all() at swapoff_all+0xe4 > boot() at boot+0x871 > reboot() at reboot+0x45 > syscall() at syscall+0x642 > Xfast_syscall() at Xfast_syscall+0xa8 > From pjd at FreeBSD.org Tue Sep 23 20:15:06 2008 From: pjd at FreeBSD.org (Pawel Jakub Dawidek) Date: Tue Sep 23 20:15:10 2008 Subject: Using a file-backed swap via md0 results in panic on reboot.. In-Reply-To: <200809231509.40279.jhb@freebsd.org> References: <200809231509.40279.jhb@freebsd.org> Message-ID: <20080923194454.GA2196@garage.freebsd.pl> On Tue, Sep 23, 2008 at 03:09:40PM -0400, John Baldwin wrote: > So if you attach an md device to a file and then use that md0 device for swap, > you will get a panic on shutdown if the swap file was in use. The reason is > that we remove swap devices _after_ unmounting the filesystems in boot(): > > if (nbusy) { > /* > * Failed to sync all blocks. Indicate this and don't > * unmount filesystems (thus forcing an fsck on reboot). > */ > printf("Giving up on %d buffers\n", nbusy); > DELAY(5000000); /* 5 seconds */ > } else { > if (!first_buf_printf) > printf("Final sync complete\n"); > /* > * Unmount filesystems > */ > if (panicstr == 0) > vfs_unmountall(); > } > swapoff_all(); > DELAY(100000); /* wait for console output to finish */ > > Is there any reason we can't move the swapoff_all()? Should we perhaps only > do it in "clean" case? Alternatively, should the swapoff_all() during > shutdown have a special flag so that it doesn't actually read in any data > from disk and just throws away the data instead? Sample panic: > > Waiting (max 60 seconds) for system process `vnlru' to stop...done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining...2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 > 0 0 done > Waiting (max 60 seconds) for system process `bufdaemon' to stop...done > All buffers synced. > Swap device ad0s1b removed. > swap_pager: I/O error - pagein failed; blkno 2097177,size 4096, error 5 > panic: swap_pager_force_pagein: read from swap failed > cpuid = 0 > KDB: stack backtrace: > panic() at panic+0x2c5 > swapoff_one() at swapoff_one+0x5bb > swapoff_all() at swapoff_all+0xe4 > boot() at boot+0x871 > reboot() at reboot+0x45 > syscall() at syscall+0x642 > Xfast_syscall() at Xfast_syscall+0xa8 The problem is this: sometimes we want to first unmount file systems and sometimes we want to first remove swap devices. In your case you have swap device residing on a file system. Imagine the opposite situation where you have a file system residing on swap device: mdconfig -t swap; newfs /dev/mdX; mount /dev/mdX The solution I have been thinking about is to call vfs_unmountall() and swapoff_all() in a loop and changed them to return number of file system unmounted or swap devices removed respectively: do { i = vfs_unmountall(); j = swapoff_all(); } while (i > 0 || j > 0); Something like that would cover both situations and even more complex (although not very realistic) situations: FS on SWAP on FS on SWAP, etc. Of course vfs_unmountall() and swapoff_all() would have to skip busy file systems/swap devices. -- Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080923/7d3221f1/attachment.pgp From imp at bsdimp.com Wed Sep 24 04:46:00 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Wed Sep 24 04:46:02 2008 Subject: 64 bit time_t In-Reply-To: <18641.24692.875414.533794@gromit.timing.com> References: <18641.9342.134166.77425@gromit.timing.com> <200809171321.45354.jhb@freebsd.org> <18641.24692.875414.533794@gromit.timing.com> Message-ID: <20080923.224537.-957831121.imp@bsdimp.com> In message: <18641.24692.875414.533794@gromit.timing.com> John Hein writes: : John Baldwin wrote at 13:21 -0400 on Sep 17, 2008: : > On Wednesday 17 September 2008 11:38:38 am John Hein wrote: : > > John Baldwin wrote at 10:40 -0400 on Sep 17, 2008: : > > > And with amd64/x86-64, it may prove to not really be necessary. : > > : > > I'm not sure I understand the "may" part. Aren't we already using 64 : > > bit time_t natively on amd64? Or maybe you're talking about 32 bit : > > compat on amd64. : > : > Yes, we are, and as newer server-class machines (at least) are predominantly : > 64-bit, for at least the server-class market it would seem that boxes will : > probably move to an amd64 kernel with a 64-bit time_t w/o requiring lots of : > rototilling to support 64-bit time_t on i386. : : Right. I'm more concerned with planning now for y2038 on 32-bit : embedded boxes that may still be around in 30 years. In this case, I : think it's easy enough for me to just change my local FreeBSD tree to : have time_t be 64 bit and recompile. : : But that doesn't help those users desperately clinging to their : 7.1-RELEASE i386 boxes 30 years from now ;) It won't be on FreeBSD/arm embedded boxes :-) When we changed from 32-bit to 64-bit on arm, it wasn't a huge deal. If you don't care about ABI compatibility, then it is a simple matter of changing the definition of __time_t in sys/i386/include/_types.h and rebuilding. After all, there's no binaries not built as part of the controlled build process that you use in certain embedded boxes that I think you might be talking about ;-0. All that system call and ioctl and structure compat stuff is interesting, but just not relevant to your problem domain... Warner From xcllnt at mac.com Thu Sep 25 18:59:44 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Thu Sep 25 18:59:55 2008 Subject: RFC: making gpart default Message-ID: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> All, I'd like to switch all architectures to gpart for the reasons given below. All current partitioning schemes are supported by gpart and work on all platforms. On top of that, ia64 and powerpc are using gpart exclusively already. Rationale: 1. Having different utilities for different schemes, which are otherwise identical in functionality, is painful. Especially since they all provide a different user interface. While this is a non-issue for the old geezers among us (we don't know any better), it is something that the younger crowd perceives as "low-quality", especially when compared to Linux' parted. However, the fact that some allow scripting while others don't is important even for the old geezers. gpart(8) is the answer. 2. We currently have multiple GEOMs in the kernel that do not provide a unified, common and/or standard interface. This is especially problematic for unified tools like the installer. In fact, we don't have unified tools that use the ctlreq interface at all, because it's virtually impossible. The fact that we have sade, is an indication that we do want a unified tool for partitioning, but without using the GEOM I/F the tool is useless to modify partitions on a disk with mounted file systems. The gpart GEOM provides a unified I/F usable by any tool for creating and modifying any kind of partitioning scheme. 3. fsck and newfs contain some logic to find out what kind of file system they're working on and invoke the appropriate back-end executable. This is good functionality, but only works for BSD labels. With GPT being used more often and non-PC hardware using different partitioning schemes, this means that the functionality isn't working in most cases. There's fundamentally no reason why newfs or fsck cannot automatically detect a partition that uses the MSDOS file system and DTRT. With gpart, we can make that functionality working across all partitioning schemes. 4. Like the above, but for mount. By checking the partition type, it's possible to have mount DTRT much more often. 5. The gpart show command can give you a unified list of disks with their partitions, irrespective of the partitioning scheme used. For the first time a single command gives us the overview we're lacking. 6. Not all "disk" devices have a geometry. Especially md(4) devices and USB mass storage devices. This is a problem for newfs_msdos. With gpart, there's always a geometry that processes can query for so that newfs_msdos et al will work without any additional options. 7. gpart breaks the 8-partition barrier for bsdlabel. 8. gpart adds VTOC information to sunlabel. With gpart default, tools like fdisk and bsdlabel continue to work on new disks and disks that have no mounted file systems. In that respect there's no difference. However, they cannot be used when file systems are mounted. In those cases gpart needs to be used. As such, the impact is limited and makes the switch much more manageable. In short: gpart is the first step towards a unified set of tools and interfaces and provides the basis for extending file system related tools by allowing us to attach real meaning to partition types. With the commit and undo feature, gpart is ready for use by next generation installers that allow us to use any partitioning scheme on any platforms. Thoughts? -- Marcel Moolenaar xcllnt@mac.com From phk at phk.freebsd.dk Thu Sep 25 19:46:25 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu Sep 25 19:46:32 2008 Subject: RFC: making gpart default In-Reply-To: Your message of "Thu, 25 Sep 2008 10:59:31 MST." <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> Message-ID: <1896.1222371977@critter.freebsd.dk> In message <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com>, Marcel Moolenaar wri tes: >All, > >I'd like to switch all architectures to gpart for the reasons given >below. I promised long time ago, that I would only use my 'GEOM Architect' hat to interfere with what people do _to_ GEOM, not what people do _with_ GEOM. The GEOM Architect therefore has only this comment: I would like to point out explicitly that Marcels proposal, to switch to gpart by default, does not prevent the use of other GEOM classes to interpret disk-layouts. I know there are users out there who use GEOM for backup and forensic purposes, who have such private geom classes, and these will not be affected by Marcels proposal in any way, except that you may need to grab a copy of geom_slice.* if Marcel obsoletes them. The following comments are my personal comments, and they may or may not be shared by the GEOM Architect, who is free to grind his teeth as he reads them: >Rationale: >1. Having different utilities for different schemes, which are > otherwise identical in functionality, is painful. On the other hand, trying to generalize widely different semantics is also not a recipe for eternal youth: Not all the worlds partitioning metadata formats have type information, some have weird ass restrictions and bootcode(-requirements) are generally magic. I didn't dare do what you've done now, but since you touched it last, I'm not worried :-) My proposed solution was to try to get the BSD label discontinued and rely entirely on the native partitioning on the relevant platforms, but that meant dealing with FAT extended partitions and going back to libdisk and I couldn't get myself to do that. >3. fsck and newfs contain some logic to find out what kind of > file system they're working on and invoke the appropriate > back-end executable. I have always found it scary that they did this based on the out-of-band information. Granted, fsck's generally choke on each others filesystems pretty conclusively, but fsck will happily trash a database stored in partition that previously contained a filesystem, provided enough magic bits survive near the start. I would prefer a scheme where fsck, like g_label, poked around a bit to find out what the partition content looked like, and started the then backend fsck_foofs with a "I *think* this is yours, but it might not be" flag. On the other hand, if fsck_foofs is explicitly started (for instance based on /etc/fstab) it has license to kill. Overall this is a fine point, but I'd hate to make it easier to trash filesystems. Lacking a reliable mechanism to keep the out-of-band filesystem type consistent with the partition content, think dd(1), g_mirror movements of data etc, I'm not too thrilled about this. >4. Like the above, but for mount. By checking the partition > type, it's possible to have mount DTRT much more often. Same argument, same code should be used, probably also for g_label(). >6. Not all "disk" devices have a geometry. Especially md(4) > devices and USB mass storage devices. This is a problem > for newfs_msdos. With gpart, there's always a geometry [...] I will argue this is wrong. For md(4), you an define a geometry if you want one, and if your plan is to move the filesystem to real media later, you had better do so correctly. For usb-sticks, the problem is that we do not even ask them about their geometry, but treat them as SCSI disks that are one-dimensional. In many cases you can infer the geometry from a preexisting MBR, but presently our code does not do this, and as we know from the Dangerously Dedicated Disk Layout, that is a pretty error-prone heuristic in the first place. We know from NanoBSD users that this is not a theoretical issue, I have countless emails where people put NanoBSD on a USB stick but find it unbootable because it presents a different geometry to the BIOS than the one we used. Dreaming up default geometries will just hide from the user that they are about to make a mistake. I don't recommend it. I would be better to fix umass/cam/da (cross out your own code) to get the correct geometry from the usb-sticks. >7. gpart breaks the 8-partition barrier for bsdlabel. We should discontinue the bsdlabel, it has too many problems and limitations. The best way is to not install more of them once sysinstall knows how to avoid it. >With gpart default, tools like fdisk and bsdlabel continue to >work on new disks and disks that have no mounted file systems. >In that respect there's no difference. However, they cannot >be used when file systems are mounted. This is going to break more scripts than you think. I think you should have talked a bit about naming of the partitions ? Are there any compatibility or transition concerns in that area ? Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From peter at wemm.org Thu Sep 25 19:50:23 2008 From: peter at wemm.org (Peter Wemm) Date: Thu Sep 25 19:50:39 2008 Subject: RFC: making gpart default In-Reply-To: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> Message-ID: On Thu, Sep 25, 2008 at 10:59 AM, Marcel Moolenaar wrote: > I'd like to switch all architectures to gpart for the reasons given > below. All current partitioning schemes are supported by gpart and > work on all platforms. On top of that, ia64 and powerpc are using > gpart exclusively already. What, if anything, do we need to do to start using it ourselves now on i386 or amd64 platforms? Just change kernel options? Also, while here, the APM (apple partition manager) module doesn't quite work with Tivo disks. Tivo drives don't have the DDR signature. Simply removing the DDR code enables FreeBSD to mount and work with tivo drives. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From xcllnt at mac.com Thu Sep 25 19:58:16 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Thu Sep 25 19:58:22 2008 Subject: RFC: making gpart default In-Reply-To: References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> Message-ID: <69B1B4C5-8E98-4014-9767-A8D5D1919A75@mac.com> On Sep 25, 2008, at 12:38 PM, Peter Wemm wrote: > On Thu, Sep 25, 2008 at 10:59 AM, Marcel Moolenaar > wrote: >> I'd like to switch all architectures to gpart for the reasons given >> below. All current partitioning schemes are supported by gpart and >> work on all platforms. On top of that, ia64 and powerpc are using >> gpart exclusively already. > > What, if anything, do we need to do to start using it ourselves now on > i386 or amd64 platforms? Just change kernel options? Yes, that's it. At this time, the following 4 lines can be added to achieve the switch: nooption GEOM_BSD nooption GEOM_MBR options GEOM_PART_BSD options GEOM_PART_MBR > > Also, while here, the APM (apple partition manager) module doesn't > quite work with Tivo disks. Tivo drives don't have the DDR signature. > Simply removing the DDR code enables FreeBSD to mount and work with > tivo drives. Yes, I saw the commit. Do you know if there's anything we can check for in case there's no DDR? I'm thinking of accepting a DDR-less APM scheme only if we find some indication that it's a Tivo disk. I guess I like the comfort of checking the DDR and prefer not to eliminate it unconditionally. This may not be possible though... -- Marcel Moolenaar xcllnt@mac.com From xcllnt at mac.com Thu Sep 25 21:24:54 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Thu Sep 25 21:25:00 2008 Subject: RFC: making gpart default In-Reply-To: <1896.1222371977@critter.freebsd.dk> References: <1896.1222371977@critter.freebsd.dk> Message-ID: <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> On Sep 25, 2008, at 12:46 PM, Poul-Henning Kamp wrote: > In message <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com>, Marcel > Moolenaar wri > tes: >> All, >> >> I'd like to switch all architectures to gpart for the reasons given >> below. > > I promised long time ago, that I would only use my 'GEOM Architect' > hat to interfere with what people do _to_ GEOM, not what people > do _with_ GEOM. > > The GEOM Architect therefore has only this comment: > > I would like to point out explicitly that Marcels proposal, to > switch to gpart by default, does not prevent the use of other GEOM > classes to interpret disk-layouts. Nod. You can freely mix and match. People are already doing that: GPT is handled by gpart, whereas MBR and BSD are handled by the MBR and BSD slicers. >> Rationale: >> 1. Having different utilities for different schemes, which are >> otherwise identical in functionality, is painful. > > On the other hand, trying to generalize widely different semantics > is also not a recipe for eternal youth: Not all the worlds > partitioning > metadata formats have type information, some have weird ass > restrictions and bootcode(-requirements) are generally magic. I have wondered about the "different semantics" and found that it's the semantics that are actually the same. No matter what the details, they all serve the same purpose: chopping up a disk in multiple (non-overlapping) regions. Granted, supporting all the details of a particular scheme may not be easy or even possible. But when we only look at the aspects that affect FreeBSD, the details end up not being that important in the end. > I didn't dare do what you've done now, but since you touched it > last, I'm not worried :-) :-) > My proposed solution was to try to get the BSD label discontinued > and rely entirely on the native partitioning on the relevant > platforms, but that meant dealing with FAT extended partitions and > going back to libdisk and I couldn't get myself to do that. Trying to use the native scheme is generally good. But I do expect that FreeBSD on platform X knows or recognizes a disk created on or used by FreeBSD on platform Y. > Granted, fsck's generally choke on each others filesystems > pretty conclusively, but fsck will happily trash a database > stored in partition that previously contained a filesystem, > provided enough magic bits survive near the start. That's why I believe we need to attach real meaning to the partition type. We should disallow a newfs_ufs on a partition that is not of type freebsd-ufs. We should disallow swapon for a partition that is not of type freebsd-swap. etc.. With gpart it's trivial to change the partition type, so it's no hassle. The protection and support this gives users certainly outweighs the hassle IMO. > Overall this is a fine point, but I'd hate to make it easier to > trash filesystems. I agree. Enforcing that partition types match the content (within reason) is a good start. Already gpart checks the partition type for the dumpon command and fails when it's an obvious mismatch. > Lacking a reliable mechanism to keep the out-of-band filesystem > type consistent with the partition content, think dd(1), g_mirror > movements of data etc, I'm not too thrilled about this. Keeping consistency can only be the responsibility of the user/admin. We give them too much power to do it for them. All we can do is enforce and/or promote consistency by taking the partition type seriously. >> 6. Not all "disk" devices have a geometry. Especially md(4) >> devices and USB mass storage devices. This is a problem >> for newfs_msdos. With gpart, there's always a geometry [...] > > I will argue this is wrong. It's not wrong. It's a consequence. As long as there are partitioning schemes and file systems that use sectors, track and/or cyclinders we need to provide it. Even under GPT do we need a geometry simply because the EFI file system is the MSDOS file system. True, GPT itself doesn't need it, but that's not the point. > For md(4), you an define a geometry if you want one, and if > your plan is to move the filesystem to real media later, > you had better do so correctly. True. Geometries create problems. > In many cases you can infer the geometry from a preexisting MBR, > but presently our code does not do this, and as we know from the > Dangerously Dedicated Disk Layout, that is a pretty error-prone > heuristic in the first place. gpart does take this into account. When the underlying provider has no geometry, gpart synthesizes one. This geometry is "soft" in the sense that it can later be adjusted to match what partitioning schemes say it is (or should be). In either case, there can be a real mismatch. If that happens gpart will let you know and it can even reject the partitioning scheme. Again, this is based on the philosophy that there's meaning. Recording a geometry without there being a meaning is pointless. > Dreaming up default geometries will just hide from the user that > they are about to make a mistake. It's not the user that can make a mistake in this case. They're merely burdened by the inconsistencies due to sloppiness and/or bad design. > I would be better to fix umass/cam/da (cross out your own code) > to get the correct geometry from the usb-sticks. I totally agree. >> 7. gpart breaks the 8-partition barrier for bsdlabel. > > We should discontinue the bsdlabel, it has too many problems > and limitations. I agree, but compatibility is not bad and NetBSD does support more than 8 partitions. >> With gpart default, tools like fdisk and bsdlabel continue to >> work on new disks and disks that have no mounted file systems. >> In that respect there's no difference. However, they cannot >> be used when file systems are mounted. > > This is going to break more scripts than you think. Likely. > I think you should have talked a bit about naming of the > partitions ? Good point. With gpart, partition types are abstracted so that the user and tools are presented with a fixed alias and not with whatever identification scheme is used for a particular partitioning scheme. For example the MBR type 165 is presented by gpart as type "freebsd". When the underlying scheme is GPT, the on-disk identification is the following UUID: 516e7cb4-6ecf-11d6-8ff8-00022d09712b Of course, gpart doesn't have aliases for all possible partition types. Just the ones that we care about in FreeBSD. For a UFS partition, the gpart type is "freebsd-ufs". For a swap partition it is "freebsd-swap". On my server (FreeBSD 7.x) "gpart show" gives: ns1% gpart show => 63 80293185 ad0 MBR (41.1GB) 63 80292807 1 freebsd [active] (41.1GB) 80292870 378 - free - (193.5KB) => 0 80292807 ad0s1 BSD (41.1GB) 0 2097152 1 freebsd-ufs (1073.7MB) 2097152 16777216 2 freebsd-swap (8.6GB) 18874368 16777216 4 freebsd-ufs (8.6GB) 35651584 44641223 5 freebsd-ufs (22.9GB) => 34 976773101 ad1 GPT (500.1GB) 34 97656250 1 freebsd-ufs (50.0GB) 97656284 879116851 2 freebsd-zfs (450.1GB) => 34 976773101 ad2 GPT (500.1GB) 34 97656250 1 freebsd-ufs (50.0GB) 97656284 879116851 2 freebsd-zfs (450.1GB) => 34 1953525101 ad3 GPT (1000.2GB) 34 1953525101 1 freebsd-zfs (1000.2GB) => 34 976773101 ad4 GPT (500.1GB) 34 976773101 1 freebsd-zfs (500.1GB) => 34 976773101 ad5 GPT (500.1GB) 34 976773101 1 freebsd-zfs (500.1GB) The actual partitioning scheme, while shown, has been made irrelevant. A UFS or ZFS partition has the same abstract type no matter how the disk is partitioning. Currently gpart has aliases the following aliases: efi freebsd freebsd-boot freebsd-swap freebsd-ufs freebsd-vinum freebsd-zfs mbr Partition types for which we don't have aliases are shown in their raw (i.e. partitioning scheme specific) repesentation (numbers, UUIDs or labels). > Are there any compatibility or transition concerns in that area ? These are the problem areas I can think of: o My amd64 had a broken/invalid BSD label and was rejected by gpart. It had to be fixed-up first. I expect that this may be more common than we'd like it to be. Thus: previously accepted BSD labels may not be accepted by gpart due to better checking. o The partition given to the dumpon command needs to be of the right type. When switching to gpart on sparc64, the dumpon command failed because there are no partition types in the current sunlabel support. Assigning the proper partition types on sparc64 after the switch to gpart did the trick. This also applies to PowerPC... o I do not have a pc98 machine. I've been testing on a memory disk and validated inter-operability, but I don't have FreeBSD booting a running on pc98. -- Marcel Moolenaar xcllnt@mac.com From peter at wemm.org Thu Sep 25 21:45:15 2008 From: peter at wemm.org (Peter Wemm) Date: Thu Sep 25 21:45:21 2008 Subject: RFC: making gpart default In-Reply-To: <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> References: <1896.1222371977@critter.freebsd.dk> <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> Message-ID: On Thu, Sep 25, 2008 at 2:24 PM, Marcel Moolenaar wrote: > On Sep 25, 2008, at 12:46 PM, Poul-Henning Kamp wrote: [..] >> pretty conclusively, but fsck will happily trash a database >> stored in partition that previously contained a filesystem, >> provided enough magic bits survive near the start. > > That's why I believe we need to attach real meaning > to the partition type. We should disallow a newfs_ufs > on a partition that is not of type freebsd-ufs. We > should disallow swapon for a partition that is not > of type freebsd-swap. etc.. > > With gpart it's trivial to change the partition type, > so it's no hassle. The protection and support this > gives users certainly outweighs the hassle IMO. Don't forget that we currently support creating file systems on raw disk devices. eg: /dev/ad1. You are currently allowed to swapon /dev/ad2. There are a lot of those out there, you can't break it because people know where you work and will come find you. :) This however, is a different issue to switching GEOM_BSD + GEOM_MBR to GEOM_PART_BSD + GEOM_PART_MBR. (I think the partition type thing could be solved by specifying the heuristic as "if the partition *has a type* and its not ufs, then disallow ufs" etc. Don't forget to include --shoot-foot=allowed override mode.) -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From phk at phk.freebsd.dk Thu Sep 25 21:57:13 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu Sep 25 21:57:21 2008 Subject: RFC: making gpart default In-Reply-To: Your message of "Thu, 25 Sep 2008 14:24:49 MST." <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> Message-ID: <11708.1222379828@critter.freebsd.dk> In message <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com>, Marcel Moolenaar wri tes: >Granted, supporting all the details of a particular >scheme may not be easy or even possible. That's my worry about the unified UI, I keep thinking of the old usenet joke about the HP Cafeteria. >Trying to use the native scheme is generally good. >But I do expect that FreeBSD on platform X knows >or recognizes a disk created on or used by FreeBSD >on platform Y. Until we have an endianes aware UFS that's only half the solution however :-/ >That's why I believe we need to attach real meaning >to the partition type. We should disallow a newfs_ufs >on a partition that is not of type freebsd-ufs. [...] Ouch. That is a level of policy that I would not want to enforce on users. This should probably be pointed out for arch@'s careful consideration: we are potentially talking about quite a lot of "it used to work, now it complains" email. >> Overall this is a fine point, but I'd hate to make it easier to >> trash filesystems. > >I agree. Enforcing that partition types match the >content (within reason) is a good start. I don't see it that way, I see it as increasing the risk when people move partitions around with g_mirror or dd. >Keeping consistency can only be the responsibility >of the user/admin. So we ask the users to keep it consistent so it can protect the users by being consistent ? I'm not quite sure I see the benefit, but I see lots of trouble: Does that mean I can't newfs /dev/da1 without partitioning ? What about DVD/ISO images which don't have parititioning, with the exception of Sparc boot-media ? In my view, enforcing out of band labels on partitions amounts to nothing but pointlessly putting signs with "coffee-machine" and "stupid ISO9000 signage guy" on stuff. It might make people thing twice, but likely only about what the point is supposed to be and why it has to be so difficult. I would strongly advice going for an inband scheme instead, we have that in g_label already. >>> 6. Not all "disk" devices have a geometry. Especially md(4) >>> devices and USB mass storage devices. This is a problem >>> for newfs_msdos. With gpart, there's always a geometry [...] >> >> I will argue this is wrong. > >It's not wrong. It's a consequence. As long as there are >partitioning schemes and file systems that use sectors, >track and/or cyclinders we need to provide it. It's not a matter of providing or not, it's a matter of constructing or not. md(4) has no geometry, and you don't know what the user is going to use the filesystem for. If you construct a synthetic default geometry, the user is not made aware that he needs to *think* about the correct geometry. >> I would be better to fix umass/cam/da (cross out your own code) >> to get the correct geometry from the usb-sticks. > >I totally agree. Then don't provide a hack to hide the hurt, let the hurt hit where it belongs. >> This is going to break more scripts than you think. > >Likely. I do find your attitude to compatibility refreshing, but will take care to be at a safe distance when the consequence strikes :-) >> I think you should have talked a bit about naming of the >> partitions ? > >Good point. >On my server (FreeBSD 7.x) "gpart show" gives: > >ns1% gpart show >=> 63 80293185 ad0 MBR (41.1GB) > 63 80292807 1 freebsd [active] (41.1GB) > 80292870 378 - free - (193.5KB) > >=> 0 80292807 ad0s1 BSD (41.1GB) > 0 2097152 1 freebsd-ufs (1073.7MB) > 2097152 16777216 2 freebsd-swap (8.6GB) > 18874368 16777216 4 freebsd-ufs (8.6GB) > 35651584 44641223 5 freebsd-ufs (22.9GB) My question was really if these partitions have the same name in /dev/ as today, ad0s1[a-d] presumably ? (I would suggest gpart always show the /dev names) Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xcllnt at mac.com Thu Sep 25 23:49:35 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Thu Sep 25 23:49:40 2008 Subject: RFC: making gpart default In-Reply-To: References: <1896.1222371977@critter.freebsd.dk> <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> Message-ID: <37A80348-515C-48F4-8F88-A77679BEEA15@mac.com> On Sep 25, 2008, at 2:45 PM, Peter Wemm wrote: > On Thu, Sep 25, 2008 at 2:24 PM, Marcel Moolenaar > wrote: >> On Sep 25, 2008, at 12:46 PM, Poul-Henning Kamp wrote: > [..] >>> pretty conclusively, but fsck will happily trash a database >>> stored in partition that previously contained a filesystem, >>> provided enough magic bits survive near the start. >> >> That's why I believe we need to attach real meaning >> to the partition type. We should disallow a newfs_ufs >> on a partition that is not of type freebsd-ufs. We >> should disallow swapon for a partition that is not >> of type freebsd-swap. etc.. >> >> With gpart it's trivial to change the partition type, >> so it's no hassle. The protection and support this >> gives users certainly outweighs the hassle IMO. > > Don't forget that we currently support creating file systems on raw > disk devices. eg: /dev/ad1. You are currently allowed to swapon > /dev/ad2. There are a lot of those out there, you can't break it > because people know where you work and will come find you. :) When there's no partitioning scheme on the disk, gpart will not be involved and the checks won't happen. -- Marcel Moolenaar xcllnt@mac.com From xcllnt at mac.com Fri Sep 26 00:26:57 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Fri Sep 26 00:27:03 2008 Subject: RFC: making gpart default In-Reply-To: <11708.1222379828@critter.freebsd.dk> References: <11708.1222379828@critter.freebsd.dk> Message-ID: <834AD44B-2372-41D2-A952-85095569BB48@mac.com> On Sep 25, 2008, at 2:57 PM, Poul-Henning Kamp wrote: >> >> Trying to use the native scheme is generally good. >> But I do expect that FreeBSD on platform X knows >> or recognizes a disk created on or used by FreeBSD >> on platform Y. > > Until we have an endianes aware UFS that's only > half the solution however :-/ File system bugs are outside the scope of gpart. It's already a much better situation if we can see the partitions on a "foreign" disk, even if we can't interpret the data in it. >>> Overall this is a fine point, but I'd hate to make it easier to >>> trash filesystems. >> >> I agree. Enforcing that partition types match the >> content (within reason) is a good start. > > I don't see it that way, I see it as increasing the risk > when people move partitions around with g_mirror or dd. There's no risk that wasn't there already. In fact, certain things won't work, which means risk is reduced. >> Keeping consistency can only be the responsibility >> of the user/admin. > > So we ask the users to keep it consistent so it can > protect the users by being consistent ? No, we don't ask the user anything. We just reward the user with certain benefits when he/she is keeping things real. Changing the partition type to match the content is for the benefit of the admin as well. It's a win-win. > I'm not quite sure I see the benefit, but I see lots of trouble: > > Does that mean I can't newfs /dev/da1 without partitioning ? When there's no partitioning scheme, then gpart is not involved and we can obvouisly not check partition types. > What about DVD/ISO images which don't have parititioning, > with the exception of Sparc boot-media ? See above. > In my view, enforcing out of band labels on partitions amounts to > nothing but pointlessly putting signs with "coffee-machine" and > "stupid ISO9000 signage guy" on stuff. While I find some of the labels redundant, it would be a mistake to think that there isn't someone out there who depends on them. > I would strongly advice going for an inband scheme instead, > we have that in g_label already. It would be nice if g_label would use the labels that people put in the partition table for entries. > If you construct a synthetic default geometry, the user is not made > aware that he needs to *think* about the correct geometry. We obviously have 2 viewpoints here: On forces the user to worry about geometry, even if there's no need for it. The other avoids the user from having to think about geometry, even if the user should. I'd rather treat our users as knowledgeable and not as being stupid. I prefer to assume that the user knows when to worry about geometry so that we can make it easy in case it doesn't matter. It's all about the ability to specify a geometry when you create the partitioning scheme. This can not be done with gpart at this time, though. >>> This is going to break more scripts than you think. >> >> Likely. > > I do find your attitude to compatibility refreshing, but will take > care to be at a safe distance when the consequence strikes :-) There's limited compatibility in replacing arcane tools each with their own UI with a unified tool that's based on a totally different paradigm. Striving for tool-compatibility would not be successful by definition. It's much easier to rewrite bsdlabel and fdisk to use the gpart ctlreq I/F directly... >> On my server (FreeBSD 7.x) "gpart show" gives: >> >> ns1% gpart show >> => 63 80293185 ad0 MBR (41.1GB) >> 63 80292807 1 freebsd [active] (41.1GB) >> 80292870 378 - free - (193.5KB) >> >> => 0 80292807 ad0s1 BSD (41.1GB) >> 0 2097152 1 freebsd-ufs (1073.7MB) >> 2097152 16777216 2 freebsd-swap (8.6GB) >> 18874368 16777216 4 freebsd-ufs (8.6GB) >> 35651584 44641223 5 freebsd-ufs (22.9GB) > > My question was really if these partitions have > the same name in /dev/ as today, ad0s1[a-d] presumably ? Yes. The device naming has not changed. -- Marcel Moolenaar xcllnt@mac.com From phk at phk.freebsd.dk Fri Sep 26 06:34:25 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri Sep 26 06:34:31 2008 Subject: RFC: making gpart default In-Reply-To: Your message of "Thu, 25 Sep 2008 17:26:54 MST." <834AD44B-2372-41D2-A952-85095569BB48@mac.com> Message-ID: <1588.1222410862@critter.freebsd.dk> In message <834AD44B-2372-41D2-A952-85095569BB48@mac.com>, Marcel Moolenaar wri tes: >On Sep 25, 2008, at 2:57 PM, Poul-Henning Kamp wrote: >It's already a much better situation if we can >see the partitions on a "foreign" disk, even if >we can't interpret the data in it. Now, don't get too carried away with your creation, we have been able to do that since day 1 with GEOM :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From trhodes at FreeBSD.org Fri Sep 26 07:58:01 2008 From: trhodes at FreeBSD.org (Tom Rhodes) Date: Fri Sep 26 07:58:12 2008 Subject: RFC: making gpart default In-Reply-To: <1588.1222410862@critter.freebsd.dk> References: <834AD44B-2372-41D2-A952-85095569BB48@mac.com> <1588.1222410862@critter.freebsd.dk> Message-ID: <20080926033126.7fdfa739.trhodes@FreeBSD.org> On Fri, 26 Sep 2008 06:34:22 +0000 "Poul-Henning Kamp" wrote: > In message <834AD44B-2372-41D2-A952-85095569BB48@mac.com>, Marcel Moolenaar wri > tes: > >On Sep 25, 2008, at 2:57 PM, Poul-Henning Kamp wrote: > > >It's already a much better situation if we can > >see the partitions on a "foreign" disk, even if > >we can't interpret the data in it. > > Now, don't get too carried away with your creation, we have been > able to do that since day 1 with GEOM :-) Killjoy. :P -- Tom Rhodes From ivoras at freebsd.org Fri Sep 26 11:13:10 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Fri Sep 26 11:13:17 2008 Subject: RFC: making gpart default In-Reply-To: <37A80348-515C-48F4-8F88-A77679BEEA15@mac.com> References: <1896.1222371977@critter.freebsd.dk> <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> <37A80348-515C-48F4-8F88-A77679BEEA15@mac.com> Message-ID: Marcel Moolenaar wrote: >> Don't forget that we currently support creating file systems on raw >> disk devices. eg: /dev/ad1. You are currently allowed to swapon >> /dev/ad2. There are a lot of those out there, you can't break it >> because people know where you work and will come find you. :) > > When there's no partitioning scheme on the disk, > gpart will not be involved and the checks won't > happen. Will there be "generic" parition types by default? (i.e. the "don't care what's on it, let me do my newfs" type) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080926/6d21210c/signature.pgp From xcllnt at mac.com Fri Sep 26 15:26:10 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Fri Sep 26 15:26:16 2008 Subject: RFC: making gpart default In-Reply-To: References: <1896.1222371977@critter.freebsd.dk> <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> <37A80348-515C-48F4-8F88-A77679BEEA15@mac.com> Message-ID: <3B2BD6FE-85D4-49F8-81A0-B2AB048ACFE8@mac.com> On Sep 26, 2008, at 4:12 AM, Ivan Voras wrote: > Marcel Moolenaar wrote: > >>> Don't forget that we currently support creating file systems on raw >>> disk devices. eg: /dev/ad1. You are currently allowed to swapon >>> /dev/ad2. There are a lot of those out there, you can't break it >>> because people know where you work and will come find you. :) >> >> When there's no partitioning scheme on the disk, >> gpart will not be involved and the checks won't >> happen. > > Will there be "generic" parition types by default? (i.e. the "don't > care > what's on it, let me do my newfs" type) There's currently no such partition type and you don't need it to do a newfs. You can always do a newfs, irrespective of partition type. -- Marcel Moolenaar xcllnt@mac.com From talon at lpthe.jussieu.fr Fri Sep 26 22:07:45 2008 From: talon at lpthe.jussieu.fr (Michel Talon) Date: Fri Sep 26 22:07:51 2008 Subject: RFC: making gpart default Message-ID: <20080926215734.GA48113@lpthe.jussieu.fr> Marcel Moolenaar wrote: > That's why I believe we need to attach real meaning > to the partition type. We should disallow a newfs_ufs > on a partition that is not of type freebsd-ufs. We > should disallow swapon for a partition that is not > of type freebsd-swap. etc.. However at present it is very convenient that one can share the swap partitions with Linux on a dual boot machine (at the price of running mkswap in the Linux boot scripts). I don't clearly see the benefit coming from enforcing the partition type. After all you should be able to do whatever you like with a chunk of your hard disk, for example cyclic buffers for inn, database space for databases working on raw disk, etc. space for ZFS which doesn't care about partition type (i have a ZFS partition which is marked of type Linux fs) and obviously swap. -- Michel TALON From xcllnt at mac.com Sat Sep 27 02:04:44 2008 From: xcllnt at mac.com (Marcel Moolenaar) Date: Sat Sep 27 02:04:51 2008 Subject: RFC: making gpart default In-Reply-To: <20080926215734.GA48113@lpthe.jussieu.fr> References: <20080926215734.GA48113@lpthe.jussieu.fr> Message-ID: <1B08091F-5807-43C0-AAD5-A693D207BF5E@mac.com> On Sep 26, 2008, at 2:57 PM, Michel Talon wrote: > Marcel Moolenaar wrote: >> That's why I believe we need to attach real meaning >> to the partition type. We should disallow a newfs_ufs >> on a partition that is not of type freebsd-ufs. We >> should disallow swapon for a partition that is not >> of type freebsd-swap. etc.. > > However at present it is very convenient that one can share the swap > partitions with Linux on a dual boot machine (at the price of running > mkswap in the Linux boot scripts). This is already implemented. > After all you should be able > to do whatever you like with a chunk of your hard disk, for example > cyclic buffers for inn, database space for databases working on > raw disk, etc. space for ZFS which doesn't care about partition type > (i have a ZFS partition which is marked of type Linux fs) and > obviously > swap. You can still do whatever you want with a partition. -- Marcel Moolenaar xcllnt@mac.com From nyan at jp.FreeBSD.org Sat Sep 27 13:08:45 2008 From: nyan at jp.FreeBSD.org (Takahashi Yoshihiro) Date: Sat Sep 27 13:08:52 2008 Subject: RFC: making gpart default In-Reply-To: <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> References: <1896.1222371977@critter.freebsd.dk> <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> Message-ID: <20080927.213040.27797529.nyan@jp.FreeBSD.org> In article <901FB1DE-BA4C-405C-8F8E-AA8CCC6A89FA@mac.com> Marcel Moolenaar writes: > o I do not have a pc98 machine. I've been testing on > a memory disk and validated inter-operability, but > I don't have FreeBSD booting a running on pc98. I tested gpart on pc98 and found some problems. 1. A disklabel checking in g_part_bsd_read() is failed. 2. g_part_pc98_dumpconf() does not print a slice name. 3. g_part_pc98_probe() is too strict. 4. It's failed to recognize a big SCSI disk as BSD slice. 5. A (bit strange) FAT slice is recoginized as BSD slice. (6. Do we need more work for libdisk?) I fixed 1, 2 and 3. The patch is: http://home.jp.freebsd.org/~nyan/geom/g_part_pc98.diff My environment is: http://home.jp.freebsd.org/~nyan/geom/dmesg.txt http://home.jp.freebsd.org/~nyan/geom/geom_pc98.txt http://home.jp.freebsd.org/~nyan/geom/g_part_pc98.txt --- TAKAHASHI Yoshihiro From rizzo at iet.unipi.it Sun Sep 28 10:19:28 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sun Sep 28 10:19:35 2008 Subject: dynamic update of usb/pci/quirks tables Message-ID: <20080928100731.GA49323@onelab2.iet.unipi.it> For some time I have been thinking of a way to solve the problem in the subject., which is hitting me almost every time i try to connect a newly bought device (scanner, phone, camera, mp3 player, even some flash keys) to FreeBSD. I was wondering what you people think about the following approach, which can be implemented by a trivial userland program (kldpatch) and probably be used with little effort or code bloat also in the loader. PROBLEM DESCRIPTION: A common problem with newer USB (but also other) peripheral is that often the kernel does not have the correct product and vendor numbers in its internal tables used for device matching or for applying quirks to certain devices. The problem arises quite often with e.g. uscanner (where the matching must be done through the table because there is no specific device class), or with various umass devices (flash keys, cameras, phones, etc.) which tend to require quirks because they do not implement all the default features used by the umass driver. APPROACH: The following approach can be used to patch the in-kernel structures used to store the match/quirks tables: + use kldfind and kldsym to locate the module, address and size for the desired data structure; + use kvm_open/kvm_read/kvm_write to read and possibly update the table. Of course patching the live kernel is dangerous, so one way to make the process a little less risky is to put some additional controls in the frontend program which implements the above. In particular, the frontend will contain a table of the form module symbol record_size record_format... which defines which modules/symbols can be patched and how each record is supposed to be structured. As an example: uscanner.ko uscanner_devs 8 2 2 4 umass.ko umass_devdescrs 16 4 4 4 2 2 if_sis.ko sis_devs 8 2 2 %s tells that uscanner has 8-byte records made of 3 numeric fields of size 2 2 4, umass.ko has 16-byte records with 5 numeric fields, if_sis has 8-byte records whose last one is a string pointer, (I could not find an instance with a fixed-size string). The frontend will accept commands of the form kldpatch if_sis.ko sis_devs write @3 0x1234 0x5678 "temporary entry" which means 'patch record 3 with the values specified". The frontend will make sure that operands are of the required size, that the table entry is within the table, and that (in case of patching constant strings) the replacement does not exceed the original length of the string. DISCUSSION: This approach does not address one case which is the "translation" of one PCI/USB/bus id into another one to make a driver believe that device X is instead device Y. It does address, however, the relatively common case where recognised IDs are in an explicit device or quirks table. I believe the same functionality can be added to the loader/kernel itself, e.g. by telling /boot/loader to load a text file with all the patches (i.e. the arguments to kldpatch) to be applied, and hooking into the kernel so that right after boot, and right after a module has been kldloaded, the patches are applied. In terms of preventing the user from doing mistakes, there are a few safety checks implemented by the internal table, which should at least prevent the user from writing outside the allowed areas. Clearly there is the issue of a possible mismatch between the record structure as known to kldpatch, and the actual record structure as defined in the .ko files -- but i don't know of a reasonably simple method to extract the structure info from the C source and pass them to kldpatch -- one way could be, perhaps, to add a special symbol to the module itself so perhaps at least the structure info can be declared within the module, and it is easier to be kept in sync. ---- comments ? cheers luigi From hselasky at c2i.net Sun Sep 28 11:55:24 2008 From: hselasky at c2i.net (Hans Petter Selasky) Date: Sun Sep 28 11:55:30 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928100731.GA49323@onelab2.iet.unipi.it> References: <20080928100731.GA49323@onelab2.iet.unipi.it> Message-ID: <200809281257.18248.hselasky@c2i.net> On Sunday 28 September 2008, Luigi Rizzo wrote: > Of course patching the live kernel is dangerous Why can't you put all the quirks in a loadable module like in USB2, that can be unloaded and loaded at any time ? --HPS From rizzo at iet.unipi.it Sun Sep 28 13:02:58 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sun Sep 28 13:03:04 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <200809281257.18248.hselasky@c2i.net> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <200809281257.18248.hselasky@c2i.net> Message-ID: <20080928130629.GA50928@onelab2.iet.unipi.it> On Sun, Sep 28, 2008 at 12:57:17PM +0200, Hans Petter Selasky wrote: > On Sunday 28 September 2008, Luigi Rizzo wrote: > > Of course patching the live kernel is dangerous > > Why can't you put all the quirks in a loadable module like in USB2, that can > be unloaded and loaded at any time ? This would be a valid approach if we were writing the system from scratch, but we aren't, so this would require changing all source modules to adapt to the new mechanism. BTW - in the base system there is a handful of users of kvm_write: /usr/src/usr.sbin/dconschat/dconschat.c /usr/src/usr.sbin/kgmon/kgmon.c /usr/src/gnu/usr.bin/binutils/gdb/kvm-fbsd.c /usr/src/gnu/usr.bin/gdb/kgdb/trgt.c users of kvm_read are many more: 8 /usr/src/gnu/usr.bin/gdb/kgdb/trgt_i386.c 8 /usr/src/gnu/usr.bin/gdb/kgdb/kthr.c 7 /usr/src/usr.sbin/pstat/pstat.c 7 /usr/src/usr.sbin/kernbb/kernbb.c 7 /usr/src/usr.bin/ktrdump/ktrdump.c 6 /usr/src/usr.sbin/kgmon/kgmon.c 6 /usr/src/sbin/dmesg/dmesg.c 6 /usr/src/gnu/usr.bin/gdb/kgdb/trgt_arm.c 5 /usr/src/tools/diag/dumpvfscache/dumpvfscache.c 4 /usr/src/usr.bin/ipcs/ipcs.c 4 /usr/src/contrib/ipfilter/lib/kmem.c 3 /usr/src/usr.sbin/iostat/iostat.c 3 /usr/src/usr.sbin/dconschat/dconschat.c 3 /usr/src/usr.sbin/asf/asf_kvm.c 3 /usr/src/usr.bin/nfsstat/nfsstat.c 3 /usr/src/usr.bin/fstat/fstat.c 3 /usr/src/tools/tools/umastat/umastat.c 3 /usr/src/lib/libmemstat/memstat_uma.c 3 /usr/src/lib/libmemstat/memstat_malloc.c 2 /usr/src/usr.sbin/ifmcstat/ifmcstat.c 2 /usr/src/usr.bin/vmstat/vmstat.c 2 /usr/src/usr.bin/fstat/fstat.h 2 /usr/src/usr.bin/bluetooth/btsockstat/btsockstat.c 2 /usr/src/libexec/rpc.rstatd/rstat_proc.c 2 /usr/src/lib/libdevstat/devstat.c 2 /usr/src/gnu/usr.bin/gdb/kgdb/trgt_sparc64.c 2 /usr/src/gnu/usr.bin/gdb/kgdb/trgt_powerpc.c 2 /usr/src/gnu/usr.bin/gdb/kgdb/trgt_ia64.c 2 /usr/src/gnu/usr.bin/gdb/kgdb/trgt_amd64.c 2 /usr/src/contrib/ipfilter/ipsend/sock.c 1 /usr/src/usr.bin/systat/main.c 1 /usr/src/usr.bin/systat/fetch.c 1 /usr/src/usr.bin/netstat/main.c 1 /usr/src/gnu/usr.bin/gdb/kgdb/trgt.c 1 /usr/src/gnu/usr.bin/binutils/gdb/kvm-fbsd.c cheers luigi From imp at bsdimp.com Sun Sep 28 20:00:55 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Sun Sep 28 20:01:02 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928100731.GA49323@onelab2.iet.unipi.it> References: <20080928100731.GA49323@onelab2.iet.unipi.it> Message-ID: <20080928.135855.1708680935.imp@bsdimp.com> uscanner_devs and sis_devs aren't quirks. They are device tables. You've repeated ignored the mapping idea that I've posted. You can't just add stuff to tables randomly and expect that to work (says someone who has actually done this to lots of drivers in the tree in the pccard era). the driver has to know what kind of device to treat it as. Putting a translation table into the kernel is much easier and you don't have to worry about hokey kludges like what you describe with 'patch'. Maybe it will work out for the other tables you want to update, but it won't work well for device tables. Warner From alfred at freebsd.org Sun Sep 28 20:19:48 2008 From: alfred at freebsd.org (Alfred Perlstein) Date: Sun Sep 28 20:19:55 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928.135855.1708680935.imp@bsdimp.com> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <20080928.135855.1708680935.imp@bsdimp.com> Message-ID: <20080928201948.GE36572@elvis.mu.org> * M. Warner Losh [080928 13:01] wrote: > uscanner_devs and sis_devs aren't quirks. They are device tables. > > You've repeated ignored the mapping idea that I've posted. You can't > just add stuff to tables randomly and expect that to work (says > someone who has actually done this to lots of drivers in the tree in > the pccard era). the driver has to know what kind of device to treat > it as. > > Putting a translation table into the kernel is much easier and you > don't have to worry about hokey kludges like what you describe with > 'patch'. > > Maybe it will work out for the other tables you want to update, but it > won't work well for device tables. I really like the idea of using a kmod to just add the new device strings.. (some form of what Hans did). -Alfred From rizzo at iet.unipi.it Sun Sep 28 21:33:07 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sun Sep 28 21:33:14 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928.135855.1708680935.imp@bsdimp.com> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <20080928.135855.1708680935.imp@bsdimp.com> Message-ID: <20080928213640.GA55310@onelab2.iet.unipi.it> Hi Warner, On Sun, Sep 28, 2008 at 01:58:55PM -0600, M. Warner Losh wrote: > uscanner_devs and sis_devs aren't quirks. They are device tables. i actually also mentioned umass which is definitely a quirks table (and to tell the truth, uscanner_devs has a flags field as well, even though it only supports 2 values). > You've repeated ignored the mapping idea that I've posted. You can't I have not ignored the mapping idea -- i explicitly mentioned it in my email, saying that my suggestion does not handle device mapping. I do like the device mapping idea, i think it is a useful one, but it is not sufficient, because it doesn't cover the case of quirks tables (i wouldn't know how to actually implement the mapping thing, but that's just my ignorance in that area of the kernel - i am sure i can figure out with some studying). This said, the kldsym/kmem based approach has the advantage that it can be implemented as a port (if you forget the /boot/loader support) and be used on a pre-existing system without having to rebuild kernels or modules. Some times this can be useful. > just add stuff to tables randomly and expect that to work (says > someone who has actually done this to lots of drivers in the tree in > the pccard era). the driver has to know what kind of device to treat > it as. i don't understand why you believe that i am suggesting to "randomly adding stuff" to the tables. I know (and I wrote it in my email) that playing with tables is dangerous and one should know what he is doing before trying. Certainly, if i have a device with unknown specs, the only way to tell what it can do is by trial and error, looking at a similar devices, and playing with mapping or quirks tables. Sure, if the code itself outside the table makes use of the actual IDs, then adding entries to the tables may not work. But, if and when someone has determined that the patching is safe (e.g. because it is a quirks table, as in the umass case, or because the actual IDs are not used anywhere else in the code, as in the case of uscanner or if_rl or probably many other places), I don't understand why we should artificially make life harder to FreeBSD users asking them to "upgrade to next release" or "patch and rebuild the kernel". Sometimes you just can't do that -- take e.g. the case of embedded platforms where you don't have a compiler available or possibly not even full kernel sources -- after all it's BSD licensed! > Putting a translation table into the kernel is much easier and you > don't have to worry about hokey kludges like what you describe with > 'patch'. > > Maybe it will work out for the other tables you want to update, but it > won't work well for device tables. I am open to all solutions that can make FreeBSD more compatible with external hardware. It will be great if we can implement your mapping idea. It will be great if there is a better way of handling quirks tables. cheers luigi From imp at bsdimp.com Sun Sep 28 22:09:38 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Sun Sep 28 22:09:44 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928201948.GE36572@elvis.mu.org> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <20080928.135855.1708680935.imp@bsdimp.com> <20080928201948.GE36572@elvis.mu.org> Message-ID: <20080928.161010.1649769915.imp@bsdimp.com> In message: <20080928201948.GE36572@elvis.mu.org> Alfred Perlstein writes: : * M. Warner Losh [080928 13:01] wrote: : > uscanner_devs and sis_devs aren't quirks. They are device tables. : > : > You've repeated ignored the mapping idea that I've posted. You can't : > just add stuff to tables randomly and expect that to work (says : > someone who has actually done this to lots of drivers in the tree in : > the pccard era). the driver has to know what kind of device to treat : > it as. : > : > Putting a translation table into the kernel is much easier and you : > don't have to worry about hokey kludges like what you describe with : > 'patch'. : > : > Maybe it will work out for the other tables you want to update, but it : > won't work well for device tables. : : I really like the idea of using a kmod to just add the new device : strings.. (some form of what Hans did). The problem is that except for the most trivial driver, that doesn't work. Most of the NIC drivers in the tree do special things based on what chip they thing they are talking to. An unknown chip may work, but it would work a lot better if the unknown chip is compatible with a specific chip, as opposed to being compatible with the driver. I'm cool with having the overriden ids also override the device description too... Warner From imp at bsdimp.com Sun Sep 28 22:18:58 2008 From: imp at bsdimp.com (M. Warner Losh) Date: Sun Sep 28 22:19:05 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928213640.GA55310@onelab2.iet.unipi.it> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <20080928.135855.1708680935.imp@bsdimp.com> <20080928213640.GA55310@onelab2.iet.unipi.it> Message-ID: <20080928.161905.-1350498240.imp@bsdimp.com> In message: <20080928213640.GA55310@onelab2.iet.unipi.it> Luigi Rizzo writes: : Hi Warner, : : On Sun, Sep 28, 2008 at 01:58:55PM -0600, M. Warner Losh wrote: : > uscanner_devs and sis_devs aren't quirks. They are device tables. : : i actually also mentioned umass which is definitely a quirks table : (and to tell the truth, uscanner_devs has a flags : field as well, even though it only supports 2 values). True. The umass stuff is definitely quirk table expansion. : > You've repeated ignored the mapping idea that I've posted. You can't : : I have not ignored the mapping idea -- i explicitly mentioned it : in my email, saying that my suggestion does not handle device : mapping. : I do like the device mapping idea, i think it is a useful one, but : it is not sufficient, because it doesn't cover the case of quirks tables : (i wouldn't know how to actually implement the mapping thing, : but that's just my ignorance in that area of the kernel - i am : sure i can figure out with some studying). I posted diffs that did everything except the maintaining the table in like 15-20 lines of code for PCI. The hard part is the interface to /boot/loader and any desire to tweak the table and rescan devices... : This said, the kldsym/kmem based approach has the advantage that : it can be implemented as a port (if you forget the /boot/loader support) : and be used on a pre-existing system without having to rebuild kernels : or modules. Some times this can be useful. I doubt that this is a good selling point. It is rare that you add new hardware to a system that's already running that's compatible with the old hardware, but needs updated driver support. I'm skeptical that the kld solution would be robust enough for people who have such situations to trust more than a source rebuild to pickup the new mapping table. : > just add stuff to tables randomly and expect that to work (says : > someone who has actually done this to lots of drivers in the tree in : > the pccard era). the driver has to know what kind of device to treat : > it as. : : i don't understand why you believe that i am suggesting to : "randomly adding stuff" to the tables. : : I know (and I wrote it in my email) that playing with tables is : dangerous and one should know what he is doing before trying. I guess we're saying the same thing: This is a large caliber gun that can be useful, but also has an unusually high foot-shooting potential. : Certainly, if i have a device with unknown specs, the only way to : tell what it can do is by trial and error, looking at a similar : devices, and playing with mapping or quirks tables. : Sure, if the code itself outside the table makes use of : the actual IDs, then adding entries to the tables may not work. Correct. This is what I do with PC Card and PCI drivers when I find hardware at garage sales that I can find no other data on. Usually a model number and version will get me enough data to know. But wireless cards seem to have busted that :-(. : But, if and when someone has determined that the patching is safe : (e.g. because it is a quirks table, as in the umass case, or because : the actual IDs are not used anywhere else in the code, as in the : case of uscanner or if_rl or probably many other places), I don't : understand why we should artificially make life harder to FreeBSD : users asking them to "upgrade to next release" or "patch and rebuild : the kernel". Sometimes you just can't do that -- take e.g. the : case of embedded platforms where you don't have a compiler available : or possibly not even full kernel sources -- after all it's BSD licensed! I think the embedded argument oversells the capability here. It assumes the embedded systems let you get to the boot loader prompt, or that it even uses /boot/loader. : > Putting a translation table into the kernel is much easier and you : > don't have to worry about hokey kludges like what you describe with : > 'patch'. : > : > Maybe it will work out for the other tables you want to update, but it : > won't work well for device tables. : : I am open to all solutions that can make FreeBSD more compatible : with external hardware. It will be great if we can implement your : mapping idea. It will be great if there is a better way of handling : quirks tables. Let's pursue both ideas. I'll implement it for usb, pci, pc card and cardbus and we'll see where it goes. Warner From rizzo at iet.unipi.it Sun Sep 28 22:23:44 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sun Sep 28 22:24:13 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928.161010.1649769915.imp@bsdimp.com> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <20080928.135855.1708680935.imp@bsdimp.com> <20080928201948.GE36572@elvis.mu.org> <20080928.161010.1649769915.imp@bsdimp.com> Message-ID: <20080928222718.GA56058@onelab2.iet.unipi.it> On Sun, Sep 28, 2008 at 04:10:10PM -0600, M. Warner Losh wrote: > In message: <20080928201948.GE36572@elvis.mu.org> > Alfred Perlstein writes: > : * M. Warner Losh [080928 13:01] wrote: ... > : > Maybe it will work out for the other tables you want to update, but it > : > won't work well for device tables. > : > : I really like the idea of using a kmod to just add the new device > : strings.. (some form of what Hans did). > > The problem is that except for the most trivial driver, that doesn't > work. Most of the NIC drivers in the tree do special things based on > what chip they thing they are talking to. An unknown chip may work, it really depends on what you are looking at -- in some drivers the "special things" are called inline in the code looking at the device IDs; in other drivers, they are implemented as functions that are called depending on the content of the quirks info, so the behaviour is (or aims to be) entirely table driven. In this respect I believe USB stuff tends to be more table-driven (probably because there is also a huge variety of devices, and the inline approach wouldn't scale well). NIC drivers are different as most of them are derived one from another by copying and modifications, and there wasn't always a lot of design involved in making them generic and configurable -- nor was it worth the effort once they are working (and i don't mean to blame anyone -- there is no point to optimize a driver for a broken piece of hardware if you can get better hw). cheers luigi From rizzo at iet.unipi.it Sun Sep 28 22:31:27 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sun Sep 28 22:31:37 2008 Subject: dynamic update of usb/pci/quirks tables In-Reply-To: <20080928.161905.-1350498240.imp@bsdimp.com> References: <20080928100731.GA49323@onelab2.iet.unipi.it> <20080928.135855.1708680935.imp@bsdimp.com> <20080928213640.GA55310@onelab2.iet.unipi.it> <20080928.161905.-1350498240.imp@bsdimp.com> Message-ID: <20080928223501.GB56058@onelab2.iet.unipi.it> On Sun, Sep 28, 2008 at 04:19:05PM -0600, M. Warner Losh wrote: > In message: <20080928213640.GA55310@onelab2.iet.unipi.it> > Luigi Rizzo writes: ... > : This said, the kldsym/kmem based approach has the advantage that > : it can be implemented as a port (if you forget the /boot/loader support) > : and be used on a pre-existing system without having to rebuild kernels > : or modules. Some times this can be useful. > > I doubt that this is a good selling point. It is rare that you add > new hardware to a system that's already running that's compatible with with umass devices (including cameras phones and mp3 players) it happens all the time, and those you can connect to many single board computers these days. Same with several scanners and usb-serial adapters. > : I am open to all solutions that can make FreeBSD more compatible > : with external hardware. It will be great if we can implement your > : mapping idea. It will be great if there is a better way of handling > : quirks tables. > > Let's pursue both ideas. I'll implement it for usb, pci, pc card and > cardbus and we'll see where it goes. ok good -- no rush, i was just trying to figure out how to address the problem, after being hit two more times last week! cheers luigi From bugmaster at FreeBSD.org Mon Sep 29 11:06:47 2008 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Sep 29 11:07:07 2008 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200809291106.m8TB6k5K040729@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total.